Solutions
Reference code for all 4 challenges. Share only after students have attempted independently.
01
Control the Car
Stage 1 — Basic movement
def choose_action(state):
return "ACCELERATE"
Stage 2 — Wall avoidance
def choose_action(state):
front = state["front_distance"]
left = state["left_distance"]
right = state["right_distance"]
if front < 30:
return "BRAKE"
if left < 20:
return "TURN_RIGHT"
if right < 20:
return "TURN_LEFT"
return "ACCELERATE"
02
Read the Sensors
Full sensor solution (reaches checkpoint 1)
def choose_action(state):
speed = state["speed"]
front = state["front_distance"]
left = state["left_distance"]
right = state["right_distance"]
front_left = state["front_left_distance"]
front_right = state["front_right_distance"]
track_angle = state["track_angle"]
if front < 18:
return "BRAKE"
if min(front, front_left, front_right) < 45 and speed > 55:
return "BRAKE"
if left < 18:
return "TURN_RIGHT"
if right < 18:
return "TURN_LEFT"
if front_left < 30:
return "TURN_RIGHT"
if front_right < 30:
return "TURN_LEFT"
if track_angle < -12:
return "TURN_RIGHT"
if track_angle > 12:
return "TURN_LEFT"
return "ACCELERATE"
03
Design Rewards
Optimised reward function
def compute_reward(state, action, next_state, event):
reward = 0.0
speed = next_state["speed"]
left = next_state["left_distance"]
right = next_state["right_distance"]
front = next_state["front_distance"]
# Progress toward checkpoints
progress = next_state["checkpoint_progress"] - state["checkpoint_progress"]
reward += progress * 50.0
# Speed bonus (only when not in danger)
if min(left, right, front) > 25:
reward += speed * 0.01
# Wall proximity penalty
if left < 15 or right < 15:
reward -= 3.0
if front < 20:
reward -= 4.0
# Events
if event == "collision":
reward -= 80.0
if event == "checkpoint":
reward += 100.0
if event == "finish":
reward += 500.0
return reward
04
Train the Agent
Instructor notes
500 episodes is the minimum for reliable convergence on the default track.
Early episodes look random — epsilon (exploration rate) starts at 1.0 and decays. That is expected.
If reward plateaus, reset training and try a denser reward function (more frequent positive signals).
The before/after replay is the clearest way to demonstrate learning to students.
Reset leaderboard: DriveAI > Instructor tab > Reset Leaderboard
Reset training: DriveAI > Instructor tab > Reset Training Data
Save / load model: DriveAI > Training tab > Save Model / Load Model
Instructor session active