Overview
DeCYVLery Bots was a 3rd-place CYVL AI Hackathon startup pitch for autonomous sidewalk delivery robots trained inside city-scale digital twins generated from CYVL LiDAR data.
Back to projectsReinforcement Learning
DeCYVLery Bots
Startup pitch and reinforcement-learning simulation prototype for autonomous sidewalk delivery robots trained in digital twins built from CYVL LiDAR street data.
Project Summary
What I built
A short look at the build, the main technical choices, and the pieces I iterated along the way.
What I Built
I took a specific chunk of a Boston street, simulated pedestrians as moving pill-shaped agents, and trained RSL-RL navigation policies on cloud A100 compute.
Technical Highlights
The design process compared four policy iterations across training length and observation stack: undertrained baseline, camera-only, camera + LiDAR, and a final LiDAR-only policy.
Design Process
Final model and training iterations
The final policy is shown first, followed by the training iterations that shaped the sensing stack and reward behavior.
Final model - Iteration 4: LiDAR-only navigation
The strongest policy removed the camera and relied on LiDAR-style observations plus GPS-style current and target coordinates for direction.
- Smoothly navigates around pedestrians while staying on the sidewalk.
- Can drive near sidewalk edges without falling off, avoiding the sidewalk-identification failure from camera-based runs.
- Does not back away from people as aggressively as the earlier camera and camera + LiDAR policies.
video
Iteration 4 - LiDAR-Only Final Model
Iteration 1 - undertrained baseline
The first training run established the simulation setup but did not train long enough to converge.
- Used the Boston street digital twin and pedestrian-agent setup.
- Did not converge because the training run was too short.
- Served as the baseline for longer training and sensing-stack experiments.
video
Iteration 1 - Baseline Training Run
Iteration 2 - camera-based pedestrian detection
The second policy trained longer and used a camera mounted on top of the robot, but camera-only observations could not accurately gauge depth in the street scene.
- Used GPS-style current and goal coordinates to provide the robot with direction of motion.
- The robot would sometimes run into pedestrians because it could not gauge depth well from camera-only observations.
- The robot sometimes fell off the sidewalk because only the sidewalk had collisions enabled and camera observations did not reliably capture sidewalk edges.
video
Iteration 2 - Camera Policy
Iteration 3 - Camera + LiDAR
Adding LiDAR improved navigation stability and reduced sidewalk-edge failures, but the camera + LiDAR policy was particularly cautious around people.
- Camera + LiDAR observations made the policy much more stable than the camera-only version.
- The robot no longer fell off the sidewalk in the same way as Iteration 2.
- It sometimes backed away from approaching pedestrians, visible near the end of the video.
video