Overview
A physical inverted pendulum balanced with reinforcement learning rather than a hand-tuned controller: I trained the policy in simulation and deployed it to real hardware with real-time embedded sensing.
Back to projectsReinforcement Learning
Partnered with Yixon Zhou
Inverted Pendulum: Sim-to-Real RL
Sim-to-real reinforcement learning project that balances a physical inverted pendulum, using ONNX deployment and magnetic encoder feedback to run the trained policy on real hardware.
Project Summary
What I built
A short look at the build, the main technical choices, and the pieces I iterated along the way.
What I Built
My partner Yixon Zhou and I designed and built the pendulum mechanism from scratch, then I trained a DQN balancing policy in Gymnasium, converted the PyTorch model to ONNX, and ran inference on a Raspberry Pi using high-frequency AS5600 encoder observations.
Technical Highlights
The project combines custom reward design, observation-noise injection for robustness, low-latency edge deployment, and mechanical iteration around encoder placement from V1 to a motor-side V2 redesign.
Demos
Demos and design process
Videos and images from the current build, earlier iterations, and the small details that shaped the project.
video
Final Balancing Demo
image
Initial Motor Packaging (No Encoder)
This first packaging version exposed localization issues that had to be addressed in the next iteration.
image
Motor Packaging (With Encoder)
Added motor rotation sensing to help localize cart position and estimate cart velocity.
image
Pole Encoder
Used to measure pole angle and estimate angular velocity.
image
Pendulum Assembly View
Overall pendulum mechanism view showing the final physical assembly.