Generating low-level robot controllers often requires manual parameters tuning and significant system knowledge, which can result in long design times for highly specialized controllers. Moreover, experiments for microrobot control in real life can be costly for reliability test, tune, and validate the controller design. To address the problem of rapidly generating low- level general controllers without domain knowledge, we propose using model-based reinforcement learning (MBRL) trained in a simulated environment. We have been making progress on MBRL along two thrusts: modeling long-term dynamics of robots and distilling compute heavy model predict control (MPC) into a reactive neural network policy. The accurate long-term predictions are done by reframing state-action data to include time dependance across trajectories (useful because less online compute and replanning is needed when able to reason into the future). The MPC distillation is done by planning offline extensively and re-weighting the controller to prioritize states that lead to the goal, rather than equally weighting all previously chosen actions, even if they were erroneous. Initial results showed the capabilities of MBRL on a simulated quadruped robot adapted to terrain and input command with only motor input, and two-axis accelerometer. Additional testing using a scaled up robot platform has supported that controllers generated based on simulation can be trained to be deployed on different robot configurations in the real world through domain randomization. The goal is to apply the most data efficient and stable methods to microrobots (hexapod, ionocraft, jumper) to accomplish tasks such as walking around obstacles on microscale robots using on-board processing through Single Chip microMote (SCuM, BPN803).
Research currently funded by: Industry Sponsored