Continuous-time reinforcement learning (CT-RL) algorithms hold great promise in real-world control applications. Adaptive dynamic programming (ADP)-based CT-RL algorithms, especially their theoretical developments, have achieved great successes. However, these methods have not been demonstrated for solving realistic or meaningful learning control problems. This work leverages various ideas from the well-established field of classical control, in particular Kleinman's algorithm, plus learning from nonlinear state/action data to achieve improved time/data efficiency and well-behaved HSV system responses.
Researchers at Arizona State University have developed a Decentralized Excitable Integral Reinforcement Learning (DEIRL) platform that has comprehensive performance evaluations toward real world flight implementations. This new algorithm is designed for control of continuous time affine-nonlinear hypersonic vehicles (HSVs). DEIRL integrates classical control principles with advanced reinforcement learning to create a decentralized control framework tailored for hypersonic vehicles. It partitions complex vehicle dynamics into manageable subproblems and introduces a reference input-based exploration method to enhance learning efficiency and stability. This model-based algorithm converges to optimal control laws while ensuring robustness against uncertainties and modeling errors, outperforming existing methods in both theory and practical application.
Potential Applications:
- Defense and military aerospace control systems.
- Commercial hypersonic transit vehicles
- Space delivery and launch vehicle guidance
Benefits and Advantages:
- Robust performance – models under uncertainties and varying initial conditions
- Efficient – Decentralized control structure reduces problem complexity and dimensionality
- Effective – Outperforms classical LQR and feedback linearization techniques
For more information about this opportunity, please see