On the ESO based reinforcement learning for pure feedback systems

The control of pure feedback system, which is widely used but has non-affine property, has always been an important and challenging problem. In order to achieve precise tracking control of pure feedback system through improving the disturbance rejection ability of existing reinforcement learning algorithm, a reinforcement learning (RL) control strategy based on extended state observer (ESO) is proposed in this paper. In the proposed method, the extended state observer can reject the total disturbances and transform the pure feedback system which is in an input-output predictor from to overcome the non-causal problem into a cascade integral form. This allows the continuous reinforcement learning strategy of the actor-critic (AC) structure not to depend on the detailed model information, which makes it practically data-driven. It is worth noting that, in order to further improve the ability to track the changing reference trajectory, a novel curvature acceleration factor is proposed, which can adjust the learning speed of the reinforcement learning controller according to the curvature of the reference trajectory. The validity of the proposed algorithm is verified by the simulation results.