my model is a double pendulum that consists of 4 states and 1 input.
The first two states describe the angles of the pendulum elements with
regard to the vertical axis. (If the double pendulum is in the upright
position the angles are zero.)
The third and fourth state describe the respective angular velocities.
The scalar input is a torque applied at the center where the pendulum is
mounted.
My goal is to stabilize the double pendulum in the upright position. i.e.
x_des = (0, 0, 0, 0)
My initial condition is set very close to the desired position: x_init =
(0.2 , 0 , 0 , 0)
There are no constraints on the input.
Now my problem is the following:
I am using GRAMPC via Matlab.
Whenever I run GRAMPC on this model the pendulum pretty much just hangs
down, or stabilizes in the downward position.
This happens both in Simulink and via function call "startMPC".
At first I thought my implementation of the dynamics is erroneous. But after
carefully checking it twice, I realized that the prediction GRAMPC makes is
actually pretty accurate.
With a horizon of 0.5 sec and a duration of 300 sec I have run the
simulation via function call. And here are the results.
Step 1: After a few seconds the pendulum falls down, however, the prediction
seems to perfectly understand the dynamics and it looks like it will be able
to get in the upright position fairly quickly.
Step 2: There is a slight change in the input, however, it is not even close
to stabilizing in the upright position. Meanwhile the prediction once again
seems to be able to compute the perfect input trajectory for the next 0.5
sec.
Step 3: The control input decreases again. The prediction window looks
almost the same as before. Additionally I have added the figure with costs,
computation time and line search step size.
I hope you can see why I believe that the implementation of the dynamics is
fine.
To me it kind of looks like the controller knows the optimal solution but is
pushing it further and further into the future.
That is why I believe that I might have some settings messed up or
overlooked. But at this point I feel like I have tried every single setting.
I would appreciate any advice or ideas on how to solve my problem.
Best,
Artur
In case it helps with the solution here are my current settings (which were
used for the screenshots)
thank you for your detailed problem description. The double pendulum is a system that is difficult to control and requires some effort in MPC tuning.
I would start with a rather short horizon, for example 0.3 sec or 0.5 sec as you have done already. It suffices to simulate the system for 3 seconds. Once it works, you can simulate it for longer time intervals. I would deactivate scaling at first, set dt = 0.001 s, MaxGradIter = 10, MaxMultIter = 1, Nhor = 50. Start without state constraints, since they make things only more complicated.
The next step is to tune the weights in the cost function. You might start without terminal costs and set e.g. the weights to 10 for the first joint position, 1 for the second joint position, 1 for the first joint velocity, 0.1 for the second joint velocity and 0.01 for the control. Afterwards, always change only one setting and look whether and how it affects the solution.
Once you have found a good parameterization of the cost function, you can tune the controller for minimal computation times. That mainly involves reducing Nhor and MaxGradIter until the controller is no longer able to stabilize the system.
Feel free to ask if you need further help.
Best regards,
Andreas Völz
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Based on your approach and after some trial and error, I finally managed to stabilize the pendulum in the upright position. The weights were given to me as part of the problem formulation so initially I did not consider them to be variable.
Moreover, it seems to me like a really small dt is necessary. Increasing it only slightly from 0.001 s renders the controller unable to stabilize the pendulum in the upright position once again.
Best regards,
Artur
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
nice to hear that your were able to stabilize the pendulum. If the weights were given, then they are hopefully already tuned for good control performance and not just some initial guesses.
Having a small sampling time for an unstable system like the double pendulum is of huge advantage, since faster feedback allows to counteract disturbances faster. If the sampling time of 0.01 s is also a requirement given to you, then I would proceed as follows:
- Solve the problem with high accuracy (if you increase the sampling time by factor 10, you should also increase MaxGradIter by factor 10), maybe even consider using ruku45 for integration or heun with high Nhor
- If this does not work, increase dt step by step from 0.001 s to 0.01 s to see where the problems begin
- Try varying the prediction horizon between 0.1 s and 1.0 s (unless this is also a design requirement given to you)
- The Matlab simulation uses a "sample-and-hold" approach, that is, the first element of the control vector is kept constant during the current sampling interval. Using a linear interpolation or the actual time-varying control during the sampling interval might have an influence if the sampling time is significant compared to the prediction horizon
Once you have found a setting, where the MPC converges and the pendulum is stabilized as desired, you can start tuning the controller for minimum computation times. Note however that you might not be able to achieve the "best possible performance" if some settings (dt, Thor, weights in the cost function, ...) are fixed, since they can have a huge influence. As GRAMPC is based on a first-order gradient method, it is, for example, important whether all quantities and their gradients are in the same order of magnitude. Such aspects are usually less important for second-order SQP methods. If problem scaling turns out to be beneficial (find out with GRAMPC scaling options), then you should finally implement it directly within the problem description (so that scaling option is off), since this is computationally faster.
Best regards,
Andreas Völz
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
can you share some insights on the tuning process, which would be of help to other GRAMPC users? For example, which settings turned out to be crucial for your problem? Which values did you finally choose for these options and which performance did you finally reach?
Best regards,
Andreas Völz
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
sorry, I have only seen your post just now.
Let me try to summarize my results quickly. I approached the problem pretty much exactly the way you have suggested. I started with a short sampling time and a short horizon. I then slowly increased MaxGradIter to 15 and the sampling time to dt = 0.01. I left Nhor at its default value 40. I had to use scaling in JScale = 50. Those settings allowed a performance of 0.5 ms which clearly beats the sampling time so I did not bother to implement the scaling in the system dynamics.
Finally, I have changed the initial conditions to consider a swing-up problem (x0 = (pi, pi, 0, 0) ). I was able to stabilize the pendulum within a second which definitely sufficed for my problem.
Thank you once again for your help. You made me understand GRAMPC a lot better.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Hello everyone,
Let me quickly introduce you to my model:
my model is a double pendulum that consists of 4 states and 1 input.
The first two states describe the angles of the pendulum elements with
regard to the vertical axis. (If the double pendulum is in the upright
position the angles are zero.)
The third and fourth state describe the respective angular velocities.
The scalar input is a torque applied at the center where the pendulum is
mounted.
My goal is to stabilize the double pendulum in the upright position. i.e.
x_des = (0, 0, 0, 0)
My initial condition is set very close to the desired position: x_init =
(0.2 , 0 , 0 , 0)
There are no constraints on the input.
Now my problem is the following:
I am using GRAMPC via Matlab.
Whenever I run GRAMPC on this model the pendulum pretty much just hangs
down, or stabilizes in the downward position.
This happens both in Simulink and via function call "startMPC".
At first I thought my implementation of the dynamics is erroneous. But after
carefully checking it twice, I realized that the prediction GRAMPC makes is
actually pretty accurate.
With a horizon of 0.5 sec and a duration of 300 sec I have run the
simulation via function call. And here are the results.
Step 1: After a few seconds the pendulum falls down, however, the prediction
seems to perfectly understand the dynamics and it looks like it will be able
to get in the upright position fairly quickly.
Step 2: There is a slight change in the input, however, it is not even close
to stabilizing in the upright position. Meanwhile the prediction once again
seems to be able to compute the perfect input trajectory for the next 0.5
sec.
Step 3: The control input decreases again. The prediction window looks
almost the same as before. Additionally I have added the figure with costs,
computation time and line search step size.
I hope you can see why I believe that the implementation of the dynamics is
fine.
To me it kind of looks like the controller knows the optimal solution but is
pushing it further and further into the future.
That is why I believe that I might have some settings messed up or
overlooked. But at this point I feel like I have tried every single setting.
I would appreciate any advice or ideas on how to solve my problem.
Best,
Artur
In case it helps with the solution here are my current settings (which were
used for the screenshots)
user.param.x0 = [0.2 , 0 , 0 , 0 ];
user.param.xdes = [0 , 0 , 0 , 0 ];
user.param.u0 = 0;
user.param.udes = 0;
user.param.Thor = 0.5;
user.param.dt = 0.01;
user.param.t0 = 0.0;
user.opt.Nhor = 40;
user.opt.MaxGradIter = 2;
user.opt.MaxMultIter = 2;
user.opt.ShiftControl = 'off';
user.opt.IntegralCost = 'on';
user.opt.TerminalCost = 'off';
user.opt.IntegratorCost = 'trapezodial';
user.opt.Integrator = 'heun';
user.opt.IntegratorRelTol = 1e-6;
user.opt.IntegratorAbsTol = 1e-8;
user.opt.IntegratorMinStepSize = 1e-16;
user.opt.IntegratorMaxSteps = 1e8;
user.opt.FlagsRodas = [0,0,0,Nx,Nx,0,Nx,Nx];
user.opt.OptimControl = 'on';
user.opt.OptimParam = 'off';
user.opt.OptimParamLineSearchFactor = 1.0;
user.opt.OptimTime = 'off';
user.opt.OptimTimeLineSearchFactor = 1.0;
user.opt.ScaleProblem = 'on';
user.opt.xScale = [1, 1, 50, 50];
user.opt.xOffset = [0, 0, 0, 0];
user.opt.JScale = 1000;
user.opt.EqualityConstraints = 'off';
user.opt.InequalityConstraints = 'off';
user.opt.TerminalEqualityConstraints = 'off';
user.opt.TerminalInequalityConstraints = 'off';
pCost = [200, 1000, 0.1, 10, 2000];
userparam = pCost;
Dear Artur,
thank you for your detailed problem description. The double pendulum is a system that is difficult to control and requires some effort in MPC tuning.
I would start with a rather short horizon, for example 0.3 sec or 0.5 sec as you have done already. It suffices to simulate the system for 3 seconds. Once it works, you can simulate it for longer time intervals. I would deactivate scaling at first, set dt = 0.001 s, MaxGradIter = 10, MaxMultIter = 1, Nhor = 50. Start without state constraints, since they make things only more complicated.
The next step is to tune the weights in the cost function. You might start without terminal costs and set e.g. the weights to 10 for the first joint position, 1 for the second joint position, 1 for the first joint velocity, 0.1 for the second joint velocity and 0.01 for the control. Afterwards, always change only one setting and look whether and how it affects the solution.
Once you have found a good parameterization of the cost function, you can tune the controller for minimal computation times. That mainly involves reducing Nhor and MaxGradIter until the controller is no longer able to stabilize the system.
Feel free to ask if you need further help.
Best regards,
Andreas Völz
Dear Andreas Völz,
thank you for your instant and detailed support.
Based on your approach and after some trial and error, I finally managed to stabilize the pendulum in the upright position. The weights were given to me as part of the problem formulation so initially I did not consider them to be variable.
Moreover, it seems to me like a really small dt is necessary. Increasing it only slightly from 0.001 s renders the controller unable to stabilize the pendulum in the upright position once again.
Best regards,
Artur
Dear Artur,
nice to hear that your were able to stabilize the pendulum. If the weights were given, then they are hopefully already tuned for good control performance and not just some initial guesses.
Having a small sampling time for an unstable system like the double pendulum is of huge advantage, since faster feedback allows to counteract disturbances faster. If the sampling time of 0.01 s is also a requirement given to you, then I would proceed as follows:
- Solve the problem with high accuracy (if you increase the sampling time by factor 10, you should also increase MaxGradIter by factor 10), maybe even consider using ruku45 for integration or heun with high Nhor
- If this does not work, increase dt step by step from 0.001 s to 0.01 s to see where the problems begin
- Try varying the prediction horizon between 0.1 s and 1.0 s (unless this is also a design requirement given to you)
- The Matlab simulation uses a "sample-and-hold" approach, that is, the first element of the control vector is kept constant during the current sampling interval. Using a linear interpolation or the actual time-varying control during the sampling interval might have an influence if the sampling time is significant compared to the prediction horizon
Once you have found a setting, where the MPC converges and the pendulum is stabilized as desired, you can start tuning the controller for minimum computation times. Note however that you might not be able to achieve the "best possible performance" if some settings (dt, Thor, weights in the cost function, ...) are fixed, since they can have a huge influence. As GRAMPC is based on a first-order gradient method, it is, for example, important whether all quantities and their gradients are in the same order of magnitude. Such aspects are usually less important for second-order SQP methods. If problem scaling turns out to be beneficial (find out with GRAMPC scaling options), then you should finally implement it directly within the problem description (so that scaling option is off), since this is computationally faster.
Best regards,
Andreas Völz
Thank you again for your advice.
I have now managed to achieve all my desired goals.
Best,
Artur
Dear Artur,
can you share some insights on the tuning process, which would be of help to other GRAMPC users? For example, which settings turned out to be crucial for your problem? Which values did you finally choose for these options and which performance did you finally reach?
Best regards,
Andreas Völz
Dear Andreas Völz,
sorry, I have only seen your post just now.
Let me try to summarize my results quickly. I approached the problem pretty much exactly the way you have suggested. I started with a short sampling time and a short horizon. I then slowly increased MaxGradIter to 15 and the sampling time to dt = 0.01. I left Nhor at its default value 40. I had to use scaling in JScale = 50. Those settings allowed a performance of 0.5 ms which clearly beats the sampling time so I did not bother to implement the scaling in the system dynamics.
Finally, I have changed the initial conditions to consider a swing-up problem (x0 = (pi, pi, 0, 0) ). I was able to stabilize the pendulum within a second which definitely sufficed for my problem.
Thank you once again for your help. You made me understand GRAMPC a lot better.