Learned Control

First, a reminder. THIS FITTER LAB IS FOR PERSONAL EDIFICATION (AND FOR FUN), IT IS NOT REQUIRED FOR SPRING20 6.302 IN ANY WAY. However....if you do try this fitter, please take advantage of evening office hours, May 25-May 29th.

Since the start of this class, we have designed feedback controllers by first developing a physics-based model of our system, and then using the model to determine a good controller. And as we moved from proportional to PID to Lead-Lag to State-Feedback to Observer-State-Feedback, our approach to controller design required progressively more comprehensive models of our system. In order to generate those models, we used physical insight, made simplifying assumptions, and relied on parameter values from measurements made on similar, but not identical, systems.

For this lab, we are going to replace modeling based on physical insight with "learning" an accurate model entirely (almost) from data. More specifically, you are going use a version of the sweeper you used in an earlier lab to measure the frequency response of your propeller arm (regardless of whether your arm is using one propeller or two), and then use rational regression to fit your frequency response measurements to a state-space model. Once you have "learned" the state-space model, you will use it, along with matlab's lqr-based optimizer and the pole-placer, to design the controller and estimator gains, K_d and L_d, for an observer-based controller. Finally, you will try the controller on your propeller arm.

After completing this lab, and you realize the power of the computational techniques used in it, you might be tempted to question why we taught you the entire rest of the class. After all, the approach below seems to be so automatic. Please do not give in to this temptation. Instead, we ask that you pay close attention to every step in this lab, and think carefully about what insights you would need to complete that step completely on your own.

So suppose you did have to start from scratch. How will you actuate and what will you sense? And since you can only measure frequency response if your system is stable and nearly linear, how will you linearize and stabilize your system? And if you stabilize your system with a classical controller, how are you going to deembed the controller from the measured frequency response? And how many frequencies should you measure, and over what range? And when it comes to generating a state-space model from the measured frequency response, how will you "regularize" the regression? And once you generate the model, how do you determine the LQR weights and estimator poles? Isn't that harder to do when the "states" are the invention of some computational procedure and not physical insight? If this list of questions does not convince you that you need every drop of 6.302 and then some, try the challenge problem, extending this approach to the Maglev lab.

Okay, maybe that is a bit overstated. But this is not: once you get this lab working, we think you will be astounded. And just to foreshadow, we have three introductory videos generated using this lab. We started by using a symmetric two-propeller system to resolve all the "FIX this" lines in the matlab scripts below, and set up a strategy for determining weights for LQR when computing controller gains, and for placing poles when computing estimator gains. The results from using the controller on the two-propeller system are shown in the leftmost video. Then we tried two other examples, an asymmetric two-propeller system and a single propeller system (center and right videos). And all we modified by hand, promise, was the nominal command needed to level each arm. Then we ran the sweeper, copied over the sweeper.csv file to the controller subdirectory, ran the matlab scripts in the controller subdirectory UNMODIFIED, uploaded the controller to the Teensy 3.6, and turned on the power.

Here are the videos of using the controllers on the propeller arms. The first video is of a symmetric two-propeller arm. The second and the third, an asymmetric two-propeller arm and a single propeller arm, both use controllers that were generated nearly automatically. The only change was the values of the nominal commands needed to balance each arm, otherwise exactly the same sketches and matlab scripts were used to generate the controllers for the symmetric two-propeller, asymmetric two-propeller, and single propeller systems. That includes the selection of Q weights and estimator poles, those were adapted to each senario algorithmically.

Below we give the steps for this lab. The sections below explain the various steps, but you should refer back to this step-by-step guide when executing each step.

1. Download the learned_control zip file from here and unzip it.
2. Make sure your propeller system is connected (either the one or the two propeller system) and power is turned on, and that the two Teensys are plugged in to the USB ports of the computers running the GUI and the Teensyduino compiler.
3. Open the 6302_sweeper_fitter sketch in the learned control directory, set the calibration for angleSensorOffset, angleSign, cmdNom1 (you only need cmdNom2 to balance the arm if you are using two propellers). You should determine the values using the props lab sketch. Exam the bottom of the sketch if you are not sure of the roles of cmdNom1 and cmdNom2.
4. Flash the Teensy 3.6 with the 6302_sweeper_fitter sketch in the learned control directory, turn on the power to your propeller system, start up the python server and open up the browser GUI. If you wait for a minute or so, you should see the sweeper collecting frequency responses for your arm. Make sure the propeller is oscillating symmetrically about horizontal.
5. In the browser GUI, click the CSV generate button so that it shows a green check. Then wait for the sweeper to run through two or more sweeps, and then click CSV generate icon again. Either your browser will ask you where to save the csv file, or it will save it somewhere automatically. In either case, make sure the CSV file is saved in, or moved to, the 6302_SSOBSONLY_control subdirectory of the learned control directory, and rename the file $sweeper.csv$.
6. Turn off the power to the propeller!
7. Run the $propSS.m$ matlab script in the 6302_SSOBONLY_control directory (THIS IS NOT the same script as the one you used the props lab!!!), but first find and fix the "FIX This" lines in $propSS.m$, $dtObs.m$, and $labFitter.m$. Again, make sure you are modifying the files in the 6302_SSOBONLY_control directory, and not anywhere else!!!)
8. The $propSS.m$ matlab script generates four files, one showing your collected frequency domain data, one showing the fit to your measured plant frequency response, and two that are repeats of the closed-loop simulations you saw in the props lab. The plots may be stacked on top of each other, so be sure you see all four. The $propSS.m$ script also creates the controller, and generates a $obs6302.h$ file just like the props lab, but in the 6302_SSOBONLY_control directory.
9. Open the 6302_SSOBONLY_control sketch in the learned control directory using Teensyduino, be SURE to copy over the calibration values you used in the sweeper, and then compile and download (or flash) the sketch on to the Teensy 3.6. When you compile the sketch, it will automaticaly read in the latest $obs6302.h$ file, so be sure you run $propSS.m$ FIRST! Then when Teensyduino downloads the sketch on to your Teensy 3.6, the Teensy will have the latest version of the controller you designed in matlab.
10. Make sure the python server is still running and the browser GUI is still plotting data, and turn on the power (but stay clear of the propeller arm!!!).
11. If your arm holds itself horizontal, your controller is probably working, and you should try setting the desired toggle slider to various negative values. If the fit model is a good match to your system, then the matlab simulations should match what you measure, and if they do not, you may need to iterate on your weights for the controller and your poles for your state estimator.
12. Look carefully at the command noise. If it is not as low as the command noise in some of our examples, you should know how to fix it.
13. If you want to take large steps with a two-propeller arm, follow the same instructions as in the props lab. But please, take a video that shows the propeller in action and the responses shown on the GUI.
14. Redo the above steps, but for a different system. Make the arm longer, add a second propeller (or remove the second one), or do something else. If you examine this strategy's effectiveness on a single propeller compared to its effectiveness on a two propeller system, what do you notice, and why? How do the fits compare? Does that tell you something about the nonlinearity of the single propeller system compared to the two propeller one?
15. This lab is not part of the 6.302 evaluation during Spring 2020. We released it on Memorial day, 5/25, so that there would be no confusion. And we hope those of you who try it will find it fun and rewarding, and will let us know what you think.

IT IS ESSENTIAL that you keep all the files from the above zip in same directory, so that any updates you make to the matlab-based state-space model or to the controller can be transferred automatically to the Teensy. Automating the transfer makes it easier for you to "miss" some of the control design details, but eliminates the tedious and error-prone process of copying controllers by hand. Since this lab requires you to design many controllers, and we want you to explore alternatives, automation seemed to be the right tradeoff.

As long as you saved the csv file generated by the sweeper (by turning on, and then later turning off, the generate \;\; CSV icon) in the 6302_SSOBSONLY_control subdirectory, and renamed it sweeper.csv, then running the matlab script propSS.m in the 6302_SSOBSONLY_control subdirectory should generate the observer-based controller (provided you fixed all the FIX this's). The propSS.m script will fit your data, generate a state-space model, design your controller, simulate your model and controller, and plot the results. The script also generates a human-readable file containing a definition of your controller, obs6302.h. When you compile and download the sketch 6302_SSOBSONLY_control on to the Teensy, the compiler will read in the obs6302.h file, and download your controller on to the Teensy.

WARNING: DO NOT MANUALLY EDIT obs6302.h in the Teensyduino IDE. You WILL OVERWRITE THE CHANGES MADE BY propSS.m (ignore this if you are doing the copy-and-paste option for using on-line matlab, but we recommend not using that option and using matlab connector instead, see the props lab).

The term "grey box" modeling is derived from the more common term "black box", where a black box refers to a system of which we have no idea of the inner workings...only its inputs and outputs. A black box model is purely input-output data-based....it is a black box; we can't see inside it. In grey-box modeling we use some physical intuition and ideas to help guide a data-driven model and we'll do that in this lab.

Measuring frequency response.

As indicated in the cartoon above, we can use frequency domain data to create a state space model of our system. The diagram also indicates that when measuring frequency response, the system is embedded in a feedback loop with a proportional and derivative compensator. That seems unnecessary, why don't we measure the open-loop frequency response of our system directly?

To measure a system's frequency response, one inputs a sinusoid, waits for the output to settle to a sinusoidal steady state, and then measures the magnitude and phase of the output. The problem with making such measurements on the propeller arm, or on any system that is NOT stable, is that the output never settles to a steady state. So instead, one combines the unstable system with a stabilizing controller, measures the closed-loop frequency response, and then "de-embeds" the effect of the feedback controller.

In the case of the propeller arm, we use a PD controller to stabilize the arm (with proportional and derivative gains K_p and K_d respectively, set near the top of the Teensy Sweeper), and then stimulate this stabilized system with sine waves of progressively higher frequencies. We then measure the phase and magnitude of the output sine wave signals of both u (our command signal) and the measured arm angle, y, and use the two sets of magnitudes and phases to deembed the PD controller, and extract the frequency response of arm alone.

The Gray Matter.

Once we have the measured arm frequency response, we can use rational regression to fit a state-space model to the measurements. We will not focus on the details of the rational regression algorithm, but will note that since the state-space model is fit to measured data, we expect that the model will include every aspect of the physical system that effects input-output behavior. That is, we no longer need a detailed physical model, and need not fret about neglecting important physical mechanisms (e.g. air resistance, arm friction, coupling between arm rotational velocity and propeller speed, etc). If the mechanism matters, it will be reflected in the measured data, and will therefore be included in a model that fits the measured data. And perhaps more importantly, if we fit our model to measured data from our own propeller arm, we will generate an "arm-specific" model that should be a better match than a nominal physics-based model.

Having an accurate model is particularly important if we intend to use the model as part of an observer-based controller, but there is a cost. State-space models generated by fitting are not likely to be resemble anything physical. That is, the states in the regression-fit model may not correspond to physical quantities like angle, velocity or acceleration. So when we use such a model in an observer-based state-space controller, and try to design controller gains using LQR, how do we set the LQR weights to get the behaviors we want? Carefully, we suppose.

Our frequency sweeper sketch uses a poportional+difference feedback to stabilize the system it is measuring, but uses very low values of K_p and K_d so that the feedback does not hide the system's properties. The sweeper generates sine waves of progressively increasing frequency as input to a PD-controller, waits a each frequency until the system achieves steady-state, and then records the amplitude and phase of the signal, u, at the output of the PD controller, as well as the amplitude and phase of the system output, y (the arm angle in our case). These values are recorded in a CSV file (Be sure to set the CSV flag in the GUI, but wait until a few points have been plotted. The first few points are often noisy!!!).

Before running the sweeper, you will need to calibrate a few parameters, ones that should be quite familiar by now. At the top of the sweeper sketch are angleSensorOffset, angleSign, cmdNom1 (and cmdNom2 if you are using two propellers), and they should be set so that the arm is nominally perfectly horizontal, and the angle sensor reports an angle of zero. You can use the calibration parameters you determined in the props lab, or you can rerun the props lab sketch to determine the offsets and nominal commands. If you are using two propellers, and did not implement two nominal commands to balance the arm, look at the bottom of the sweeper sketch to see how cmdNom1 and cmdNom2 are used.

Running the Sweeper

After calibrating the nominal commands and sensor offsets, start the sweeper (download the sweeper sketch to the control Teensy, make sure the monitor Teensy is running the USB repeater, run the python server on the monitor laptop, open a browser on the monitor laptop, connect and wait). You will notice that NOTHING HAPPENS IMMEDIATELY IN THE GUI! This is because the sweeper starts by determining a very low frequency steady-state, which can take tens of seconds, and the GUI is ONLY PLOTTING CONVERGED magnitude and phase.

Once the arm starts oscillating, make sure it is oscillating symmetrically about horizontal, and if not, adjust the value of cmdNom1 and restart the sweeper. If the oscillations are symmetric about horizontal, watch your system while sweeping, and you will see that for a narrow range of frequencies, the Magnitude and/or MagCMD will increase substantially. That is, you will be driving your propeller arm at, or near, a closed-loop resonance frequency. If your PD-controlled system's natural frequencies are nearly imaginary, the oscillation amplitude at resonance will be so large that your arm will swing wildly, and you not get good frequency response data. If this happens, you may need to increase the derivative gain, Kd, (but do not increase it by more than fifty percent!) to keep the oscillation amplitude of your arm within a reasonable range.

Once the sweeper is running with the arm oscillating symmetrically about horizontal, and the response near the resonance frequency is not too large, and you have let the sweeper plot a few data points (the first few are often noisy), then you are ready to record frequency responses for use in model generation. Click on the "Generate CSV" button, which should then switch to displaying a green check, and let the sweeper run through its frequency points two or three times. It will take several minutes to run one full data-collection cycle (a sweep through all frequencies). We encourage you to let it run at least three cycles in a row, because labfitter.m averages multiple sweeps, and that reduces noise.

Collecting Frequency Response Data

Once the sweeper is running reliably, set the generate CSV to ON in the GUI, record at least 2 SUCCESSIVE SWEEP CYCLES (the fitter averages the results to reduce noise). A typical sweep showing several cycles is shown in the figure above. Notice the green check mark near the generate CSV switch in the GUI.

When you have recorded enough data, set the generate CSV switch off (red x). Then, either your browser will ask you where to save the csv file, or it will automatically save the csv file somewhere. In either case, make sure the CSV file is saved, and is then moved to,t the 6302_SSOBSONLY_control subdirectory of the learned control directory. Once you are sure the CSV file is in the correct directory, rename it sweeper.csv.

Once you have saved the frequency response sweeper.csv file in the 6302_SSOBSONLY_control subdirectory, you are almost ready to try running the matlab script that fits a state-space model to your data, computes controller and estimator gains for an observer-based state-space controller, and generates the include file that describes the controller to the Teensy 3.6.

Fitter FIX this.

To run the fitter, run the propSS.m matlab script in the 6302_SSOBONLY_control directory (THIS IS NOT the same script as the one you used the props lab!!!), but first find and fix the "FIX This" lines in propSS.m and labFitter.m. Again, make sure you are modifying the file in the 6302_SSOBONLY_control directory, and not anywhere else!!!)

To fix propSS.m, you will need to decide on number of poles and zeros for your model, but BE CAREFUL OF OVERFITTING!!! With more degrees of freedom, you can always match the data better, but you risk "fitting" noise, in which case, the resulting model will not behave reasonably. Of course, if you have exact responses for every frequency from zero to \infty, the fitter could find the number of poles and zeros automatically, but access to an infinite range of noise-free data is not realistic. Instead, use your insight into the physical system to decide on a reasonable number of poles and zeros. You might not know the exact location of your poles and zeros, but you probably know how many you have.

To fix the issue in labFitter.m, you must untangle the plant frequency response from the frequency response of the PD-controlled system. Can you divide the frequency response of the transfer function from y_d to arm angle, (which is also the closed-loop frequency response G(j \omega)) by the transfer function from y_d to the motor command input u? Does that mean you can divide their frequency responses? Does that division yield the "plant" frequency response H(j\omega), as in \Theta(j\omega) = H(j\omega) U(j\omega)? Is that division being done correctly in the labFitter.m script?

Controller and Observer gains.

State-space models generated by fitting can be quite accurate, but they are not likely to resemble anything physical. Specifically, the states in a regression-fit state-space model do not usually correspond to physical quantities like angle, velocity or acceleration. So when we use such a model in an observer-based state-space controller, and try to use discrete-time LQR to design the controller, how do we set the dlqr Q weights to get the behaviors we want? Do any of the other matrices, \textbf{Cd} or \textbf{Bd}, give us a hint about the mapping between the computationally-determined states and the physical ones?

Be sure to examine the poles of your controller and estimator. If YOU HAVE TROUBLE FINDING OBSERVER GAINS THAT GIVE YOU FAST OBSERVER CONVERGENCE, REFIT YOUR MODEL WITH FEWER ZEROS! A model with extra zeros can "hide" internal structure from the measured output, y = \textbf{C}_d \textbf{x}. This question of observability, again a topic we defer to your next control class.

The propSS output graphs.

The propSS.m matlab script generates four files, one showing your collected frequency domain data, one showing the fit to your measured plant frequency response, and two that are repeats of the closed-loop simulations you saw in the props lab. Matlab may stack the plots on top of each other, so be sure you move the plots around so you can see all four. The propSS.m script also generates a obs6302.h file, just like the props lab did, but keep in mind that this file is in the 6302_SSOBONLY_control directory.

Run the sweeper on your arm, save the data to the file sweeper.csv in the 6302_SSOBONLY_control directory, and run propSS.m to fit a model your data. Examine the generate CT model, described by \textbf{A}, \textbf{B}, and \textbf{C}. You can get the poles of the model using the eig command, how can you get the zeros (Hint, look at the latest update of the last section of the observer prelab, in the problem with six controllers). Also, examine the DT model, described by \textbf{Ad}, \textbf{Bd}, and \textbf{Cd}. What are its poles, and how do they relate to the CT poles. Finally, how good is the fit? Does the frequency response of the model match the data for both magnitude and phase?

You might also look at the formulas used to generate synthetic data in the labFitter.m file, do you understand what the formulas are doing?

Finally, how are the states in your generated model related to the arm's angle, rotational velocity, and rotational acceleration? How did you decide which state to weight heavily?

The 6302_SSOBSONLY_control.ino sketch in the 6302_SSOBSONLY_control directory is a three-state observer-based controller. Before downloading the sketch to the Teensy 3.6, transfer the calibration values from the frequency sweeper to the controller, as shown in the image below.

Once you have transferred the calibration, download your sketch on to the Teensy 3.6, make sure the monitor GUI shows the arm angle changes as you move the arm, and then turn on the power. Since there are no longer any measured states, the GUI only has three plots. The leftmost plot compares the measured, estimated and desired y, the middle plot shows u, or the delta motor command, and the rightmost plot shows the three estimated states. But since the states are determined by the fitter, their meaning requires "interpretation".

Be aggressive! Crank up the LQR weights (except for R, the superego to Q's ID), just make sure the observer stays ahead. If you have a two-prop arm, the model match and performance can be quite impressive, as you can see in the image below and the videos that follows.

Does your grey-box-model generated observer-based controller do a good job of controlling the arm? How fast are your step responses? How did you pick the number of poles and zeros. What happens if you use too many poles or too many zeros? Compare the extracted A, B, and C matrices to the ones from the physics-based nominal propeller model (you can see both systems if you look at the top of propSS.m). Are the matrix entries similar, and if not, why. And are the poles similar? Then, in designing your controller and observer, how did you pick your gains? If you force the observer to converge very fast, how does that effect the system? What can you do to minimize the command noise? How well does your controller work compared to the one based on a nominal physical model?

Add (or remove) a propeller, or lengthen the arm, or try using unequal arm lengths for a two-prop system, or add a weight using the magnets. Rerun the sweeper, feed the sweeper data into the fitter/controller, download the generated controller, and see how it works (we did it on the systems at the top of this lab). You may discover that it is necessary to change the number of poles and zeros to get a good fit for your new system. You may have to adjust the nomCmd1 and nomCmd2 terms in the sweeper and controller sketches.

Refit the model, and redesign the controller, then try it on your different system. Did you have to change your number of poles and or zeros? Did the fitter predict an unstable system? What about your gain-picking strategy? How well does your system work?

What about using the above strategy for Maglev? You have a controller that allows you to take small steps without dropping the magnet, can you use that controller, measure the frequency response, and deembed the controller? Can you fit a model to that frequency response? Can you use the model to determine feedback and observer gains? Can you float a magnet using a discrete-time state-space controller? How well does it perform?

Getting this to work is NOT EASY, you have to rethink everything. How to set up the sweeper, how to collect the right data, what frequencies to use, what gains to use in the nominal controller. High gains can "hide" the system specifics, remember "hiding" the system specifics is why we use feedback in the first place. So keep your gains as low as you can. And remember, if you rotate any potentiometers between running the sweeper and running the observer-based controller, then your controller is running on A VERY DIFFERENT SYSTEM than the one you modeled. That will not go well!