Example: how will the algorithm "Learn" the basal rate?

In this article, we will show how the artificial pancreas algorithm will not only control the basal rate of the insulin pump but also learn the basal rate. We will discuss two software capabilities from the functional decomposition, namely: the "Control basal rate" capability and the "Learn basal rate" capability. We will think of the "Control basal rate" capability as a higher-level function for now, and discuss the "Learn basal rate" function in more detail.

For starters, remember these important things:


 * The basal rate will be an adaptive variable, i.e. it will be used by a software process as a constant for the purpose of controlling the basal rate command. In the state where the basal rate is controlled by the CGMS feedback, the basal rate will be adjusted from it's usual constant value to the basal rate actually needed.  Based on CGMS data, the pump basal rate is increased above its constant value when it was insufficient to keep BG values steady.  The difference between the basal rate needed right now and the stored value for the basal rate is the basal rate error signal.


 * When we control the basal rate with CGMS data, we can also learn the basal rate at the same time. The same error signal is used as an input into the "Learn basal rate" function. When it sees that the "Control basal rate" function is driving the basal rate command higher than the stored basal rate value, then it will adjust the stored value of the basal rate for future use -- but it will only "nudge" the value in the right direction. It will learn slowly, at least ten times slower than the control function.  This is preferred for adaptive variables because it will reliably learn in the correct direction when there are multiple events that consistently have the same error in the same direction, whereas it will not learn much in the wrong direction when an non-typical event shows an error signal that is uncharacteristic of typical events.  So the "Learn basal rate" function will have to be initialized reasonably close to the basal rate that is needed, and then the algorithm will ensure that the basal rate adaptive variable will track the basal rate actually needed as it evolves over time.


 * We can learn the basal rate at certain times, namely when have not bolused in a while so that we have a "balance" between BG with no insulin on board. Our artificial pancreas executive state machine will know when this is true, and so it call run the software to control the basal rate and learn the basal rate at the right time.

The basal rate variable = f(t)
The basal rate is really a pattern. It is a function of time: you want to know what basal rate to use right now, at say 1pm. The basal rate will be stored in memory with a value for every hour of the day. So, you need to know the time of day in order to look up the basal rate value for that part of the day. When we look at a curve of the basal rate needed as a function of the time of day, we could expect it to vary from hour to hour and also from person to person. There are some open-loop basal rate patterns which are interesting to discuss. We may use them as a starting point, but we will let the algorithm deviate from them at it learns.

We will also assume that the pattern must be continuous, with no large discontinuities in the basal rate data structure from one point of time to the next. To keep it this way, we cannot just adjust one part of the curve, or we may accidentally create a discontinuity. So we will modify the basal rate values via weight factors (center weighted) for blending of new values with old.

The basal rate as an adaptive variable
Let's take the general figure from the Adaptive Variables, Adaptive Filtering article and use it as a template for the basal rate description.

In this case:
 * the bubble Control the Basal Rate is the "process to control' in the generic diagram;
 * the Basal rate is the input variable to the process to control;
 * the Basal rate command is the output variable from the process to control;
 * the Basal rate error is the difference between basal rate command and the basal rate signal, which is an indicator that the the actual BG from CGMS is different from the target BG value and therefor the process for controlling the basal rate has decided to increase the basal rate command to compensate;
 * the Integrate BRE signal and determine basal rate adjustment process integrates the error signal to accumulate the total exposure of magnitude and duration of the error caused by the shortcomings of the predicted basal rate relative to the basal rate really needed. If the exposure is large enough to demand a change to the basal rate, then it will output an incremental change to the basal rate.  This increment is a rate-of-change function for the adaptation rate. Note that the rate of change is limited such that the variable will adapt slowly, and will only learn a portion of the error at any one event.  It takes several consistent events to slowly adapt the constant value up to its desired value;
 * the Determine threshold/trigger criterion will prevent the adaption unless a threshold is reached. This can be individual for each direction, as it will be natural to be more aggressive about reducing insulin and less aggressive about increasing it;
 * the Adjust basal rate will increase or decrease the basal rate within boundaries depending on how large and how long the basal rate error signal is observed. Note that the basal rate constant value will be updated only for future use.



"Learning" the basal rate
So, let's walk through the figure and discuss how it works. First let's consider the "control basal rate" functionality. This will be described in detail later, but for now let's just say that we will make an error signal from the actual BG measured from the CGMS and our target value for BG. The error signal will be calculated as actual-minus-target, such that a positive error signal indicates that the BG value is above target and needs therefore to increase the insulin delivered. We will need some signal processing to avoid sudden changes in the basal rate command, but in general the basal rate command will be increased proportionately to the integral of the error signal. When we have the opportunity between boluses to control the basal rate with CGMS data, we can increase the basal rate actually delivered to fill the gap between the insulin needed to balance and hold BG steady relative to the insulin rate predicted. The function will of course work in both directions, and will deny insulin to balance when required.

The software capability to "learn the basal rate" will be broken down into sub-functions. The first function is to perform integration of the error signal and calculate the magnitude of the resulting change to the basal rate, if any. The integration function will give us a number combining both the magnitude and duration of the error signal over the entire exposure event in this state. When we are done monitoring how far off the prediction actually was, we can then change the basal rate values (only once) so they can be used next time.

We don't want the algorithm to change the basal rate unless the change it needs is large enough to warrant doing so. The process "Determine trigger/threshold criterion" will control this. We also want the rate-of-change of the adaption to be controlled (one large mistake should not cause a big change to the basal rate) so the maximum and minimum values for the basal rate change will be enforced. We also don't want the final basal rate be larger than a maximum value either, and that will be enforced by the process that updates the basal rate. While enforced by the software, all of these max and min control limits will be made by an external PC that a CDE or test engineer can set up manually.

The process "Update basal rate" will actually change some of the values in memory for the basal rate pattern. It will only change it once, and only when other applications are ready to do so. This process will change the basal rate value for 6 hours (or more or less depending on what future modeling and experience will tell us) with a weight factor for each hour. We know we need to change the basal rate an hour before we want it to have impact to the blood glucose level, and we want to "blend" in the changes with the baseline curve to avoid a discontinuity. So the weight factors for updating the basal rate pattern, with one weight factor for each one-hour data point in the pattern, will be something like: {5%, 10%, 15%, 15%, 10%, 5%} where the 15% value is one hour before the actual BG levels were high enough to mandate a change to the pattern.

Example: Low BG at 4 am
For example, suppose the basal rate at 4 am was proving to be too high by CGMS values below target and trending down. It was set with one hour increments starting at 1 am with these values: 0.9, 0.85, 0.85, 0.9, 0.9, and 0.85 units of insulin per hour respectively. The CGMS sees low blood sugars at 4 am, for which the artificial pancreas algorithm reduces the basal rate to zero for 1.5 hours and then to 0.75 for the next two hours before going back to 1.0. This change takes 2 units of insulin out of the basal insulin profile over that time period. The adaptive algorithm will want to take about 1/3 (again to be determined later with more experience with the algorithm) of that out of the basal rate, or 0.6 units for tomorrow at the same time. Over a 6 hour period, this would have the following impact to the basal rate pattern stored in memory to use in the future:



Note that one "low" event did not completely drive the new basal rate to its corresponding lower rate. If so, that would have taken 2 units insulin out of the basal rate over those 6 hours. Instead, by learning slowly, it will take a few low events over a few days, or a pattern of intermittent lows over several days, to drive the basal rate fully down to its new pattern. The basal rate function learns and adapts gracefully, not all at once. Remember, the process that actually controls the instantaneous basal rate dialed it all the way down right away, as needed for each "low" event for each one that occurred: it is only the learning of the new basal rate that takes several days to catch up to the best basal rate for future use.

This learning and adaption can happen to different parts of the algorithm all day long. At night (when our pump user is not bolusing) it will happen more often. During the day, our pump user may have to skip meals sometimes to keep the executive state machine in the "Control basal rate with CGMS feedback" mode to learn the basal rate for that part of the day. Also remember, our pump user (or the CDE) can still hook-up to a computer and set the basal rate to any values that are needed.