Thank you, @Dragan1. I understand what you write and hope that @Kevin_McMahon would come back an give a comment on this.
1 Like
If I understand correctly, the paper summarizes a conference about AP tech, and the immediate context of the quotation is the accuracy parameters such devices should have to meet. Always a risk translating professional jargon into ordinary speech but the point seemed to be “People are already managing to do ok with devices that aren’t that rigorous.” I.e., the context is not simply “let’s do an empirical study to measure pump accuracy”–someone wanted to find the biggest variance from spec for the sake of argument. You’d need to look at the original study to evaluate whether ±200% was really a significant datapoint or just an outlier.
I don’t have time to hunt it down but there was a similar point about CGM accuracy in the same context–that the fingersticks the CGM results are being truth-tested against are none too accurate themselves (something about which there has been much discussion on TUD and elsewhere) and the rule that in any divergence the fingerstick is presumed correct may also need to be interrogated. I think they’re on firmer ground there, though I’m not ready to throw out my ContourNext just yet.
1 Like
Hi Everyone,
Sometimes we can just disagree and we don’t have to embrace each other’s opinions. This is my last comment on the subject as it’s the same as my previous reply.
The specifics of the comment are limited to a 1u bolus so not sure why there are examples of anything but (ie - a 5u bolus).
Steve and I stand by everything we wrote in the book as I said previously. So glad you enjoyed the book. However, I’m considering this discussion closed as I have many more tasks on my plate.
Bottom line is that t1d is rife with inherent variability so we should not stress about this fact but rather embrace it and incorporate strategies that understand that numbers are just an estimate and not written in stone.
Good luck,
Kevin
2 Likes
Even for an outlier, thats one heck of error. IIRC Animas (and I would imagine all the others) are quoting accuracy of plus/minus 5%. Statically for a normal distribution you should expect ~99% of readings within plus/minus 3 SDs. So even if the claimed variation is based on one SD (and it’s more likely to be 3), an outlier of +200% would be 67 SDs. I used an online calculator and the BIGGEST error it would give me a value for was +70%. Assuming SD is 5%, the probablity of such an error occuring as an outlier would be 3.2 e-14, which is much less than one in a trillion boluses. For a 200% error, there probably aren’t enough atoms in the unverse!
I would bet a significant sum that this figure is an error or a misquote.
Joerl
1 Like
If people are truly interested in what Alan Stevens of the FDA meant when he said the error rate for one unit delivery by a pump could be in that range, someone could try asking him: alan.stevens@fda.hhs.gov.
I found his email address here: http://www.fda.gov/downloads/MedicalDevices/.../UCM217457.pdf
The way the comment reads, it appears to me it is based on adverse events reported to the FDA, but that’s just my best guess.
EDIT: I found the transcript of the meeting where Mr. Stevens made the comments. Here is what he said and it appears to be based upon the adverse events associated with insulin pumps reported to the FDA:
So the first thing we should think about is accuracy. How does that affect accuracy? So for delivery rates above one unit, these products typically fall within the range of +/- 5 percent delivery. When we get below one unit, and at the very low end, and as you can see here, well below .5 units, we get up to anywhere from +/- 30 percent all the way up to +/- 200 percent we’ve seen values where they’re even reported. You know, I went back and looked at some of the submissions, and the farther back we go, we don’t have a lot of solid data at the very low end of the spectrum. And some of the issues are the test method that’s used or the standards that exist call for delivering 25 consecutive boluses and then reporting the minimum and maximum deviation as well as the average, the mean deviation. And when we’re talking about very low quantities, particularly, you know, .05 or .025 units delivered every 10 to 15 minutes, every single bolus, the accuracy and the certainty around that measurement becomes important. And even the test methods themselves, there is measurement uncertainty that needs to be considered.
http://www.fda.gov/downloads/MedicalDevices/NewsEvents/WorkshopsConferences/UCM257106.pdf
3 Likes
This is not a matter of different opinions at all - we are just trying to point to fairly obvious factual errors in the book - nothing else. As far as I am concerned, these are relatively minor points - no need to extend this discussion any further. I and I think many of us here fully agree with your bottom line and I thank you and Dr. Ponder for your outstanding work and contributions to modern approaches to T1D management.
Best,
Dragan
1 Like
Yes, that’s an important point. If you’re talking about some absolute error or variance in the pump, it’s going to be proportionally higher the smaller the increments become. And there is no question that this is true:
Someone else around here pointed out that if the definition of insanity is repeating the experiment and expecting different results, the definition of T1 seems to be repeating the experiment and never getting the same results. Which can drive you insane after a while if you’re expecting perfect predictability.
2 Likes
Indeed. However, applied to 1U the worst-case error of +/-200% is equally implausible. There is absolutely no way a pump would be able to deliver -1U or +3U when 1U is requested. This is simply not feasible, not even as a far far out outlier. My suspicion is that the original quote attributed to Alan Stevens from FDA is simply a misquote: it is possible that Alan verbally referred to one “unit” as being one smallest increment a pump is able to produce (which is 0.025U for my MM pump), not 1 U. Whoever transcribed the discussion in that meeting probably misunderstood this statement and so 1U (instead of a “smallest unit”, or a “smallest step”) unfortunately ended up in that online publication and then made it into the Sugar Surfing book.
The variability for 0.025U could indeed be much larger (although I am sure +/-200% still refers to very very rare outliers, including bubbles, capillary effects etc). As an exercise, lets suppose the variance for 0.025U is +/-100%. A 1U bolus consists of 1/0.025 = 40 such steps, which averages out the variance of the sum to about +/-15%, This is a bit more plausible, although still fairly large. The true variance for 0.025U is probably closer to +/-30%, which would imply around +/-5% variance for a 1U bolus, which seems reasonable.
In any case, I think we are at the end of this discussion. My recommendation is to not be overly concerned about the book statement, which is simply incorrect. There is so much other variability in T1D, but the pump accuracy is not in the top of that list, not even close.
3 Likes
RE-POSTING this comment so people can see the source of the original statement by Mr. Stevens.
I found the transcript of the meeting where Mr. Stevens made the comments. Here is what he said and it appears to be based upon the adverse events associated with insulin pumps reported to the FDA:
So the first thing we should think about is accuracy. How does that affect accuracy? So for delivery rates above one unit, these products typically fall within the range of +/- 5 percent delivery. When we get below one unit, and at the very low end, and as you can see here, well below .5 units, we get up to anywhere from +/- 30 percent all the way up to +/- 200 percent we’ve seen values where they’re even reported. You know, I went back and looked at some of the submissions, and the farther back we go, we don’t have a lot of solid data at the very low end of the spectrum. And some of the issues are the test method that’s used or the standards that exist call for delivering 25 consecutive boluses and then reporting the minimum and maximum deviation as well as the average, the mean deviation. And when we’re talking about very low quantities, particularly, you know, .05 or .025 units delivered every 10 to 15 minutes, every single bolus, the accuracy and the certainty around that measurement becomes important. And even the test methods themselves, there is measurement uncertainty that needs to be considered.
http://www.fda.gov/downloads/MedicalDevices/NewsEvents/WorkshopsConferences/UCM257106.pdf
3 Likes
No fair! Math! But seriously I appreciate your applying some rigor to the discussion. And yeah, I think we can agree that the general point is correct even if some particulars got munged in transit.
1 Like
@Shadow2, thank you for locating the original source of this. I can understand now a high percentage error when talking about very small actual boluses. When delivering a 0.025 bolus, a +200% error means delivering 0.075 bolus, an error of +0.05 units. For almost all of us, an error of this scale falls well within the “noise” of actual site absorption, carb counting variability, or even climbing a few more stairs during a day.
@Siri, this was a legitmate question to raise but given this additional context does not leave open an important question in my mind. Sugar Surfing’s main thesis still stands: blood glucose management in diabetics can benefit a great deal from looking at blood glucose trends in a more dynamic and less static way.
This is a statistic that didn’t need to be included and should have included a few more statements to clarify and contextualize. The authors can still take considerable pride in a substantial contribution to people who use insulin.
3 Likes
Thanks @Terry4 for your slight, but significant adjustment of terminology [quote=“Terry4, post:31, topic:47906”]
…contribution to people who use insulin.[/quote]
The methods and ideas in the book can be helpful to anyone using insulin for diabetes management. I have been using similar methods myself (by my own experimentation) for much of at least the last 2 of the 3 years I’ve been using insulin with CGM support.
@Thas, you’re right, my choice of words was informed by a more inclusive view of diabetes. Thank you for seeing that! I was not always of this mind. My participation at TuD helped me learn to respect the path of others with diabetes, no matter the type.
1 Like
Hey, I do not have/have not read this book. But am well aware of the bolus being too great/too little at times. See my endo. soon, & really need to speak with him re: the pump settings. Another thought: getting that preset level of insulin (apidra for me) all night affects my BS levels as well. Have already changed the Basal rates 2-3 times, since June. Just a work in progress.
I agree with @Dragan1:
Thanks to all of you who participated and cleared things up.
3 Likes