94% = 25%, revisitedPosted by elliott on 6/04/04 at 08:07 (151992)
In the past, my view that Dornier's one-year 94% success rate translates into more like a 1-in-4 chance that the Machine Actually Cures (MAC) you has been subject to some ridicule on these boards. Perhaps my previous attempts to elucidate fell short. Let me try again with a very simple explanation:
At 3 months, the control group exhibited a 'success' rate in the 40-percent range. Even if that rate would remain unchanged by one year, that would mean that the MAC rate would be around one in two (94%-40% = 54%). Suppose the control group were kept blinded until one year, and continued to improve so that at one year their success rate was 69%, a figure conveniently selected after the fact but eminently reasonable based on other data. Then at one year, the MAC rate is 94 - 69 = 25%, or one in four, as claimed.
It is a fact and not Dornier's fault that in PF studies, many will get better due to placebo effect and natural course of healing. At the same time, it is unrealistic to say that the machine is curing 94% of the people who get treatment. If you are one of those who are less likely to fall for placebo (e.g., you're in a wheelchair and can't walk) and are less likely to heal in the next few months or year (e.g., you've had it for many years without any signs of improvement), then the MAC rate, i.e., the odds of the machine curing you, may be more like 25%. Which I think is still pretty good--a lot better than nothing--but then you can see that price does matter. This reasoning may also explain why the experience on this board has fallen far short of expectations.
Re: 94% = 25%, revisitedelliott on 6/04/04 at 08:23 (151995)
Just to clarify my last paragraph, what I'm saying is, if you are one for whom the placebo effect and likelihood of healing are very minimal or nonexistent, then your success rate is more like 25%.
Re: 94% = 25%, revisitedPauline on 6/04/04 at 09:48 (151996)
And you claim to know nothing about math, not true.
Re: 94% = 25%, revisitedBill jr on 6/04/04 at 11:51 (152010)
The 92% (not 94%) referred to in the article was the reduction in pain of the treated group, not to a success rate. If you look at the article, 76 patients were in the treated group. The average pain level for the group before treatment was 7.7 on a scale of 0-10. The article surveyed the treated group at one year, 50 patients responded. The average pain score for those 50 patients was 0.6 on a scale of 0-10, essentially pain free.
Based on your point-of-view, the treatment provided complete pain relief at one year for 65.8% of the patients treated (65.8% = 50/76). We do not know the results for the remaining 26 patients because they were lost to follow-up.
The results are very good and demonstrate the ESWT is an effective, long term alternative to surgery. Based on this study, patients treated by ESWT can expect a 65.7% chance of being painfree at one year, post treatment.
I hope that we see future studies with a larger number of patients.
Re: 94% = 25%, revisitedDr. Z on 6/04/04 at 12:06 (152011)
Hi Bill jr.
In the abstract section the authors talk about 94% as success rate ( R/M) scoring.. The 92% is the average reduction in pain after 12 months. ( VAS)
Re: 94% = 25%, revisitedelliott on 6/04/04 at 12:09 (152012)
No, it is indeed 94%. Yes, there were other data, but I am referring to the definition of success based on the 4-point Roles & Maudsley scoring (defined as moving up from 3 or 4 to 1 or 2). Dr. Z has used this measure as a sole definition of success. While I personally think this measure in isolation may be too soft, there is precedent.
Thanks for your comments. Can I ask who you are? E.g., do you work for Dornier?
Re: 94% = 25%, revisitedBill jr on 6/04/04 at 12:24 (152015)
Sorry about the mistake, but why do you care about the RM score. It seems to me that talking about the number of patients who are pain free is a more reasonable measure of the success of ESWT.
I don't work for Dornier.
Re: 94% = 25%, revisitedelliott on 6/04/04 at 12:49 (152016)
Dr. Z, I always ask you to read things carefully, but I'm not going to fault you for this one, because you have to read so carefully that it is really the authors' fault for not wording it more clearly.
Coincidentally and regrettably, there happen to be two 94%s: one is for the R & M scoring, which actually was a secondary efficacy endpoint but got a lot of publicity as a stand-alone success rate; the other, a primary efficacy endpoint, is the percent showing at least 60% improvement in their VAS scores for pain during first few minutes of walking in the morning. Both figures were 94%. The abstract of the article we overpaid for was not as clear as it should have been. It meant the VAS scoring, corresponding with the 56% given at 3 months, confirmed elsewhere in the article (not to be confused with a 57% meaning something else). Of course, it takes a very careful reading of the text. Some sentences you have to read verrrry slowly and carefully (e.g., last sentence on pg 295 continuing on to 296), where it first gives the percent improvement at 3 months and then in the next few words switches to a success rate at 12 months. Terrible sentence. The article gives 91% reduction (not 92%) in VAS pain from baseline (see top sentence in right column of page 193 and Table 5); it probably was mistakenly rounded up to 92% in other abstracts. Regarding the two 94%s, more clear is the penultimate paragraph in this link:
Re: 94% = 25%, revisitedDr. Z on 6/04/04 at 13:17 (152019)
Jim and Elliott,
That is my point and the R/M is does exactly this. It points out from the patient's point of view Excellent, Good, Fair Poor and defines each of these as to what they mean. Pain levels with activity are used to determination as to sucess.
Re: 94% = 25%, revisitedelliott on 6/04/04 at 13:19 (152020)
You'll have to ask Dr. Z and others that question. In fairness to Dornier, if someone goes from wheelchair to walking with moderate pain, that's a miracle but would not be included in success using your definition. Another reason may be that 66% doesn't sound impressive enough. Of course, I'd say it's too generous, because it includes the placebo effect and natural healing. I might add that all one-year figures should be viewed cautiously, since there are no comparable figures for the control group, which was unblinded at 3 months.
Don't work for Dornier. OK, then. Can you tell us anything else about why you're here? I gather you're not just a PF sufferer who wandered onto the site.
Re: 94% = 25%, revisitedDr. Z on 6/04/04 at 13:23 (152021)
I took the 92% from memory. It was a number from a graph that Dornier gave to Dr. Z in the beginning. It refers to the average reduction in pain ( 92% ) . VAS is so subjective that it really has no meaning to Dr. Z. R/M in my opinion is what counts. Bill Jr. pointed out that lack of pain is what counts and the R/M is the closest measurement of lack of pain
Re: 94% = 25%, revisitedelliott on 6/04/04 at 13:28 (152022)
Pauline, on the contrary, I said I was a math major. Regardless, I'm trying to make the point that this is just subtraction not requiring sophisticated math: take the placebos and natural healers out of the 94% and you're left with around 25%, the true power of the device.
Re: 94% = 25%, revisitedelliott on 6/04/04 at 15:03 (152027)
Regarding the 91% vs. 92%, it may be that your figure is right and there is an error in the paper. The baseline VAS was 7.7, and the 1-year VAS was .6; see Table 5. Then the 'average reduction in pain' as you call it, i.e., the improvement from baseline, can be calculated as (7.7 - 0.6)/7.7 = 0.922, or 92%. No doubt the two starting numbers are rounded figures. In order to make this division as small as possible, let's use the largest value allowable for .6, that is, .65, which of course lowers the numerator. The smallest value that can be produced is then (7.65 - 0.65)/7.65 = 0.91503, which still rounds up to 92%. A mathematical way of looking at it is this: in order to get this to round down to 91%, it must be true that
(X - .65)/X < .915.
Since X is positive, we can multiply both sides of the above inequality by X and the new inequality will still hold, i.e.,
X - .65 < .915X,
which simplifies to .085X < .65, which means X < 7.65, a contradiction to the assumption that the starting number was legitimately rounded to 7.7.
I wonder if there are any other data in that paper that are suspect...
Re: 94% = 25%, revisitedelliott on 6/04/04 at 15:22 (152028)
I think a weakness of R & M is that a patient who starts as a low Fair and improves a bit to a high Fair may feel inclined to tell the investigator he/she is now a Good, for otherwise there is no way to indicate the smaller improvement. (As an aside, due to the large significance of a jump between each point, R & M may necessitate a larger study group than otherwise in order for the results to be significant). In other words, I'm saying a case can be made that maybe 4 points is too few. That's a reason why a 10-point scale (or even a 100-point scale) has some advantages. And it's not really that complicated. Investigator asks patient to rank his pain, both at baseline and at endpoint, from 0 to 10, with 0 being painfree and 10 being the worst. More points allows smaller changes to be captured. Of course, R & M has more concretely defined categories attached to the numbers, an advantage for R & M. In any case, if real curing is going on, several measures should be impressive, not just R & M. I say the more data given, the better.
Re: 94% = 25%, revisitedEd Davis, DPM on 6/04/04 at 15:24 (152029)
We are getting overly fixated on a particular study. Studies have signficant limitations. Say, we took patients with a two year history of intractable PF and got similar figures after following them for two more years, then the numbers would be more compelling.
It would be really cool if we could take the thousands of patients that we treat collectively and hire someone to run efficacy studies. That just is not reality in the world of healthcare. Such studies are of value and important but most of healthcare, I would venture to say, is based on collective empirical observation. When you have patients with multi-year histories of PF ,refractory to multiple modalities and see them being cured after application of ESWT and that happens doezens of times; then several dozen fellow practitioners relate similar experiences, that is really how we make up our mind what works.
I have seen thick achilles tendons, fibrotic, weak, degenerating and literally watch them go back to normal in front of my eyes -- that convinces me. I don't need a new study on ESWT for every body part -- I have seen it work, visually (ultrsonography) watching thinning and normalization of fasciae on scores of patients -- that is what counts.
I read the new paper by Buchbinder with amazement because if you have several hundred million dollars and dozens of years to do ironclad studies on all that we do in healthcare, it would be a wonderful world indeed. It is not reality. It probably never will be. If we LET the insurance companies set requirements for coverage that are based on a level of evidence that any reasonable practitioner knows will never arrive, then we have handed them a major victory and reason to deny valid treatments to millions of people.
Re: 94% = 25%, revisitedelliott on 6/04/04 at 16:15 (152032)
Dr. Ed, there are a lot of injustices in the world. Even if we acknowledge that the lack of insurance coverage for ESWT is one of them, you have made this (and Buchbinder) your one major pet peeve. You don't spend much time complaining about other injustices. As righteous and caring a caregiver as you no doubt are, you happen to have a personal financial stake in ESWT. This makes you less than an ideal advocate for it, as your impartiality can be questioned due to potential financial gain. The fact that you are a doctor does not change this either: until at least recently, U.S. doctors historically have as a group behaved in questionable ways that happened to further increase their salaries, with the annual increases well beyond those in other sectors. In other words, as a group they could not be trusted. Had they not behaved that way, they might have been less clamped down on than they now are and allowed more freedom to offer care as they saw fit. Mandating insurance coverage for ESWT helps you and the owners of the devices but passes on the expense to us beneficiaries at large in the form of higher insurance premiums--as if they're not high enough already.
ESWT treatment is expensive, and its potential use widespread, so it would be wise to ascertain ESWT efficacy before it is covered by insurance. Let me ask you: if it really is so that 94% = 25%, should it still be covered by insurance? I personally feel that the ESWT evidence to date is still far from clear, and that includes the Dornier FDA study. If you feel there is some injustice in the process, life is full of them.
Re: 94% = 25%, revisitedEd Davis, DPM on 6/04/04 at 16:57 (152036)
The thing that we have to ask is what is the 'standard of care' of treatment of a disease entity in the community? When it comes to PF -- we have orhtotics, night splints and now ESWT which all can be called into question from a research based approach. If insurance stopped covering night splints and orhtotics, you would see me complain here just as ardently. Some insurance companies don't cover orthotics and patients pay out of pocket. Few could pay out of pocket if all three modalities were removed from coverage based on Buchbinder's criteria.
If money was the issue for doctors we would have no trouble embracing surgery. Plantar fascial release surgery takes about 5 to 10 minutes, pays the surgeon 700 to $900. The anesthesiologist gets paid $300 to 800 and the hospital several thousand and you have no problem with that! You seem to have no trouble with the fact that insurers pay for this procedure. Why?
Re: 94% = 25%, revisitedBill jr on 6/05/04 at 07:21 (152076)
I don't agree with your logic. People can have PF for many years. It is not possible at the time of failed treatment to know whether your PF will resolve in one year or whether you are chronic. Your logic implies that there is a 25% chance of success when the truth is that there is greater than 67% chance. If you don't have the treatment then the chance of being better is much lower due to the natural course of PF, of placebo effect. Are you willing to wait an additional year to learn whether you are one of the unlikely who would respond to ESWT?
Additionally, your reasoning could equally be applied to surgery showing that the success of surgery if also low.
It is really a timing issue. You decide when PF is chronic 6 months, 1 year, 5 years, and then test ESWT or surgery and judge success.
Re: 94% = 25%, revisitedBill jr on 6/05/04 at 07:27 (152077)
I understand your concern about the control group being unblinded at 3 months . However it is not reasonable to expect study participants to remain blinded for one year. Remember study participants were restricted from all conservative therapies for the length of the study. This means that all of the study's participants where prohibited from taking pain medicine, orthotics, message, etc... until the study ended. I doubt that anyone would live with PF pain for one year, untreated, for the sake of science. Call me crazy, but I just don't see it happening.
Re: 94% = 25%, revisitedEd Davis, DPM on 6/05/04 at 14:12 (152107)
You make a good point -- one that places limits on the 'ease' of doing studies. Experimenting on human beings has its limits. How can you take a large group of people with PF, tell them to do nothing but suffer so that they will become the 'control' group.
I recently had a similar conversation with one of the docs in my area that performs spinal implant stimulators. We know that they work. Forty nine out of 50 states Workers Comp. systems pay for their use in chronic pain patients. Yes they are expensive. WA State is the only state in the US where the medical director disallows payment using similar arguments. Realistically, we would need to get several hundred patients in severe pain and implant, surgically dummy spinal stimulators in half of them. Then we could do the study. It is just not going to happen.
Re: 94% = 25%, revisitedEd Davis, DPM on 6/05/04 at 14:25 (152109)
You have brought up some interesting points. The 6 month figure has been used as an arbitrary number. I beleive that one can have the pathology present for years and depending on vocation/avocation, shoegear choices, etc. start manifesting symptoms at a certain point. About 3/4 of patients ask me the same question -- why now? It cannot be answered.
I think that as ultrasonography gradually becomes more of the diagnostic standard for PF we will have a better guide because what we are truly treating with ESWT is a pathologic fascia, ie. plantar fasciosis. The criteria for when to use ESWT, is really better answered by the question...'is fasciosis present?' To do this we need two criteria: the time element (possibly) but a ultrasonographic picture showing us fasciosis. That is, a fascia that is pathologically thickened along with other signs such as a hypoechoic area at the area of pathology. Rompe has done research to show the mean thickness of helath fascia to be 3.5 mm. I think more studies along that line would be helpful.
Re: 94% = 25%, revisitedelliott on 6/05/04 at 23:11 (152138)
Bill, what I thought I made clear is that a good chunk of that 67% are people who would have gotten better anyway. Does the machine get credit for that? Correct, you don't know in advance if you're one of them, but if the value added by the machine is only 25%, you might decide in advance whether it's worth shelling out $3000 for it. Part of the problem is there's no specific time for when PF is considered chronic. I know a runner who had a severe case for four years, got better with conservative means, and is now running again.
Re: 94% = 25%, revisitedelliott on 6/05/04 at 23:18 (152139)
I never said everything about our health care system makes sense. But a night splint is around $50, an orthotic $300, ESWT $3000. PF in particluar is widespread and I don't fault them for wanting to be reasonably confident that the new technology works before endorsing it. Regarding surgery, the insurance system leaves the appropriateness of surgery up to doctors.
Re: 94% = 25%, revisitedBS on 6/06/04 at 08:03 (152147)
My orthotics were $700. I'm still having trouble with them after 3 adjustments and over a year trying to wear them.
Re: 94% = 25%, revisitedEd Davis. DPM on 6/06/04 at 12:28 (152151)
That does not explain WHY a third party will trust doctors only with surgery but not with non-surgical modalities. Why should there be virtually no burden of proof for surgical coverage yet when doctors embrace a means to cure thousands of people, preventing surgery, you are taking a strong stand!
Re: 94% = 25%, revisitedEd Davis. DPM on 6/06/04 at 19:53 (152168)
I don't think that anyone has any problem coming up with a definition of when PF is intractable and ready for surgery or ESWT. I, persoanlly, think that it should be a combination of time and ultrsonographic evidence. The vast majority of patients we see with PF get better without either. The reason ofr this site is that either patients visit in whom treatment was incomplete, innapropriate or they actaully have intractable cases.
Re: 94% = 25%, revisitedBill jr on 6/07/04 at 07:24 (152197)
I understand your argument about the 25%. I disagree with your reasoning. We will have to agree to disagree. However, from what I see, your reasoning can be applied to all PF treatments making treating PF a losing proposition. I don't know of any PF treatment that has a value greater than 50% since there is a high placebo effect.
Are you advocating doing nothing unless the treatment is paid for by insurance? If so, why should the insurance companies pay for something that you would not pay for yourself?
In terms of your runner friend, would he have paid for ESWT if it had a 67% chance of reducing the duration of his severe PF from four years to one year? Or more interestingly, why did he live with severe PF for four years when he could have had surgery?