Jan R, Elliott, and Dr Z postsPosted by Scott R on 2/20/04 at 22:45 (144878)
Dr Z, someone complained and i looked at the files that were deleted and i don't see where they should be deleted, especially all of them. So I'm reposting the ones that don't mention Dr Z (i assume those are the ones you objected to).
Here they are:
Jan R posted:
The strength of a study is defined by its statistical power. If you see a high between-group difference with low standard deviation, you may have a high power though your sample size may be quite low.
Calculation of a sample size also depends on the variable you declare as your main outcome measure.
As you deny adequate treatment to 50% of your patients in a placebo-controlled trial I find it questionable from an ethical standpoint to expand patient enrollment over the absolutely necessary minimum (statistically seen).
What clearly is favourable is a multi-center design!
Of course, the more patients you enroll per group, the better. But you have to weigh advantages and disadvantages. If you have a clear result in 50 patients, you don´t have to examine 500 patients.
It would be more favourable in my opinion to repeat the exact treatment design at a second, independent location.
Dr. Jan R:
I understand what you're saying, as well as the ethical and practical considerations. But with new treatments one usually doesn't know what the standard deviation is until after conducting the study. What I'm saying is, if one year later the treatment group success rate is, say, 85%, and the placebo group success rate is 75%, then 85% of 25 is 21 (rounded) people, and 75% of 25 is 19 (rounded) people. Sure, the binomial distribution has quite a small standard deviation, and the difference between 75% and 85% can be statistically significant, but I just wouldn't want to be making any sweeping statements as to ESWT efficacy based on a difference of 2 people between the groups.
Jan R posted:
This is why one should define which between-group difference at which point of follow-up is considered statistically significant (before the trial starts).
If you then find a difference you have to determine whether this difference is clinically relevant at all.
Usually only differences of at least 20% are considered both significant and relevant (Should be defined before the trial starts).
Regarding standard deviations: That is why you perform a pilot study. It is this pilot study with its results (mean and SD) that then enables you to calculate sample size sufficient for a power of more than 80%.
Look at the Haake trial (BMJ 2003): No pilot trial before, sample size calculation pure speculation. 100s of patients enrolled in a protocol that would have shown in less than 50 patients to be ineffective.
Regarding FU of one year: As to plantar fasciitis I am not aware of any controlled trial focusing on 12-month follow-up as main follow-up interval. With tendinopathies and their great tendency for self-healing, between-group differences at much shorter notice have been chosen for main outcome measure.So, we cannot provide evidence on 12 months after treatment.
Dr. Jan R, good points. I think this discussion is very helpful.
Concerning the Haake trial, inefficiencies and all, what I found most interesting was that it is perhaps the only (large) study that kept the control group blinded for an entire year, and (ethical issues aside) the one-year success rate for this group was an impressive 75% (this even though a main criticism of the study was that not enough appropriate conservative care was continued throughout). So in my mind, ceteris paribus (as if this were so, because it's not), a study claiming, say, a one-year success rate of 81% may not have achieved all that much over placebo.
Concerning your first and last points, namely
'one should define which between-group difference at which point of follow-up is considered statistically significant (before the trial starts)'
'Regarding FU of one year: As to plantar fasciitis I am not aware of any controlled trial focusing on 12-month follow-up as main follow-up interval. With tendinopathies and their great tendency for self-healing, between-group differences at much shorter notice have been chosen for main outcome measure. So, we cannot provide evidence on 12 months after treatment',
very well said indeed. These two basic points seem to be either violated or ignored in much of the discussion on heelspurs. Both here and on a popular ESWT web site (the one that always shows up first in any web search), it is repeatedly pointed out (at least with full disclosure, unlike many annoying anonymous or semi-anonymous posts on this board claiming all kinds of things) that the one-year FDA Dornier Epos results showed a 94% success rate. Ignoring that the FDA officially only followed the study for 3 months (and so a more accurate wording would be 'one-year followup on the FDA study'), this violates both of your points at once: it emphasizes a one-year period (without a control group to compare it with, I may add), and also conveniently selects after the fact what was just ONE of the SECONDARY endpoints, the Roles and Maudsley pain score. A justification might be that the R & M pain score has become a standard endpoint in ESWT studies for PF, but this does not change the fact that it violates your first point. It also happens to be true that the Ossatron to which it is often compared had a far more selective criterion for success and, to its credit, stuck with it at the end, even on their web site. So it could be that if you compare apples to apples, Ossatron might come out the more impressive machine.
There is one other aspect of the FDA trials that has bothered me for some time. When they chose the patients for the ESWT trials, they chose patients who were most likely to be able to heal themselves, by other methods (such as stretching, rest, ice, etc). The patients only had to have PF for one year. Why didn't they pick some people who had PF for 10 years? They also would not take anyone who had any surgery. Yet, we continoulsly hear that ESWT will cure patients that have had surgery. In fact, it has been said on these pages that patients who have had surgery are some of the easiest to cure! Sorry, I don't see it happening on this message board. I think the FDA did everyone a great disservice, by not taking patients who represented the majority of people who have PF. I personally think they (FDA) should look at their original criteria, and then conduct a second group of trials for both the Ossatron, and the Dornier, with patients that represent a majority of PF sufferers.
Am I off base here?
BrianG, sorry, I'm going to have to disgree somewhat. The problem is not with the FDA study, but rather with the marketing and advertising that take place afterward, leading many to believe their chances are much higher than the evidence dictates. But if I were going to do an initial FDA study on PF, I'd start with similar criteria as the FDA did. You first want to see what ESWT can accomplish with 'pure' PF, and then and only then you start to expand the horizons by including or excluding groups, e.g., long-term and post-surg patients. I'm with you that the more studies of this sort, the better, and hopefully we'll see them in time. Keep in mind also that this issue is complicated by the fact that a large percentage get better with time anyway, so peeling away the amount attributable to ESWT is problematic.
BTW, FYI: the minimum was 6 months [conservative care], not 1 year, and both FDA studies *did* include people who had symptoms for 10 years and longer; check the links. Regarding duration, a new study has found little difference in success rates between those under and those over the 2-year limit:
See, they're working on it!
As I read it, you are correct. I know it was a few years ago, and I was just going by memory. Something I probably shouldn't do! ) I really did think that the majority of the patients only had PF for 1 year. I still don't see the healing rates on this forum, any where near what the FDA would lead us to believe.
Re: Note to elliott about statistical significanceScott R on 2/20/04 at 23:19 (144882)
Elliott i wanted to point out that the proper use of statistics (and doing the calculations in revverse to pre-determine your study size) should take into account small sample sizes as in your 19 verses 21 patient example. Using your example, here's how to determine if the difference is significant:
treated non-treated total
------- ----------- -------
success 21 19 40
failure 4 6 10
total n1=25 n2=25 50
proportion p1=.84 p2=.76 p=.80
z score= (p1-p2)/(sqrt(p*(1-p)*(1/n1+1/n2)) = 0.71
The z score has to be at least 1.96 to have 95% confidence, so your example is very far from having statistical significance and the design of the study should take this into account.
Re: Note to elliott about statistical significanceEd Davis, DPM on 2/21/04 at 00:36 (144888)
I too am unsure why exctly caused deletion of the posts.
For the benefit of readers, could you please take the time to explain, in lay terms the meaning of the discussions over statistics. I think many here would greatly appreciate that.
Ed Davis, DPM
Re: Note to elliott about statistical significanceDr. Z on 2/21/04 at 01:09 (144891)
I may have deleted too many. My purpose was too avoid the repeat situation that happened with Elliott and myself in the past. I am in agreement with what has been placed back into this board. I really am trying to avoid these long debates that block the purpose of the board and that is what I was trying to do. Its my first day so I am learning. I
Re: Note to elliott about statistical significanceDr. Z on 2/21/04 at 09:14 (144908)
I have a new computer ED. I would be happy to e-mail you the threads that were deleted. Please give me your e-mail address again.
Re: Note to elliott about statistical significanceEd Davis, DPM on 2/21/04 at 12:06 (144922)
Hi Dr. Z:
It is (email removed)