Update on seminar series

If you are here looking for the next scheduled seminar, sorry for being so quiet!

We have teamed up with the Clinical Trial Service Unit (CTSU) and aim to bring a new schedule of statistical primers, seminars and paper discussions starting from September 2010. I will circulate more details in September.

In the meantime, if you have an idea for a seminar or would like to present some of your own work, please do email me and let me know!

Nada

The power of graphs in meta analysis

We met today (April 26) to discuss the paper 'More than numbers: The power of graphs in meta-analysis' by Leon Bax and colleagues.

We went through the different plots types (short powerpoint presentation below). A few of the points I picked up on included:

- Limitations of the funnel plot, including the need for at least 25 studies or more to determine whether any studies are 'missing'.
- The need for accurate and complete trial registers to estimate publication bias
- Reporting the funnel plot statistic instead of including the plot in a paper
- The usefulness of L'Abbe plot, which can be extended to other studies with continuous variables. It was also pointed out that L'Abbe plot demonstrates whether the different studies report a constant risk reduction/increase which may be useful. There is a 'bubble plot' in Excel which can be used to draw a L'Abbe plot.

Many of the attendees said that they generally didn't report any plots other than a forest plot in a meta-analysis. Paul G. said that he sometimes tries a L'Abbe plot, and has also used 'Rosenthal's file drawer N' method to estimate publication bias by estimating how many studies of no effect would be needed to change the summary estimate.

We had some concerns about the simulation studies in this paper. The researchers who scored the graphs' ability to demonstrate hetereogeneity or bias may have benefitted from a training period, and may have preferred the forest plot due to familiarity. We thought that it may have been better to have more raters scoring fewer graphs, and agreed that most of the different plots were poor at assessing publication bias (Table 4).

Our next meeting is scheduled for May 19.

Meta Analysis Plots

April 21 meeting options

Dear all,

I hope that some of you will be able to join us next week for the
stats and epidemiology journal club on Wednesday April 21. The
meeting will be held at 1pm in the 2nd floor meeting room in the
Rosemary Rue Building.

I have attached three possible papers for discussion. If you are
interested in coming, please send me back your vote for which paper
you'd like to discuss. The options are:

1. Bias in identifying and recruiting participants in cluster
randomised trials: what can be done? Sandra Eldridge and colleagues,
BMJ. 2009 Oct 9;339:b4006.
A discussion of how to design cluster randomized trials to minimize
selection bias

2. More Than Numbers: The Power of Graphs in Meta-Analysis. Leon Bax
and colleagues, Am J Epidemiol. 2009 Jan 15;169(2):249-55
What graphs should we use to report the results from a meta analysis?
A comparison for some of the popular options and limitations in each.

3. Power for studies with random group sizes. Walter T. Ambrosius,
Stat Med. 2010 Mar 11.
How to appropriately conduct a power calculation for observational studies

Let me know by next Monday and I will circulate the winning paper! Just email me at nada.khan@dphpc.ox.ac.uk.

Meeting on 17 March 2010 - Capture/Recapture

Thanks to Geraldine Surman and Matthias Pierce from NPEU for a great talk on capture/recapture methods and applications in their own projects. I've uploaded their talk rfom Wednesday, which includes some Stata code and may be of interest to other people who wish to apply this method in their own research. Both Geraldine and Matthias are happy for people to contact them to discuss capture/recapture in further detail. Next meeting is scheduled for 21 April.

Capture Recapture Mar 10

Meeting on 20 January - Missing data and Mendelian randomization

Dear all,

Hope you enjoyed Nicola Fitz- Simon's discussion on missing data and Mendelian randomization. I've uploaded her slides (see below) so do take a look if you missed the meeting. Our next meeting is scheduled for Wednesday February 17.

Missing Data and Mendelian Random is at Ion

December 16 discussion - Am J Epi paper by Kurth et al

Today we discussed the paper by Kurth et al in the Am J Epi . Some slides on our discussion can be found just below this post. Our next meeting is on Wednesday, January 20. Nicola Fitz Simons will be leading a discussion on missing data and Mendelian randomization. Happy Christmas!


December Meeting

Next meeting - 16 December, 1pm

Our next meeting is on 16 December 2009 (Wednesday) at 1pm in the 2nd floor meeting room. The paper up for discussion is:

Kurth et al
American Journal of Epidemiology, 2006; 163: 262-270
Results of Multivariable Logistic Regression, Propensity Matching, Propensity Adjustment and Propensity-based Weighting under Conditions of Nonuniform Effect

I've put together a few slides to get our discussion going - I thought that some points for discussion could include the benefits and disadvantages of different adjustment methods, and a consideration of what question we're asking of the data when we conduct analysis.

Hope to see you there! I'll bring some Christmas treats for us to snack on during the meeting. Let me know if you can't get a hold of the paper full text.

Summary of our second meeting on Oct 26

Many thanks to Karen Smith for her clear and informative seminar on the use and application of interrupted time series analysis. Karen has kindly allowed me to publish her Powerpoint slides online, so if you are interested in seeing her talk, it's just below this post. Our next meeting is on Wednesday, November 18 at 1pm. Nicola Fitz-Simons, who is based at NPEU, will be leading a seminar based on some of her methodological research.
Interrupted Time Series

Seminar by Karen Smith

The next journal club will be led by Karen Smith, senior medican statistician at the Centre for Statistics in Medicine here in Oxford. Karen will be leading a discussion on interrupted time series analysis based on a paper she worked on, which was recently published in the BMJ. The paper can be found here:

Effect of withdrawal of co-proxamol on prescribing and deaths from drug poisoning in England and Wales: time series analysis

Please do join us for this talk and discussion - just to note that Karen will be giving her talk on Monday, October 26, NOT October 21 as we had previously scheduled. All seminars will take place from 1-2pm, 2nd floor meeting room, Rosemary Rue building.

Testing for baseline balance in clinical trials

There are a number of interesting points in this paper. In particular, section 4: Some misconceptions about balance.

To paraphrase, consider a two arm trial in which 200 patients (100 male and female) are allocated at random to one of two treatments. The resulting proportions of male and females in each arm after randomization are different (this is far more likely to be the case than being exactly equal (50 males and females in each arm) regardless of how well the randomization was performed. Exact balance in the covariates is probably more indicative of a non-randomzied trial.

Does this matter, yes if the covariate in question (gender in this case) has an effect on the outcome,

Suppose a model for the outcome y is;

y(1= treatment) = mu + beta1*x + theta + error

y(0 = control) = mu + beta1*x + error

where theta is the true treatment effect and beta1 is coefficient for the confounding covariate and x is the covariate (gender). A naive estimate of theta is the difference of means between the two groups.

theta.hat = ybar(1) - ybar(0)

which will be biased by the value of

beta1(xbar(1) - xbar(0))

The magnitude of which is dependent on the distribution of x across the treatment groups and the value of beta1.

The problems with testing for imbalance:

It is common for people to judge balance via some kind of significance test of the group means. There are problems with this as is described in Senn's paper. Firstly, all that matters is the observed data not the wider population distribution over repeated identical trials. So the question of "is that a real difference" is meaningless and arises from the confusion between identifying a sample and population. Secondly, is one of why should a difference of two standard errors define imbalance. As described earlier any difference in the distribution of x is potentially relevant when beta1 is not zero and does not become nullified if p = 0.1, and as the p-value is dependent on the sample size it is possible for a difference in means to be balanced in terms of the p value in a small trial whereas in a larger trial the exact same difference would be identified as imbalanced.

The biggest problem with the significance test is not conditoning on an important covariate just because the p-value is > 0.05. If the effect of the covariate on the outcome is substantial the bias will be important. However, if balance is obtained in the important covariates it does not follow that you can ignore it and perform an unconditional analysis. Although, the estimate of the means will be unbiased, the standard errors will not be. The effect of the covariate on the variance is in fact maximised with the unconditional analysis.

Conclusion

This is from Senn's paper directly;

Thus, to sum up, a conditional analysis of an unbalanced experiment produces a valid inference; an unconditional analysis of a balanced experiment does not. Question; what is the value of balance as regards validity of an inference? Answer: none.