The power of graphs in meta analysis

We met today (April 26) to discuss the paper 'More than numbers: The power of graphs in meta-analysis' by Leon Bax and colleagues.

We went through the different plots types (short powerpoint presentation below). A few of the points I picked up on included:

- Limitations of the funnel plot, including the need for at least 25 studies or more to determine whether any studies are 'missing'.
- The need for accurate and complete trial registers to estimate publication bias
- Reporting the funnel plot statistic instead of including the plot in a paper
- The usefulness of L'Abbe plot, which can be extended to other studies with continuous variables. It was also pointed out that L'Abbe plot demonstrates whether the different studies report a constant risk reduction/increase which may be useful. There is a 'bubble plot' in Excel which can be used to draw a L'Abbe plot.

Many of the attendees said that they generally didn't report any plots other than a forest plot in a meta-analysis. Paul G. said that he sometimes tries a L'Abbe plot, and has also used 'Rosenthal's file drawer N' method to estimate publication bias by estimating how many studies of no effect would be needed to change the summary estimate.

We had some concerns about the simulation studies in this paper. The researchers who scored the graphs' ability to demonstrate hetereogeneity or bias may have benefitted from a training period, and may have preferred the forest plot due to familiarity. We thought that it may have been better to have more raters scoring fewer graphs, and agreed that most of the different plots were poor at assessing publication bias (Table 4).

Our next meeting is scheduled for May 19.

Meta Analysis Plots