Undertaking Systematic Reviews with the Cochrane Schizophrenia Group
Interested parties contact the Cochrane Schizophrenia Group to register themselves as reviewers and propose a title for a protocol to be developed. Registered reviewers are encouraged to attend a training workshop for 'Developing a protocol for a review', and a later one on ‘Introduction to analysis.’
Protocols are expected to be submitted within six months of registering a title. The CSG editorial base and the designated key editor provide ongoing support as necessary throughout the development of the protocol. Following submission, the protocol is peer reviewed by an anonymous editor and copy-edited and processed for publication at the editorial base.
Reviews are generally expected to be submitted for publication one year after publication of the protocol. The editorial process as described for the protocol is repeated, and reviewers are additionally encouraged to seek further expert peer review.
Reviewers are responsible for periodic updating of their reviews, at least every two years. If obvious studies become available in the interim, the key editor would encourage earlier incorporation of these data. All updates are handled as described above for the initial submission of a review.
Reviewers are encouraged to contact the editorial base for feedback and information throughout the process of completing protocols and reviews. The work of completing a review is challenging and can become daunting or frustrating at times, and editorial base staff can provide the resources or expertise to deal with problems as they arise, whether they relate to data extraction or downloading software. Our reviews are written in a programme called Review Manager (RevMan) which can be downloaded for free (see the links below)
In addition to supporting the review completion process, the editorial base provides information of interest to the reviewer concerning projects and events organised by either the CSG or The Cochrane Collaboration. This may include notification of training events, launches of new software, or publications relevant to the work of individual reviewers.
The Links below should contain all of the relevant documents that an author will require to get started with the Cochrane Schizophrenia Group.
Cochrane Schizophrenia Group Registration Form
Protocol Registration Form
Analysis Registration Form
Examples and Guidelines For Reviewers
CSG Review Style Guidelines
CSG Model Review - Aripiprazole For Schizophrenia
Study Based Registers - A Brief Introduction
Setting Up and Maintaining a Study Based Register
Cochrane Schizophrenia Group Translation network
Cochrane Newcomers Guide
Cochrane Collaboration Web Page
Cochrane Handbook for Systematic Reviews of Interventions (formerly the Reviewers' Handbook)
Cochrane Library Web Page
Meerkat User Guide (pdf)
For more information about resources available to reviewers at the CSG editorial base, please contact the review group co-ordinator (RGC) by email - Bethany.York@nottingham.ac.uk
Deltakelse i Cochrane-samarbeidet som oversiktsforfatter: Du kan melde deg inn i den oversiktsgruppen som er mest relevant i forhold til ditt helsefaglige interesseområde. Hvis du ikke finner en gruppe som ser relevant ut, registrer deg som interessert i å delta i en ny gruppe. Som medlem av en oversiktsgruppe får du støtte og opplæring i arbeidet med en systematisk oversikt, og et internasjonalt publikum når arbeidet ditt publisr i The Cochrane Library.
- Cochrane Handbook for Systematic Reviews of Interventions - offisiell håndbok for utarbeiding av Cochrane-oversikter
- RevMan homepage - dokumentasjon og veiledning for programvaren som benyttes for å lage og oppdatere Cochrane-oversikter.
- Cochrane Style Resource – sjekk din Cochrane-oversikt opp mot de offisielle språklige retningslinjene
- Using Individual Patient Data - Power Point-presentasjon
- Browse the list of Cochrane Review Groups
- Re-publishing of reviews - forklaring av prosedyrer og søknadsskjema for å få tillatelse til å publisere din Cochrane-oversikt I et annet vitenskapelig tidsskrift QUOROM statement checklist (PDF-dokument). Sjekkliste over ting som oversiktsforfattere kan framheve I oversikten for å forsikre leserne om påliteligheten av funnene.
Opplæring ansikt til ansikt
Open Learning Materials – lær framgangsmåten I passe store, nettbaserte moduler Dette er et supplement til Cochrane Reviewers' Handbook – til hjelp I prosessen med å ferdigstille en oversikt.
Opplæringstilbud fra andre organisasjoner:
Undertaking Systematic Reviews of Research on Effectiveness – omfattende veileder fra NHS Centre for Reviews & Dissemination
Methods used in reviews
Access to specialised register by reviewers
The Cochrane Schizophrenia Group's Register, based in Nottingham, UK, is available to all reviewers by contact with the editorial office. The Trials Search Co-ordinator conducts searches and sends results to reviewers. At present (November 2008) the Group's Register contains 10232 coded studies.
Additional search strategies
Citation searching: This can be undertaken either electronically or manually. The methods by which reports are selected for citation searching are made explicit and reproducible.
Personal contact: The methods by which individuals were selected for contact are made explicit and reproducible.
Pharmaceutical industry: The methods by which companies are selected for contact and, in turn, how they select material for the review are made explicit and reproducible. It is important that pharmaceutical companies are not contacted directly but through the editorial base. CSG works to foster good relationships with pharmaceutical companies in an attempt to obtain the largest dataset possible. Individual approaches may lead to rejection, duplication of effort and the perception that CSG's efforts are ill-coordinated.
The Cochrane Schizophrenia Group encourages selection of trials to be performed independently and in parallel by at least two reviewers. The Group has no policy on whether those selecting studies should be blinded to author, institution and journal. Inter-rater agreement may be calculated using Cohen's kappa. Disagreement may be resolved by discussion or by contacting the authors of the study.
Assessment of methodological quality
The Cochrane Schizophrenia Group encourages the use of quality rating of trials within reviews based on the three quality categories as described in the Cochrane Collaboration Handbook (Clarke 2002). Reviewers are advised to include studies of categories A and B. If quasi-randomised trials, (category C), are included in the review, reviewers are required to perform a sensitivity analysis to detect any bias introduced by the more bias-prone category C studies. If additional scales are used they should be described.
The editors recommend data to be independently extracted by at least two reviewers. Any disagreement should be resolved by discussion and all decisions documented. Where necessary, the authors of the studies can be contacted to help resolve issues. Summary data from published or unpublished reports are the most frequent source for this Group's reviews. Where individual patient data are sought, every effort is made to ensure completeness. In an effort to minimise publication bias, reviewers are encouraged to include unpublished data.
It is recommended that reviewers enter data into RevMan independently of each other.
1. Incomplete data.
For particular outcomes, where data is felt to be too prone to bias because of poor levels of follow up, reviewers pre-state the proportion of drop out at which the data is thought to be unacceptable.
2. Dichotomous - yes/no - data.
2.1 Statistics: For homogenous binary outcomes, for example 'admitted' or 'not admitted', an odds ratio (OR), usually the Peto OR, with the confidence interval (usually the 95% CI) is estimated. Where possible, the number needed to treat statistic (NNT) and its confidence interval is also calculated. For heterogeneous data the OR is calculated using the DerSimonian-Laird random effects model.
2.2 Intention to treat: Where possible data is presented on a 'once-randomised-always-analyse' basis. For outcomes, such as relapse, the authors assume certain outcomes for those lost to follow up. This is often the negative assumption - for example, those who were lost to follow up all relapsed. The assumption may vary depending on the outcome but all assumptions are pre-stated and tested in a sensitivity analysis. Deaths are not counted as losses of contact and losses to follow up are not counted as deaths.
3. Continuous - scale - data
3.1 Intention to treat: In this instance this is impossible, unless individual patient data are available. The reviewer has to use what data are reported. 'Last observation carried forward' (LOCF) data are preferred to a 'completer analysis' only. It is important not to forget that at some point there is such a great loss that it is not worth presenting data at all (see 1. Incomplete data).
3.2 Normal data: Mental health continuous data is often not normally distributed. To avoid the pitfall of applying parametric tests to non-parametric data the following standards are applied to all data before inclusion: i. standard deviations/errors and means have to be obtainable; ii. when a scale starts from a finite number (such as 0), the standard deviation, when multiplied by 2, was less than the mean (as otherwise the mean was unlikely to be an appropriate measure of the centre of the distribution - Altman 1996). Endpoint scores on scales have a finite starting point and top score; therefore this rule can be applied to them.
Data that does not meet the second standard are not entered on the RevMan software (which assumes a normal distribution). However, data not meeting these standards can be reported in the 'Other data types' of the result section if they had been analysed with appropriate non-parametric tests.
Change data is even more problematic as it does not have finite end points - whereas the scale from which the data is derived ranges from 0-100, the change in that scale may go from -100 to +100. Although most change scores are likely to be skewed it cannot be proven from the rule outlined above. The reviewers should pre-state how they are to manage such data - at present there does not seem to be an ideal method. If, however, both change and endpoint data are presented for the same outcome, within the same study, or across different studies, the latter is presented in preference.
3.3 Rating scales: A wide range of instruments is available to measure mental health outcomes. These instruments vary in quality and many are not valid, or even ad hoc. For outcome instruments some minimum standards have to be set. They could be that: i. the psychometric properties of the instrument should have been described in a peer-reviewed journal; ii. the instrument should either be a. a self report, or b. completed by an independent rater or relative (not the therapist); and iii. the instrument should be an unmodified global assessment of an area of functioning.
Reviewers are encouraged to look carefully at all data in case the results of one or two trials are clearly not in accord with the others. As well as inspecting the graphical presentations, reviewers are advised to check the differences between the results of each included trial using the Chi-square test of heterogeneity. If heterogeneity is present the reviewers undertake a sensitivity analysis to the presence or absence of these data. Reviewers are advised to interpret a significance level less than 0.10 as evidence of heterogeneity. All data from studies that have been selected are presented.
5. Addressing publication bias
Data from all identified and selected trials can be entered into a funnel graph (trial effect against trial size) in an attempt to investigate the likelihood of overt publication bias or other variables associated with skewed funnel plots.
6. Tables and figures
Where possible, data is entered into RevMan in such a way that the area to the left of the line of no effect indicates a favourable outcome for the intervention of interest.
Reporting of reviews
Reviewers are encouraged to undertake the discussion and conclusions section viewing the data from the perspectives of not only clinicians and researchers, but also policy makers, managers and consumers of care. Inclusion of cost data and a health economic evaluation is supported.
After consultation with an editor these are submitted to the Co-ordinator and recorded within the Group module. Protocols should be submitted within one year after registration of a title.
Two editors, one with primary responsibility for supporting the authors and one anonymous, provide ongoing support and peer review. Furthermore, all protocols are sent for semi-structured lay and peer review. Reviewers are welcome to seek further expert peer review. Once available on the Cochrane Library comments are welcomed either directly to the reviewers, through the editorial base or the Cochrane Library. Reviewers have one year between submission of protocol and submission of the review.
Two editors continue to provide support and peer review. All reviews are sent for semi-structured lay and peer review. Reviewers are encouraged to seek further expert peer review. Once available on the Cochrane Library comments are welcomed either directly to the reviewers, through the editorial base, or the Correspondence Manager of the Cochrane Library.
Reviewers are responsible for periodical updating of their reviews, at least once a year. If obvious studies become available in the interim, the key editor would encourage earlier incorporation of these data. All updates are handled as set out above for the initial submission of a review.
Should a review become clearly out of date and the reviewers not be able to update their work, the key editor would negotiate with the reviews in order to re-allocate authorship of the updated versions. So far no disputes have arisen on this matter and it has been entirely possible to resolve update issues by negotiation. Should dispute arise editors not directly involved with the review would be asked to adjudicate.
All reports of controlled trials relevant to the scope of this group, whether randomisation is described or implied, are acquired, coded and included. The Cochrane Schizophrenia Group's Register therefore contains some quasi-random studies and trials that do not make the allocation procedure explicit. Currently it also contains non-clinical studies where physiological outcomes are recorded in preference to clinical measures. It does not contain studies where before and after (A-B-A) design has been used. The Cochrane Schizophrenia Group's Register contains 11608 coded studies containing 14143 references (November 2009). The register is maintained on Meerkat 1.5. This version of Meerkat stores references as studies. When an individual reference is selected through a search, all references which have been identified as the same study are also selected.
The Group also maintains a large register of citations (~100,000) and papers (~7767) that have been inspected and found not relevant to the work of this Group.