| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|
|
|
|||||||
|
|||||||||
Scientific Section |
Liverpool University Dental Hospital and School of Dentistry, UK
Dr J. E. Harrison, Department of Clinical Dental Sciences, Liverpool University Dental Hospital and School of Dentistry, Pembroke Place, Liverpool L3 5PS, UK. E-mail:
Jayne.Harrison{at}rlbuh-tr.nwest.nhs.uk
Abstract
Aim: To test the hypothesis that there is insufficient evidence available, from clinical trials, to allow evidence-based decisions to be made on the effectiveness of orthodontic treatment.
Objectives: To identify reports of orthodontic clinical trials and assess their demographic characteristics.
Design: A retrospective, observational study.
Setting: The American Journal of Orthodontics and Dentofacial Orthopedics, British Journal of Orthodontics, and European Journal Orthodontics.
Data source: Clinical trials published between 1989 and 1998.
Method: A hand-search was performed to identify all clinical trials. The journal and year of publication, research method, interventions, and sample size of the trials reported were recorded.
Results: One-hundred-and-fifty-five trial reports were identified of which 56 (36.1%) were published from 1989 to 1993 and 99 (69%) from 1994 to 1998. Ninety-nine (69%) reports were published in the AJO-DO, 18 (11.6%) in the BJO and 38 (24.5%) in the EJO. Eighty-five (54.8%) were reports of randomized controlled trials and 70 (45.2%) of controlled clinical trials. The interventions most frequently assessed were bonding materials (21.9%), growth modification treatments (21.3%), and oral hygiene procedures (9.0%). The median sample size was 32 (IQR 19.5, 50).
Conclusion: There is sufficient evidence available from clinical trials to warrant doing systematic reviews of orthodontic clinical trials to aid decision-making.
Introduction
The randomized controlled trial (RCT) was originally developed for use in agricultural research and was modified by Hill in 1937, for use in humans.1,
2 The first report of the RCT being used in clinical medicine was in a study that assessed the effectiveness of streptomycin in the treatment of pulmonary tuberculosis.3 Since then the RCT has become the standard method to assess new drugs before they are licensed for general use. The RCT has also been used widely by many medical and some dental disciplines to assess differences in treatment effect between alternative procedures. However, orthodontics has lagged behind and it is only in the last few years that the results of some large RCTs, assessing competing orthodontic interventions and treatment strategies, have been published in the orthodontic literature.4–9
The aim of our study was to test the hypothesis that there is insufficient evidence available from clinical trials on the effectiveness of orthodontic treatment to allow evidence-based decisions to be made.
Our objectives were to:
Materials and methods
Identification of clinical trials
The principal investigator successfully completed the Cochrane Collaboration Oral Health Group hand-searching test search for the identification of randomized clinical trials (RCTs) and controlled clinical trials (CCTs).10 I hand-searched the American Journal of Orthodontics and Dentofacial Orthopedics (AJO-DO), British Journal of Orthodontics (BJO), and European Journal Orthodontics (EJO) to identify all papers that reported randomized or controlled clinical trials published between 1989 and 1998 inclusive.11
Assessments
The following information on each publication was recorded:
(Appendix 1); We then assessed the intra-examiner reliability by reclassifying a random 10 per cent sample of the trials identified in each journal.
Statistical analysis
Descriptive statistics were used to assess the distribution of trials published in each journal. Any differences in categorical data were evaluated with the chi-squared (
2) test. Odds ratios (OR) and 95 per cent confidence intervals (95% CI) were used to assess differences in dichotomous variables. Intra-examiner reliability for the classification systems used was evaluated with the Kappa statistic13 and percentage agreement.
Results
Reliability
The percentage agreement of the classification system ranged from 94 to 100 per cent with Kappa scores of 0.88–1.0. This suggests that the intra-examiner reliability was very good.
Journal and year of publication
A total of 155 papers reporting clinical trials were published in AJO-DO, BJO, and EJO between 1989 and 1998 (Table 1). Over two-thirds of the papers were published in AJO-DO, about a tenth in BJO and a quarter in EJO. This represents 7.0, 4.1, and 6.7 per cent of all papers published in AJO-DO, BJO, and EJO, respectively. These differences were not statistically significant (2 = 4.7, df = 2, P > 0.05). It was evident that, over the two 5-year periods, the number of reports of clinical trials had nearly doubled (Table 1).
|
Subject of clinical trials and interventions assessed
We found that most (148/155, 95.5%) of trials assessed either clinical materials or therapeutic interventions, and a smaller number (7/155, 4.5%) were concerned with diagnostic or educational subjects. Although there were more reports of clinical trials assessing therapeutic interventions (86/155, 55.5%) than clinical materials (62/155, 40.0%) there were seven times as many papers concerned with therapeutic interventions (576) than with clinical materials (82). The reports of clinical trials on therapeutic interventions therefore represented a significantly smaller proportion of the total number of clinical studies than those on clinical materials (OR 8.2; CI 4.8, 13.9). A more detailed examination of the interventions revealed that trials related to bonding materials or regimes (34/155, 21.9%), treatments to bring about growth modification (33/155, 21.3%), and oral hygiene procedures (14/155, 9.0%) were the most frequently reported. The remaining reports were concerned with different treatment mechanics, glove wearing, analgesics, etc.
Sample size
The median sample size was 32 (IQR 19.5, 50; see Figure 1). Importantly, of the 155 clinical trials only nine (5.8%) reports justified the sample size and/or contained a power calculation.
|
The publication rate of clinical trials in AJODO, BJO, and EJO has seen a dramatic increase with the number of reports of clinical trials (RCTs and CCTs) published nearly doubling between 1989–93 and 1994–98. Bonding materials and procedures, treatments to bring about growth modification and oral hygiene regimes were the most commonly assessed interventions.
Comparison with other specialities
Similar studies, that have assessed papers published in other journals have been carried out14–21 and revealed that the percentage of RCTs published was generally very low. However, when journals of other dental specialities, for example, cariology and periodontology were examined, up to a quarter of papers in their specialist journals were reports of RCTs. This is because many of the trials compared mouthwashes, toothpastes, or toothbrushes, and these may be more amenable to assessment in the context of a clinical trial.
Journal and year of publication
It was interesting to find that there has been a steady increase in the number of clinical trials published in these journals. This may have happened because of increased awareness by clinicians and researchers together with recent publication of the results from some large RCTs.4–9 However, one problem with orthodontic research is the time lag between the start of an orthodontic RCT and publication of the results. This is illustrated by the fact that it is only since 1997 that the results of some of these trials have been published. There are several reasons for this delay, including the recruitment of patients, length of treatment and follow-up, and the publication process.
Research method and controls used
The need for objective assessment of clinical treatments using more powerful prospective research methods was brought to the attention of the orthodontic profession over a decade ago.18 Although RCTs do provide the least biased assessment of differences in treatment effect between two or more treatments,22 they are expensive and time consuming. For these reasons, other research methods have been suggested as alternatives for assessing the relative merits of competing orthodontic treatment strategies.23,24 Such methods can provide valuable data; however, they should not be considered as easier routes to quicker answers and only used when there are compelling reasons that preclude the implementation of an RCT.25
Subject of clinical trials and interventions assessed
Subject.
One particularly interesting finding was that proportionally more clinical trials evaluated materials than treatment procedures. This may be due to materials being considered similar to drugs, for which there is legislation requiring them to be assessed in a clinical trial before they enter widespread clinical use, whereas treatment procedures are more akin to operations and can be introduced without formal testing. This deficiency of clinical trials of treatment procedures has also been identified in studies looking at the methods used in clinical studies to assess new and established operations.26
If we explore these differences further, we can suggest that drug trials find the drug that ‘does the greatest good for the greatest number of patients’ because their effect is largely independent of the clinician prescribing them. If this is extended to orthodontic clinical trials we can see them as having a third dimension by trying to establish which treatment ‘does the greatest good, for the greatest number of patients by the greatest number of operators’. This is because the outcome of orthodontic treatment is related to the clinician providing the treatment. If this concept is then applied pragmatically to the provision of orthodontics, we should not be looking for the technique that produces the best outcome for patients treated under ideal circumstances by a single operator in a dental hospital or university clinic. Instead, trials should ideally determine which technique produces, on average, the best outcome for the most patients when used by the main providers of orthodontic treatment. In the UK, these are orthodontists and dentists working in the General Dental Service27 and District General Hospitals (DGHs) within the National Health Service. However, from a worldwide perspective, most orthodontic treatment is probably carried out by orthodontists and dentists working in private practice. An ongoing clinical trial, on growth modification, has attempted to manage this problem by involving orthodontists working in several DGHs, rather than just in dental hospitals (O’Brien, British Orthodontic Conference 1998, 2000).
Interventions. The most frequently assessed interventions were those concerned with bonding procedures, growth modification and oral hygiene procedures. There are several reasons why these interventions may have been assessed more frequently in clinical trials than other orthodontic interventions.
Bonding trials.
The trials assessing bonding procedures included those that assessed different adhesives,28 methods of curing,29 and tooth preparation procedures.30 Bonding materials and procedures are probably more amenable to testing in a clinical trial and can take advantage of the split mouth technique, thereby using internal controls. From the patients’ point of view, such trials may be more acceptable to enter because they are usually direct comparisons of a standard treatment (adhesive) and a new one so participants do not run the risk being allocated to a no treatment control. It is also possible that the manufacturers of the materials being assessed may be more willing to provide financial support for trials assessing bonding materials.
Growth modification.
Another key area of clinical trial research has been growth modification. The trials have either been direct comparisons of different appliances31 or interventions,8 or a comparison of interventions with a no (or delayed) treatment control.6
Oral hygiene procedures.
Trials investigating the effectiveness of oral hygiene procedures have included those that have assessed mouthwashes,32 toothbrushes,33 and programmes to provide oral hygiene instruction.34 Mouthwashes and toothbrushes can also be viewed as being similar in nature to drugs and therefore considered more amenable to assessment in a clinical trial. These interventions are also suitable for assessment in crossover trials where all patients receive or use all interventions in a random or quasi-random order.
Sample size
The number of patients participating in a trial should not be determined by administrative convenience alone. If trials have too few participants they will lack power. Such trials may be considered as unethical because patients have been recruited when there was very little chance of the results being valuable.35 However, trials involving too many participants may also be considered unethical because they subject more patients than necessary to the possibly inferior treatment and require extra resources.36
Unfortunately, we found that many trials were inadequately powered to detect all but the largest differences in treatment effect. This is not uncommon and attention has been drawn to this in medical research.35,37–39 Arguably, for most orthodontic trials, subtle differences in treatment effect will be expected and therefore a larger sample size will be required to detect these differences. This could mean that most orthodontic RCTs would have to be multi-centred in order to be able to recruit sufficient patients within a suitable time. This has obvious funding implications for any RCT and highlights the need for a great deal of planning and co-operation to run an orthodontic clinical trial successfully. Nevertheless, one such trial has addressed these problems and is underway in the UK (O’Brien, British Orthodontic Conference 1998, 2000).
Conclusions
We concluded the following:
Appendix 1: classification system for the subject of clinical trials published in the AJO-DO, BJO, and EJO
Acknowledgments
I would like to thank Professors Lennon and Ashby for their supervision of and comments on the work that has contributed to this paper. I would also like to express my appreciation to the referees of this paper and the Editor for their tremendous help in transforming previous drafts of this paper into this version.
References
Doll R. Controlled trials: the 1948 watershed. BMJ 1998; 317: 1217–1220.
Yoshioka A. Use of randomisation in the Medical Research Council’s clinical trial of streptomycin in pulmonary tuberculosis in the 1940s. BMJ 1998; 317: 1220–1223.
Medical Research Council (MRC). Streptomycin treatment of pulmonary tuberculosis. BMJ 1948; 2: 769–782.
Baumrind S, Korn EL, Boyd RL, Maxwell R. The decision to extract: Part I—interclinician agreement. Am J Orthod Dentofacial Orthop 1996; 109: 297–309.[CrossRef][Medline]
Baumrind S, Korn EL, Boyd RL, Maxwell R. The decision to extract: Part II—Analysis of clinicians’ stated reasons for extraction. Am J Orthod Dentofacial Orthop 1996; 109: 393–402.[CrossRef][Medline]
Tulloch JFC, Phillips C, Koch G, Proffit WR. The effects of early intervention on skeletal pattern in Class II malocclusion: A randomized clinical trial. Am J Orthod Dentofacial Orthop 1997; 111: 391–400.[CrossRef][Medline]
Tulloch JFC, Phillips C, Proffit WR. Benefit of early Class II treatment: Progress report of a two-phase randomised clinical trial. Am J Orthod Dentofacial Orthop 1998; 113: 62–72.[CrossRef][Medline]
Ghafari J, Shofer FS, Jacobsson-Hunt U, Markowitz DL, Laster LL. Headgear versus functional regulator in the early treatment of Class II, Division 1 malocclusion: a randomised clinical trial. Am J Orthod Dentofacial Orthop 1998; 113: 51–61.[CrossRef][Medline]
Keeling SD, Wheeler TT, King GJ, Garvan CW, Cohen DA, Cabassa S, et al. Antero-posterior skeletal and dental changes after early Class II treatment with bionators and headgear. Am J Orthod Dentofacial Orthop 1998; 113: 40–50.[CrossRef][Medline]
10 Cochrane Collaboration Oral Health Group. Oral Health Review Group Journal hand-searchers’ manual. Manchester: Cochrane Oral Health Group, 1999.
11 Dickersin K, Scherer R, Lefebvre C. Identifying relevant studies for systematic reviews. BMJ 1994; 309: 1286–1291.
12 Harrison JE, Ashby D, Lennon MA. An analysis of papers published in the British and European journals of orthodontics. Br J Orthod 1996; 23: 203–209.[Abstract]
13 Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics 1977; 33: 159–174.[CrossRef][Medline]
14 Feinstein AR. Clinical biostatistics XLIV. A survey of the research architecture used for publications in general medical journals. Clin Pharmacol Ther 1978; 24: 117–125.[Medline]
15 Fletcher RH, Fletcher SW. Clinical research in general medical journals. N Engl J Med 1979; 301: 180–183.[Abstract]
16 Haines SJ. Randomised clinical trials in the evaluation of surgical intervention. J Neurosurg 1979; 51: 5–11.[Medline]
17 Morris RW. A statistical study of papers in the Journal of Bone and Joint Surgery. J Bone Joint Surg Br 1984; 70: 242–246.
18 Tulloch JFC, Antczak-Bouckoms AA, Tuncay OC. A review of clinical research in orthodontics. Am J Orthod Dentofacial Orthop 1989; 95: 499–504.[CrossRef][Medline]
19 Roberts CT, Semb G, Shaw WC. Strategies for the advancement of surgical methods in cleft lip and palate. Cleft Palate Craniofac J 1991; 2: 141–149.
20 Pollock AV. Surgical evaluation at the cross-roads. Br J Surg 1993; 80: 964–966.[Medline]
21 Gluud C, Nickolova D. Quality assessment of reports on clinical trials in the Journal of Hepatology. J Hepatol 1998; 2: 321–327.
22 Tulloch JFC. Bias and variability in clinical research. Clin Orthod Res 1998; 1: 94–96.[Medline]
23 Johnson LE. Let them eat cake: the struggle between form and substance in orthodontic clinical investigation. Clin Orthod Res 1998; 1: 88–93.[Medline]
24 Vig KWL, Weyant R, Vayda D, O’Brien K, Bennett E. Orthodontic process and outcome: efficacy studies—Strategies for developing process and outcome measures: a new era in orthodontics. Clin Orthod Res 1998; 1: 147–155.[Medline]
25 Ellenberg SS. Studies to compare treatment regimens: the randomised clinical trial and alternative strategies. JAMA 1981; 246: 2481–2482.
26 Gross M. A critique of the methodologies used in clinical studies of hip-joint arthroplasty published in the English-language orthopedic literature. J Bone Joint Surg Am 1988; 70: 1364–1371.
27 O’Brien KD, Corkill CM. The specialist orthodontic practitioner. The 1989 survey. Br Dent J 1990; 168: 471–475.[Medline]
28 Fricker JP. A new self-curing resin-modified glass-ionomer cement for the direct bonding of orthodontic brackets in vivo. Am J Orthod Dentofacial Orthop 1998; 113: 384–386.[Medline]
29 Armas Galindo HR, Sadowsky PL, Vlachos C, Jacobson A, Wallace D. An in vivo comparison between a visible light-cured bonding system and a chemically cured bonding system. Am J Orthod Dentofacial Orthop 1998; 113: 271–275.[CrossRef][Medline]
30 Sadowsky PL, Retief DH, Cox PR, Hernandez-Orsini R, Rape WG, Bradley EL. Effects of etchant concentration and duration on the retention of orthodontic brackets: an in vivo study. Am J Orthod Dentofacial Orthop 1990; 98: 417–421.[Medline]
31 Illing HM, Morris DO, Lee RT. A prospective evaluation of Bass, Bionator and Twin Block appliances. Part I—the hard tissues. Eur J Orthod 1998; 20: 501–516.
32 Brightman LJ, Terezhalmy GT, Greenwell H, Jacobs M, Enlow DH. The effects of a 0.12 per cent chlorhexidine gluconate mouthrinse on orthodontic patients aged 11 through 17 with established gingivitis. Am J Orthod Dentofacial Orthop 1991; 100: 324–329.[Medline]
33 Trimpeneers LM, Wijgaerts IA, Grognard NA, Dermaut LR, Adriaens PA. Effect of electric toothbrushes versus manual toothbrushes on the removal of plaque and periodontal status during orthodontic treatment. Am J Orthod Dentofacial Orthop 1997; 111: 492–497.[CrossRef][Medline]
34 Denes J, Gabris K. Results of a 3-year oral hygiene programme, including amine fluoride products, in patients treated with fixed orthodontic appliances. Eur J Orthod 1991; 13: 129–133.
35 Freiman JA, Chalmers TC, Smith H Jr, Kuebler RR. The importance of beta, type II error and sample size in the design and interpretation of the randomized controlled trial. Survey of 71 ‘negative’ trials. N Engl J Med 1878; 299: 690–694.
36 Altman DG. How large a sample?. BMJ 1980; 281: 1336–1338.
37 Altman DG. Practical Statistics for Medical Research. London: Chapman & Hall, 1991. pp. 455–460
38 Moher D, Dulberg CS, Wells GA. Statistical power, sample size and their reporting in randomised controlled trials. JAMA 1994; 272: 122–124.
39 Dickinson K, Bunn F, Wentz R, Edwards P, Roberts I. Size and quality of randomised controlled trials in head injury: review of published studies. BMJ 2000; 320: 1308–1311.
Received July 30, 2001; accepted July 24, 2002
This article has been cited by other articles:
|
|
 |
|
  L. P. Y. Hichens, J. R. Sandy, H. N. Rowland, A. G. McNair, S. Clark, D. Hills, P. Huntley, S. Ransome, M. Forty, J. Peak, et al. Practical aspects to undertaking research in the primary care setting: experience from two studies J. Orthod., December 1, 2005; 32(4): 262 - 268. [Abstract] [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
