HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS		QUICK SEARCH:   [advanced] Author: Keyword(s): Year:  Vol:  Page:

This Article Abstract Full Text (PDF) Alert me when this article is cited Alert me if a correction is posted Citation Map Services Email this article to a friend Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Download to citation manager Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Google Scholar Articles by Travess, H. C. Articles by Williams, A. C. Search for Related Content PubMed PubMed Citation Articles by Travess, H. C. Articles by Williams, A. C.

The development of a patient-centered measure of the process and outcome of combined orthodontic and orthognathic treatment

H. C. Travess^*, J. T. Newton, J. R. Sandy and A. C. Williams

^* John Radcliffe Hospital, Oxford,
GKT Dental Institute, London, and
Bristol Dental Hospital, UK

Address for correspondence: Dr A. Williams, Child Dental Health, University of Bristol Dental School, Lower Maudlin Street, Bristol, BS1 2LY, UK. Email: a.c.williams{at}bristol.ac.uk

Received October 24, 2003; accepted January 28, 2004

	Abstract

Top Abstract Introduction Materials and method Results Discussion Conclusions Contributors References

 
Objective: The aim of this study was to develop a patient-based measure of the process and outcome of combined orthodontic and orthognathic care in the National Health Service in the UK.

Design: Identification of relevant dimensions through qualitative methods, design of form, determination of psychometric properties of the scale, specific readability, reliability and validity.

Setting: NHS hospitals in the South West Region.

Subjects: The sample comprised patients who had received combined orthodontic and orthognathic treatment between 01 January1998 and 31 December 2000. Twenty-six participants (a 25% response rate) took part in four focus group meetings. Thirty subjects (65% response rate) took part in a pilot study to test the properties of the questionnaire.

Main outcome measures: Six broad themes emerged from the focus groups. These formed the basis of the sections in the questionnaire.

Results: The questionnaire developed had a Flesch reading ease score of 72.9 or US grade level 4 equivalent to aged 9–10 years. Test–retest reliability gave kappa values for most questions that exceeded 0.4. Criterion validity of the measure was established by comparing responses to the questionnaire over two periods with a telephone interview on a sample of 30 patients. Criterion related validity was poor for nine of the 16 items. By contrast the construct validity of the questionnaire was satisfactory.

Conclusion: A patient-based measure of the process and outcome of combined orthodontic and orthognathic treatment has been developed. This has sufficient validity and reliability for use in inter-center audit projects.

Key words: Orthodontics, orthognathic surgery, patient-based outcomes, qualitative research

	Introduction

Top Abstract Introduction Materials and method Results Discussion Conclusions Contributors References

 
The introduction of Clinical Governance has led to an increasing emphasis on assessing the quality and effectiveness of patient care under the National Health Service (NHS). The Government has also directed that the NHS should be responsive to the needs of consumers and that patients should be involved in shaping the future delivery of care.¹ These issues have led to a need to develop measures that reflect, not only the clinical results of treatment, but also the quality of patients’ experiences of treatment.

Previous measures of patient satisfaction have been based on tools developed by clinicians. It cannot be assumed, however, that patients’ perceptions of quality of care are similar to those of clinicians. For example, Burke & Croucher,² in a survey of patients in general dental practice, showed that there was a mismatch between dentists and patients in the criteria considered to be important in treatment delivery. Furthermore, patients are more likely to respond to questionnaires that examine subjects of interest to them.³ Clearly, then, it is important to develop any health care measures with patients’ views, rather than just those of clinicians.

Over the last decade, qualitative methods have been used more frequently in areas such as health services research and health technology.^4,5 Qualitative data are typically gathered from in-depth interviews and focus group work. The main advantage of this approach is that it allows complex issues to be probed and answers to be clarified in a more relaxed atmosphere with more structured methods.⁶ Focus groups are a form of group interview that capitalize on communication between research participants in order to obtain data. The method is particularly useful for exploring people’s knowledge and experiences of a procedure, mainly because it facilitates the expression of ideas and experiences that might be left under-developed in an interview.⁷

As orthodontic treatment becomes more widely available in the United Kingdom there has been an increase in the demand for orthognathic surgery for these cases not treatable by orthodontics alone. Most of this treatment is undertaken within the NHS by hospital orthodontists, and oral and maxillofacial surgeons working together. Little is known about the standards of care being achieved, in part because there are no agreed measures for auditing the process or outcome of care.

Data on patients’ opinions of the outcomes achieved with combined orthodontic and orthognathic surgical treatment have been sought, but with questionnaires developed by clinicians.^8–14 The little that is known about patients’ evaluation of orthodontic–orthognathic treatment in this area has focused on patients’ motivations for treatment.^15–17 To our knowledge their opinions about the process of delivery of care have not been previously evaluated.

This article describes the methods used in developing a patient-based measure for auditing the quality of care received by patients undergoing combined orthodontic–orthognathic treatment under the NHS in the SouthWest Region. Since little is known about the experience of patients undergoing orthognathic surgery within the NHS, qualitative research methods were used at the beginning of this study to identify the range of possible issues of concern.

	Materials and method

Top Abstract Introduction Materials and method Results Discussion Conclusions Contributors References

 
The development of the questionnaire took place in three phases. In the first, qualitative phase, items to form the questionnaire were generated from focus group discussions. The second phase comprised the design of the questionnaire format and pilot testing for acceptability. In the final phase, quantitative research methods were used to test the validity or reliability of the measure.

Phase one: focus group study
As part of a regional study of the delivery of orthognathic surgical treatment, patients who had received combined orthodontic and orthognathic surgical treatment during the period 01 January 1995 and 31 December 2000 at eight hospitals in the South West NHS region in the United Kingdom (UK) were identified from clinic lists, local databases and the relevant surgical operating books. The following were excluded:

• syndromic patients;
  
• cleft lip and palate patients;
  
• distraction osteogenesis cases;
  
• patients who had undergone genioplasty or another procedure not involving full movement of the mandible or maxilla
  

A series of focus group meetings were held in the following locations: Bristol, Bath, Taunton and Plymouth. Each meeting was held in a non-clinical environment outside the hospital. The chosen locations were easily accessible by private and public transport.

Each meeting was held on a week-day evening to allow ease of access and improve attendance. Potential participants for the focus group meetings were initially chosen from the list of patients who had undergone surgery within the past 2 years (1 January 1999 to 30 December 2000) and who lived within easy traveling distance of the location of the focus group meetings using the technique of purposive sampling.⁴ This selects participants because they are likely to represent the whole range of possible views on a topic, rather than because they are representatives of the study population. Subjects were chosen to reflect the range of different surgical procedures that are used in orthognathic surgery; a typical gender distribution for orthognathic patients (two female: one male) and the total age range of our sample at operation (age 15–55 years). There was a poor response rate to the initial request for volunteers to take part in the study and, therefore, the sample frame for the focus group study was extended to also include patients who were operated on during the period (1 January 1998 to 30 December 1998). Potential participants were contacted by letter and invited to take part in a focus group to explore their views of combined orthodontic orthognathic treatment. A study information sheet and consent form were included with the letter of invitation. Written consent was obtained from each participant. Patients were invited to attend the meeting closest to their home and travel expenses were reimbursed by the research team.

An independent facilitator, who was unaware of the issues surrounding orthodontic–orthognathic care, convened each meeting. A variety of questioning styles were used as recommended by Britten.¹⁸ The discussions were allowed to be flexible according to the experience of the participants. Each meeting was tape-recorded, with the participants’ consent and an observer took written field notes of the proceedings. Following each meeting, the tapes were transcribed into Microsoft Word^© documents with the names of participants changed to preserve anonymity. The transcripts were then analysed as described below and themes surrounding the delivery of combined orthodontic–orthognathic surgery were identified. Following ‘grounded theory’,¹⁹ whereby themes are allowed to emerge from the data, further focus group meetings were held until no new themes emerged from the data.

Analysis. Each transcript was then analysed independently by hand, the two researchers involved also worked independently. The themes surrounding the process and outcome of care that emerged from the data were identified using the technique of constant comparison.¹⁹ Codes were assigned to each unit of speech—defined as a continuous period of speech by an individual person—which ended when a different person spoke. Each code represented a theme. The first speech unit was read and a code, or codes, created to reflect the content of the speech unit. The next speech unit was then compared with the first unit to see if it fitted the first code or required a new code to be created. This process was continued throughout each transcript. Analysis of themes occurring in the group interviews proceeded in parallel with meetings, focus groups were arranged until no new topics arose, as advised by Morgan.⁷

The researchers then compared identified themes and a common agreed set of themes was created. For each theme, the meanings were defined and common-code descriptions assigned. The researchers independently rated the frequency of occurrence of each theme for all the focus group transcripts using QSR NUD•IST^© software. Frequency tables of the occurrence of each theme were created. The inter-examiner agreement in the rank order of occurrence of themes was calculated using Spearman’s rank correlation coefficient.

Phase two: development of the questionnaire
The key themes identified from the focus groups were used to form the basis of a questionnaire to audit patient perception of the delivery and outcome of combined orthodontic–orthognathic surgical treatment. Questions were devised within each category to be included in the questionnaire. The number of items in each section mirrored the importance that each section was given in the focus groups by the participants. To improve response rates,²⁰ a bold yellow colour and logo was used for the front cover of the questionnaire (Figure 1). The first version of the questionnaire was piloted on a sample of 15 patients attending a joint maxillofacial and orthodontic clinic at a hospital within the South West Region who had recently completed combined orthodontic–orthognathic treatment. Subjects were invited to complete the questionnaire with the researcher present and to assess each question for ease of understanding. The time taken by each participant to complete the questionnaire was recorded. The researcher made note of any questions or sub-sections that participants faltered on, left out or failed to answer. The Flesch Reading Ease score and Flesch–Kincaid grade of the questionnaire was tested using Microsoft Word^© software. The Flesch reading ease score was targeted at 60.0 or above and the Flesch–Kincaid grade level at grade seven or below (equivalent to a reading age of 12–13 years).

View larger version (27K): [in this window] [in a new window]   Figure 1 Front cover design of the questionnaire

Phase three: testing the validity or reliability of the questionnaire
Forty-six patients, comprising the subjects who took part in the focus group meetings together with 20 patients who showed an interest in participating in the study, but were unable to attend a focus group meeting were invited to take part in the pilot study to test the questionnaire. All participants were at least 3 months post-surgery at the time of the study. The maximum time since surgery for this sample was 3 years 9 months.

Patients identified for phase three of the study were contacted by post with a study information sheet explaining the nature of the research. Each respondent was asked to complete a consent form allowing the researchers to access their medical records in addition to consenting to the study. Subjects who indicated that they wished to participate were sent a questionnaire (T1) together with a postage-paid reply envelope for its return. Subjects who failed to return the questionnaire were sent a reminder letter, together with another questionnaire. This was followed up with a second reminder letter.

After a period of at least 3 weeks after the return of the first questionnaire (T1), each respondent was sent a second copy of the questionnaire (T2) to complete at home. A reminder letter and questionnaire were sent to respondents who failed to return the second questionnaire (T2) as described previously. The responses to the questionnaires T1 and T2 were coded and entered into SPSS^© for analysis. The reliability of the response to each question was calculated using kappa and weighted kappa values were calculated as appropriate for each question. To give an overall assessment of reliability, the mean kappa score for each section of the questionnaire was calculated.

The face and content validity of the questionnaire were assumed to be good since the questions reflected the range of experiences and concerns of individuals who had undergone combined orthodontic–orthognathic treatment as identified from the focus group work. To assess criterion-related validity, all patients who had completed at least one questionnaire were contacted by a telephone (researcher HT) on a weekday evening.

The researcher interviewed each participant using a structured interview schedule (Figure 2). If the question required a scale answer then this was described in full before the participant answered. For other questions, a free answer response was recorded. This was then followed by the researcher giving the interviewee a list of other possible responses, which had been presented as tick boxes in the original questionnaire. The data from the telephone survey were coded and entered into a database in SPSS^© for analysis. The percentage agreement between the responses given in the questionnaire (T1) and those in the telephone interview were calculated.

View larger version (27K): [in this window] [in a new window]   Figure 2 Telephone interview sheet used in criterion validity study

View larger version (28K): [in this window] [in a new window]   Figure 3 Data collection sheet used in the construct validity study

	Results

Top Abstract Introduction Materials and method Results Discussion Conclusions Contributors References

Phase one: focus group study
A total of four focus group meetings were held. Table 1 shows the number of subjects invited to the focus group meetings and the number that attended within each area. Six of the participants were male and 20 were female, the age range was 18–50 years (median 25 years). Ten participants (37%) had bi-maxillary surgery; 10 underwent a mandibular procedure and the remaining six subjects had maxillary surgery.

• reasons for surgery;
  
• orthodontic treatment;
  
• surgery—in patient issues;
  
• post-operation problems at home;
  
• information that helped;
  
• benefits of treatment.
  

Each theme was then divided into sub-themes that further characterized the broad theme (Figure 4). Table 2 shows the results of the inter-rater reliability study for the frequency of occurrence of sub-themes within each broad theme. The level of agreement between the two raters was high for all themes except ‘orthodontic treatment’ and ‘post-operative recovery period’. Discussion between the two raters revealed differences in opinion about whether post-operative orthodontics should be included in this category. This was reconciled by coding all statements, which referred to orthodontic treatment under the ‘orthodontic treatment’ theme regardless of when they occurred in the treatment process. The two raters also differed slightly in their interpretation of the post-operative recovery period. It was agreed that the ‘post-operative recovery’ theme should be limited to issues surrounding the delivery of care, which occurred after the patient had been discharged from hospital following their surgery.

View larger version (26K): [in this window] [in a new window]   Figure 4 Themes and sub-themes identified from the focus groups using the constant comparison technique19

That was why I did it, the only reason I did it. Because I have never ever liked the way I looked. (Participant in focus group 2; coded 1.1—Reasons for surgery/facial appearance)

Another common reason was because they were experiencing difficulties eating:

I always choose, or I always chose things that were easy to eat. I would never, ever have a piece of meat, like a steak, which involved a lot of chewing. (Participant in focus group 2; coded 1.3—Reasons for surgery/difficulty eating)

Orthodontic treatment. Although the experience of undergoing orthodontic treatment was, in general, seen as less difficult than the surgical treatment, several participants reported both pain and inconvenience from the appliance:

I had wires gouging into me for two years, you know and so your life revolves around little bits of wax and the ulcers and the smell of the ulcers and you know, the inside of my mouth was just cut to pieces. (Participant in focus group 1; coded 2.2—Orthodontic treatment/pain; 2.3—Orthodontic treatment/ulcers; 2.14—Orthodontic treatment/wax)

Wearing retainers at the end of active treatment was more of an issue to some participants than the fixed appliances:

Because the retainer is actually in your mouth, so to speak and to eat, I found it much more difficult with that in my mouth than I did with the actual sort of braces on the outside. (Participant in focus group 3; coded 2.8—Orthodontic treatment/trouble with retainers; 2.23—Orthodontic treatment/eating problems; 2.24—Orthodontic treatment/speech problems)

In-patient issues. The experience of in-patient treatment and surgery was an important issue to participants; indeed, it was often the first matter that participants wished to discuss. The participants recalled this as a time of marked physical ill health and discomfort. This was often exacerbated by the participant having unexpected symptoms after the operation. In general, patients felt better when they could anticipate negative events such as pain, numbness and swelling. A typical example was a comment from a participant who experienced post-operative difficulties breathing. The participant found this distressing because they had not been warned that this might happen:

It was quite frightening at times, when you couldn’t breathe and you couldn’t get enough air through your mouth. I hadn’t realized that my nose would be quite so affected. (Participant focus group 2; coded 3.3—In patient issues/could not breather post-op)

Other problems focused on the lack of specialist nursing care available on the ward:

There was no help with eating. I couldn’t eat a thing and there was no special diet or anything ... I didn’t eat anything for a week. (Participant in focus group 1; coded 3.32 In-patient issues/problems with eating) Yes, it was my son that said to the nurse ‘how’s my mum going to take that tablet, surely you’ve got one that’s dissolvable’ and they looked at me and looked at the tablet and said ‘oh yeah, I suppose you could really couldn’t you’, so it was, yes, what you’re saying is right because afterwards you can’t say very much. (Participant from focus group 2; coded 3.5—In patient issues/not given liquid pain killers, probs with pain relief; 3.8—In patient issues/lack of knowledge and attention from ward staff; 3.8—In patient issues/unable to talk post-op)

Problems at home following the operation. The short-term inconveniences at home following the operation included ongoing pain and swelling. Tiredness was a common complaint. Several participants felt that it took them longer to recover from the operation than they had anticipated:

The orthodontist said about 4 weeks, but there was no way I could have managed to cope with my job after 4 weeks. (Participant in focus group 2; coded 4.1—Post-op problems/recovery period longer than expected; 4.11—Post-op problems/swelling)

Some participants experienced a delayed reaction to their surgery:

Yeah, I had a bit of an identity crisis actually about six weeks afterwards oddly enough, and I was really thrilled with the result and I couldn’t tell you to this day why I was upset, but I looked in the mirror and I was just so upset because it wasn’t me. (Participant from focus group 3; coded 4.7—Post-op recovery/change in appearance; 3.16—In patient issues/frightened looking at yourself after operation)

Information that helped the participants. Participants were asked how the delivery of care could have been improved. Many participants would have wished to meet with a patient who had undergone a similar procedure prior to making their decision to undertake the treatment:

I think I actually said to [Name of surgeon] ‘is there anyone I can talk to who has had it before? Even like on the phone, just to get an idea’, and he said ‘no not really’. (Participant in focus group 1; coded 5.5—Information/wanted to meet someone who had it done; 5.2—Information/knew someone who had it done)

Benefits of treatment. Participants in the present study reported a range of benefits from undergoing combined orthodontic–orthognathic surgical treatment. These included increased self-esteem, confidence, attractiveness and functional aspects:

I smile a lot more now and my posture is a lot different and my eyes are different and it’s really quite strange actually, I didn’t think it was going to be quite such a change. (Participant in focus group 3; coded 6.1—Benefits of treatment/look better overall; 6.4—Benefits of treatment/comfortable smiling, straight teeth)

I’d go through it again tomorrow if I had to. It has made such a huge difference in my life that I would go through it all over again. My appearance has changed so much ... I haven’t made a conscious effort to change the way I am, but other people have noticed I’m totally different. (Participant from focus group 3; coded 6.1—Benefits of treatment/look better overall; 6.2—benefits of treatment/confidence; 6.11—benefits of treatment/would have it done again)

Not all participants, however, felt that they had benefited from treatment:

I felt like they had lied about that, because although they told me about the numbness just before the operation they did say that 99% of people do get feeling back so don’t worry about it, but then I don’t know a few months ago they asked how much feeling I have got back and I said well my jaw is still frozen and they said ‘Oh yes, people don’t get the feeling back in their jaw’ and it’s like well that isn’t quite the same as saying 99% will get the feeling back. And that is a major thing. You know, eating if you can’t feel it. (Participant in focus group 3; coded 6.13—Benefits of treatment/post-op problems; 3.31—Surgery in-patient issues/unexpected issues for consent; 4.2—Post-op problems/time unable to eat; 4.3—Post-op problems/time before numbness wears off)

Phase two: development of the questionnaire
Twenty-five subjects took part in the pre-pilot studies to test the acceptability of the questionnaire. The time taken by the subjects included in the pre-pilot studies to complete the questionnaire ranged between 10 and 40 minutes (median 15 minutes). The ease of administration was found to be good, although some subsection questions were omitted during the pre-pilot. The questionnaire layout was therefore adjusted before the pilot study to reduce this. The overall Flesch—Kincaid grade level of the questionnaire used in the pilot study was 4.7 (equivalent to the reading ability of a 9–10 year old). The introductory pages were graded at 7.0 (12–13 year old reading ability). The Flesch Reading Ease scores for the questionnaire and the introductory pages were 72.9 and 70, respectively. This suggests that 60–70% of the population would be able to understand the document.

Phase three: testing the reliability and validity of the questionnaire
The study to test–retest the reliability of the questionnaire. Thirty participants (65% response rate) completed the questionnaire at times T1 and T2 to test–retest the reliability of the questionnaire. Six participants were male and 24 were female. The age range was from 18 to 57 years. Fourteen participants had undergone single jaw surgery, whilst 16 had a bi-maxillary procedure. The median and range of kappa scores for questions in each section of the questionnaire are shown in Table 3. The kappa scores show good to excellent agreement²¹ for all sections of the questionnaire except section J. This section was about information given to patients before and during treatment.

	Discussion

Top Abstract Introduction Materials and method Results Discussion Conclusions Contributors References

During the qualitative phase of the study, participants identified several motivating forces for undergoing treatment. These included functional, social and psychological reasons. A similar range of motivations has been identified by numerous researchers both in the UK and the USA, using a variety of methodological approaches.^{9,10,14,17,22–25} These observations provide some degree of validation to the use of focus groups to ascertain patients’ perceptions of treatment. The advantage of using qualitative methods was demonstrated in both the range of issues discussed and the ability of participants to identify apparently contradictory positions.²⁶ For example, participants identified a great many benefits of treatment, but also identified some post-operative problems that the researchers had not appreciated.

The focus group discussions in the present study identified a number of issues to patients surrounding the delivery of orthognathic treatment that are not commonly described in the literature. For example, some of the participants in the qualitative study recounted that it felt difficult to breath in the period immediately after maxillary impaction and that this was a frightening feeling. This may have been a consequence of reduction in the antral space, or reduced sensation in the linings of the nasal passages and sinuses. Patients are routinely counseled about the possibility of lingual paraesthesia after mandibular osteotomy. Few are told, however, about the changes in sensory perception that may occur following a maxillary impaction, possibly because clinicians are unaware of the potential impact that these may have on the patient.

Several previous studies have examined patients’ reasons for seeking combined orthodontic–orthognathic treatment but, most of these studies have been based on clinician-based questionnaires. Very few studies that would be comparable to the present study have used qualitative techniques to probe patients’ reasons for surgery. Ronis et al.¹⁵ used telephone interviews to develop a questionnaire looking at the benefits and risks of treatment. They did not, however, examine the process of care in any great detail, unlike the present study. The lack of a clear description of the methods of analysis in the latter study also makes it difficult to reach clear conclusions. Broder et al.¹⁷ used qualitative methods to compare the motivations of patients who chose to have combined treatment with those of patients who opted for orthodontic treatment alone. This is in contrast to the present study, which focuses on patients who have completed combined orthodontic and orthognathic treatment. Furthermore, Broder et al.¹⁷ interviewed patients who were yet to have treatment, in contrast to the present study, which is entirely retrospective.

We were interested in patients’ experiences of in-patient care after orthognathic surgery, an aspect where there has been little previous research. Participants reported being distressed by the side effects of surgery, which they had not been warned about by clinicians. For example, patients can have difficulty breathing through their noses following a Le Fort I procedure. This finding is in agreement with other research,²⁷ which has shown that patients who are informed of likely problems and given information about coping with them, report less distress. Several participants in our study also reported being surprised at how long it took them to recover from their operation. It is difficult to put these findings into context with little previous research addressing this important area of the process of treatment and recovery. The participants expressed a wide range of views concerning information that they wished they had received before starting treatment. This reflects the complexity of patients’ information requirements and the need to tailor information to meet individual needs.

Perceived benefits of undergoing orthognathic surgery
The benefits of treatment identified by participants in the present study were similar to those identified in previous studies.^14,15,28 Ronis et al.¹⁵ suggested that the patient’s satisfaction with treatment was related to their motivations for and expectations of the treatment.

The present study suggests that patients’ perceptions of treatment are qualitatively different from those of the clinician. This is illustrated in the participants’ discussions in relation to the orthognathic surgery. In most instances, the surgery was seen as the main focus of their attention:

I think the worst thing for me was having to decide so far in advance that I was going to go ahead with it because you knew that once the braces were there that was it and if you were looking ahead two years you were thinking about this op for two years. I think that was the worst bit for me really, the waiting for it. (Participant from focus group 4)

Several participants alluded to the protracted nature of their treatment and the frustrations that this caused. The focus of their attention was always on the surgery before the event and on completing treatment after the surgery. The clinician’s knowledge of the process of care is generally far greater than that of the majority of patients and they approach this from an entirely different angle. The delivery of combined orthodontic–orthognathic surgical treatment can be divided into five distinct phases, each with the own clinical goals. These are:

• pre-surgical orthodontic preparation;
  
• surgery;
  
• post-surgical management;
  
• post-surgical orthodontic finishing;
  
• retention.
  

Clinical goals required before surgery and before debond necessitate the stages of treatment in the order they are performed. The patients, however, seem to perceive the treatment as a whole.

Limitations of the study
The results of the study must be interpreted in the light of its limitations. As is common for all sampling methods in qualitative research, a degree of bias is suspected in the selection of patients attending the focus groups. Bias intervenes in the type of patient who is willing to devote time to discussing their treatment. This tends to be those who are polarized in their views—those who are extremely happy with their treatment and wish to ‘give something back’, and those who are very unhappy with their treatment and feel an opportunity to discuss their concerns is available. It is important to appreciate that the aim of qualitative research is to identify as wide a range as possible of the views that exist within the population under study. Indeed, subjects were selected for the present study because they had different demographic characteristics and had undergone a range of surgical procedures, and were therefore likely to represent a wide variety of treatment experiences. Once the range of views and issues have been identified from qualitative work these can then be used to form the basis of a survey of a larger representative population.

Interviewer bias is another potential problem when undertaking qualitative research. In the present study, this was minimized by using a facilitator who was not a clinician and not involved in the treatment of patients undergoing combined orthodontic–orthognathic treatment. The facilitator was also not involved in the data analysis. To further reduce bias, the transcripts of each focus group meeting were analysed by two independent researchers, one of whom had not attended the focus group meetings and was therefore unfamiliar with the discussions that took place. There was a very good correlation between raters for the rank order of occurrence of sub-themes within each broad theme grouping. There were, however, some differences in interpretation, and these were clarified before the final analysis of the frequencies. Although there may not have been 100% agreement in the coding of speech units, we can be confident that the 2 raters identified similar themes as being important to participants.

As with all orthodontic treatment,²⁹ there was a preponderance of females in all the focus groups meetings. Qualitative researchers tend to place less emphasis on the representativeness of the sample, placing more importance on the range of views expressed by participants. The fact that the range of views expressed by participants is both broad and inclusive of findings from research using different methods suggests that the goal of identifying a comprehensive range of patient views has been achieved.

Another disadvantage of the methods used in the present study was the time that had elapsed, since some participants had undergone surgery, which was as long as 3 years in some cases. This illustrates well the difficulties faced by qualitative researchers in recruiting subjects to take part in this type of study because of the amount of time and effort involved in participating. Although the focus group meetings in this study were arranged in the evenings for the convenience of participants and several attempts were made to contact non-respondents, the response rate to the focus group study was low. This meant that the inclusion criteria for the study had to be extended to include subjects who had been operated on several years previously, which may have affected their recall of treatment events. Since the aim of this study was to identify the range of issues to patients, rather than the depth of their feelings about these issues, recall bias is unlikely to have a significant impact on the qualitative part of this study. It is possible, however, in the present study that recall bias could have affected the testing of the criterion validity of the questionnaire, particularly in the recall of the number of emergency appointments. The effect and direction of such bias is difficult to assess in the small sample that was included in the pilot study. It would be important, however, in larger population studies to consider the time that has elapsed since surgery when assessing patient perception of the quality of treatment delivery.

The low response rate in the present study also illustrates the difficulties involved in undertaking studies of patient perception of a relatively low volume procedure, such as orthognathic surgery, in an area such as the South West Region where hospitals are widely dispersed. Ideally, focus groups should have been arranged in every major town where orthognathic surgery is undertaken within the Region to enable subjects from every hospital unit to participate in the study thus increasing the sample size. The cost and time required to run meetings at 13 different centers would have been prohibitive, however, and since no new issues emerged after the third meeting, the research team felt that a sufficient range of data had been collected for the purposes of this study.

Properties of the questionnaire
The test–retest reliability of the questionnaire was shown to be good for the measure as a whole. Some individual items showed lower levels of inter-test–retest reliability; however, these items tend not to vary across participants, but it is still important to measure them since they were identified as relevant aspects of care in the focus groups. There were 3 items in section B in the questionnaire, where agreement was low. Although it was decided that these items should be included in the final questionnaire responses to these questions in inter-center studies should be interpreted with caution. The questionnaire has also been shown to be a reliable measure for participants who completed treatment at least 6 months previously. This would suggest that the instrument is suitable for use as a measure of patient-centered outcomes at a point distant from the delivery of care.

Internal validity was tested in two ways in this study, namely through telephone interviews and data collection from the medical records. Our results show only fair levels of agreement between the questionnaire and the criterion of the telephone interview, but with a large range (between 0 and 1.0). There are two possible interpretations of these findings. The first explanation could be that the measure is genuinely not valid. The construct validity, however, showed moderate agreement between many items in the questionnaire and data from medical records. One would expect criterion validity to be agreed. The second explanation is that there could be methodological differences between telephone interviews and written questionnaires that may result in a low agreement. It is known that telephone interviews have higher rates of non-response, result in less complete information and more ‘don’t know’ responses.⁴ An alternative method of testing validity, would have been to assess the questionnaire against a similar measure taken form the published literature. This was not possible in the present study because there was no comparable measure available.

Construct validity was assessed by comparing data from the medical records with responses to the questionnaire. There was moderate agreement between most items suggesting good construct validity. One item showed poor agreement and should be interpreted with caution. This item (G6), asked about the number of emergency appointments attended by respondents throughout treatment. There are two explanations for this. Either emergency appointments are not always recorded in the notes or, alternatively, this could be due to recall bias.

Difficulties auditing patient perception of the quality of orthognathic treatment. The present study used a retrospective method to assess all aspects of the treatment process and identified a pool of patients who had completed treatment within a 5-year time frame. Cunningham et al.³⁰ surveyed a similar target population with a clinician-devised questionnaire but extended the time scale to include patients up to 16 years post-surgery. Unfortunately, the process and outcome of any treatment cannot be assessed until after treatment is complete. This is particularly difficult when assessing the process and outcome of combined orthodontic and orthognathic treatment, since this can take up to 3 years or more to complete if the retention phase is taken into account. If the present measure is to be used routinely in the clinical situation for audit purposes, then it would be sensible to standardize the point in treatment when patients’ views are surveyed. It is worth noting, however, that all the subjects in the present study were able to talk about reasons and benefits of having treatment in great detail despite it being some time since their treatment was completed.

	Conclusions

Top Abstract Introduction Materials and method Results Discussion Conclusions Contributors References

 
A range of previously unrecognized issues and concerns to patients undergoing combined orthodontic–orthognathic treatment were identified in this study using qualitative research methods. These issues have been incorporated into a patient-based measure suitable for auditing patient perception of the process and outcome of combined orthodontic orthognathic care within the NHS. The measure has undergone rigorous testing, and has been shown to be acceptable to patients and have sufficient reliability and validity for use in inter-center audit studies of patient perceptions of the delivery of orthognathic treatment in the UK based on larger, representative populations.

	Contributors

Top Abstract Introduction Materials and method Results Discussion Conclusions Contributors References

 
Helen Travess was responsible for recruitment of the participants, data collection, data analysis, designing the questionnaire and drafting the article. Tim Newton was responsible for facilitating the focus groups, giving expert advice and statistical support for the data analysis, drafting, critical revision and final approval of the article. Alison Williams was responsible for study design, obtaining funding, logistic, administrative, and technical support and data interpretation, drafting, critical revision and final approval of the article. Jonathan Sandy was responsible for providing logisitic, administrative and technical support, and drafting, critical revision and final approval of the article. Alison Williams is the guarantor.

	References

Top Abstract Introduction Materials and method Results Discussion Conclusions Contributors References

 
1 Department of Health. The New NHS—Modern, Dependable. London: HMSO, 1997.

2 Burke L, Croucher R. Criteria of good dental practice generated by dental practitioners and patients. Int Dent J 1996; 46: 3–9.[Medline]

3 Edwards P, Roberts I, Clarke M, et al. Increasing response rates to postal questionnaires: systematic review. Br Med J 2000; 324: 1183–92.

4 Bowling A. Research Methods in Health. Investigating Health and Health Services. Buckingham: Open University Press, 1997.

5 Mays N, Pope C. Assessing quality in quantitative research. Br Med J 2000; 320: 50–2.[Free Full Text]

6 Kitzinger J. Introducing focus groups. Br Med J 1995; 311: 299–302.[Free Full Text]

7 Morgan DL. Successful Focus Groups: advancing the state of the art. Newbury: Sage Park, 1993.

8 Flanary CM, Barnwell GM, Alexander JM. Patient perceptions of orthognathic surgery. Am J Orthod Dentofac Orthop 1985; 88: 137–45.

9 Kiyak HA, Hohl T, Sherrick P, West Ra, McNeill RW, Bucher F. Sex differences in motives for and outcomes of orthognathic surgery. J Oral Surg 1981; 39: 754–64.[Medline]

10 Kiyak HA, McNeill RW, West RA, Hohl T, Bucher F, Sherrick P. Predicting psychologic responses to orthognathic surgery. J Oral Maxillofac Surg 1982; 40: 150–5.[Medline]

11 Kiyak HA, Hohl T, West RA, McNeill RW. Psychologic changes in orthognathic surgery patients: a 24 month follow up. J Oral Maxillofac Surg 1984; 42: 506–12.[Medline]

12 Kiyak HA, McNeill RW, West RA. The emotional impact of orthognathic surgery and conventional orthodontics. Am J Orthod 1985; 88: 224–34.[Medline]

13 Kiyak HA, Vitaliano PP, Crinean J. Patients’ expectations as predictors of orthognathic surgery outcomes. Hlth Psychol 1988; 7: 251–68.

14 Ostler S, Kiyak HA. Treatment expectations versus outcomes among orthognathic surgery patients, Int J Adult Orthodont Orthognath Surg 1991; 6: 247–55.

15 Ronis DL, Callan MA, Vig KWL, Vig PS, McNamara JA. Developing measures of patient perceptions of orthognathic surgery. Int J Adult Orthodont Orthognath Surg 1994; 9: 87–94.

16 Olson RE, Laskin DM. Expectations of patients from orthognathic surgery. J Oral Surg 1980; 38: 283–5.[Medline]

17 Broder H, Phillips C, Kaminetzky S. Issues in decision making: should I have Orthognathic Surgery? Semin Orthod 2000; 6: 249–58.

18 Britten N. Qualitative interviews in medical research. Br Med J 1995; 22: 251–3.

19 Glaser BG, Strauss AL. The Discovery of Grounded Theory: strategies for qualitative research. New York: Aldine publishing Co., 1967.

20 Dillman DA. Mail and Telephone Surveys: the total design method. New York: Wiley, 1978.

21 Altman DG. Practical Statistics for Medical Research. London: Chapman and Hall, 1991.

22 Kiyak HA, West RA, Hohl T, McNeill RW. The psychological impact of orthognathic surgery: a 9 month follow up. Am J Orthod 1982; 81: 404–12.[Medline]

23 Phillips C, Broder HL, Bennett ME. Dentofacial disharmony: motivations for seeking treatment. Int J Adult Orthodont Orthognath Surg 1997; 12: 7–15.

24 Forssell H, Finne K, Forssell K, Panula K, Blinnikka L-M. Expectations and perceptions regarding treatment: a prospective study of patients undergoing orthognathic surgery. Int J Adult Orthodont Orthognath Surg 1998; 13: 107–13.

25 Nurminen L, Pietila T, Vinkka-Puhakka H. Motivation for and satisfaction with orthodontic surgical treatment: a retrospective study of 28 patients. Eur J Orthod 1999; 21: 79–87.[Abstract/Free Full Text]

26 Gibson BJ. The place of qualitative research in Public Health Dentistry. J Publ Hlth Dent 2002; 62: 3–4.

27 Rosengarten M. The behaviour of the pre-school child at the initial dental visit. J Dent Res 1961; 40: 673.

28 Cunningham S, Hunt NP, Feinmann C. Perceptions of outcome following orthognathic surgery. J Oral Maxillofac Surg 1996; 34: 210–13.

29 Khan RS, Horrocks EN. A study of adult orthodontic patients and their treatment. Br J Orthod 1991; 18: 183–94.[Abstract]

30 Cunningham S, Crean SJ, Hunt NP, Harris M. Preparation, perceptions and problems: a long term follow up study of orthognathic surgery. Int J Orthod 1996; 11: 41–7.

This article has been cited by other articles:

				B. J. Houlberg Dental Residents' Perceptions of Practice and Patient Management Training During Postgraduate Education J Dent Educ., June 1, 2008; 72(6): 643 - 652. [Abstract] [Full Text] [PDF]

				J. R. Sandy Northcroft Memorial Lecture 2006 The future of specialist training J. Orthod., September 1, 2007; 34(3): 177 - 184. [Abstract] [Full Text] [PDF]

				L. Hichens, H. Rowland, A. Williams, S. Hollinghurst, P. Ewings, S. Clark, A. Ireland, and J. Sandy Cost-effectiveness and patient satisfaction: Hawley and vacuum-formed retainers Eur J Orthod, August 1, 2007; 29(4): 372 - 378. [Abstract] [Full Text] [PDF]

				J. Stirling, G. Latchford, D. O. Morris, J. Kindelan, R. J. Spencer, and H. L. Bekker Elective orthognathic treatment decision making: a survey of patient reasons and experiences J. Orthod., June 1, 2007; 34(2): 113 - 127. [Abstract] [Full Text] [PDF]

				A. McNair, P. Gardiner, J. R. Sandy, and A. C. Williams A qualitative study to develop a tool to examine patients' perceptions of NHS orthodontic treatment. J. Orthod., June 1, 2006; 33(2): 97 - 106. [Abstract] [Full Text] [PDF]

This Article Abstract Full Text (PDF) Alert me when this article is cited Alert me if a correction is posted Citation Map Services Email this article to a friend Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Download to citation manager Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Google Scholar Articles by Travess, H. C. Articles by Williams, A. C. Search for Related Content PubMed PubMed Citation Articles by Travess, H. C. Articles by Williams, A. C.