|
|
||||||||
Article |
Liverpool University Dental Hospital and School of Dentistry, UK
Address for correspondence: J. E. Harrison, Orthodontic Department, Liverpool University Dental Hospital and School of Dentistry, Pembroke Place, Liverpool L3 5PS, UK. Email: Jayne.Harrison{at}rlbuh.nhs.uk
Received November 27, 2003; accepted April 8, 2004
| Abstract |
|---|
|
|
|---|
Design: A retrospective, observational study.
Setting: PILs available from professional organizations and commercial companies.
Materials and methods: Twenty-six orthodontic PILs were assessed. The entire text of each leaflet was reproduced in Microsoft Word, 2000. Readability statistics were obtained via the Tools menu. The design elements of each leaflet were assessed. The leaflets were sent to the Plain English Campaign for assessment of their eligibility for the Crystal Mark.
Outcome measures: Leaflet and sentence length, passive percentage, Flesch Reading Ease score, Flesch Kincaid Grade Level, design percentage and eligibility for the Plain English Campaigns Crystal Mark.
Results: Overall, nearly half of the leaflets (42.3%) were rated as fairly difficult or difficult to read. However, the BOS PILs were significantly better than the AAO leaflets in all but one outcome with the BOS leaflets being rated as standard or fairly easy to read, meaning that 7080% of the UK population would be able to understand them. None of the PILs were eligible for the Plain English Campaigns Crystal Mark.
Conclusions: The orthodontic PILs assessed were difficult to read and none were eligible for the Plain English Campaigns Crystal Mark. However, the BOS leaflets were much easier to read and better designed than those produced by the AAO making them a useful tool to improve patients understanding of different treatment options and allowing them to be used in the informed consent process.
Key words: Readability, orthodontic, patient information leaflets (PILs), Crystal Mark
| Introduction |
|---|
|
|
|---|
Evidence suggests that a patients overall satisfaction with a clinician is increased if s/he is given, and understands, information and clinical advice.3
Nanda and Kierl4
highlighted this by stating that successful orthodontic treatment depended not only on the knowledge and skills of the clinician, but also on the cooperation of the patient and parents. A high proportion of parents of children undergoing orthodontic treatment, have been found to be unaware of its potential negative outcomes e.g. relapse and caries.5
Patients tend to forget or misunderstand much of what is discussed during a consultation.6
In general, people only retain about 20% of what they hear, but this may increase by up to 50% if there is additional visual or written input.7
George et al.8
demonstrated that patients favoured written information and that patients who were given leaflets were more satisfied with their treatment as a whole.
Leaflets are cheap to produce and can save patients the embarrassment of asking questions directly of a professional.9
They can be used to reinforce what has been discussed10
and can be referred to by patients away from the stressful environment of the consultation room.11
Weinman12
confirms the desire, use and value of leaflets by patients, showing that 75% of patients wanted written information and that 80% read the leaflets. Disappointingly though, the design of health information leaflets is poor.13,
14
In the interaction of the 2 elements, the design elements enhance the readability of the primary structural elements.15,
16,
17
Outside factors, such as stress, may also influence patients learning.18
Readability formulae assess the structural elements of the text and are designed to measure the reading difficulty. There are over 50 published readability formulae3
that produce a score or number that indicates how readable that piece of text is. Most are based on the premise that long words and/or sentences make text harder to understand.19
The reading abilities vary widely across the population so it is important that information is pitched at a suitable level for it to be understood by the maximum number of patients.20
The Adult Literacy and Basic Skills Unit21
found 1 in 5 of the UK population is functionally illiterate. In the USA, the average reading level is 8th grade (around 14 years of age), with 1 in 5 adults reading at the 5th grade level (around 11 years old) or below.22
Many researchers have found that PILs and information on web pages tend to be written at too high a level for the general public.20,
23
27
However, despite an improvement in the readability of leaflets recently, the aim of having them written at the level of 6th grade (around 12 years old) has not yet been achieved.28
No assessment of the readability of orthodontic PILs has been conducted; therefore, this study was designed to address this issue.
| Aims |
|---|
|
|
|---|
| Objectives |
|---|
|
|
|---|
| Methods |
|---|
|
|
|---|
|
Reading statistics
The following reading statistics for each leaflet were obtained via the Tools menu:
Design elements
The design elements of each leaflet were assessed using a 20-item checklist compiled from guidelines from Coey,31
the National Institute for the Blind32
and the Centre for Health Information Quality.33
Each leaflet was given a percentage score (design percentage) expressing the number of criteria that were satisfied (see Appendix 2). A record was also made of the number of leaflets satisfying each criterion.
Plain English Campaign Crystal Mark
The leaflets were sent to the Plain English Campaign (www.plainenglish.co.uk/crystalmark.html) for assessment of their eligibility for the Crystal Mark. The initial assessment was free and included looked for:
Statistical analysis
The mean score and standard deviation, of each leaflet, for each reading statistic were calculated. The weighted mean difference (WMD), with associated 95% confidence intervals (95% CI) and p values, were used to assess differences in the mean scores of the AAO and BOS leaflets. The scores for each group of leaflets (by publisher) were ranked for each test. The overall score of each group of leaflets was then ranked to give the best and worst performing leaflets.
| Results |
|---|
|
|
|---|
|
|
Sentence length
The mean number of words per sentence for all leaflets was 14.3 (SD=1.5). Again, there was a highly statistically significant difference (p<0.00001) between the mean number of words per sentence in the AAO and BOS leaflets, with the AAO leaflets having more words per sentence [WMD (95% CI)=2.93 (2.50, 3.36)].
All of the leaflets examined fell within the range of the sentence length recommended by the Plain English Campaign, i.e. 1520 words. However, the PILs from the BOS rank best with the shortest sentences above the AAO leaflets, which were ranked in 5th place (see Table 4
). The worst performing leaflet was that from Stafford Miller.
|
This was the only test where PILs from the AAO performed better than those from the BOS. All leaflets were rated as excellent or good in terms of clarity (see Appendix 2). The two best leaflets were commercial publications from Oral B and Stafford Miller. The worst performing leaflets on this test were those from the BOS.
Flesch Reading Ease (FRE)
The mean FRE score for all leaflets was 58.9 (SD=13.3). There was a highly statistically significant difference (p<0.00001) between the mean FRE scores of the AAO and BOS leaflets, with the BOS leaflets having a higher score [WMD (95% CI)=26.88 (22.31, 31.45)], suggesting that they are easier to read.
Flesch Kincaid Grade Level (FKGL)
The mean FKGL for all leaflets was 8.5 (SD=2.1). There was a statistically significant difference (p=0.00001) between the mean FKGL of the AAO and BOS leaflets, with the BOS leaflets having a lower FKGL [WMD (95% CI)=4.31 (3.65, 4.97)], again suggesting that the BOS leaflets are easier to read.
Reading difficulty
Overall, the leaflets ranged from being fairly easy to fairly difficult to read, with just half of them being fairly difficult or difficult to read, meaning that less than 40% of the UK population would understand them (Table 5a,
b
).
|
|
Design percentage
All leaflets scored highly on the design element checklist, but only half (10/20) of the criteria were satisfied by all leaflets (Table 6
). The 3 criteria that were least frequently satisfied were:
|
Again, there was a statistically significant difference between PILs from the AAO and BOS with the leaflets from the BOS having a better design [WMD 3.30 (95% CI 4.93, 1.67)]. The leaflets from the BOS were in a triple-folded A4 format, whereas those from the AAO were small booklets of similar size that were not written or designed to be browsed through.
Plain English Campaigns Crystal Mark
Feedback from the Plain English Campaign revealed that none of the 26 PILs were eligible for the Crystal Mark.
Overall findings
There were significant differences between the PILs published by the AAO and BOS in all assessments made. The BOS PILs were easier to read and better designed than those produced by the AAO and outperformed them in all but one assessment i.e. the passive percentage test. PILs from the BOS were ranked in joint second place overall (with Cyber Sign) above those from the AAO that were ranked in sixth place.
| Discussion |
|---|
|
|
|---|
The length of the leaflets varied considerably. Although length, per se, does not affect reading ease the longer leaflets may be off-putting, meaning that fewer patients and/or parents would read them. This would then make them less effective as an information tool.
Limitations of study methods
Readability formulae should only be used as a guide for assessing reading difficulty of a text34
as they do not take into account other factors that can influence the comprehension of a text, e.g. the use of active and passive verbs, the way the information is organized and looks on the page, and the readers motivation and level of prior knowledge.35
Blinkhorn and Verity36
pointed out that dentistry has evolved a professional vocabulary that may be incomprehensible to the layman and that readability formulae may therefore under-estimate the difficulty of a text. Interestingly, they found that all 14 year olds studied could pronounce and understand the word orthodontics, but only 14% knew the meaning of the word appliance, which was used widely in the 26 PILs examined in this study. The term brace could easily be substituted and may be more easily understood.
The rejection of all 26 PILs by the Plain English Campaign highlights the problem with readability tests in that they do not guarantee the readability of a leaflet.
Comparison with other studies
Our study shows that a similarly high percentage of the AAO PILs were written at too high a level compared with other studies but that the BOS leaflets were far more readable than most PILs (see Table 7
). Interestingly, despite the need for specialized language, the BOS PILs were easier to read than general dental practice leaflets in the UK24
and websites seem to be written at a level that is easier to read than PILs.20
One of the design criteria assessed was the Royal National Institute for the Blinds recommended 12-point minimum font size,32
which was satisfied by all of the BOS PILs compared with only 10.6% of medical practice leaflets assessed in a similar study.13
|
Patients must understand what that treatment involves, the alternatives available, and the risks and benefits of various treatment options when giving informed consent to treatment. Leaflets can only help give patients information if they are readable and understandable. Using patients and the public in lay reader panels can greatly assist in the production of readable PILs that are suited to their users.
Errors in communication can prompt allegations of malpractice.24
It is therefore necessary to document which PILs have been given to patients during the consent process in order to help reduce misunderstandings between the patient and clinician. Sections for this are now included in some consent forms.43,
44
The accessibility of information is an important consideration in modern-day practice. Under the Disability Discrimination Act (1995),45
orthodontists could be prosecuted for not providing information in other accessible formats, for example, Braille, computer disc or audiotape. Thought must also be given to patients whose first language is not English, so PILs need to be available in other languages to ensure that patients are able to give their informed consent.
Implications for research
The move towards evidenced-based dentistry requires that there is sound evidence of the efficacy of the interventions we provide for our patients. It is important, therefore, to be able to demonstrate knowledge gain from PILs to be able to justify their use. This can be done in an appropriately designed, randomized, controlled trial. From the standpoint of clinical governance, the design of leaflets needs to be evidence-based and peer reviewed. The quality of the information in the PILs was not assessed in this study, but may be an important consideration for future research.
| Conclusions |
|---|
|
|
|---|
The use of the Crystal Mark to clearly denote readable PILs is recommended.
| Contributors |
|---|
|
|
|---|
| Appendices |
|---|
|
|
|---|
| Appendix 1: Explanation of the assessment criteria used to assess the leaflets |
|---|
|
|
|---|
Flesch Kincaid Grade Level (FKGL)
The FKGL gives a result in terms of United States (US) school grades. It uses mean sentence and word length to determine the readability level.
Table A1
compares the interpretation of the FRE and FKGL scores.
|
Sentence length
The sentence length test is simply the total number of words in a document divided by the total number of sentences to give an average sentence length. An average sentence length of between 15 and 20 words is ideal according to the Plain English Campaign.
Passive percentage
Passive verbs cause several problems as they:
With an active verb, the 3 parts of a sentence appear in the order: subject then verb then object. For example:
Anne (subject) read (verb) the leaflets (object). Read is an active verb here. The sentence says who is doing the reading before it says what is being read.
With a passive verb, the order is reversed: object then verb then subject.
The leaflet (object) was read (verb) by Anne (subject).
Read is a passive verb here. The sentence says what is being read before it says who is doing the reading.
By making the sentence passive, the words was and by were introduced making the sentence clumsier. You should aim to make 8090% of verbs active.
To derive the passive percentage the number of passive verbs in a document is divided by the total number of sentences and multiplying the result by 100. See Table A2
.
|
| Appendix 2: Design percentage checklist |
|---|
|
|
|---|
Design element checklist
|
| References |
|---|
|
|
|---|
2 Witt E, Bartsch A. The effect of information and communication in the orthodontic consultation. Part 1: the imparting of information. Fortsch Kieferorthop 1993; 54: 18795.[CrossRef]
3 Ley P. Communicating with Patients. London: Chapman and Hall, 1988.
4 Nanda RS, Kierl MJ. Prediction of co-operation in orthodontic treatment. Am J Orthod Dentofac Orthop 1992; 102: 1521.[Medline]
5 Pratelli P, Gelbier S, Gibbons DE. Lack of knowledge among parents, of the implications of receiving orthodontic treatment. Comm Dent Hlth 1996; 13: 21114.
6 Ley P. Comprehension, memory and success of communications with the patient. J Inst Hlth Educ 1972; 10: 239.
7 Gauld VA. Compliance and recall. J Roy Coll Gen Pract 1981; 83: 298300.
8 George C, Waters W, Nicholas J. Prescription information leaflets: a pilot study in general practice. Br Med J 1983; 287: 11936.
9 Ormrod J, Robinson M. How readable are health education leaflets? Hlth Visitor 1994; 67: 4245.
10 Moll JMH. Doctor patient communication in rheumatology. Studies of visual and verbal perception using educational booklets and other graphic material. Ann Rheum Dis 1986; 45: 198209.
11 Bishop P, Kirwan J, Windsor K. The Arc Patient Literature Evaluation Project. Chesterfield: Arthritis and Rheumatism Council, 1996.
12 Weinman J. Providing written information for patients: psychological considerations. J Roy Soc Med 1990; 83: 3035.[Medline]
13 Albert T, Chadwick S. How readable are practice leaflets? Br Med J 1992; 305: 12668.
14 Bennett J, Bridger P. Communicating with patients. Br Med J 1992; 305: 1294.
15 Gilliland J. Readability. London: Hodder and Stoughton, 1976.
16 Irwan VD, Wilson NW. An Evaluation of Dental Health Literature. St Paul: Bruce Publishing Co., 1942.
17 NIH. Pretesting in Health Communications. NIH publication No. 81:1493. Bethesda: Government Printing Office, 1980.
18 Redman B. The Process of Patient Education. St Louis: Mosby, 1988.
19 Ceccio JF, Ceccio CM. Effective Communication in Nursing Theory and Practice. Chichester: Wiley and Sons, 1982.
20 Smart JM, Burling D. Radiology and the Internet: a systematic review of patient information resources. Clin Radiat 2001; 56: 86770.
21 Adult Literacy and Basic Skills Unit. Making it Happen: improving basic skills within the Health Service. London: ALBSU, 1994.
22 Doak C, Doak L, Root J. Teaching patients with low literacy skills, 2nd edn. New York: Lippincott-Raven, 1996.
23 Bakdash MB, Odman PA, Lang AL. Distribution and readability of periodontal health education literature. J Periodont 1983; 54: 53841.
24 Newton JT. The readability and utility of general dental practice patient information leaflets: an evaluation. Br Dent J 1995; 178: 32932.[CrossRef][Medline]
25 Alexander RE. Readability of published dental educational materials. J Am Dent Ass 2000; 131: 93742.
26 Payne S, Large S, Jarrett N, Turner P. Written information given to patients and families by palliative care units: a national survey. Lancet 2000; 355: 1792.[CrossRef][Medline]
27 DAlessandro DM, Kingsley P, Johnson-West J. The readability of pediatric patient education materials on the World Wide Web. Arch Pediat Adolesc Med 2001; 155: 80712.
28 Freda MC, Damus K, Merkatz IR. Evaluation of the readability of ACOG patient education pamphlets. Obstet Gynecol 1999; 93: 7714.
29 Flesch R. A new readability yardstick. J Appl Psychol 1948; 32: 2213.[CrossRef]
30 Plain English Campaign. Available at: www.plainenglish.co.uk (accessed 22 February 2002.
31 Coey L. Readability of printed educational materials used to inform potential and actual osteomates. J Clin Nurs 1996; 5: 35966.[Medline]
32 Royal National Institute for the Blind. Making Print Legible. London: RNIB, 1990.
33 Centre for Health Information Quality (CHIQ). Available at: http://www.hfht.org/chiq/(accessed 22 February 2002.
34 Meade CD, Smith CF. Readability formulas: cautions and criteria. Patient Educ Counsel 1991; 17: 1538.
35 Cutts M. The Plain English Guide. Oxford: Oxford University Press, 1996.
36 Blinkhorn AS, Verity JM. Assessment of the readability of dental health education literature. Comm Dent Oral Epidemiol 1979; 7: 1958.[CrossRef][Medline]
37 Hearth-Holmes M, Murphy PW, Davis TC, et al. Literacy in patients with a chronic disease: systemic lupus erythematosis and the reading level of patient education materials. J Rheumatol 1997; 24: 23359.[Medline]
38 Davis TC, Mayeaux EC, Fredikson D, Bocchini JA, Jackson RH, Murphy PW. Reading ability of parents compared with reading level of pediatric patient education materials. Pediatrics 1994; 93: 4608.
39 Thomson AM, Cunningham SJ, Hunt NP. A comparison of information retention at an initial orthodontic consultation. Eur J Orthod 2001; 23: 16978.
40 Humphris GM, Duncalf M, Holt D, Field EA. The experimental evaluation of an oral cancer information leaflet. Oral Oncol 1999; 35: 57582.[CrossRef][Medline]
41 Mayeaux EC, Murphy PW, Arnold C, Davis TC, Jackson RH, Sentell T. Improving patient education for patients with low literacy skills. Am Fam Physician 1996; 53: 20511.[Medline]
42 Kenney T, Wilson RG, Purves IN, et al. A PIL for every ill? Patient information leaflets (PILs): a review of past, present and future use. Fam Pract 1998; 15: 4719.
43 Patient Agreement to Orthodontic Treatment. Available at: www.bos.org.uk (accessed 22 February 2002.
44 Consent Form 3. Available at: www.doh.gov.uk (accessed 22 February 2002.
45 Disability Discrimination Act. Department for Education and Employment. Code of Practice. Rights of Access. Goods, facilities and premises. London: Stationary Office; 1995.
This article has been cited by other articles:
![]() |
H. H. Keller, D. Smith, C. Kasdorf, S. Dupuis, L. Schindel Martin, G. Edward, C. Cook, and R. Genoe Nutrition Education Needs and Resources for Dementia Care in the Community American Journal of Alzheimer's Disease and Other Dementias, March 1, 2008; 23(1): 13 - 22. [Abstract] [PDF] |
||||
![]() |
A. T. Ha and S. A. Gansky HIPAA Notice of Privacy Practices Used in U.S. Dental Schools: Factors Related to Readability or Lack Thereof J Dent Educ., March 1, 2007; 71(3): 419 - 429. [Abstract] [Full Text] [PDF] |
||||
![]() |
A. C. Williams, H. Shah, J. R. Sandy, and H. C. Travess Patients' motivations for treatment and their experiences of orthodontic preparation for orthognathic surgery J. Orthod., September 1, 2005; 32(3): 191 - 202. [Abstract] [Full Text] [PDF] |
||||
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |