Article Text

Original research
Evaluation of the usability, content, readability and cultural appropriateness of online alcohol and other drugs resources for Aboriginal and Torres Strait Islander Peoples in New South Wales, Australia
  1. Rebecca Amanda1,2,
  2. Kritika Rana1,2,3,
  3. Paul Saunders3,4,
  4. Marguerite Tracy5,6,
  5. Nicole Bridges7,
  6. Prakash Poudel8,
  7. Amit Arora1,2,3,9,10
  1. 1School of Health Sciences, Western Sydney University, Penrith, NSW, Australia
  2. 2Health Equity Laboratory, Campbelltown, NSW, Australia
  3. 3Translational Health Research Institute, Western Sydney University, Penrith, NSW, Australia
  4. 4School of Medicine, Western Sydney University, Penrith, NSW, Australia
  5. 5General Practice Clinical School, Sydney Medical School, The University of Sydney, Sydney, NSW, Australia
  6. 6Drug Health Services, Cumberland Hospital, Western Sydney Local Health District, North Parramatta, NSW, Australia
  7. 7School of Humanities and Communication Arts, Western Sydney University, Kingswood, NSW, Australia
  8. 8Office of Research and Education, Canberra Hospital, Canberra Health Services, ACT Government, Canberra, ACT, Australia
  9. 9Discipline of Child and Adoloscent Health, Sydney Medical School, The University of Sydney, Westmead, NSW, Australia
  10. 10Oral Health Services, Sydney Local Health District and Sydney Dental Hospital, Surry Hills, NSW, Australia
  1. Correspondence to Professor Amit Arora; a.arora{at}westernsydney.edu.au

Abstract

Objectives This study aimed to analyse the usability, content, readability and cultural appropriateness of alcohol and other drugs (AODs) resources for Aboriginal and Torres Strait Islander Peoples in New South Wales (NSW), Australia.

Outcome measures The content of 30 AOD resources for Aboriginal and Torres Strait Islander Peoples was analysed according to the following criteria: general characteristics; elements of graphical design and written communication; thoroughness and content; readability (Flesch-Kincaid grade level (FKGL), Gunning Fog index (Fog), Simplified Measure of Gobbledygook and Flesch Reading Ease); and cultural appropriateness.

Results Most resources displayed good usability, depicted by the use of headings and subheadings (n=27), superior writing style (n=19), relevant visuals (n=19) and use of colour support (n=30). However, some resources used at least one professional jargon (n=13), and many did not provide any peer-reviewed references (n=22). During content analysis, 12 resources were categorised into the alcohol group and 18 resources in the other drugs group. Impact of alcohol during pregnancy and breast feeding (n=12) was the most common included topics in the resources related to alcohol, while the physical impact of drugs (n=15) was the most discussed topics among the other drugs group. Based on the FKGL readability score, 83% of resources met the recommended reading grade level of 6–8 by NSW Health. Many resources (n=21) met at least half of the cultural appropriateness elements of interest. However, less than one-third were developed in collaboration with the local community (n=9), used local terms (n=5), targeted the local community (n=3), included an Aboriginal voice (n=2) and addressed the underlying cause (n=1).

Conclusions Many AOD resources are developed specifically for Aboriginal and Torres Strait Islander Peoples, but their usability, content and readability differed, and they were not culturally appropriate for all communities. Development of a standardised protocol for resource development is suggested.

  • epidemiology
  • primary care
  • substance misuse
  • public health

Data availability statement

Data sharing not applicable as no datasets generated and/or analysed for this study.

http://creativecommons.org/licenses/by-nc/4.0/

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

Statistics from Altmetric.com

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

STRENGTHS AND LIMITATIONS OF THIS STUDY

  • This study has used rigorous methodology to conduct a comprehensive appraisal of the resources based on a wide range of criteria: resource characteristics; elements of graphical design and written communication; thoroughness and content; readability; and cultural appropriateness.

  • Four commonly used and validated indices, Flesch-Kincaid grade level, Gunning Fog index (Fog), Simplified Measure of Gobbledygook and Flesch Reading Ease), were used to evaluate the readability of the resources.

  • The evaluation methods used in this study were developed in discussion with a steering group with expertise in alcohol and drugs (AODs), digital communication and codesigned with an Aboriginal and Torres Strait Islander representation.

  • Only resources written in English language were included in this study, which may exclude resources that are written in local language and developed specifically for a local community.

  • Thoroughness and content between AODs groups differ depending on the commonly discussed topic, which may exclude some elements that may influence the intended readers’ opinion on harmful AOD use and treatment available.

Introduction

Harmful use of alcohol and other drugs (AODs) is a global public health issue, that is, responsible for many preventable chronic illnesses, hospitalisations and unintentional deaths.1 According to the WHO, alcohol consumption is responsible for at least three million deaths each year and harmful use of alcohol accounted for 5.1% of the global burden of disease.2 In 2019, the United Nations reported that drug use disorders were responsible for half a million deaths and 18 million years loss of healthy life.3

In Australia, recent data show that 7.5% of the total disease burden is attributable to AOD use disorders.4 Moreover, this issue disproportionately affects certain population groups in Australia, such as Aboriginal and Torres Strait Islander Peoples and individuals with mental health conditions.5 In 2018, harmful use of AOD collectively contributed to 17.4% of the total disease burden among Indigenous Australians, almost 10% more than non-Indigenous Australians.4 6 In this paper, the term ‘Aboriginal and Torres Strait Islander Peoples’ and ‘Indigenous Australians’ refer to the First Nations Peoples of Australia. The term ‘Indigenous peoples’ refers to First Nations Peoples globally.

Aboriginal and Torres Strait Islander Peoples comprise 3.3% of the Australian population.7 While the gaps in burden of disease between Indigenous and non-Indigenous Australians have decreased in recent years, Indigenous Australians still experience worse health than their non-Indigenous Australians counterparts.6 Unfortunately, these gaps in health are likely to start before birth and remain throughout their lives.6

For Indigenous Australians, the concept of health is not limited to physical, mental and social well-being, but also cultural, spiritual and ecological well-being of the individual and the community.8 Factors posing risk of harmful use of AOD among Aboriginal and Torres Strait Islander Peoples are associated with psychological distress from ongoing impacts of colonisation and intergenerational trauma, substance use by peers, family members or partner, and availability of substances.9 In addition, the cultural connections of Aboriginal and Torres Strait Islander Peoples are often misunderstood, misinterpreted or undervalued in the current policies and approaches in Australia, which may contribute to systemic discrimination, isolation, social exclusion and limit the access to healthcare.10 On the other hand, health knowledge, cultural engagement and supportive environments are protective factors against AOD use disorders and its related harms.9

Health education, through evidence-based written materials (such as leaflets and posters) and verbal communication, is a vital part of the AOD demand reduction initiatives.5 11 However, health literacy plays a fundamental role in understanding such information.12 Health literacy is characterised as the ability to obtain, read and make sense of health information and use the health information in ways that promote and maintain their health.13 14 This ability is essential to understand and complete healthcare forms, such as consent forms or other health-related resources, and to make decisions and manage one’s health.11 15 The inability to read and understand resources can have a negative effect on health outcomes and quality of care provided by family or carers.16 17 Individuals with low health literacy are less likely to be involved in their health management, thus affecting their access to healthcare and quality of care.18–20 The 2018 Australian National Health Survey reported that 17% of Australians were unable to appraise health information.21 It is suggested that Aboriginal and Torres Strait Islander Peoples may be at higher risk of low health literacy as educational attainment and school-based numeracy and literacy scores have been found to be significantly lower among Indigenous peoples compared with the general population.22

The concept of health literacy has been broadened beyond an individual’s capability, into a more complex interaction with the healthcare systems (eg, healthcare providers, health policies, health education and resources).14 23 For instance, healthcare providers are recommended to refrain from using medical jargon when delivering health information to aid the patient’s understanding. Prior research noted that the mode of information, such as spoken and written information, may influence a patient’s retention.12 24 While patients with low literacy may struggle with written information, it has important benefits compared with other modes of information. Written information may promote treatment adherence as it remains available for later reference and for review by family and carer.12 24 In addition, including visuals aids or informative images may improve usability and quality of written information.24 25

To date, the evidence on the readability of AOD resources for Aboriginal and Torres Strait Islander Peoples appears to be limited. Studies conducted previously mainly focused on the readability analysis for the general population and were not specific to AOD resources.16 26–28 Thus, analysis of AOD resources for Aboriginal and Torres Strait Islander Peoples is necessary to identify important elements when developing resource materials, which will help future health resource development and potentially prevent and delay harmful AOD use as well as to reduce the gaps in health outcomes between the Indigenous and non-Indigenous Australians. This study aimed to analyse the usability, content, readability and cultural appropriateness of readily available online AOD resources for Aboriginal and Torres Strait Islander Peoples in New South Wales (NSW), Australia.

Methods

A desktop search was conducted between October and December 2021 to obtain all AOD resources that were readily available online in NSW. These resources were obtained from key organisations working in the health and well-being of Aboriginal and Torres Strait Islander Peoples or key organisations that offer support and information services about AOD. Initial search terms included “alcohol”, “drugs”, “Aboriginal and Torres Strait Islander Peoples” and “Indigenous Australians” to capture all relevant resources.

Eligibility criteria

The eligibility criteria for this audit were as follows:

Inclusion criteria

  • Written in English language.

  • Resources developed specifically for Aboriginal and Torres Strait Islander Peoples with or at risk of AOD issues and/or their carers, families and communities.

  • AOD resources produced by governmental and not-for-profit organisations in NSW.

  • AOD resources which are readily available via the internet.

Exclusion criteria

  • Resources that do not contain key messages related to AOD use and/or its management.

  • Resources that only include information on how to access AOD service.

Development of evaluation method

The content of each included resource was appraised based on five main criteria: resource characteristics, elements of graphical design and written communication, thoroughness and content, readability, and cultural appropriateness. Topics in each criterion were developed in discussion with a steering group where there was not a standard assessment tool available. Four experts from three different fields of expertise (alcohol and drugs, digital communication, and an Aboriginal and Torres Strait Islander person) were invited and agreed to join the steering group. All experts had no part in the original development of the educational brochures.

Resource characteristics

Characteristics of each included resource were documented, including the title, publisher, publisher type, target group and format. The format of AOD resources was recorded according to the type of resources (eg, booklets, brochures, fact sheets, postcards, posters and storybooks) and the number of pages (ie, in a PDF format).

Elements of graphical design and written communication

Elements of graphical design and written communication were appraised based on the use of headings and subheadings, typography, the percentage of bulleted text, the percentage of visual to written text, writing style, active voice, the use of professional jargons, type of illustrations, relevance of the illustrations, infographics, and use of colour supports.

For the evaluation of some elements of graphical design and written communication, the Suitability Assessment of Materials (SAM) evaluation criteria was used. Developed to systematically assess printed materials and illustrations in a timely manner, the SAM evaluation criteria allow healthcare providers to assess reading materials based on 22 factors and rate them as ‘2=superior,’ ‘1=adequate’ and ‘0=not suitable’ according to classifications established by Doak et al.29 As many topics from the SAM evaluation criteria overlaps with the existing topics used in this study, only some elements were chosen or adapted from it. Three topics, including ‘typography,’ ‘writing style’ and ‘type of illustrations’, were analysed as per the SAM evaluation criteria and were documented as ‘superior,’ ‘adequate’ and ‘not suitable’.29 For example, ‘writing style’ was rated as ‘superior’ if the resource utilised mostly conversational style and active voice, and simple sentences. Three topics of ‘use of headings and subheadings’, ‘active voice’ and ‘relevance of the illustrations’ were also adapted from the SAM evaluation criteria. These were marked as ‘yes’ if the resource used any headings and subheadings, used any active voice, or provided relevant illustrations to the topic. Kool’s macrocoherence and microcoherence model of communication also was used to evaluate the use of headings and subheadings and percentage of bulleted text in each resource.30

Topics of ‘the percentage of visual to written text’, ‘infographics’ and ‘use of colour supports’ were used to evaluate the use of visual stimulation to achieve the principle of dual code theory. The dual code theory postulates that simultaneous verbal and visual cognitive systems stimuli will help readers to understand the presented information better.25 Each included resource were marked as ‘yes’ for any use of medical or technical jargon as it could impede the reader’s understanding of the corresponding health information.31 32

Thoroughness and content

To evaluate the thoroughness and content, the AOD resources were further categorised based on whether the primary focus was on alcohol or other drugs. Resources that covered both were classified into the group, to which the majority of its content belonged.

Each included resource was analysed based on the presence or absence of common elements of its group’s classification. For instance, the subtopic of impact of alcohol during pregnancy and breast feeding was analysed in the alcohol group, but not in other drugs group, as most resources in the alcohol group discussed the detrimental effect of alcohol among pregnant and breastfeeding women. Resources that had alcohol as the main content were evaluated based on the presence or absence of five key topics. These topics were background information, impact of alcohol, safe limit, contact information for getting help and supporting evidence. The topic of ‘impact of alcohol’ overarched into four subtopics: physical health, mental health, social impacts, and during pregnancy and breast feeding. The ‘other drugs’ as the main content were evaluated based on the presence or absence of eight key topics: background information, impact of drugs, overdose and withdrawal, treatment options, contact information for getting help, associated laws, information for family and carers, and supporting evidence. The topic of ‘impact of drugs’ was categorised into three subtopics of physical health, mental health and social impacts.

Readability

Four readability indices, the Flesch-Kincaid grade level (FKGL),33 the Gunning Fog index (Fog),34 the Simplified Measure of Gobbledygook (SMOG)35 and the Flesch Reading Ease,36 were used to evaluate the readability of the resource (ie, understandability of health information in each included resource). The formulas for the indices are outlined in table 1. These four indices are widely accepted to assess the readability of health education resources and are validated for calculating readability.29 37

Table 1

Formulas of the four readability indices

To obtain the readability score from the four readability formulas, the title and content of each included resource was converted to plain text (ie, without abbreviations, bullet points, illustrations and text boxes) and entered into an automated online programme.38 In addition, the word and sentence counts produced by the online programme were manually matched to the plain text.

The FKGL, Fog and SMOG readability indices yield a numerical value that represents the reading grade level or the level of education required to understand the corresponding text.33 On the contrary, the Flesch Reading Ease formula produces a score between 0 and 100. A lower score suggests a more difficult text to comprehend; whereas a higher score suggests that it is easier to comprehend the corresponding text.36 For instance, Flesch Reading Ease score of 85 indicates an easy text, in which a year 6 student can understand. During the analysis, the Flesch Reading Ease score of included resources was converted into the average of reading grade levels based on table 2. Subsequently, reading grade levels obtained from four readability indices were compared with each other. The reading grade levels from these indices are based on the US grade levels and are equivalent to grade levels in Australia.39

Table 2

Interpretation of the Flesch Reading Ease36

Cultural appropriateness

Culturally targeted messages have been shown to influence the receptivity of preventive programmes and health promotion among Indigenous Australians and Indigenous peoples worldwide.40 41 The cultural appropriateness aspect of each included resource was appraised based on the presence or absence of 10 criteria: avoids stereotypes, addresses underlying cause, uses local terms, includes an Aboriginal voice, targets local community, developed in collaboration with local community, directing people to culturally appropriate services, cultural match in logic, language and experience, diversity of imagery, and culturally relevant artwork or images. All criteria were modified from the adapted DISCERN instrument by the Cultural and Indigenous Research Centre Australia (CIRCA) research team, the SAM evaluation criteria and in discussion with an Aboriginal and Torres Strait Islander person from the steering group.42

Patient and public involvement

Patients and the public were not involved in this research.

Results

Included in this study was a total of 30 AOD resources readily available in NSW produced from governmental and not-for-profit organisations that are specific to Aboriginal and Torres Islander Peoples.

Resource characteristics

Characteristics of included resources were documented in table 3. Most resources (n=14) were published by governmental organisations such as Australian Government Department of Health, NSW Health and NSW Police Force. Eleven resources were published by not-for-profit organisations such as Aboriginal Drug and Alcohol Network of NSW, Aboriginal Drug and Alcohol Residential Rehab Network, Aboriginal Health and Medical Research Council of NSW, Alcohol and Other Drugs Knowledge Centre, and The University of Sydney Matilda Centre. The other five resources were collaborative publication between a governmental organisation and a non-governmental organisation (Australian Government Department of Health and Continence Foundation of Australia; Australian Government Department of Health and The University of Sydney Matilda Centre and NSW Health and Australian Drug Foundation). The most common publishers noted in this study were NSW Health (n=11) and Alcohol and Other Drugs Knowledge Centre (n=8). Of the 30 included resources, 12 had a target group such as the community, families and friends, individuals who are using drugs, men, parents and carers, women, and young people. All included resources ranged from 1 to 16 pages in length, with an average of 4.4 pages. Twelve of the included resources were published in the format of fact sheets, followed by four brochures, four storybooks, three postcards and three posters.

Table 3

Characteristics of Aboriginal and Torres Strait Islander Peoples’ specific alcohol and other drugs resources

Elements of graphical design and written communication

The majority of the included resources (n=27) used headings and subheadings (see table 4). Although none of the resource’s typography was rated as superior, most resources (n=25) were rated adequate, whereas the typography of a booklet and four storybooks were rated not suitable. Of the 30 resources, 9 contained 50% or more bulleted text, whereas 13 used no bulleted text.

Table 4

Elements of graphical design and written communication

Fourteen resources contained 50% or more visual to written text, while 4 resources contained no visuals. The writing style of most included resources (n=19) were appraised as superior. Conversely, five and six resources were appraised as adequate and not suitable, respectively. The majority of resources (n=24) used active voice in the written text. Thirteen resources used at least 1 term classed as professional jargon in the written text, and 16 identified jargon terms were noted in online supplemental material 1.

Of the 30 included resources, 19 resources contained some type of illustrations that were relevant to the topic (example resource: https://cracksintheice.org.au/pdf/cracks-in-the-ice-indigenous-what-is-cracks-in-the-ice.pdf).43 Fifteen resources displayed superior illustrations, while four resources were considered to be adequate. Less than one-third of included resources (n=9) provided infographics, and all resources used some kind of colour support. Overall, a brochure titled ‘Cracks in the ice: Information for staying safe’ performed the best in the graphical design and written communication aspect. It demonstrated an adequate typography, a superior writing style, superior illustrations and covered all elements of interest.

Thoroughness and content

Thoroughness and content of included alcohol resources and other drugs resources are presented in tables 5 and 6, respectively. Twelve resources had alcohol as the main content, whereas 18 resources had other drugs as the main content.

Table 5

Thoroughness and content of alcohol resources (n=12)

Table 6

Thoroughness and content of other drugs resources (n=18)

Of all alcohol resources, the 3 most covered elements were the impact of alcohol during pregnancy and breast feeding (75.0%), safe limit (66.7%) and physical health impact of alcohol (50.0%). On the other hand, only 16.7% of resources provided supporting evidence. A booklet titled ‘Yarning about alcohol and pregnancy’ provided the most comprehensive information. The eight-page booklet published by NSW Health covered almost all elements of interest but did not provide any supporting evidence. While a poster titled ‘What is FASD?’ only covered the element of impact of alcohol during pregnancy and breast feeding.

The most covered element among other drugs resources was the physical health impact of drugs (83.3%). Four other elements: background information, mental health impact of drugs, social impact of drugs and contact information for getting help were covered in more than half the resources. On the contrary, treatment options and supporting evidence were covered the least (33.3%). Of the 10 elements, 4 resources (22.2%) covered 7 or more elements of interest. A two-page fact sheet titled ‘Facts about heroin’ provided the most comprehensive information, covering all elements of interest. On the contrary, the two fact sheets titled ‘Detox, rehab and COVID-19’ only covered two elements: treatment options and contact information for getting help.

Readability

Figure 1 illustrates the reading grade level of each AOD resource based on four readability indices: FKGL, Fog, SMOG and Flesch Reading Ease. Generally, assessment of the same AOD resource using four readability indices resulted in four different grade levels. For instance, a one-page poster titled ‘Grog is no good for our babies’ attained a FKGL score of 2.6, an Fog score of 4.7, an SMOG score of 3.8 and a Flesch Reading Ease grade level of 6.

Figure 1

Readability indices scores of the included alcohol and other drugs resources.

Overall, FKGL scores were consistently at the same grade level or lower as the Fog and Flesch Reading Ease. A one-page fact sheet titled ‘fetal alcohol spectrum disorder (FASD) among Aboriginal and Torres Strait Islander people’ attained the highest reading grade levels across all four indices, with an FKGL score of 10.6, an Fog score of 11.2, an SMOG score of 9.7 and an average Flesch Reading Ease grade level of 14.5. Conversely, a two-page postcard titled ‘What is FASD?’ attained the lowest FKGL, SMOG and Flesch Reading grade level scores of 1.7, 3.5 and 5, respectively. This resource also achieved the second lowest Fog scores of 4.1. Both resources covered the topic of FASD and included 50% or fewer elements of interest, but the later resource had a higher percentage of bulleted text and visual to written text, achieved a superior writing style, used active voice and contained no professional jargons. A resource titled ‘Facts about heroin’ which contained the most comprehensive information received an FKGL score of 6.3, an Fog score of 8.7, an SMOG score of 6.5 and an average Flesch Reading Ease grade level of 8.5.

Cultural appropriateness

The cultural appropriateness aspect of all included resources is provided in table 7. All resources avoided stereotypes and were culturally match in logic, language and experience. Twenty-seven out of 30 resources included culturally relevant artwork or images. Similarly, majority of the included images (n=22) represented diversity in communities, in terms of age, appearance, gender and lifestyle. A brochure produced by the Australian Government Department of Health and The University of Sydney Matilda Centre was the only resource that acknowledged intergenerational trauma as one of the underlying causes of drugs misuse. On the other hand, another resource stated that alcohol use may contribute to separation from culture.

Table 7

Cultural appropriateness of Aboriginal and Torres Strait Islander Peoples’ specific alcohol and other drugs resources (n=30)

Three resources specified a local Indigenous community, namely Arnhem Land, Bandjalang and Koori. Moreover, five resources used local expressions, such as binjal, womba and yarndi in the written text. Of all included resources, 21 directed people to culturally appropriate services. Nine resources were developed in collaboration with local community, two of which included the voices of Aboriginal and Torres Strait Islander Peoples.

Discussion

This study appraised online AOD resources available in NSW for Aboriginal and Torres Strait Islander Peoples on the basis of resource characteristics, elements of graphical design and written communication, thoroughness and content, readability and cultural appropriateness. It was noted that resources written in the format of fact sheets have higher literacy demand as they are more likely to contain more information, more likely to use professional jargon, less likely to use active voice and contain fewer illustrations. However, some resources demonstrated that it is possible to have low literacy demand while containing comprehensive information and fulfilled most elements of graphical design and communication. Although most resources fulfilled half or more elements of interest in cultural appropriateness, they may not contain culturally targeted messages that meet the needs of all Aboriginal and Torres Strait Islander Peoples and communities.

While many included resources demonstrated good usability, there is a room for improvement in the areas of typography, using more illustrations, increasing the use of bulleted text, and reducing professional jargon. Monton et al reported that the typography subcategory assessed by the SAM score was rated as one of the highest, however, none of the resources in this study achieved a ‘superior’ typography.44 Typography is an important element to ensure that the text is reader-friendly. For instance, a written resource is difficult to read when it is written in small print, hard-to-read fonts or all capital letters.29 On the other hand, it is easier to read when the key points are emphasised (ie, bold, colour) and when the text is interspersed with blank space.29 45 Other studies evaluating health resources with SAM score emphasised the scarcity of ‘Superior’ resources, particularly regarding the graphics.44 46 47 Similarly, only half of the resources in this study demonstrated ‘superior’ illustrations as per SAM score, and one-third of the included resources did not contain any illustrations. This was concerning as including visual explanations have been shown to enhance the readers’ understanding and retention of information.48 Readability is further enhanced when presented with bullet points and by avoiding the use of professional jargon. Thus, elements of graphical design and written communications are essential to aid readers’ readability and comprehension.

A way to predict the level of reading difficulties of written resources is by using readability tools, which assess the physical characteristics of the text such as word length, sentence length and syllable count.29 37 Previous research, on the readability of written health resources intended for the general population, discovered that most resources are too difficult to read for most patients, requiring grade level 9 and higher.26 28 44 49 On the contrary, written health resources for Aboriginal and Torres Strait Islander Peoples are generally written at a lower average readability of grade level 6.50 However, these findings were not entirely consistent with this study. Although most included AOD resources for Aboriginal and Torres Strait Islander Peoples meet the NSW Health recommendation of grades 6–8 level, some resources were written above the recommended grade level.51 To ensure effective communication of written resources on AOD, providers should tailor the educational information to match the health literacy and reading skills of the target population and aim for the recommended reading levels.16 45

Nevertheless, relying on readability tools alone to improve written resources may make the included information too simple and insufficient, as it does not consider the content of the text or difficult vocabularies.45 This was observed in this audit, in which a resource with the lowest readability titled ‘What is FASD?’ contained only one key content. At the same time, a resource titled ‘yarning about alcohol and pregnancy’ demonstrated that it is possible to provide a comprehensive information and still achieve low readability scores. Pretesting materials for their cultural suitability and readability allows the publisher to make appropriate adjustments based on the feedback and discover possible cultural challenges that are otherwise overlooked. Therefore, pretesting prior to dissemination is an essential step to maximising the accessibility and suitability of information provided.16 45

Culture is one of the contributing factors to the social and emotional well-being of Aboriginal and Torres Strait Islander Peoples.10 Previous studies signify the importance of involving Aboriginal and Torres Strait Islander organisations during the development process.40 42 Even so, only nine included resources clearly mentioned some form of collaboration with Aboriginal and Torres Strait Islander organisations. Similarly, a report by the CIRCA found that almost half of the recommended quality resources did not clearly disclose the community consultation and were only apparent after in-depth interviews.42 This is concerning as the community consultation is necessary to not only ensure culturally appropriate messages, but also ensure that the specific needs of the target community are addressed.40 42

Cultural diversity among Aboriginal and Torres Strait Islander communities constructs a consequential challenge in developing culturally sensitive educational resources that caters to all communities.40 This could be the foremost reason on the scarcity of health resources that target a specific local community, make use of local terms, address the underlying cause of AOD use or include an Aboriginal voice. Gould et al found that organisations often compromised by keeping the information general, avoiding stereotypes and using diverse images, even though some communities may end up being under-represented.40 For example, including more real stories or Aboriginal voice may make the resource more engaging, but choosing the role models can be challenging as some communities do not allow showing names or pictures of the deceased.40 42 50 Hence, organisations should focus on developing resources that are specific to a local community and avoid generalising the messages.

Limitations of this study

To the best of our knowledge, this study is the first to appraise the usability, readability, content and cultural appropriateness of AOD resources for Aboriginal and Torres Strait Islander Peoples in NSW. However, five main limitations were identified. First, this study was limited to online AOD resources, which exclude hard copy resources, especially AOD resources that are developed by local Aboriginal Medical Services and only available in their local clinic, and resources that are disseminated through the social media. Second, the findings of this study do not represent resources that are produced in other states or territories in Australia, as the search was limited to resources available in NSW. Third, only resources written in English language were included in this study, which may exclude resources that are written in local language and developed specifically for a local community. Fourth, elements of the thoroughness and content was chosen based on the commonly discussed elements in their respective groups, which may exclude some elements that may potentially influence the intended readers’ opinion. Lastly, this study did not assess end-users view, such as through a qualitative investigation with Aboriginal and Torres Strait Islander Peoples or organisations.

Implications and recommendations

All assessed categories in this study (ie, usability, readability, content and cultural appropriateness) are essential and should be considered when developing AOD resources. It will likely be important to develop a wide range of resources on a variety of topics on AOD while providing comprehensive information to different audiences, including individuals with or at risk of AOD issues, individuals seeking AOD treatment, individuals undergoing AOD treatment, and families and carers. Furthermore, collaboration and partnerships with Aboriginal and Torres Strait Islander organisations and communities are vital to identify and address specific needs of their communities. The findings of this study also highlight the importance of a multifaceted evaluation process during the resource development phase. Despite the abundance of recommendations from many peer-reviewed journals, there is no specific national protocol for developing targeted resources that are evidence based, comprehensible, user-friendly and culturally acceptable across diverse communities.50 Lastly, this study noted the need for health literacy data of Aboriginal and Torres Strait Islander Peoples, as it was not reported in the 2006 Australian Bureau of Statistics survey or the 2018 Australian National Health Survey.20 21

Conclusion

Overall, resources developed by government organisations performed better across all categories. Even so, some AOD resources designed for Aboriginal and Torres Strait Islander Peoples in NSW are not optimal in terms of their usability, content thoroughness and readability, and cultural appropriateness. Involvement of Aboriginal and Torres Strait Islander Peoples during the resource development and evaluation process may improve their usability, the comprehensiveness of information, readability and culturally appropriateness. In addition, development of a national or standardised protocol for developing resources for Aboriginal and Torres Strait Islander Peoples may benefit government and not-for-profit organisations in developing future health resources that will contribute to reducing disparities in health outcomes.

Data availability statement

Data sharing not applicable as no datasets generated and/or analysed for this study.

Ethics statements

Patient consent for publication

References

Supplementary materials

  • Supplementary Data

    This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

Footnotes

  • Twitter @M_C_Tracy

  • Contributors AA, KR and PP conceived and designed the study. RA, KR and AA conducted the data collection. RA conducted the initial data analysis with help from KR, AA, PS, MT, NB and PP on the evaluation methods. RA drafted the initial manuscript. AA is the guatantor. All authors contributed to the revision of the manuscript and approve the final manuscript.

  • Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.

  • Competing interests None declared.

  • Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.

  • Provenance and peer review Not commissioned; externally peer reviewed.

  • Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.