Table 1

Overview of coding guide for the news item

Part 1: health clam
Credited response A response…
 Completeproviding a complete account of the claim (ie, that a regular intake of corn reduces the risk of type II diabetes).
 Mostly completeproviding a mostly complete account of the claim, with some significant words lacking (ie, ‘regular’ and/or ‘type II’), and/or referring to amount of intake (eg, ‘much’ or ‘more’ corn).
Non-credited response A response…
 Wrongwhere the claim relates to the topic of the news report, but is otherwise wrong.
 Vaguewith no reference to corn and/or type II diabetes.
 Otherwhich is irrelevant or a ‘do not know’ response.
Part 2: information requests
Credited response A response relating to…
 Methods how the study was conducted, including study design, subjects, procedures and measurements.
 Data/statistics what was observed in the reported study, or about statistical tests used to analyse the data.
 Theory/agent why the reported effects might have occurred, including questions about the properties of the presumed causal agent and/or possible underlying mechanisms.
 Social contextthe credentials and bias related to who did the study or funded it and where it was conducted or published.
 Relevancethe importance or applicability of the study findings, or the impact of the study.
 Related researchwhether the findings have been replicated or fit results from previous research.
 Ambiguousthe study described in the news report that is ambiguous because it fits under two or more scientific categories.
Non-credited response A response
 Future studiesindicating the need for one or more future studies, either in general, or relating specifically to one of the scientific research categories (methods, data, etc).
 Disbeliefindicating that the student does not believe that the study has been conducted.
 Wrongrelating to the topic of the news report, but is otherwise wrong.
 Vagueonly vaguely referring to the scientific categories (methods, data, etc).
 Otherwhich is irrelevant or is a ‘do not know’ response.
  • When coding part 1, the raters applied one variable and used values starting with ‘1’ and ‘0’ to indicate whether the response included an acceptable account of the health claim or not. For part 2, the raters applied eight variables. Seven of these were labelled to reflect the scientific research categories (methods, etc), raters used the values ‘1’ and ‘0’ to indicate whether the response included an acceptable request for information within the specific category. For the eight variable, non-credited responses to part 2, values starting with ‘0’ were used to indicate type of response. Blank responses (part 1 or part 2) were coded with the value ‘99’.