REPLY - CoMiSS validation study protocol
Letter to the Editor of BMJ Open - 8 August 2018
We thank MM Vanderschuren et al. for their interest in our published protocol for a validation study of the Cow’s Milk-related Symptom Score (CoMiSS®) [1] and would like to provide our reply to the concerns raised.
CoMiSS was designed in 2015 by a paediatric expert group in order to provide an easy-to-use tool to facilitate a greater awareness for cow’s milk protein allergy (CMPA), particularly in regions where allergy services are not easily accessed or where community awareness for CMPA is low. [2] Since its introduction, CoMiSS has been translated into 14 languages and is used in 16 countries around the world. A pooled data analysis of three cross-sectional studies in children with CMPA confirmed that the CoMiSS score is useful in monitoring the response to treatment with hypoallergenic formula or a cow’s milk protein (CMP)-free elimination diet in infants and children with CMPA. [3]
It is important to highlight that CoMiSS was not designed as a diagnostic tool for CMPA and has not been validated for this purpose. [2] The published protocol outlines a prospective, single-blinded study which aims to assess the sensitivity and specificity of a change in CoMiSS score from baseline to 2 weeks after commencing a hypoallergenic diet, using a single-blinded oral food challenge (OFC) with cow’s milk-based formula as the diagnostic reference. [1] While a double-bli...
REPLY - CoMiSS validation study protocol
Letter to the Editor of BMJ Open - 8 August 2018
We thank MM Vanderschuren et al. for their interest in our published protocol for a validation study of the Cow’s Milk-related Symptom Score (CoMiSS®) [1] and would like to provide our reply to the concerns raised.
CoMiSS was designed in 2015 by a paediatric expert group in order to provide an easy-to-use tool to facilitate a greater awareness for cow’s milk protein allergy (CMPA), particularly in regions where allergy services are not easily accessed or where community awareness for CMPA is low. [2] Since its introduction, CoMiSS has been translated into 14 languages and is used in 16 countries around the world. A pooled data analysis of three cross-sectional studies in children with CMPA confirmed that the CoMiSS score is useful in monitoring the response to treatment with hypoallergenic formula or a cow’s milk protein (CMP)-free elimination diet in infants and children with CMPA. [3]
It is important to highlight that CoMiSS was not designed as a diagnostic tool for CMPA and has not been validated for this purpose. [2] The published protocol outlines a prospective, single-blinded study which aims to assess the sensitivity and specificity of a change in CoMiSS score from baseline to 2 weeks after commencing a hypoallergenic diet, using a single-blinded oral food challenge (OFC) with cow’s milk-based formula as the diagnostic reference. [1] While a double-blind, placebo-controlled food challenge (DBPCFC) is considered superior to an OFC [4], the difference in validity mainly applies to older children and adults where subjective symptoms are more prevalent. [5, 6] In infants under 2 years of age, an OFC based on objective symptoms and protocolised outcome criteria is an accepted approach which minimises the burden for infants. [7] A DBPCFC would require 2 challenges while not necessarily providing additional clinical accuracy in this age group. For example, in the EuroPrevall study of infants with probable CMPA a correct diagnosis of non-IgE-mediated CMPA was not adequately captured despite use of the DBPCFC in the diagnostic process. [8, 9] Weighing up these factors, we have opted to include an OFC in the current study design which is in line with current clinical guidelines. [7]
As any clinical tool, CoMiSS has its limitations. During the design process, several clinical conditions were deliberately not included. For example, rectal bleeding as a mono-symptomatic presentation was omitted as it should prompt an urgent paediatric assessment. Similarly, growth failure in early infancy requires a specialist referral for a more detailed assessment, and CoMiSS was not deemed useful in this scenario. Furthermore, as CoMiSS was not designed as a diagnostic tool it cannot provide a differential diagnosis between CMPA and other common paediatric conditions. However, it may prompt the primary health care provider to consider CMPA in the diagnostic process.
At the time of study conception, CoMiSS was already a widely used clinical tool and modifications of the score for the purposes of a validation study were not practical. The validation study was therefore designed to gain further insights into CoMiSS in its current form as a possible diagnostic tool, if interpreted in conjunction with the clinical response to a trial of a CMP-free elimination diet. The main hypothesis of the trial is that a significant fall in CoMiSS, in parallel with improved symptoms on a CMP-free diet, would support a diagnosis of CMPA. As a secondary endpoint, the usefulness of the baseline CoMiSS as a diagnostic marker for CMPA will also be assessed. Finally, the relative contribution of each individual score component in identifying infants with CMPA will be assessed. Once these data are available, informed decisions can be made on possible refinements and design modifications of the CoMiSS tool.
Authors:
1. Yvan Vandenplas, Vrije Unversiteit Brussel, Brussels, Belgium
2. Christophe Dupont, Université Paris Descartes, Paris, France
3. Philippe Eigenmann, University Hospitals of Geneva, Geneva, Switzerland
4. Arne Høst, Odense University Hospital, Odense, Denmark
5. Mikael Kuitunen, University of Helsinki, Helsinki, Finland
6. Carmen Ribes-Koninkx, La Fe University Hospital, Valencia, Spain
7. Neil Shah, Great Ormond Street Hospital for Children, London, United Kingdom
8. Hania Szajewska, The Medical University of Warsaw, Warsaw, Poland
9. Andrea von Berg, Marien-Hospital, Wesel, Germany
10. Ralf G Heine, Nestlé Health Science, Vevey, Switzerland
REFERENCES
1. Vandenplas Y, Mukherjee R, Dupont C, Eigenmann P, Høst A, Kuitunen M, Ribes-Koninkx C, Shah N, Szajewska H, von Berg A, Heine RG, Zhao ZY, on behalf the Chinese CoMiSS Investigator Team. Protocol for the validation of sensitivity and specificity of the Cow's Milk-related Symptom Score (CoMiSS) against open food challenge in a single-blinded, prospective, multicentre trial in infants. BMJ Open 2018;8:e019968.
2. Vandenplas Y, Dupont C, Eigenmann P, Host A, Kuitunen M, Ribes-Koninckx C, Shah N, Shamir R, Staiano A, Szajewska H, Von Berg A. A workshop report on the development of the Cow's Milk-related Symptom Score awareness tool for young children. Acta Paediatr 2015;104:334-9.
3. Vandenplas Y, Steenhout P, Järvi A, Garreau AS, Mukherjee R. Pooled Analysis of the Cow's Milk-related-Symptom-Score (CoMiSS) as a Predictor for Cow's Milk Related Symptoms. Pediatr Gastroenterol Hepatol Nutr 2017;20:22-26.
4. Sampson HA, Gerth van Wijk R, Bindslev-Jensen C, Sicherer S, Teuber SS, Burks AW, Dubois AE, Beyer K, Eigenmann PA, Spergel JM, Werfel T, Chinchilli VM. Standardizing double-blind, placebo-controlled oral food challenges: American Academy of Allergy, Asthma & Immunology-European Academy of Allergy and Clinical Immunology PRACTALL consensus report. J Allergy Clin Immunol 2012;130:1260-74.
5. Koletzko S, Niggemann B, Arato A, Dias JA, Heuschkel R, Husby S, Mearin ML, Papadopoulou A, Ruemmele FM, Staiano A, Schäppi MG, Vandenplas Y, European Society of Pediatric Gastroenterology Hepatology and Nutrition. Diagnostic approach and management of cow's-milk protein allergy in infants and children: ESPGHAN GI Committee practical guidelines. J Pediatr Gastroenterol Nutr 2012;55:221-9.
6. Muraro A, Werfel T, Hoffmann-Sommergruber K, Roberts G, Beyer K, Bindslev-Jensen C, Cardona V, Dubois A, duToit G, Eigenmann P, Fernandez Rivas M, Halken S, Hickstein L, Høst A, Knol E, Lack G, Marchisotto MJ, Niggemann B, Nwaru BI, Papadopoulos NG, Poulsen LK, Santos AF, Skypala I, Schoepfer A, Van Ree R, Venter C, Worm M, Vlieg-Boerstra B, Panesar S, de Silva D, Soares-Weiser K, Sheikh A, Ballmer-Weber BK, Nilsson C, de Jong NW, Akdis CA; EAACI Food Allergy and Anaphylaxis Guidelines Group. EAACI food allergy and anaphylaxis guidelines: diagnosis and management of food allergy. Allergy 2014;69:1008-25.
7. Fiocchi A, Brozek J, Schünemann H, Bahna SL, von Berg A, Beyer K, Bozzola M, Bradsher J, Compalati E, Ebisawa M, Guzman MA, Li H, Heine RG, Keith P, Lack G, Landi M, Martelli A, Rancé F, Sampson H, Stein A, Terracciano L, Vieths S. World Allergy Organization (WAO) Diagnosis and Rationale for Action against Cow's Milk Allergy (DRACMA) Guidelines. World Allergy Organ J 2010;3:57-161.
8. Schoemaker AA, Sprikkelman AB, Grimshaw KE, Roberts G, Grabenhenrich L, Rosenfeld L, Siegert S, Dubakiene R, Rudzeviciene O, Reche M, Fiandor A, Papadopoulos NG, Malamitsi-Puchner A, Fiocchi A, Dahdah L, Sigurdardottir ST, Clausen M, Stanczyk-Przyluska A, Zeman K, Mills EN, McBride D, Keil T, Beyer K. Incidence and natural history of challenge-proven cow's milk allergy in European children - EuroPrevall birth cohort. Allergy 2015;70:963-72.
9. Koletzko S, Heine RG. Non-IgE mediated cow's milk allergy in EuroPrevall. Allergy 2015;70:1679-80.
In the south Brazilian city of Porto Alegre - where about 1.5 million people live – two primary care units added out-of-hours (OOH) services in March and April of 2017 to increase patient access, as described by Kelly et al(1,2). OOH appointments in these two facilities are offered from 6 to 10pm. Differently from the traditional primary care model in Brazil, in which units serve people living in a specific catchment area with pre-scheduled appointments, during OHH all patients are welcomed without the need for scheduling. Medical, nursing, and dental care, as well as immunizations, are offered.
In the first 14 months since the start of the program, we recorded 33,964 appointments in the two clinics during OOH. During the same period, 51,181 consultations were provided in the same two units during traditional hours (and about 1.7 million primary care consultations in the city). Even though women are the most frequent users of both daytime and OOH services, the proportion of men ( 35.5% vs. 32.4% during the day) and of people of working age (54.96% vs. 47,7% during the day) was higher in OOH vs. regular daytime services.
However, the most interesting aspect of OOH services is the high number of rapid testing for HIV, syphilis, and hepatitis performed during these extended hours: 3,868 tests during OOH vs. 3,405 tests during usual hours – or 11.3% of the overall appointments during OOH vs. 6.6% during regular hours. Since the regular shift lasts 11 hours (...
In the south Brazilian city of Porto Alegre - where about 1.5 million people live – two primary care units added out-of-hours (OOH) services in March and April of 2017 to increase patient access, as described by Kelly et al(1,2). OOH appointments in these two facilities are offered from 6 to 10pm. Differently from the traditional primary care model in Brazil, in which units serve people living in a specific catchment area with pre-scheduled appointments, during OHH all patients are welcomed without the need for scheduling. Medical, nursing, and dental care, as well as immunizations, are offered.
In the first 14 months since the start of the program, we recorded 33,964 appointments in the two clinics during OOH. During the same period, 51,181 consultations were provided in the same two units during traditional hours (and about 1.7 million primary care consultations in the city). Even though women are the most frequent users of both daytime and OOH services, the proportion of men ( 35.5% vs. 32.4% during the day) and of people of working age (54.96% vs. 47,7% during the day) was higher in OOH vs. regular daytime services.
However, the most interesting aspect of OOH services is the high number of rapid testing for HIV, syphilis, and hepatitis performed during these extended hours: 3,868 tests during OOH vs. 3,405 tests during usual hours – or 11.3% of the overall appointments during OOH vs. 6.6% during regular hours. Since the regular shift lasts 11 hours (from 7am to 6pm) and OOH lasts 4 hours, this also shows that rapid testing was performed twice as fast during OOH. Therefore, it seems that there is a specific demand for rapid testing during OOH, suggesting an important role of these extended hours managing communicable diseases, still a very serious problem in our country(3).
Another important difference is OOH resolvability. While in the traditional format the number of consultations that result in a referral to a medical specialty is on average 16.9%, in OOH this number is 10.64%.
As reported by Kelly et al.(1), implementation of the OOH strategy in Porto Alegre aimed to attract to the primary care setting those people who sought emergency departments without needing urgent care. Not many primary care facilities offer extended hours in Brazil, and thus we need to keep monitoring our OOH units. However, our goal is to have eight working OOH units (one per district of the municipality) by the year 2020 (a third unit was added in June 2018), increasing access and strengthening primary health care in our city.
References
1. Kelly SJ, Piercy H, Ibbotson R, Fowler Davis SV. Who attends out-of-hours general practice appointments? Analysis of a patient cohort accessing new out-of-hours units. BMJ Open. 2018 Jun 9;8(6):e020308. doi: 10.1136/bmjopen-2017-020308.
2. Harzheim E, Marchezan Júnior N, Pfeil JN, Molina-Bastos CG, Frank T, Nascimento DM, Sturmer PL. Extending primary care hours in Brazil as health policy [comment]. Available at https://www.bmj.com/content/357/bmj.j2142/rr
3. Souza MFM, França EB, Cavalcante A. Burden of disease and health situation analysis: results of the Global Burden of Disease (GBD) Brazil network. Rev Bras Epidemiol. 2017 May;20Suppl 01(Suppl 01):1-3. doi: 10.1590/1980-5497201700050001.
Braithwaite et al. [1] claim that kanuka honey is an effective
treatment of rosacea. Both their study medication as well as their study
design and statistical analysis are seriously flawed and do not allow this
conclusion. First, the study medication is inappropriate to show an effect
of kanuka honey. The verum was not kanuka honey (as the title suggests),
but a mixture of kanuka honey and 10% glycerine (Honevo). As control...
Braithwaite et al. [1] claim that kanuka honey is an effective
treatment of rosacea. Both their study medication as well as their study
design and statistical analysis are seriously flawed and do not allow this
conclusion. First, the study medication is inappropriate to show an effect
of kanuka honey. The verum was not kanuka honey (as the title suggests),
but a mixture of kanuka honey and 10% glycerine (Honevo). As control, an
entirely different preparation was chosen, i.e. Cetomacrogol a mitigating
and protective cream containing macrogol cetostearyl ether 22, cetiol,
sorbitol and the preservative ascorbic acid, used for dry and sensitive
skin, eczema and itching. This induced an obvious risk of bias and lack of
blinding. Moreover, if the specific effects of kanuka honey (versus
another honey) were to be tested, a different honey should have been
tested with the same amount of glycerine. Second, the study would have
been powered for a 25% response in the control and a 50% response in the
Honevo group. However, the observed effect sizes were much lower (17.4%
and 34,3%, respectively). Even if subsequently p-values less than 0.05 were calculated, no conclusions can be drawn due to this a priori lack of
statistical power. Thus, both the title and the conclusion of the
publication are not justified by the study design and data.
References
1. Braithwaite I, Hunt A, Riley J, et al. Randomised controlled trial
of topical kanuka honey for the treatment of rosacea. BMJ Open
2015;5:e007651. doi:10.1136/bmjopen-2015-007651
We welcome this systematic review on interventions to improve the
governance of patient safety in emergency care settings. This work
highlights the value of information contained in adverse events derived
from medical record review and incident reporting systems. The authors
identified four qualities shared by effective incident reporting systems:
(i) staff education on the importance and learning purpose of reporting,...
We welcome this systematic review on interventions to improve the
governance of patient safety in emergency care settings. This work
highlights the value of information contained in adverse events derived
from medical record review and incident reporting systems. The authors
identified four qualities shared by effective incident reporting systems:
(i) staff education on the importance and learning purpose of reporting,
(ii) multiple and constantly available reporting options, (iii) a short
reporting form to minimise burden, and (iv) structural feedback by
presenting descriptive statistics, findings of incident root-cause
analyses and improvement actions (p. 10, [1]).
Our experience of implementing an Emergency Medicine-specific
incident reporting system for use across Australasian hospitals supports
these findings. The Emergency Medicine Events Register has supported site
champions at participating hospitals to educate fellow staff about
incident reporting and provides an on-line (www.emer.org.au), streamlined
data collection form tailored to emergency medicine, and automated
feedback including descriptive summaries of incidents [2]. Reporting an
incident takes just under 6 minutes. Other measures were taken to support
reporting, such as the ability to claim continuing professional
development points and statutory protection of the data from discovery
(qualified privilege) [2].
The Emergency Medicine Events Register is a voluntary, anonymous
system that is managed by an independent third party (the Australian
Patient Safety Foundation). The decision for anonymity and independent
management was made to enhance system uptake by doctors. Lack of medical
engagement in incident reporting is often ascribed to fear and mistrust of
incident reporting systems and their misuse by hospital administrators and
regulators [3] and a perception that error is inevitable and should be
managed 'in-house' or by self-regulation [4]. Hesselink et al's 2016
review also supported our model, with the proviso that implementation of a
non-anonymous peer review process had improved reporting rates in one
study [5]. Hesselink et al (2016) concluded that "anonymity of reporting
and management of incident reports by an independent third party may not
be necessary if an incident reporting and peer review process is perceived
to be safe" (p. 10).
We agree with this proposition but believe that, mostly, doctors'
perceptions are that incident reporting is not yet considered safe, and
therefore that anonymity is an essential element to an incident reporting
system. Anonymity imposes limitations on the type of feedback that can be
provided - it may only be generic, such as case studies [6]. However,
anonymous systems can support hospital-based incident reporting systems
that have a strong focus on accountability (as opposed to system learning)
and, in general, poor uptake by doctors [7,8].
Meaningful engagement of doctors has been identified as one of three
keys to unlocking the full potential of incident reporting systems [9].
Ownership of patient safety concerns by specialties, such as Emergency
Medicine, is a necessary step to developing sustainable system-wide
improvements. Specialty led systems that engage trainees and fellows in
data collection and analysis, are integrated with professional
organisations and their related activities (such as training and
education) and are supported by third party patient safety organisations,
can promote national and international patient safety agendas [10].
However, the barriers to doctors reporting incidents are significant.
Emergency medicine-specific barriers to incident reporting include:
difficulties in training all staff, the need for one-on-one support for
site champions, perceived duplication with hospital-based systems and a
view that reporting was only a consultant-level activity [2]. Such
barriers inform the continued development of the system. One such recent
development in the Emergency Medicine Events Register has been an
expansion in scope that now allows for consumer reporting [11].
References
1. Hesselink G, Berben S, Beune T, et al. Improving the governance of
patient safety in emergency care: a systematic review of interventions.
BMJ Open, 2016;6:e009837.
2. Schultz TJ, Crock C, Hansen K, et al. Piloting an online incident
reporting system in Australasian emergency medicine. Emerg Med Australas,
2014;26:461-7.
3. Waring JJ. Beyond blame: cultural barriers to medical incident
reporting. Soc Sci Med, 2005;60:1927-35.
4. Rosenthal M. How doctors think about medical mishaps. In:
Rosenthal M, Mulcahy L, Lloyd-Bostock S, eds. Medical Mishaps. Buckingham:
: Open University Press. 1999. 141-53.
5. Reznek MA, Barton BA. Improved incident reporting following the
implementation of a standardized emergency department peer review process.
Int J Qual Health Care, 2014;26:278-86.
6. Deakin A, Schultz TJ, Hansen K, et al. Diagnostic error: Missed
fractures in emergency medicine. EMA - Emerg Med Australas, 2015;27:177-8.
7. Hobgood C, Weiner B, Tamayo-Sarver JH. Medical error
identification, disclosure, and reporting: Do emergency medicine provider
groups differ? Acad Emerg Med, 2006;13:443-51.
8. Rowin EJ, Lucier D, Pauker SG, et al. Does error and adverse event
reporting by physicians and nurses differ? Jt Comm J Qual Patient Saf,
2008;34:537-45.
9. Mitchell I, Schuster A, Smith K, et al. Patient safety reporting:
a qualitative study of thoughts and perceptions of experts 15 years after
'To Err is Human'. BMJ Qual Saf, 2015, published online first: doi:10.1136/bmjqs-2015-004405
10. Jones DN, Benveniste KA, Schultz TJ, et al. Establishing national
medical imaging incident reporting systems: issues and challenges. J Am
Coll Radiol, 2010;7:582-92.
11. Emergency Medicine Events Register. Consumer Report.
2016.http://www.emer.org.au/consumer-report.html (accessed 23rd March 2016).
Is there an association between church membership and lower sexually
transmitted disease (STD) incident among selected Danish religious
cohorts? A large body of literature has shown a positive link between
religion and health, particularly it's role in reducing lifetime risk for
cardiovascular disease and selected cancers (1, 2). However, little
research has looked at the effects of religion on sexual behavior and
incide...
Is there an association between church membership and lower sexually
transmitted disease (STD) incident among selected Danish religious
cohorts? A large body of literature has shown a positive link between
religion and health, particularly it's role in reducing lifetime risk for
cardiovascular disease and selected cancers (1, 2). However, little
research has looked at the effects of religion on sexual behavior and
incidence of STDs.
The authors in this study sought to investigate this association
among Danish Seventh Day Adventists (SDAs) and Baptists. They
hypothesized that religious societies in the selected Danish cohorts will
have a lower incidence of STDs. Choosing a retrospective cohort design,
the authors followed two selected cohorts from 1977 to 2009 by linking
national registers of inpatient and outpatient care. They then compared
STDs incidents for syphilis, gonorrhea and chlamydia among the selected
cohorts and the general Danish population. The results showed lower
incidence of STDs were observed for both cohorts.
The study selected a large Danish religious cohort of 3,119 SDAs and
1,856 Baptists, setting it apart from other similar studies with small
sample sizes (3, 4, 5, 6). Furthermore, the study took necessary steps to
ensure that identification of the selected cohort was done correctly and
that the study represented the Danish population. However, the study has a
few limitations that need to be addressed. Most of the study data was
derived from the national patient register and unfortunately these records
were not designed for meeting the specific objectives of the study;
therefore, it is very possible that some of the variables needed for the
study were not made available in the records. Furthermore, the patient
recorded data may have been completed by different health care professions
and therefore prone to consistency and/or inaccuracy. Additionally, the
article mentions that only 26 out of 46 Baptist communities took part in
the study making these individual prone to selection bias.
It is also not clear why the authors chose to observe the STDs
syphilis, gonorrhea and chlamydia. Were these the most frequently reported
STDs during this time or were they selected based on available data from
the national patient registers? What about other STDs such as HPV
(discussed in article conclusion but not part of study analysis), genital
herpes and HIV? The authors also chose to adjust for age and gender which
are important confounders but many other external factors (race,
individual sexual beliefs and attitudes, friends, family and peer
influences, social norms and culture, etc) were not taken into
consideration. Lastly, church membership and/or affiliation tell us very
little about how religious a person is and may therefore provide very
little information regarding the link between religion and STD incident. There is a need to further examine the link between religion and STDs
incidents by possibly conducting a prospective cohort study that looks at
the association between religious practice (church attendance,
participation/activity), attitudes, and beliefs, and how these relate to
STDs incident.
References
1. Wallace JM, Forman TA. Religion's role in promoting health and
reducing risk among American youth. Health Educ Behav, 1998;25:721-41.
2. Krup et al. Association between sexually transmitted disease and
church membership. A retrospective cohort study of two Danish religious
minorities. BMJ Open, 2016;6:10128
3. Landor A, Simons LG, Simons RL, Brody GH,
Gibbons FX. The Role of Religiosity in the Relationship Between
Parents, Peers, and Adolescent Risky Sexual Behavior. Journal of Youth and
Adolescence, 2011;40(3), 296-309.
4. Rostosky SS, Wilcox BL, Wright MLC, et al. The impact of
religiosity on adolescent sexual behavior: a review of the evidence. J
Adolesc Res, 2004;19:677-97.
5. McCree DH, Wingood GM, DiClemente R, et al. Religiosity and risky
sexual behavior in African-American adolescent females. J Adolesc Health,
2003;33:2-8.
6. Lefkowitz ES, Gillen MM, Shearer CL, et al. Religiosity, sexual
behaviors, and sexual attitudes during emerging adulthood. J Sex Res,
2004;41:150-9.
We strongly endorse the conclusion of Cornelius et al. regarding the necessity
for improved communication of benefit-risk information to medication
prescribers and users. Based on our extensive experience in evaluation
and communication of medication-associated risks, we have previously
proposed improvements in the processes for selecting and effectively
communicating adverse drug reactions and interactions in HPI (1-3)....
We strongly endorse the conclusion of Cornelius et al. regarding the necessity
for improved communication of benefit-risk information to medication
prescribers and users. Based on our extensive experience in evaluation
and communication of medication-associated risks, we have previously
proposed improvements in the processes for selecting and effectively
communicating adverse drug reactions and interactions in HPI (1-3).
In our view, their article exemplifies the opacity underlying risk
information in current regulatory Healthcare Professional Information
(HPI) and points to the necessity for its readers to be familiar with past
and present regional standards and approaches for its creation, especially
when comparing HPI across regions. We would like to draw attention to
potential additional reasons for the observed adverse reaction profile
differences, and to the need their analysis illustrates for an
understanding of the specialized rules, terminologies, and other aspects
of HPI creation to avoid misinterpreting its content.
The decision whether to list an undesirable event as an ADR in HPI
("ADR for the purposes of HPI") involves judgement about both the strength
of evidence for a causal association with the drug and the significance of
that potential ADR for the prescribing decision. In general, the causal
evidence threshold is not fixed, but rather adjusted according to the
clinical importance of the event, being lower for those with greater
potential for severe harm to the patient. However, we do not believe this
to be a major cause of the substantial differences in ADR enumeration
between US and EU HPI in the subset described. Rather, given the age
range of the products studied, it appears that these primarily reflect
differences between the EU approach to HPI of including only ADRs (i.e. those with evidence supporting a causal association) vs. the then-
prevalent US approach of including both ADRs and adverse events with only
a temporal relationship to drug, which resulted in the inclusion in US HPI
of many reported adverse events, especially from clinical trials, without a reasonable causal association with drug. In 2006, FDA ended this decades-long practice and once again began to require that ADRs meet the
"reasonable causal association" evidentiary standard to be included in
HPI.
Older US HPIs in particular may also have used the term "adverse
event" to encompass both actual "adverse reactions" and "adverse events
without reasonable causal association". Furthermore, the commonly seen
explanation for displaying adverse reactions/events as being those
occurring in e.g. ? 5% of patients and more frequently than with placebo
does not necessarily represent a criterion for categorising an event as an
ADR, but may simply reflect the criterion for including it in that table
or list. More specific criteria, such as "reported in ? 5% and at least
twice as frequently than with placebo" represent surrogate causality
criteria.
The degree of confidence regarding an ADR's causal association is
only exceptionally described in HPI, and it is often unclear whether or to
what extent listings may contain items other than "ADRs for the purposes
of HPI". It should also be noted that HPI readers unfamiliar with
pharmaceutical regulatory definitions may not appreciate that terms such
as adverse reaction, suspected adverse reaction, etc. may have markedly
different meanings in different contexts, e.g. for obligatory reporting of
adverse events to regulatory authorities vs. for informing the medical
community or the public. For example, an event from a clinical trial that
is deemed a "suspected adverse reaction" (SAR) for the purposes of
mandatory regulatory reporting may not meet criteria for inclusion in HPI.
Given the foregoing and other limitations of the safety information
presented in the HPI, it is essential to consult sources other than HPI
for additional information and interpretation regarding actual and
potential risks. For example, several regulatory authorities have begun
publishing detailed "assessment reports" on the internet. In those
jurisdictions, more recently created regulatory product information thus
consists of 3 layers: patient-directed information; "traditional" HPI
(such as the EU SmPC and US PI); and the more detailed information in
public assessment reports.
Finally, the following caveats must always be borne in mind when
assessing the benefits and risks of medicines. For a new medicine, it is
essential to understand that the safety profile in the HPI is purely
provisional, as only the most common adverse reactions have any reasonable
likelihood of detection in clinical trials. The totality of risks,
especially uncommon ones, can only be ascertained after exposure of much
larger patient populations with clinical characteristics that often differ
from those in clinical trials, creating an additional challenge for
assessing causality with the typically limited case data available. In
both clinical trial and licensed clinical use, there is almost always a
significant "background noise" of undesirable occurrences that occur in
the target patient population and are unrelated to the drug. These events
typically account for the bulk of reported safety data, creating a "needle
-in-the-haystack" situation that may obscure the smaller subset of
attributable harms (ADRs). Various informatics methods have been developed
to address this issue, but clinical judgement remains the final
determinant of causality and hence HPI inclusion.
Given the importance of presenting clear, accurate, and complete
safety information to all medication users, it is disappointing that the
international harmonisation of many aspects of drug development and
licensing achieved during the past 25+ years remains elusive for this
critical regulatory function.
References
Drazen J et al. Institute of Medicine (IOM) 2007. Adverse Drug Event
Reporting: The Roles of Consumers and Health-Care Professionals, Workshop
Summary. Washington, DC: The National Academies Press.
Fontaine AL. Current Requirements and Emerging Trends for Labelling as a
Tool for Communicating Pharmacovigilance Findings. Drug Safety, 2004; 27:
579-589
Kahn SN. Signal Detection and Evaluation in Pharmacovigilance: Analytical
and Regulatory Aspects. Regulatory Affairs Focus, 2006; 11(4); 14-19.
Messrs Schmidt and Schmidt, in their letter published in BMJ Open on
24 March 2016, assert that the study we reported about the effects of
Kanuka honey in rosacea has a number of important flaws including problems
with the experimental treatments, study design, and statistical analysis.
With respect to the study medications the aim of the study was not to
estimate the effect of a preparation of Kanuka honey with...
Messrs Schmidt and Schmidt, in their letter published in BMJ Open on
24 March 2016, assert that the study we reported about the effects of
Kanuka honey in rosacea has a number of important flaws including problems
with the experimental treatments, study design, and statistical analysis.
With respect to the study medications the aim of the study was not to
estimate the effect of a preparation of Kanuka honey with another honey
but to compare the properties of a honey treatment for rosacea against a
control skin treatment. In this respect, the choice of treatment arms is
clearly appropriate. Glycerine as an excipient, a pharmacologically
inactive component, in the Kanuka honey preparation allows both ease of
administration to the skin and also prolonged contact of the Kanuka honey
with the skin. The use of excipients for this purpose in dermatological
products is common. The control arm was Cetomacrogol cream. As outlined in
the paper this is a non-ionic compound which itself comprises a number of
excipients and which is also often used as a delivery agent for other
dermatological agents. The study therefore compares, as much as possible,
the potential active properties of Kanuka honey with a dermatologically
neutral agent.
We agree that it is not possible to mask participants as to the
active or control therapies in a study of this nature, due to the
characteristic smell and taste of honey, and that this could potentially
lead to bias in the estimates of treatment effects. We have not only
highlighted this potential bias in the study report but have also been
scrupulous in other aspects of the study design such as masked
randomisation and masked assessment of the primary outcome variable of
rosacea severity. This was achieved through the same masked investigator
making the visual assessments of rosacea severity without any
communication with the study participants.
With regard to the comment about statistical power, the paper
presents a point estimate and confidence interval for the difference
between the active and comparator arms of the trial and because the lower
limit of the confidence interval is above the null, the issue of Type II
error does not arise. Readers can judge for themselves the relevance of
the lower or upper confidence limits of the difference. We did not specify
or set out to carry out a Bayesian analysis of the experimental result in
light of the anticipated rates of response. Even if we had done so, the
point estimate of the relative response achieved, a relative risk of 2,
was actually identical to that anticipated in the power calculation. We
further note that the point estimate and confidence interval for the
response rate in the active treatment group, 24/68, is 35.5% (24.1 to
47.8); and in the comparator group 12/69, it is 17.4% (9.3 to 28.4). Both
of these are concordant with the response rates anticipated in the sample
size calculation of 50% and 25% respectively.
Conflict of Interest:
The authors of this letter undertook the kanuka honey study, which is the subject of this correspondence. The study was funded by HoneyLab, who supplied the Kanuka honey product (Honevo).
We have read with great interest the article “What insights do patients and caregivers have on acute kidney injury and post-hospitalisation care? A single-center qualitative study from Toronto, Canada” by Silver et al.
The article sheds light on the fact that most of its participants prioritised chronic conditions that ‘progress over time’ over AKI. These co-morbidities often include heart failure, hypertension and Type 2 diabetes mellitus, which are treated with non-steroidal anti-inflammatory drugs, diuretics and metformin respectively( 1). There is considerable data that these drugs are nephrotoxic and should therefore, be deprescribed or temporarily with-held or dose-adjusted in patients with AKI.
However, from unpublished research at our hospital, there is often reluctance to stop these drugs, suggesting that this misconception may be shared by physicians as well. As this is a common clinical problem with considerable morbidity, Think Kidneys Campaign (NHS collaboration of various trusts) have developed a checklist for medication optimization in patients with AKI (2), but its use is sparse at least from our experience.
Studies have shown that in-hospital mortality due to AKI far exceeds that due to these long-standing, chronic conditions. In a large nation-based study, in-hospital mortality of AKI was found to be 12.32% (3), with an increase in number of absolute deaths from 2001 to 2011. This is in contrast to the 3% in-...
We have read with great interest the article “What insights do patients and caregivers have on acute kidney injury and post-hospitalisation care? A single-center qualitative study from Toronto, Canada” by Silver et al.
The article sheds light on the fact that most of its participants prioritised chronic conditions that ‘progress over time’ over AKI. These co-morbidities often include heart failure, hypertension and Type 2 diabetes mellitus, which are treated with non-steroidal anti-inflammatory drugs, diuretics and metformin respectively( 1). There is considerable data that these drugs are nephrotoxic and should therefore, be deprescribed or temporarily with-held or dose-adjusted in patients with AKI.
However, from unpublished research at our hospital, there is often reluctance to stop these drugs, suggesting that this misconception may be shared by physicians as well. As this is a common clinical problem with considerable morbidity, Think Kidneys Campaign (NHS collaboration of various trusts) have developed a checklist for medication optimization in patients with AKI (2), but its use is sparse at least from our experience.
Studies have shown that in-hospital mortality due to AKI far exceeds that due to these long-standing, chronic conditions. In a large nation-based study, in-hospital mortality of AKI was found to be 12.32% (3), with an increase in number of absolute deaths from 2001 to 2011. This is in contrast to the 3% in-hospital mortality of T2DM (4). Hence, it only seems logical that immediate and effective treatment of AKI take precedence over treatment of chronic, stable conditions in the short-term.
In summary, we believe that the finding of your study that AKI is a ‘low-priority concern’ holds true not just for patients, but potentially also for physicians, and therefore, does have considerable consequences in terms of safe prescribing.
1. Farooqi S, Dickhout JG. Major comorbid disease processes associated with increased incidence of acute kidney injury. World Journal of Nephrology 2016; 5(2): 139–146.
2. Griffith K, Ashley C, Blakeman T, Fluck R, Lewington A, Selby N, et al. ‘Sick day’ guidance in patients at risk of Acute Kidney Injury: an Interim Position Statement from the Think Kidneys Board 2015.
3. Chertow GM, Burdick E, Honour M et al. Acute Kidney Injury, Mortality, Length of Stay, and Costs in Hospitalized Patients. Journal of the American Society of Nephrolog 2005; 16(11): 3365-3370
4. Hyperglycemia: An Independent Marker of In-Hospital Mortality in Patients with Undiagnosed Diabetes. The Journal of Clinical Endocrinology & Metabolism 2002, 87(3): 978–982
In the UK I have observed that the number of cataract operations is undulating in parallel with changes in the number of deaths. The number of deaths is showing highly unusual on/off switching which may arise from a hidden infectious outbreak. This on/off switching can be revealed by applying a rolling 12-month total to your data. Are you able to apply such a rolling total to your data to see if there are any hidden patterns in the trends? A list of publications regarding on/of switching in deaths and medical conditions can be found at http://www.hcaf.biz/2010/Publications_Full.pdf I hope that this will prove to be helpful. Kind Regards
To the editor:
We read with interest the protocol for the validation of the Cow’s Milk-related Symptom Score (CoMiSS) against open food challenge by Vandenplas et al.(1) They have proposed this symptom score as a resource for primary healthcare providers, aiming to increase awareness of cow’s milk allergy (CMA)-related symptoms to facilitate an earlier diagnosis. The score was developed to increase awareness of mainly non-IgE mediated CMA, a disorder of increasing interest in recent literature publications.(2,3) However, dysmotility, dysmotility-related symptoms, blood in stool and failure to thrive (weight ≤2.5 Z-scores, drift from growth curve of >2 percentiles over 6 months or weight-for-length -2 Z-scores)(4) are excluded from the score. These are considered key features in non-IgE mediated allergy.(5) In contrast, severe respiratory symptoms and urticaria are included in the score. In our opinion, using the CoMiSS for diagnosing CMA will therefore hamper diagnosing non-IgE mediated CMA. The authors stress that the design of this study might not enable a reliable distinguishment between IgE-mediated CMA, non-IgE mediated CMA and non-allergic cow’s milk-related symptoms. In addition, in our experience we believe that healthcare professionals might even get confused in the differential diagnosis regarding non-allergic cow’s milk-related symptoms. For example pyloric stenosis, gastroesophageal reflux disease, hemorrhagic disease of the newborn or infective gastro...
To the editor:
We read with interest the protocol for the validation of the Cow’s Milk-related Symptom Score (CoMiSS) against open food challenge by Vandenplas et al.(1) They have proposed this symptom score as a resource for primary healthcare providers, aiming to increase awareness of cow’s milk allergy (CMA)-related symptoms to facilitate an earlier diagnosis. The score was developed to increase awareness of mainly non-IgE mediated CMA, a disorder of increasing interest in recent literature publications.(2,3) However, dysmotility, dysmotility-related symptoms, blood in stool and failure to thrive (weight ≤2.5 Z-scores, drift from growth curve of >2 percentiles over 6 months or weight-for-length -2 Z-scores)(4) are excluded from the score. These are considered key features in non-IgE mediated allergy.(5) In contrast, severe respiratory symptoms and urticaria are included in the score. In our opinion, using the CoMiSS for diagnosing CMA will therefore hamper diagnosing non-IgE mediated CMA. The authors stress that the design of this study might not enable a reliable distinguishment between IgE-mediated CMA, non-IgE mediated CMA and non-allergic cow’s milk-related symptoms. In addition, in our experience we believe that healthcare professionals might even get confused in the differential diagnosis regarding non-allergic cow’s milk-related symptoms. For example pyloric stenosis, gastroesophageal reflux disease, hemorrhagic disease of the newborn or infective gastroenteritis all of which cannot be distinguished by this current published score.
The authors indicate that they based the trial design on current best clinical practice. That is, in their opinion, a cow’s milk protein-free elimination diet followed by an open food challenge (OFC).(6) However, in current clinical guidelines, double blind placebo controlled challenges (DBPCFC) are considered the gold standard to prove or exclude allergy.(5,7,8) Although the authors point out this limitation themselves, it remains unclear why the OFC was chosen over the DBPCFC. To solve this, they single-blinded the study so OFC and CoMiSS results will not be determined by the same investigator. This, however, causes a substantial risk of inter-observer variability, especially regarding atopic dermatitis.(9) Since the primary study objective is to validate the difference in CoMiSS before and after treatment with an amino-acid based formula against the OFC as a predictor of CMA, this inter-observer variability is undesirable. Of further importance, the score will not be validated against the gold standard DBPCFC. The secondary study objectives are to determine the sensitivity and specificity of the change in CoMiSS and to investigate whether the CoMiSS at baseline can predict the OFC outcome. Since the primary objective is to validate the score to the OFC outcome, it seems contradictory to investigate whether the score can predict the OFC outcome as a secondary objective.
We congratulate the authors in their development of a symptom score as a clinical resource for primary healthcare providers in this allergic disorder of infancy and childhood. However, the symptom score appears to have been validated in paediatric departments in secondary and tertiary care institutions against an inadequate reference standard. The primary healthcare providers should be educated, not only in paediatrics and allergy, but also in using the score, and be able to subsequently educate the patients’ parents. The eventual conversion of these results to primary healthcare and the possible increased awareness of non-IgE mediated CMA therefore remain major sources of concern and perhaps further development of this symptom score is required.
References
1. Vandenplas, Y. et al. Protocol for the validation of sensitivity and specificity of the Cow’ s Milk-related Symptom Score (CoMiSS) against open food challenge in a single-blinded, prospective, multicentre trial in infants. BMJ Open 8, (2018).
2. Candy, D. C. A. et al. A synbiotic-containing amino acid-based formula improves gut microbiota in non-IgE-mediated allergic infants. Pediatr. Res. 0, 1–10 (2017).
3. Berni Canani, R. et al. Formula selection for management of children with cow’s milk allergy influences the rate of acquisition of tolerance: A prospective multicenter study. J. Pediatr. 163, 771–777.e1 (2013).
4. van den Elzen, A., Sibbles, B. & Niewenhuis, E. Failure to thrive: van symptoom naar diagnose. Prakt. Pediatr. 48–52 (2007).
5. Venter, C. et al. Better recognition, diagnosis and management of non-IgE-mediated cow’s milk allergy in infancy: iMAP—an international interpretation of the MAP (Milk Allergy in Primary Care) guideline. Clin. Transl. Allergy 7, 26 (2017).
6. Koletzko, S. et al. Diagnostic Approach and Management of Cowʼs-Milk Protein Allergy in Infants and Children. J. Pediatr. Gastroenterol. Nutr. 55, 221–229 (2012).
7. Muraro, a. et al. EAACI Food Allergy and Anaphylaxis Guidelines: Diagnosis and management of food allergy. Allergy Eur. J. Allergy Clin. Immunol. 69, 1008–1025 (2014).
8. Richtlijn Diagnostiek van Koemelkallergie bij Kinderen in Nederland. NVK (2012).
9. Schmitt, J. et al. Assessment of clinical signs of atopic dermatitis: A systematic review and recommendation. J. Allergy Clin. Immunol. 132, 1337–1347 (2013).
REPLY - CoMiSS validation study protocol
Letter to the Editor of BMJ Open - 8 August 2018
We thank MM Vanderschuren et al. for their interest in our published protocol for a validation study of the Cow’s Milk-related Symptom Score (CoMiSS®) [1] and would like to provide our reply to the concerns raised.
Show MoreCoMiSS was designed in 2015 by a paediatric expert group in order to provide an easy-to-use tool to facilitate a greater awareness for cow’s milk protein allergy (CMPA), particularly in regions where allergy services are not easily accessed or where community awareness for CMPA is low. [2] Since its introduction, CoMiSS has been translated into 14 languages and is used in 16 countries around the world. A pooled data analysis of three cross-sectional studies in children with CMPA confirmed that the CoMiSS score is useful in monitoring the response to treatment with hypoallergenic formula or a cow’s milk protein (CMP)-free elimination diet in infants and children with CMPA. [3]
It is important to highlight that CoMiSS was not designed as a diagnostic tool for CMPA and has not been validated for this purpose. [2] The published protocol outlines a prospective, single-blinded study which aims to assess the sensitivity and specificity of a change in CoMiSS score from baseline to 2 weeks after commencing a hypoallergenic diet, using a single-blinded oral food challenge (OFC) with cow’s milk-based formula as the diagnostic reference. [1] While a double-bli...
In the south Brazilian city of Porto Alegre - where about 1.5 million people live – two primary care units added out-of-hours (OOH) services in March and April of 2017 to increase patient access, as described by Kelly et al(1,2). OOH appointments in these two facilities are offered from 6 to 10pm. Differently from the traditional primary care model in Brazil, in which units serve people living in a specific catchment area with pre-scheduled appointments, during OHH all patients are welcomed without the need for scheduling. Medical, nursing, and dental care, as well as immunizations, are offered.
In the first 14 months since the start of the program, we recorded 33,964 appointments in the two clinics during OOH. During the same period, 51,181 consultations were provided in the same two units during traditional hours (and about 1.7 million primary care consultations in the city). Even though women are the most frequent users of both daytime and OOH services, the proportion of men ( 35.5% vs. 32.4% during the day) and of people of working age (54.96% vs. 47,7% during the day) was higher in OOH vs. regular daytime services.
However, the most interesting aspect of OOH services is the high number of rapid testing for HIV, syphilis, and hepatitis performed during these extended hours: 3,868 tests during OOH vs. 3,405 tests during usual hours – or 11.3% of the overall appointments during OOH vs. 6.6% during regular hours. Since the regular shift lasts 11 hours (...
Show MoreBraithwaite et al. [1] claim that kanuka honey is an effective treatment of rosacea. Both their study medication as well as their study design and statistical analysis are seriously flawed and do not allow this conclusion. First, the study medication is inappropriate to show an effect of kanuka honey. The verum was not kanuka honey (as the title suggests), but a mixture of kanuka honey and 10% glycerine (Honevo). As control...
We welcome this systematic review on interventions to improve the governance of patient safety in emergency care settings. This work highlights the value of information contained in adverse events derived from medical record review and incident reporting systems. The authors identified four qualities shared by effective incident reporting systems: (i) staff education on the importance and learning purpose of reporting,...
Is there an association between church membership and lower sexually transmitted disease (STD) incident among selected Danish religious cohorts? A large body of literature has shown a positive link between religion and health, particularly it's role in reducing lifetime risk for cardiovascular disease and selected cancers (1, 2). However, little research has looked at the effects of religion on sexual behavior and incide...
We strongly endorse the conclusion of Cornelius et al. regarding the necessity for improved communication of benefit-risk information to medication prescribers and users. Based on our extensive experience in evaluation and communication of medication-associated risks, we have previously proposed improvements in the processes for selecting and effectively communicating adverse drug reactions and interactions in HPI (1-3)....
Messrs Schmidt and Schmidt, in their letter published in BMJ Open on 24 March 2016, assert that the study we reported about the effects of Kanuka honey in rosacea has a number of important flaws including problems with the experimental treatments, study design, and statistical analysis.
With respect to the study medications the aim of the study was not to estimate the effect of a preparation of Kanuka honey with...
Dear Editor,
We have read with great interest the article “What insights do patients and caregivers have on acute kidney injury and post-hospitalisation care? A single-center qualitative study from Toronto, Canada” by Silver et al.
The article sheds light on the fact that most of its participants prioritised chronic conditions that ‘progress over time’ over AKI. These co-morbidities often include heart failure, hypertension and Type 2 diabetes mellitus, which are treated with non-steroidal anti-inflammatory drugs, diuretics and metformin respectively( 1). There is considerable data that these drugs are nephrotoxic and should therefore, be deprescribed or temporarily with-held or dose-adjusted in patients with AKI.
However, from unpublished research at our hospital, there is often reluctance to stop these drugs, suggesting that this misconception may be shared by physicians as well. As this is a common clinical problem with considerable morbidity, Think Kidneys Campaign (NHS collaboration of various trusts) have developed a checklist for medication optimization in patients with AKI (2), but its use is sparse at least from our experience.
Studies have shown that in-hospital mortality due to AKI far exceeds that due to these long-standing, chronic conditions. In a large nation-based study, in-hospital mortality of AKI was found to be 12.32% (3), with an increase in number of absolute deaths from 2001 to 2011. This is in contrast to the 3% in-...
Show MoreIn the UK I have observed that the number of cataract operations is undulating in parallel with changes in the number of deaths. The number of deaths is showing highly unusual on/off switching which may arise from a hidden infectious outbreak. This on/off switching can be revealed by applying a rolling 12-month total to your data. Are you able to apply such a rolling total to your data to see if there are any hidden patterns in the trends? A list of publications regarding on/of switching in deaths and medical conditions can be found at http://www.hcaf.biz/2010/Publications_Full.pdf I hope that this will prove to be helpful. Kind Regards
To the editor:
Show MoreWe read with interest the protocol for the validation of the Cow’s Milk-related Symptom Score (CoMiSS) against open food challenge by Vandenplas et al.(1) They have proposed this symptom score as a resource for primary healthcare providers, aiming to increase awareness of cow’s milk allergy (CMA)-related symptoms to facilitate an earlier diagnosis. The score was developed to increase awareness of mainly non-IgE mediated CMA, a disorder of increasing interest in recent literature publications.(2,3) However, dysmotility, dysmotility-related symptoms, blood in stool and failure to thrive (weight ≤2.5 Z-scores, drift from growth curve of >2 percentiles over 6 months or weight-for-length -2 Z-scores)(4) are excluded from the score. These are considered key features in non-IgE mediated allergy.(5) In contrast, severe respiratory symptoms and urticaria are included in the score. In our opinion, using the CoMiSS for diagnosing CMA will therefore hamper diagnosing non-IgE mediated CMA. The authors stress that the design of this study might not enable a reliable distinguishment between IgE-mediated CMA, non-IgE mediated CMA and non-allergic cow’s milk-related symptoms. In addition, in our experience we believe that healthcare professionals might even get confused in the differential diagnosis regarding non-allergic cow’s milk-related symptoms. For example pyloric stenosis, gastroesophageal reflux disease, hemorrhagic disease of the newborn or infective gastro...
Pages