A Nationwide Flash-Mob Study for Suspected Acute Coronary Syndrome

Angel M.R. Schols, MD, PhD; Robert T. A. Willemsen, MD, PhD; Tobias N. Bonten, MD, PhD; Martijn H. Rutten, MD; Patricia M. Stassen, MD, PhD; Bas L. J. H. Kietselaer, MD, PhD; Geert-Jan Dinant, MD, PhD; Jochen W.L. Cals, MD, PhD


Ann Fam Med. 2019;17(4):296-303. 

In This Article


Study Design

We performed a 2-week, nationwide, prospective, observational, flash-mob study among FPs. The study protocol was approved by the Medical Ethical Committee of Zuyderland Medical Centre (The Netherlands; 17-N-119) and prospectively registered at www.trialregister.nl (NTR6789).

The flash-mob steering committee invited FPs and organizations from their professional and social networks to act as ambassadors for the study (Figure 1). These ambassadors were asked to actively spread the word and distribute study materials among FPs nationwide. All ambassadors as well as the steering committee members used traditional professional networks as well as their social media channels to advertise the study. In addition, information about the study was distributed via personal communication, scientific journals, and our website (www.huisartshartweek.nl).[13,14] Furthermore, study materials were sent to specialized FPs, 8 departments of family medicine, >100 FP out-of-hours services, and ambulance services and were included in a Dutch FP scientific journal, Huisarts en Wetenschap, distributed to approximately 14,000 professionals.

Figure 1.

Implementation of the flash-mob method in the present study.
aSpecialized FPs receive a special 2-year training in heart and vascular disease or acute medicine.

Setting and Participants

After the Dutch College of General Practitioners annual conference where the study was promoted, all patients referred by FPs for suspected ACS from November 20, 2017 to December 3, 2017 were eligible for inclusion. During that time, all FPs in the Netherlands could register patients they referred during practice hours and out-of-hours shifts using a short 1-page case report form (CRF) either on paper or online (taking <2 min). In the Netherlands, FPs perform a gate-keeping function, and hospital care can only be accessed after referral by an FP. More than 95% of all episodes of care are completely covered in primary care.[15]

We defined ACS as acute myocardial infarction or unstable angina pectoris. The study was originally planned for 1 week, which we extended to 2 weeks to increase the sample size. We excluded patients who were unable to provide informed consent, were not seen in person by an FP (telephone referral), or were eventually not referred to hospital. All patients were informed about the study by the participating FP and had to provide oral informed consent.

Case Report Form

The CRF consisted of 10 consultation items including MHS items, whether the attending FP immediately suspected a serious diagnosis, whether the patient reported that the pain felt like pressure, the duration of the symptoms, signs of ischemia on ECG if performed, and the attending FP's assessment of ACS probability on a scale of 1 to 10. In addition, we registered the patient's name, sex, date of birth, name of the patient's own FP, whether the attending physician was an FP (if so, years of work experience) or FP trainee, and the time the patient was seen (daytime/out-of-hours and date). FPs were not asked to calculate the MHS.

Reference Standard

The final diagnosis was obtained by contacting all FPs of included patients via post/mail followed by at least 2 telephone/fax reminders at least 6 weeks after the initial consultation. We asked FPs to select the most probable final diagnosis from a short list—ACS, stable angina, other cardiac diagnosis, or noncardiac diagnosis—and to base their answer on the (discharge or other) letter from the cardiologist.

Statistical Analysis

We performed a univariate analysis to detect possible ACS predictors. The odds of having ACS were compared between patients with and without each possible predictor by calculating odds ratios with 95% CIs and Pearson χ 2 tests for each possible (dichotomous) predictor. The variable sex-adjusted age dichotomized age in line with the original MHS variable using a cut-off of age ≥65 years for women and age ≥55 years for men.

The MHS and the FP assessment of ACS probability were dichotomized into positive (high risk) and negative (low risk) results, with a cut-off value of ≤2 for MHS (based on previous MHS research) and ≤5 for FP assessment (based on practical usability).[1–3] For the MHS, we performed additional analyses using a cut-off value of ≤1 (also based on previous MHS research) because this cut-off value showed greater sensitivity and negative predictive value (NPV), therefore decreasing the chance of missing patients with ACS in the referred population.[1] For each strategy, the sensitivity, specificity, positive predictive value (PPV), and NPV were calculated with corresponding 95% CIs according to the efficient-score method (corrected for continuity).[16,17] See the Supplemental Appendix (http://www.AnnFamMed.org/content/17/4/296/suppl/DC1/) for the sensitivity and specificity of all other possible cut-off values for the FP assessment.

The diagnostic ability of the MHS and FP assessments to discriminate between patients with and without ACS was assessed by plotting sensitivity against 1-specificity on a receiver operating characteristic curve for both strategies and calculating the area under the curve (AUC).

We performed sensitivity analyses for cases with missing MHS items and imputed a 0 score for missing MHS items for patients with chest pain, assuming these missing items would have led to a negative score for that specific item. Statistical analyses were performed using IBM SPSS Statistics Data Editor version 23 (International Business Machines Corp) and the Vassarstats website: http://vassarstats.net/clin1.html.