International Delphi Expert Consensus on Safe Return to Surgical and Endoscopic Practice

From the Coronavirus Global Surgical Collaborative

Horacio J. Asbun, MD, FACS; Mohammad Abu Hilal, MD, PhD, FACS, FRCS; Filipe Kunzler, MD; Domenech Asbun, MD; Jaap Bonjer, MD, PhD, MBA, FRCSC; Kevin Conlon, MD, FRCSI, FACS, FRCSG, FTCD; Nicolas Demartines, MD, FACS, FRCS; Liane S. Feldman, MDCM, FACS, FRCS; Salvador Morales-Conde, MD, PhD; Andrea Pietrabissa, MD, FACS; Aurora D. Pryor, MD, FACS; Christopher M. Schlachta, BSc, MDCM, FRCSC, FACS; Patricia Sylla, MD, FACS, FASCRS; Eduardo M. Targarona, MD, PhD, FACS; Yolanda Agra, MD, PHD, MHRs; Marc G. Besselink, MD, Msc, PhD; Mark Callery, MD, FACS; Sean P. Cleary, MD, FACS; Luis De La Cruz, MD; Philippe Eckert, MD; Chad Evans, BA; Ho-Seong Han, MD, MS, (Gen Surg), PhD; Daniel B. Jones, MD, MS, FASMBS; Tong Joo Gan, MD, MHS, FRCA, MBA; Daniel Koch, MD; Keith D. Lillemoe, MD, FACS; Davide Lomanto, MD, PhD, FAMS; Jeffrey Marks, MD, FACS; Brent Matthews, MD, FACS; John Mellinger, MD, FACS; William Scott Melvin, MD, FACS; Eduardo Moreno-Paquentin, MD, FACS; Claudio Navarrete, MD; Timothy M. Pawlik, MD, PhD, MPH; Patrick Pessaux, MD, PhD; Walter Ricciardi, MD, MPH, MSc; Steven Schwaitzberg, MD, FACS; Paresh Shah, MD, FACS; Joseph Szokol, MD; Mark Talamini, MD, FACS; Ricardo Torres, MD; Alessandro Triboldi; Suthep Udomsawaengsup, MD, FRCST, FACS; Federica Valsecchi, PhD; Jean-Nicolas Vauthey, MD, FACS; Michael Wallace, MD; Steven D. Wexner, MD, PhD, FACS, FRCS, FRCS; Michael Zinner, MD, FACS; Nader Francis, MBChB, FRCS, PhD


Annals of Surgery. 2021;274(1):50-56. 

In This Article


These recommendations were produced following modified Delphi methodology.[6] The Executive Committee - EC (HA, NF, MAH, FK, DA) served as the organizers of the consensus and guided the Delphi process through the steps outlined below. The Steering Committee (SC) was composed of members of the CVGSC. The SC in conjunction with other international experts made up the Expert Group (EG). Selection of members of the EG was based on their peers' recommendations and their leadership positions across different specialties and societies. Due to the global nature of the pandemic and broad range of topics discussed, diversity of participant background was essential (see supplemental table 2, EG members represented surgeons, anesthesiologists, gastroenterologists, governmental policymakers, and patient advocates/representatives from 15 different countries in North and South America, Europe, and Asia.

Identification of Topic Domains and Formulation of Questions

General domains and associated questions relevant to the pandemic's effect on the delivery of surgical services were initially proposed by the SC on April 27, 2020. Published position statements on how to resume surgical services were reviewed and used to help formulating the areas of interest and associated questions. These domains and questions were approved by members of the EC and further defined during a virtual meeting between the SC and EC. For standardization purposes, the domains and questions were formulated in the setting of three phases of the COVID-19 pandemic as defined by the CVGSC (Table 1). The answers to these questions would form the basis for the recommendations produced by the process.

Formulation of Statements

The EG members were divided into subgroups organized by domain topics and led by a designated chair. Each subgroup formulated statements addressing the questions in their domains. The statements were produced taking into account the literature and guidelines that were available at the time the manuscript was drafted. Statements were then submitted to the EC, which did not participate in the formulation of these statements.


The EC compiled a synthesis of the statements received from each subgroup of experts. In some cases, the wording of the statements was modified by the EC to have more uniform syntax across the statements and to eliminate redundant proposals. Potentially conflicting statements were left as-is and highlighted during subsequent rounds of voting and discussion. The EC did not alter the content of the statements, and any concerns regarding the effect of adjusting statements were discussed with and approved by the subgroups that wrote them.

First Round of Voting (D1) – May 10, 2020. The statements were then distributed to all experts for a first round of Delphi voting (D1). EG members voted to agree or disagree with the statements, and thus qualify the statements as valid by expert opinion. A dichotomous polling method was chosen over a Likert process because the final goal was to assess if there was agreement with the statements or not. Using a Likert process may have added more variability of opinion to the subjects for which no science is available. The binary system would force the experts to be more definitive in their decision. Beside each statement, a section for comments was available. The authors of the statements and the resultant votes/comments remained anonymous. This approach was utilized to avoid bias created by undue influence of individuals or subgroups on others. The EC did not partake in voting.

Consensus was achieved when a statement reached at least 80% of votes in agreement. Statements with less than 80% agreement and related comments in D1 were returned to the expert subgroup that formulated them. Per methodology, statements that reached consensus in the first round were not sent for a second round because if they reached the 80% threshold they would not undergo any major modification that would require a second voting process. However, any minor concerns could be addressed in the virtual meeting. The subgroups had the option to revise statements that did not reach 80% agreement based on feedback, or to recommend discarding them based on excessive need for modification.

Second Round of Voting (D2) – May 16, 2020. Revised statements were sent for a second round of Delphi voting (D2). The same process of anonymous submission, voting, and commenting was undertaken. Statements that did not reach 80% approval were marked for further review.

Virtual Meeting (VM3) – May 22, 2020. A virtual meeting with the entire group of experts was held for final discussion (VM3) of statements that did not reach approval in D2. This meeting also allowed experts to bring to attention any other issues or statements they felt required further consideration. Any adjusted statements were anonymously voted on after discussion, with the same 80% threshold for approval.

Voting for D1 and D2 was carried out through electronic questionnaires on the online platform SurveyGizmo.[7] The video teleconference during VM3 was held on Zoom,[8] and voting during that conference was through Poll Everywhere.[9]

Data analysis was based on percentage response rates for each statement in each round of voting. After completion of all voting and statement formulation, the manuscript was drafted with the recommendations and sent to all members for revision, input, and approval before submission for publication. A certification of reading and acknowledgment was electronically collected from all 44 authors. Voting in D1/D2 and manuscript review was mandatory for all EG members.

Throughout the process, 2 surgeon researchers (FK, DA) were involved in collecting and organizing data, communicating with experts/committee members, and creating and distributing the electronic questionnaires. These individuals did not partake in the selection of experts, organization of subgroups, formulation of questions or statements, or in voting.