Evaluating the VHA's Staffing Methodology: A Reliable Approach

Evaluation Efforts

The VA Office of the Inspector General (OIG) has included an evaluation of the implementation and effect of Staffing Methodology in its Comprehensive Assessment Program, under which routine surveys and assessments are conducted across VHA annually. It has reported on findings related to Staffing Methodology and nurse staffing more generally in the reports for individual facilities, as well as for VHA as a whole. In general, the OIG has monitored compliance with the steps specified in the plan and with the staffing recommendations generated by the unit and facility-based expert panels, finding that, at various points in time, facilities may be on target, over, or below recommended staffing levels. In addition to VA OIG reports, the Government Accountability Office (GAO, 2008) has addressed issues related to nurse staffing in the VHA, bolstering the need for a standardized approach to determining nurse staffing levels.

The VA Office of Nursing Services commissioned an Evaluation Center through the VA Quality Enhancement Research Initiative program, with the primary focus on evaluating the Staffing Methodology directive and its effect across VHA to date. This evaluation has had three primary approaches: telephone interviews with nurse executives and their teams from 21 VHA facilities (one in each regional network), including review of key documents related to Staffing Methodology; interrupted time series analysis of staffing levels across in patient care throughout VHA from October 1, 2009 to present; and a survey of all VHA nurse executives to assess the degree to which Staffing Methodology is integrated with the budgeting or resource allocation process at each VHA facility. While all three components of the evaluation are ongoing, preliminary findings from the first component, the qualitative telephone interviews with nurse executives and their teams, have been reported.

Methods

Out of a total of 125 VA medical centers with inpatient care units, 21 were purposively selected, one in each of the 21 regional networks (Veterans Integrated Service Network) to ensure geographic representation. The selected facilities were intended to be representative of VA medical centers across the system. Nurse executives in each medical center were contacted by email to schedule telephone interviews with the evaluation team. Nurse executives were encouraged to include any relevant members of their team in the interview. Interviews lasted approximately 60 minutes and were audio-recorded and transcribed to ensure accurate capture of the data. Transcripts were then sent to each nurse executive to verify accuracy and correct any errors. Each interview was conducted by at least two members of the evaluation team, one to facilitate the interview using a semi-structured interview guide, and the other to take notes. The evaluation team held a debriefing session following each interview to capture relevant themes and information. Preliminary analyses using a structured template were conducted by at least two evaluation team members using interview notes supplemented by the transcripts and audio-recordings. In addition, the nurse executives and their teams were asked to send copies of the organizational chart for their facility, including the approved FTEs for each unit and the nursing service as a whole, as well as copies of the narratives and recommendations from the unit and facility-based expert panels, to the extent possible. These supplementary materials were used to triangulate interview data and assess the types of products generated by the Staffing Methodology process.

In addition, data were extracted from national VHA databases for all facilities with inpatient units between October 1, 2009 and July 30, 2014 to create a NHPPD for all nursing personnel across all inpatient units and aggregated to the entire VHA. Changes were analyzed over time in NHPPD across VHA using interrupted time series analyses.

Findings were tested for statistical significance using January 1, 2012 as the date for the intervention in the interrupted time series analysis. The Staffing Methodology Directive was issued October 1, 2010, with implementation mandated by January 1, 2011.

This evaluation is being conducted as a nonresearch activity as described in VHA Handbook 1058.05, under the direction of the VA Chief Nursing Officer and the Office of Nursing Services.