Chapter 9: Evaluation

INTRODUCTION

Nurses are often eager to start implementing practice changes found through critique and synthesis of the evidence. Taking time to collect baseline data before implementation is critical in order to later identify whether the practice change worked as intended, to identify how well the implementation strategies worked, and to demonstrate an impact through improved outcomes.

BASELINE DATA COLLECTION

It is important to create a comprehensive evaluation plan for the pilot and to collect baseline data prior to implementing any practice change (Puterman et al., 2013; Russell, Wallace, & Ketley, 2011). Evaluation is meant to maximize the benefits of a change process and reduce unnecessary harm (American Evaluation Association, 2012). Evaluation of evidence-based practice (EBP) usually contrasts baseline with post-implementation data (i.e., pre-/post-comparison) from the pilot and also includes trended process and outcome measures for integration and sustainability.

SAMPLE SIZE

An important decision when planning for evaluation is to determine the sample size. Elements of the project purpose statement include the specific patient population and pilot unit, clinic, or setting. It is helpful to know approximately how many patients the pilot area typically treats with the clinical condition related to the topic. The pilot group may be a subset of those patients. Consider the patient volume typically seen in a reasonable amount of time (e.g., one month). The following questions should be considered when planning the pilot evaluation:

• Which patients from the unit, clinic, or setting should be included in the sample?

• What data or indicators are critical to collect?

• How will those indicators be operationalized (i.e., how are the indicator defined and measured)?

• Is the measure sensitive to the practice change?

• How will the data be collected (i.e., data source) so that the data are sensitive to the practice change?

• How long will the pilot last?

• How frequently will data be collected?

• How frequently will data be reported back to clinicians?

• What is the goal (i.e., what value indicates success)?

A sample of at least 25–50 patients is desirable for each pre- and post-implementation group. However, the sample size must be feasible (Puterman et al., 2013; Russell et al., 2011). Completing the pilot in a timely manner will help maintain the team’s interest (Russell et al., 2011).

SAMPLE PLAN

Using consecutive patients for the sample will help capture typical patients or a representative group from the unit or clinic and the usual variability of patients receiving care without oversampling unusual patients (Puterman et al., 2013). For example, collecting data every Wednesday on an inpatient unit may include the same patient for more than ...