You are here

Article Text

Article menu

Download PDFPDF

Narrative review
Theory-based and evidence-based design of audit and feedback programmes: examples from two clinical intervention studies
  1. Sylvia J Hysong1,2,
  2. Harrison J Kell3,
  3. Laura A Petersen1,2,
  4. Bryan A Campbell4,
  5. Barbara W Trautner1,2
  1. 1Center for Innovations in Quality, Effectiveness and Safety, Michael E. DeBakey VA Medical Center, Houston, Texas, USA
  2. 2Baylor College of Medicine, Houston, Texas, USA
  3. 3Educational Testing Service, Morrisville, Pennsylvania, USA
  4. 4Normal Modes, Durham, North Carolina, USA
  1. Correspondence to Dr Sylvia Hysong, Center for Innovations in Quality, Effectiveness and Safety, Michael E. DeBakey VA Medical Center, Houston TX 77030, USA; Hysong{at}bcm.edu

Abstract

Background Audit and feedback (A&F) is a common intervention used to change healthcare provider behaviour and, thus, improve healthcare quality. Although A&F can be effective its effectiveness varies, often due to the details of how A&F interventions are implemented. Some have suggested that a suitable conceptual framework is needed to organise the elements of A&F and also explain any observed differences in effectiveness. Through two examples from applied research studies, this article demonstrates how a suitable explanatory theory (in this case Kluger & DeNisi's Feedback Intervention Theory (FIT)) can be systematically applied to design better feedback interventions in healthcare settings.

Methods Case 1: this study's objective was to reduce inappropriate diagnosis of catheter-associated urinary tract infections (CAUTI) in inpatient wards. Learning to identify the correct clinical course of action from the case details was central to this study; consequently, the feedback intervention featured feedback elements that FIT predicts would best activate learning processes (framing feedback in terms of group performance and providing of correct solution information). We designed a highly personalised, interactive, one-on-one intervention with healthcare providers to improve their capacity to distinguish between CAUTI and asymptomatic bacteruria (ASB) and treat ASB appropriately. Case 2: Simplicity and scalability drove this study's intervention design, employing elements that FIT predicted positively impacted effectiveness yet still facilitated deployment and scalability (eg, delivered via computer, delivered in writing). We designed a web-based, report-style feedback intervention to help primary care physicians improve their care of patients with hypertension.

Results Both studies exhibited significant improvements in their desired outcome and in both cases interventions were received positively by feedback recipients.

Summary A&F has been a popular, yet inconsistently implemented and variably effective tool for changing healthcare provider behaviour and, improving healthcare quality. Through the systematic use of theory such as FIT, robust feedback interventions can be designed that yield greater effectiveness. Future work should look to comparative effectiveness of specific design elements and contextual factors that identify A&F as the optimal intervention to effectuate healthcare provider behaviour change.

Trial registration number NCT01052545, NCT00302718; post-results.

  • Audit and feedback
  • Health services research
  • Quality improvement methodologies

https://doi.org/10.1136/bmjqs-2015-004796

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    Background

    Audit and feedback (A&F) is a common intervention for changing healthcare provider behaviour and improving healthcare quality.1 ,2 A&F can be defined as ‘a summary of clinical performance (audit) over a specific period of time, and the provision of that summary (feedback) to individual practitioners, teams, or healthcare organizations.’3 A&F is an appealing intervention because of its seemingly straightforward approach to behaviour change: by receiving current information about their performance (‘knowledge of results’), providers are more motivated to improve.4 ,5 A&F has been used to improve a wide range of behaviours in clinical practice across many settings,2 making it highly flexible. Further, A&F is often the first step in awarding financial incentives to providers; consequently, the design and implementation of an A&F intervention can materially impact quality of care, as well as costs for patients, providers and healthcare organisations.6 Thus, the nature and characteristics of both the outcome and the feedback intervention must be considered to achieve the desired results.

    Studies within healthcare including a Cochrane review suggest that A&F can improve patient care effectively;2 however, others have found that A&F varies in effectiveness and have attributed this inconsistency to the design of specific interventions.2 Many A&F studies inadequately describe their interventions and do not detail their underlying causal processes, making it difficult to compare feedback interventions across different healthcare settings.1

    Theory and conceptual frameworks are important yet underused tools in healthcare for successfully designing behaviour-change interventions. Michie and Abraham7 posit that, to design interventions that truly change behaviour, one must rely on empirically supported theories of behaviour change that identify both psychological antecedents and operating mechanisms of behaviour change. This contention is echoed in the A&F literature. Foy and colleagues suggest a conceptual framework is needed to organise the elements of A&F and explain observed differences in effectiveness.1 Ivers et al8 contend that using theory in designing, implementing or interpreting A&F interventions can help clarify the behavioural causal pathways and their underlying mechanisms, resulting in more effective feedback interventions. Unfortunately, explicit use of theories to design A&F interventions is rare, with little consensus as to appropriate theories.3

    Another potential problem is that not all theories are well suited to aid in intervention design; at minimum, the theory must clearly describe the mechanism behind the behaviour change and be supported by independent work.7 For example, Rogers' Diffusion of Innovation Theory9 and Bandura's Social Cognitive Theory10 have been reported as the two theories most often used in A&F intervention design. Yet, the suitability of these theories for this purpose has been questioned due to their lack of specificity.3 Similarly, implementation-oriented frameworks such as PARIHSi can guide contextual factors to be considered to ensure successful implementation; however, they may not be able to help identify the kinds of features an intervention requires to be optimally effective in a given context or across multiple contexts. Often, it is necessary to draw from numerous theories to accomplish that goal. In the case of A&F, Kluger and DeNisi11 have already conducted such a synthesis to produce Feedback Intervention Theory (FIT). Specifically designed for explaining feedback's operating mechanisms, FIT draws from cognitive and motivational theories to formulate its tenets, including control theory,12 ,13 goal-setting theory,14 multiple-cue probability learning paradigm,15 social cognition10 and learnt helplessness theory, to offer clear propositions for how feedback works and what cues are likely to improve feedback effectiveness. Through two examples from applied research studies using FIT-based feedback interventions, the present article demonstrates how FIT can be systematically applied in healthcare settings to design better feedback interventions.

    Feedback intervention theory: a brief primer

    According to FIT, people regulate their behaviour by comparing it to goals, standards or benchmarks to which they have committed.14 When they observe a discrepancy between the standard and their actual behaviour, they try to resolve it by adjusting their level of effort to increase the likelihood that the benchmark will be met; the degree of increased effort an individual will undertake can be altered by providing performance feedback. For example, physicians who prescribe a particular medication for a given condition are likely to continue prescribing it until some external influence, such as a patient request for a cheaper alternative, compels them to change. Since individuals' attention is limited, only gaps receiving attention have the potential for change; for example, an alert in the computerised prescription order-entry system could bring a cheaper medication to the physician's attention, thereby increasing the likelihood of switching to the new medication. Thus, feedback interventions work by providing new information that redirects recipients' locus of attention either away from the task (eg, towards ourselves or towards irrelevant tasks) or towards the task. Kluger and DeNisi propose that three factors determine how effectively this attentional shift occurs: characteristics of the feedback itself (or ‘feedback intervention cues’), the nature of the task (‘task characteristics), and personality and situational variables.11 Feedback cues determine the direction towards which attention will likely shift; task characteristics determine how susceptible the task is to attentional shifts, and personality and situational variables determine how the feedback recipient chooses to change once the attentional shift occurs.

    According to FIT, feedback cues that redirect the locus of attention towards the task itself reinforce extant motivational processes, thereby strengthening feedback's effect on task performance; similarly, cues that redirect attention towards details of the task tend to promote learning and also strengthen feedback's effect on task performance; conversely, information that shifts attention away from the task diverts cognitive resources away from trying to improve, thereby weakening feedback's effect.11 Figure 1 graphically depicts the basic tenets of FIT.

    Figure 1
    Figure 1

    Schematic of tenets of Feedback Intervention Theory.

    Kluger and DeNisi,11 Hysong16 and Ivers et al2 independently found via meta-analysis that feedback has a generally moderate effect on performance. Additionally, the Kluger and DeNisi and the Hysong meta-analyses explicitly tested the tenets of FIT and identified several feedback intervention characteristics that enhance feedback effectiveness, including correct solution information (information about how to do the task correctly), velocity (change from last period of measurement), graphical or written presentation formats, inclusion of normative information, neutral tone and other goal-setting components. Characteristics found to decrease feedback effectiveness included discouraging tone, praiseful tone and verbal feedback.11 Table 1 summarises and defines characteristics predicted by FIT to impact feedback effectiveness, and the effect reported by each meta-analysis. As shown in this table, these three independent meta-analyses exhibit several parallels. Although theory and evidence seem to disagree about which elements of feedback are ineffective, there appears to be good congruence about the more effective elements of feedback. Due to its versatile framework and demonstrable improvements in A&F effectiveness both inside and outside healthcare settings,17–19 FIT is a suitable guide for the design of A&F interventions across a variety of contexts.

    View this table:
    Table 1

    Factors predicted to impact feedback effectiveness by Feedback Intervention Theory and by Cochrane systematic review

    The following case examples demonstrate how the principles of FIT were applied to design A&F interventions for two different healthcare scenarios, both of which led to improvements in two different healthcare quality outcomes. As the theory provides the most guidance about the effectiveness of feedback intervention cues, and feedback design is the element over which organisations and researchers have the most control, the cases concentrate on the feedback cues selected in the design of the feedback intervention.

    Case 1: Reducing inappropriate diagnosis of catheter-associated urinary tract infection via case-specific feedback

    We employed FIT to design an A&F intervention in the United States Department of Veterans Affairs Veterans Health Administration (VHA) to improve internal medicine resident and long-term care personnel's capacity to distinguish between asymptomatic bacteriuria (ASB) and catheter-associated urinary tract infection (CAUTI). ASB is frequently misdiagnosed as CAUTI, often leading to unnecessary use of antibiotics, in turn leading to resistant organisms, unnecessary cost, prolonged hospital stays and infections such as Clostridium difficile-associated disease.20 Details of the CAUTI study's design, including the A&F intervention, are published elsewhere;21 highlights are summarised here.

    Participants

    The A&F intervention was applied to 169 healthcare providers who make the decision to order a urine culture and prescribe antibiotics for a positive urine culture in two types of hospital units: acute-care medicine wards and long-term care units. In the acute-care medicine wards, internal medicine residents (trainees), usually interns or second-year residents, were generally responsible for making decisions about prescribing antibiotics for positive urine cultures, under supervision from a rotating pool of attending physicians. In the extended-care ward, the A&F intervention targeted the nurse practitioners, physician assistants and staff physicians who comprised the providers responsible for prescribing antibiotics for positive urine cultures.

    Intervention

    The intervention was based on a treatment algorithm derived from the Infectious Diseases Society of America guidelines for the non-treatment of ASB. This treatment algorithm describes steps providers should take when confronted with a patient with an indwelling urinary catheter and a subsequent positive urine culture.22 All participants' cases involving positive urine cultures were identified via a comprehensive chart review and examined by trained research staff for errors in diagnosis and treatment of ASB cases (defined as deviations from the treatment algorithm). For each clinician, the team identified a subset of cases that would teach the guideline principles clearly, keeping the ratio of erroneously to correctly managed cases to approximately 1:2. Research staff contacted the clinical providers involved to deliver the feedback intervention face to face, which consisted of a one-on-one feedback session reviewing each ASB case. For each case, an interactive hyperlinked PowerPoint presentation was used as an explanatory device during the feedback meeting (see online supplementary appendix A) and given to participants to keep.

    Intervention design considerations

    In developing the algorithm to be used for classifying cases in the study, we learnt that clinicians routinely caring for patients with urinary catheters use mental models that often deviate from guidelines when classifying cases of catheter-associated bacteriuria as either ASB or CAUTI.22 Thus, designing an intervention that shifted attention to the details of the task and fostered better learning of the guideline so that physicians' mental models could be altered was paramount. This drove our design over other factors, such as labour intensiveness or wide-scale deployment capabilities (all participants were physicians at local hospitals within 1.5 miles of each other and the research team). To accomplish our goal, we opted for an interactive, computer-based presentation of feedback, which FIT predicts would be less likely to shift attention away from the task than simple verbal presentation. The presentation's interactive nature was also intended to shift attention to the details of the task: it showed the physician the specific decision path he or she took, highlighted correct and incorrect decisions (a step towards correct-solution information) and provided correct solution information (one of the most effective cues according to the meta-analytic findings sizes reported earlier) by revealing the correct course of action when a participant clicked on a series of yes-or-no links in the presentation. Finally, because Ivers2 showed that providing both verbal and written feedback yielded better results than written feedback alone, research staff reviewed the treatment algorithm verbally with participants as they followed along on the interactive presentation; however, the research staff used a written, standardised but branching script to accomplish this task. The standardised script ensured that every participant received the same relevant feedback for each decision point in the diagnosis/treatment of ASB/CAUTI. Table 2 summarises how each feedback design characteristic was operationalised for the study.

    View this table:
    Table 2

    Operationalisation of feedback design characteristics Case 1

    Measures

    Reactions to feedback

    After delivering personalised feedback to individual participants, research assistants completed Smither and colleagues' 11-item scale to capture feedback recipients' reactions to their feedback, specifically targeting recipients' perceptions of the value of the feedback, the recipient's sense of obligation to use the feedback and extent to which the recipient was concerned with others' perceptions of his/her performance.24 Feedback recipients also completed questions about the clarity, usability, tolerability and degree of acceptance of feedback received.

    Effectiveness of feedback intervention

    Postintervention chart monitoring at the ward level captured the effectiveness of the A&F intervention. For 6 months after the feedback intervention was implemented, charts for each ward were monitored for adherence to the treatment algorithm, specifically, rates of urine-culture orders and inappropriate use of antibiotics.

    Results

    Reactions to feedback

    Most feedback recipients reacted positively to feedback received, indicating that they were open and accepting of the feedback and motivated to improve; mean feedback acceptance scores were high (M=5.18/7, SD=0.26), as were tolerability (M=5.81, SD=0.07) and usability scores (M=5.27, SD=0.04).

    Based on research assistants' reports of recipients' reactions, the PowerPoint presentation received positive reactions, as it organised the guidelines for ASB and CAUTI classification and treatment in a simple, understandable way without loss of information fidelity. Though feedback recipients occasionally tried to justify their original decisions, the feedback session quickly turned to the evidence and a learning opportunity for the recipient.

    Effectiveness of feedback intervention

    The overall rate of urine-culture ordering in wards where participants received A&F decreased significantly during the intervention period (from 41.2 to 23.3 per 1000 bed-days; (incident rate ratio (IRR), 0.57; 95% CI 0.53 to 0.61; p<0.0001) and further during the maintenance period (12 per 1000 bed-days; IRR, 0.29; 95% CI 0.26 to 0.32; p<0.0001). At the comparison site, urine cultures ordered did not change significantly across all three periods. Overtreatment of ASB at the intervention site reduced significantly (1.6 to 0.5 per 1000 bed-days; IRR, 0.35; 95% CI 0.22 to 0.55, p<0.0001) during the intervention period; these reductions persisted (0.4 per 1000 bed-days; IRR 0.24; 95% CI 0.13 to 0.42) during the maintenance period. Overtreatment of ASB at the comparison site was similar across all time periods.25

    Case 2: Helping physicians provide guideline-recommended hypertension care via summary feedback

    In this second case, we used FIT to design an A&F intervention to help physicians meet their goals for controlling patients' hypertension, as part of a randomised controlled trial of financial incentives to improve delivery of guideline-recommended hypertension care.26 ,27 Despite evidence that hypertension treatment is effective, the condition is undertreated by physicians and controlled in less than one-fourth of US citizens.28 Hypertension is a risk factor for stroke, coronary artery disease and congestive heart failure.

    Although the intervention of interest in this hypertension study was financial incentives, A&F constituted an important component of the financial-incentive intervention, as it served as the principal tool for justifying the size of the incentive awarded to any recipient. Moreover, A&F served as the comparable activity for the control group. All participants (regardless of study arm) received A&F on their hypertension care; only the various financial-incentive configurations tested in the study differentiated the study arms. The details of the financial-incentive intervention have been published elsewhere;26 only the A&F design is discussed here.

    Participants

    Eighty-three primary care physicians from 12 geographically dispersed VHA medical centres participated in this study. Additionally, non-physician clinicians (n=42) in trial groups receiving practice-based incentives (see below) also participated.

    Intervention

    Over 20 months, participants received web-based A&F reports every 4 months. Each report presented participants with percentage scores for patients receiving hypertension therapy, patients meeting goal blood pressure (BP) levels during last visit and patients receiving hypertension therapy that also met goal BP levels during last visit. Percentage scores were given, both for patients the physician treated during each period and for patients treated by other physicians at the same clinic during that period (in aggregate). Additionally, suggested performance goals for the subsequent performance period were also included.

    Intervention design considerations

    Several factors drove the design of this intervention. The nature of the task was an important consideration. FIT predicts that less complex tasks such as ordering guideline-recommended medication in the presence of consistently elevated BP29 are more susceptible to being positively influenced by feedback (see table 1); consequently, maintaining attention on the task and activating motivational processes (rather than trying to foster learning, as in the previous case) was a sufficient approach for this intervention. Practical matters also heavily influenced the design: this study involved hospitals geographically dispersed across the country, where often no prior relationship existed between the site and the research team. Target feedback recipients were busy primary care clinicians with little time to participate in a research study; in addition, feedback in this case was the control intervention, rather than the intervention of interest. Consequently, ease of delivery and minimal participant burden were key in this case—we sought to design feedback that could be deployed easily from a remote location, reviewed easily by participants and quickly show where gaps in their performance lay.

    To accomplish our goal, we designed a web-based written report providing summary-based feedback. Each report depicted scores about each hypertension management aspect of interest; this provides some degree of correct-solution information by shifting attention to the specific aspect of hypertension management requiring most attention (eg, ordering new medication vs adjusting existing treatment). We also provided achievable goals for participants to strive towards for the following period, which work to activate recipients' motivational processes. Reports also contained group feedback and velocity information in the form of line graphs depicting individual and clinic score histories for hypertension treatment and meeting goal BP levels. Both velocity and group feedback cues improve feedback effectiveness by activating recipients' motivation processes and keeping attention focused on the task. Table 3 summarises how each feedback design characteristic was operationalised for the study. Appendix B presents an example of the feedback report used.

    View this table:
    Table 3

    Operationalisation of feedback characteristics for Case 2

    Results

    Reactions to feedback

    A random subset of 28 physicians were interviewed at the 16-month mark to gain more in-depth knowledge of their hypertension management experiences during the study. Among the topics discussed was the quality of feedback received and suggestions for improvement. Interview respondents reported finding the feedback reports useful, as they were more detailed and focused than the performance measure reports they normally receive as part of their regular work duties:31I think it [the RCT feedback report used in this study] is a more accurate reflection, I don't find the [VA-generated] facility reports very useful.—Physician, Site #9We get reports but it's buried amongst, like, ‘Do you look at the diabetic's foot’ or whatever. I can't say I've really scrutinized those either. Yours is more targeted.—Physician, Site #2

    Effectiveness of feedback intervention

    Mean (SD) total payments over the study were US$4270 (US$459), US$2672 (US$153) and US$1648 (US$248) for the combined, individual and practice-level interventions, respectively. Relative to controls who received A&F alone, only the individual-incentives group exhibited significant changes in either BP control or appropriate response to uncontrolled BP. Change in guideline-recommended medication use was not significantly different than in the control group. The effect of the incentive was not sustained after a washout. However, we observed an average 11% increase (SD=2.87) in use of guideline-recommended antihypertensive medications between periods 1 and 5; this included the control group, who only received A&F reports.27

    Discussion

    A&F has been a popular tool employed to change healthcare provider behaviour and improve healthcare quality; however, it has suffered from inconsistent, atheoretical implementation, leading to mixed levels of effectiveness.1 FIT provides a practical, evidence-based framework with which to design feedback interventions. We have presented two different real-world feedback interventions designed using the principles of FIT, both positively accepted by their intended recipients and demonstrating significant improvements in desired clinical behaviours in the presence of well-designed feedback.

    In both cases, feedback was presented in writing, via computer, using a standardised structure whose content was individualised for each participant. In addition, Case 1 featured very specific correct solution information; whereas Case 2 featured individualised goal setting and norms. Correct solution information and goal setting are two of the most powerful feedback design features for maximising effectiveness,11 ,16 and in some cases their combined use is warranted. In these two cases, however, the designers' decision to use only one or the other feature, as well as the designers' respective selections, were driven by the context in which the A&F intervention was being used (consistent with the PARIHS framework) and by the features of the desired outcome. In Case 1, the desired outcome was correct diagnosis and subsequently appropriate treatment of ASB many possible decisions could have led to a wrong outcome; consequently, providing correct solution information about where the physician should have made a different choice was imperative. Conversely, in Case 2 the desired outcome was BP control and antihypertensive medication prescription; in this case the path to the correct outcome was far clearer and simpler; consequently, the more important design feature was to provide the motivational effect of goal setting to remind providers to carry out the desired outcome they already know they should. These findings demonstrate the need for considering both the desired outcome and feedback characteristics when designing a feedback intervention for maximum effect.

    The two cases presented here covered two different healthcare scenarios; consequently, the A&F interventions featured rather different characteristics. So how do A&F designers decide which components to choose for their interventions? The commonalities between the two cases can prove instructive. Both cases selected components that were individually tailored to the feedback recipient (consistent with FIT and Hysong et al),32 and that directed attention to the details of the task, consistent with FIT. The nature of which components to use (eg, written/computerised feedback, correct solution information, goal setting), in both cases, was driven by the situational and organisational context. The role of context on feedback effectiveness is an area for which FIT does not provide much guidance and is a limitation of the theory. However, newer research and theories can be drawn upon to fill this gap and are excellent opportunities for future research.33 For example, depending on whether feedback recipients tend to be more performance versus mastery oriented, research suggests feedback should be customised to align with those recipient characteristics.34 Recent research has also recommended encouraging feedback-seeking behaviours from employees and focusing on creating a culture of feedback where employees feel safe at giving and receiving feedback and have opportunity for reflection; this can take the form of both formal and informal forms of feedback.34 ,35

    Limitations

    The studies presented in the two cases, although guided by FIT to design a robust intervention that would be successful, were not designed for purposes of comparative effectiveness or to tease apart the effects of specific feedback elements; rather they were guided by FIT to design the intervention most likely to result in successful behaviour change, given the contextual constraints of the respective studies. Thus, the presentation of the two cases, while useful for presenting annotated examples of the design considerations for building successful, theory-based interventions using FIT, cannot speak to the matter of comparing the relative effectiveness of specific design elements of feedback.

    The main limitation in our case presentation concerns limitations of the theory itself; FIT adopts an inherently cognitive perspective towards feedback; consequently, recipient motivations and contextual (situational) variables are outside the scope of the theory; additionally, because it adopts a cognitive perspective it is inherently individually based and does not address feedback in team contexts,36 as is the case with newer frameworks.37 Despite these limitations, newer research and theories (eg, DeShon et al37) have built upon the work of FIT precisely due to its clear and specific propositions and rationales; also, where the scenario lends itself to it, FIT can provide specific direction for designing A&F interventions, as it did with the two cases presented here.

    Conclusion

    We conclude that FIT can be effectively used to design feedback interventions to more effectively improve provider performance and quality of healthcare. Future work should focus on expanding understanding of specific contextual cues that impact feedback effectiveness (whether for individuals or teams) to provide more specific guidance towards A&F intervention design.

    References

    1. ↵
      1. Foy R,
      2. Eccles MP,
      3. Jamtvedt G, et al
      . What do we know about how to do audit and feedback? Pitfalls in applying evidence from a systematic review. BMC Health Serv Res 2005;5:50. doi:10.1186/1472-6963-5-50
      OpenUrlCrossRefPubMed
    2. ↵
      1. Ivers N,
      2. Jamtvedt G,
      3. Flottorp S, et al
      . Audit and feedback: effects on professional practice and health care outcomes. Cochrane Database Syst Rev 2012;(6):CD000259. doi:10.1002/14651858.CD000259.pub3
    3. ↵
      1. Colquhoun HL,
      2. Brehaut JC,
      3. Sales A, et al
      . A systematic review of the use of theory in randomized controlled trials of audit and feedback. Implement Sci 2013;8:66. doi:10.1186/1748-5908-8-66
      OpenUrlCrossRefPubMed
    4. ↵
      1. Karraker RJ
      . Knowledge of results and incorrect recall of plausible multiple-choice alternatives. J Educ Psychol 1967;58:11–14. doi:10.1037/h0024113
      OpenUrlCrossRefWeb of Science
    5. ↵
      1. Locke EA
      . Motivational effects of knowledge of results: Knowledge or goal setting? J Appl Psychol 1967;51:324–9. doi:10.1037/h0024771
      OpenUrlCrossRefPubMedWeb of Science
    6. ↵
      1. Petersen LA
      . Pay for performance: what we measure matters. Virtual Mentor 2013;15:570–5. doi:10.1001/virtualmentor.2013.15.7.ecas2-1307
      OpenUrlCrossRefPubMed
    7. ↵
      1. Michie S,
      2. Abraham C
      . Interventions to change health behaviours: Evidence-based or evidence-inspired? Psychol Health 2004;19:29–49. doi:10.1080/0887044031000141199
      OpenUrlCrossRefWeb of Science
    8. ↵
      1. Ivers NM,
      2. Sales A,
      3. Colquhoun H, et al
      . No more ‘business as usual’ with audit and feedback interventions: towards an agenda for a reinvigorated intervention. Implement Sci 2014;9:14. doi:10.1186/1748-5908-9-14
      OpenUrlCrossRefPubMed
    9. ↵
      1. Rogers EM
      . Diffusion of innovations. 4th edn. New York: Free Press, 1995.
    10. ↵
      1. Bandura A
      . Social cognitive theory of self regulation. Organ Behav Hum Decis Process 1991;50:248–87. doi:10.1016/0749-5978(91)90022-L
      OpenUrlCrossRefWeb of Science
    11. ↵
      1. Kluger AN,
      2. DeNisi A
      . The effects of feedback interventions on performance: a historical review, a meta-analysis, and a preliminary feedback intervention theory. Psychol Bull 1996;119:254–84. doi:10.1037/0033-2909.119.2.254
      OpenUrlCrossRefWeb of Science
    12. ↵
      1. Annett J
      . Feedback and human behavior. Harmondsworth, Middlesex, UK: Penguin Books, 1969.
    13. ↵
      1. Podsakoff PM,
      2. Farh JH
      . Effects of feedback sign and credibility on goal setting and task performance. Organ Behav Hum Decis Process 1989;44:45–67. doi:10.1016/0749-5978(89)90034-4
      OpenUrlCrossRefWeb of Science
    14. ↵
      1. Locke EA,
      2. Latham GP
      . A theory of goal setting and task performance. Englewood Cliffs, NJ: Prentice Hall, 1990.
    15. ↵
      1. Balzer WK,
      2. Doherty ME,
      3. O'Connor R Jr.
      . Effects of cognitive feedback on performance. Psychol Bull 1989;106:410–33. doi:10.1037/0033-2909.106.3.410
      OpenUrlCrossRefWeb of Science
    16. ↵
      1. Hysong SJ
      . Meta-analysis: audit & feedback features impact effectiveness on care quality. Med Care 2009;47:356–63. doi:10.1097/MLR.0b013e3181893f6b
      OpenUrlCrossRefPubMedWeb of Science
    17. ↵
      1. Alder GS
      . Examining the relationship between feedback and performance in a monitored environment: a clarification and extension of feedback intervention theory. J High Technol Manag Res 2007;17:157–74. doi:10.1016/j.hitech.2006.11.004
      OpenUrlCrossRef
    18. ↵
      1. Geister S,
      2. Konradt U,
      3. Hertel G
      . Effects of process feedback on motivation, satisfaction, and performance in virtual teams. Small Group Res 2006;37:459–89. doi:10.1177/1046496406292337
      OpenUrlAbstract/FREE Full Text
    19. ↵
      1. King PE
      . When do students benefit from performance feedback? A test of feedback intervention theory in speaking improvement. Commun Q 2015;64:1–15.
      OpenUrl
    20. ↵
      1. Trautner BT,
      2. Grigoryan L
      . Approach to a positive urine culture in a patient without urinary symptoms. Infect Dis Clin North Am 2014;28:15–31. doi:10.1016/j.idc.2013.09.005
      OpenUrlCrossRefPubMed
    21. ↵
      1. Trautner BW,
      2. Kelly PA,
      3. Petersen N, et al
      . A hospital-site controlled intervention using audit and feedback to implement guidelines concerning inappropriate treatment of catheter-associated asymptomatic bacteriuria. Implement Sci 2011;6:41. doi:10.1186/1748-5908-6-41
      OpenUrlCrossRefPubMed
    22. ↵
      1. Trautner BW,
      2. Bhimani RD,
      3. Amspoker AB, et al
      . Development and validation of an algorithm to recalibrate mental models and reduce diagnostic errors associated with catheter-associated bacteriuria. BMC Med Inform Decis Mak 2013;13:48. doi:10.1186/1472-6947-13-48
      OpenUrlCrossRefPubMed
      1. Trautner BW,
      2. Petersen N,
      3. Hysong SJ, et al
      . Overtreatment of asymptomatic bacteriuria: identifying provider barriers to evidence-based care. Am J Infect Control 2014;42:653–8. doi:10.1016/j.ajic.2014.02.003
      OpenUrlCrossRefPubMed
    24. ↵
      1. Smither JW,
      2. London M,
      3. Roukema Richmond K
      . The relationship between leaders’ personality and their reactions to and use of multisource feedback. Group Organ Manag 2005;30:181–210. doi:10.1177/1059601103254912
      OpenUrlCrossRef
    25. ↵
      1. Trautner BW,
      2. Grigoryan L,
      3. Petersen NJ, et al
      . Effectiveness of an antimicrobial stewardship approach for urinary catheter-associated asymptomatic bacteriuria. JAMA Intern Med 2015;175:1120–7. doi:10.1001/jamainternmed.2015.1878
      OpenUrlCrossRefPubMed
    26. ↵
      1. Petersen LA,
      2. Urech T,
      3. Simpson K, et al
      . Design, rationale, and baseline characteristics of a cluster randomized controlled trial of pay for performance for hypertension treatment: Study Protocol. Implement Sci 2011;6:114. doi:10.1186/1748-5908-6-114
      OpenUrlCrossRefPubMed
    27. ↵
      1. Petersen LA,
      2. Simpson K,
      3. Pietz K, et al
      . Effects of individual physician-level and practice-level financial incentives on hypertension care: a randomized trial. JAMA 2013;310:1042–50. doi:10.1001/jama.2013.276303
      OpenUrlCrossRefPubMedWeb of Science
    28. ↵
      1. Wang TJ,
      2. Vasan RS
      . Epidemiology of uncontrolled hypertension in the United States. Circulation 2005;112:1651–62. doi:10.1161/CIRCULATIONAHA.104.490599
      OpenUrlFREE Full Text
    29. ↵
      1. Hysong SJ,
      2. Amspoker AB,
      3. Petersen LA
      . A novel method for assessing task complexity in outpatient clinical-performance measures. J Gen Intern Med 2016;31(Suppl 1):28–35. doi:10.1007/s11606-015-3568-z
      OpenUrlCrossRefPubMed
      1. Hysong SJ,
      2. Simpson K,
      3. Pietz K, et al
      . Financial incentives and physician commitment to guideline-recommended hypertension management. Am J Manag Care 2012;18:e378–91.
      OpenUrlPubMed
    31. ↵
      1. Petersen LA,
      2. SoRelle RS,
      3. Broussard Smitham KK, et al
      . Unintended Consequences of Pay for Performance to Improve Management of Hypertension: A qualitative examination of study participants’ experiences. Paper presented at the 2014 AcademyHealth Annual Research Meeting. San Diego, CA: June 8-10, 2014.
    32. ↵
      1. Squires JE,
      2. Graham ID,
      3. Hutchinson AM, et al
      . Identifying the domains of context important to implementation science: a study protocol. Implement Sci 2015;10:1–9. doi:10.1186/s13012-014-0195-8
      OpenUrlCrossRefPubMed
    33. ↵
      1. Hysong SJ,
      2. Best RG,
      3. Pugh JA
      . Audit and feedback and clinical practice guideline adherence: making feedback actionable. Implement Sci 2006;1, 9.
    34. ↵
      1. Anseel F
      . Providing and seeking feedback in the workplace. Paper presented at the 2015 Annual Meeting of the European Association of Work and Organizational Psychology, May 13–16, Oslo, Norway.
    35. ↵
      1. London M
      . The power of feedback: giving, seeking and using feedback for performance improvement. London: Routledge; 2014.
    36. ↵
      1. Hysong SJ,
      2. Knox MK,
      3. Haidet P
      . Examining clinical performance feedback in patient-aligned care teams. J Gen Intern Med 2014;29(Suppl 2):S667–74. doi:10.1007/s11606-013-2707-7
      OpenUrlCrossRefPubMed
    37. ↵
      1. DeShon RP,
      2. Kozlowski SW,
      3. Schmidt AM, et al
      . A multiple-goal, multilevel model of feedback effects on the regulation of individual and team performance. J Appl Psychol 2004;89:1035–56. doi:10.1037/0021-9010.89.6.1035
      OpenUrlCrossRefPubMedWeb of Science

    Footnotes

    • Contributors SH conceived and wrote the first draft of the paper and designed the A&F interventions presented in each case. BWT was the principal investigator of the study presented in Case 1; LAP was the principal investigator of the study presented in Case 2. HJK aided in the design of Case 1's audit & feedback intervention and designed the ‘reactions to feedback’ component of Case 1; BAC edited subsequent drafts of the paper and provided material intellectual input. All authors provided critical intellectual input and material edits for the paper.

    • Funding Veterans Administration, Health Services Research and Development Program (CDA 07-0181), (IIR 04-349) and (IIR 09-104); and National Heart Lung and Blood Institute (1R01HL079173-01), (R01HL079173-S2).

    • Competing interests None declared.

    • Ethics approval Institutional review boards for Baylor College of Medicine and Michael E. DeBakey VA Medical Center.

    • Provenance and peer review Not commissioned; externally peer reviewed.

    • ↵i PARIHS—Promoting Action on Research Implementation in Health Services.