Effects-Driven IT Development
(formerly known as Evidence-Based IT Development)

The aim of this project is to investigate how the effects of the use of a system could play a prominent role in the contractual definition of IT projects and how contract fulfilment could be determined on the basis of proven utility value and measured effects. The hypothesis is that by substituting system functionality with measurable, agreed-upon effects of using the system, the contract will provide more appropriate means for managing the customer-vendor relationship and working systematically toward meeting customer goals.

The idea of effects-driven IT development is generally applicable to all large-scale IT projects but will in this project be investigated in the context of Electronic Patient Record (EPR) systems. Effects-driven IT development aims at establishing strategic partnerships between vendor and customer in order to develop state-of-the-art EPR solutions with proven utility value and a measured effect on the clinical work the system supports. Measurable effects are defined in relation to clinical work practices within documentation and decision making. Examples of measurable effects could include:

  • The physician can complete the daily rounds as a “one-man-show” (without an escorting nurse), and all information and coordination with other clinical staff is done using the EPR system.
  • Mental workload is reduced when clinical staff assesses the status of a patient during the daily patient conference.
  • The amount of information missing during the nursing conference, which initiates every nursing shift, is reduced by 90% and nurses experience an unambiguous patient plan.

Effects may also be viewed at a national and political level or in relation to the hospital’s overall strategic goals. Our primary focus is on effects on the clinical work with a subordinate interest in how these effects relate to strategic and political goals.

The project will investigate how and to which extent vendor and customer can change their focus from IT functionality to one of measurable effects and the development of EPR solutions with proven utility value. Effects-driven development is rooted in preferences for effects over products and processes, for measurement over expectations and estimates, and for effects-based contracts over functionality contracts. Our ultimate goal is to develop an effects-based commercial contract model where the customer’s payments are dependent on measurable effects of using the vendor’s system.

Effects-driven development seems promising especially for complex and business critical projects that require establishment of strategic, long-term, mutually beneficial relationships characterized by trust, mutual learning, and cooperation between vendor and customer. This is indeed the case for EPR solutions in Denmark.

Research questions

  • How can desired effects be identified and specified in collaboration with users and how can methods for measuring those effects be devised?
  • How can realistic experiments be conducted in order to measure effects of system usage during real work processes?
  • How can effects specific to user's work processes be related to overall strategic and political goals?
  • How can an overall partnership, necessary for an effects-driven approach, be established between customer and vendor?
  • How can IT projects be based on effects-based contracts and what are the conditions for and consequences?

Empirical methods

Overall, the project employs an action-research approach. More specifically, we have used:

  • Observation
  • Interviews
  • Questionnaires, including TLX
  • Self observation by clinicians
  • Document (and log) analysis

Current partners

  • Hospitals, health care centers, and EPR units in the Zealand Region and the Greater Copenhagen Region
  • CSC Scandihealth
  • Roskilde University

Publications

> M. Hertzum & J. Simonsen: Effektdrevet IT-udvikling – Resultater fra klinisk-proces projektet. Pdf
> J. Simonsen & M. Hertzum: Evidence-Based IT Development: Toward a New Contract Model for EPR Projects. Pdf
> J. Simonsen & M. Hertzum: A PD Strategy for EPR Systems: Evidence-Based IT Development. Pdf
> EPJ-enheden - Roskilde Amt, CSC Scandihealth A/S, Roskilde Universitetscenter/Datalogi: Erfaringsrapport: Klinisk proces projekt. Pdf (100 sider, 1,5 MB)
>J. Møller-Jensen, I.L. Pedersen, & J. Simonsen: Measurement of the Clinical Usability of a Configurable EHR. Pdf
> A. Barlach & J. Simonsen: Which Parts of a Clinical Process EPR Needs Special Configuration. Pdf
> Barlach, A. & J. Simonsen: Effect specifications as an Alternative to Use Cases. Pdf
> Simonsen, J & M. Hertzum: Participative Design and the Challenges of Large-Scale Systems: Extending the Iterative PD Approach. Pdf
> Hertzum, M. & J. Simonsen: Positive effects of electronic patient records on three clinical activities Pdf
> Simonsen, J. & M. Hertzum: The Role of Ethnography in the Organizational Implementation of IT On-line article