Highlights of “Usability Engineering in the Wild: How Do Practitioners Integrate Usability Engineering in Software Development?”

Back in 2014, I published this study together with a co-author. To make it more accessible, I have written the following summary highlighting the essential findings and implications of the study.

In the paper, we presented an explorative study about how usability engineering is conducted and understood in practice. Our research question was: “How do practitioners perceive and integrate usability engineering in software development?”

Through 12 qualitative semi-structured interviews with participants from 12 different software development organizations, we wanted to get an understanding and extending existing research about how usability is conducted and integrated into practical software development. By extending existing research, the intention is to support the ongoing process of refining and developing new methods and approaches for supporting practical usability engineering.

Findings

We found that:

  1. By practitioners, usability is used as an ambiguous term often mixed with other concepts, especially user experience (UX). Usability was by some described as a good feeling when designing. What makes usability ambiguous is that the practitioners rarely set and defined the objectives and purposes of usability. Regarding UX and usability, UX was linked to the overall design. Usability was linked to individual functionality and narrow design aspects, for example, the design of an interface object such as a button. Missing usability competencies in some organizations made it challenging to customize evaluations and decide when and what to evaluate.
  2. Usability is not easily explicable and was found difficult to justify and make credible. Initial designs are based on domain knowledge, and user input is used for adjustments, feedback, and visions. Visions for usability are set, but measurable goals are not.
  3. Informal and ad-hoc evaluations are common because developers need ongoing feedback, that cheaply, easily, and fast can be implemented. The practitioners did not follow a clear plan, or organized evaluation steps. The organizations had built up a body of principles and standards for individual software types – local de facto standards. These local de facto standards are developed because it’s easy, fast and useful, to keep following the same principles used in previous projects.
  4. Reporting of usability problems is done in a wide variety of ways, with the common denominator being lightweight. This can take the form of simple notes, presentations, workshops and user stories. This fits into the light hierarchies, the fast pace of projects, and makes it possible to act upon identified problems during the development process. Problem-solving is done through discussion with colleagues, and by getting inspiration from other sources such as blog-posts and online forums.
  5. All organizations followed an agile development approach. The advantage is that this approach is dynamic and allows changes during development. The disadvantage is that the focus on sprint completion can become the main factor leaving out proper or any usability engineering.

In conclusion, we found that local de facto standards highly drive practical usability engineering, and the practitioners conduct usability engineering informal and lightweight. This includes setting goals, including users, evaluating, and reporting usability problems. A majority of the development projects are completed within short time spans, and the participants need ongoing feedback that can be acted upon during development. Especially these factors are central, as such fast-paced environments do not leave much time for planned and organized usability engineering. Especially the local de facto standards were seen as an efficient approach towards creating robust designs fast. Agile development is a double-edged sword. It allows dynamics between developers and designers, but the focus on sprint completion can take away attention from usability engineering. Missing and limited usability competencies also have an impact. Some examples are ineffective inclusion of users, as well as challenges planning and customizing evaluations to specific conditions.

Implications

Some implications of the study are:

  1. Developing scales for prioritizing different user needs could be used for structuring, and facilitating discussions and decision-making when prioritizing usability problems and which product features to include. We support the idea of such straightforward scales. They are easy to understand, learn, and can be used by teams consisting of people, with a wide variety of backgrounds and roles.
  2. Developing approaches and facilitation for workshops seems relevant. Workshops are already popular because it’s a dynamic approach, but practitioners have problems using workshop sessions optimal. This is especially true when including users, because the practitioners experienced problems both running the right activities, and directing the users towards providing relevant feedback. We see a potential for optimizing an already popular method, and using existing guide lines when conducting workshops.
  3. Local de facto standards were used as local interaction design patterns. Future methods for documenting, sharing, and communicating local de facto standards within organizations could support building up a corpus of local design patterns.

Bornoe, N., & Stage, J. (2014). Usability Engineering in the Wild: How Do Practitioners Integrate Usability Engineering in Software Development?. In International Conference on Human-Centred Software Engineering (pp. 199-216). Springer Berlin Heidelberg. DOI: 10.1007/978-3-662-44811-3_12