X

Easy Access to WSO2's Online Resources During COVID-19 Lockdown.   Read More

White Paper

WHITE PAPER

4/2019

Evaluating the User Experience of Your Enterprise Products: A Framework for Developers and Testers

By Minoli Perera and Dakshika Jayathilaka

1. Introduction

In reality, developers, testers, and user experience (UX) designers have different focus and expertise. However, with the growth of digital technology, developers and testers will need to broaden their expertise to include the UX field as it plays a vital role when creating enterprise-level products and services.

Currently, there is no framework created for developers and testers to evaluate UX in enterprise software. The proposed framework provides a simple checklist for developers and testers to evaluate feature/product-wise UX aspects qualitatively and quantitatively. The checklist is created around Nielsen's Usability Heuristics for User Interface Design along with additional items to capture different aspects in UX. This will help to identify, in advance, the actual experience gaps that end users may face in real deployments. This will reduce rework, saving developer and user time and effort.

This white paper will introduce a developer/tester focused framework to evaluate UX in enterprise products.

2. Framework

The framework is built around four quantitative metrics and seven qualitative metrics to evaluate UX. The quantitative metrics are associated with checkpoints that define the metrics.

The table below shows the checkpoints that are associated with each metric.

Metrics Description Options/Checkpoints
Time on task Total time taken to complete the task -
Success rate Successful completion rate of the task
  • Direct - completed successfully on the first attempt
  • Indirect - failed the first attempt but completed within next attempts
  • Failed - Could not complete the task
Difficulty score Difficulty level in completing the task
  1. -  Very Easy
  2. -  Easy
  3. -  Moderate
  4. -  Difficult
  5. -  Very Difficult
Number of errors Number of errors the user experienced when completing the task -
Findability Ease of finding relevant functionality, content or information
  • Recognition rather than recall
  • Flexibility and efficiency of use
Simplicity A clean, understandable, and minimalistic design, which provides the most straightforward way to do the task
  • Aesthetic and minimalist design
  • Clear call to action
  • Clear and consistent navigation
Intuitiveness Ease of use of the system to the degree where the user is able to interact without applying knowledge consciously
  • Clear call to action
  • Match between system and the real world
  • User control and freedom
  • Happy path work
  • Error prevention
  • Clear and consistent navigation
Consistency Similar elements/functions having a consistent look and functionality
  • Consistency and standards
  • Clear and consistent navigation
Guidance Directing and guiding users to perform tasks
  • User guidance
  • Visibility of the system status
  • Help and documentation
  • Error recognition
  • Error recovery
Accessibility and Compatibility (Optional) Comply with accessibility standards and cross-browser support
  • Comply with accessibility standards
  • Cross-browser compatibility
Responsiveness (Optional) The system renders well when viewed on devices of different screen sizes
  • Responsive design

3. Checkpoints

Checkpoints provide clear guidelines when evaluating UX in products. Each checkpoint is associated with check items. See below for the list of checkpoints with relevant check items.

1. Clear and consistent navigation

This indicates a quick and easy flow of events through the system. Users should be able to have the ability to go back.

Check items:

  1. Is there a quick and easy flow through the system?
  2. Can the user go back to the previous action?

2. Happy path work

Was the user able to perform the action/task featuring no exceptional or error conditions?

Check items:

  1. Was the user able to perform the action/task featuring no exceptional or error conditions?

3. Visibility of system status

The system should keep users informed about what is going on, through appropriate feedback within a reasonable timeframe. Progress should be indicated for multi-step workflows.

Check items:

  1. Is the user informed about what is going on?
  2. Is the user given relevant feedback after a particular action, e.g. notification, status bar, and loading indicator?
  3. Does the action give feedback within a reasonable context (time and place)?

4. Match between the system and the real world

The system should speak the users' language, with words, phrases, and concepts familiar to the user, rather than system-oriented terms. Follow real-world conventions, making information appear in a natural and logical order.

Check items:

  1. Does the design match with real-world conventions and concepts?
  2. Is the labeling convention that is used in the system familiar to you?
  3. Does the information appear in natural and logical order?

5. User control and freedom

Users often choose system functions by mistake and will need a clearly marked "emergency exit" to leave the unwanted state without having to go through an extended dialogue. Should support undo and redo.

Check items:

  1. Is there a clear exit to the task, e.g. Cancel?
  2. Can the user undo the action?

6. Consistency and standards

Users should not have to wonder whether different words, situations, or actions mean the same thing. Consistency should be maintained throughout the user flow.

Check items:

  1. Does the flow maintain the same patterns and consistency?

7. Error prevention

Users should be prevented from problems even before their occurrence in the first place. For this, either eliminate error-prone conditions or check for them and present users with a confirmation option before they commit to the action.

Check items:

  1. Is all relevant validations provided?
  2. Is there a confirmation option before committing to actions?

8. Error recognition

Error messages should be expressed in plain language (no codes), precisely indicating the problem.

Check items:

  1. Is the error message expressed in plain language?
  2. Does the error message precisely indicate the problem?

9. Error recovery

This verifies whether the messages constructively suggest a solution or not.

Check items:

  1. Are the messages constructively suggesting a solution?

10. Recognition rather than recall

Minimize the user's memory load by making objects, actions, and options visible. The user should not have to remember information from one part of the dialogue to another. Instructions for use of the system should be visible or easily retrievable whenever appropriate.

Check items:

  1. Are the objects, actions, and options clearly visible?
  2. Are the instructions for the task visible or easily retrievable when necessary?

11. Flexibility and efficiency of use

Accelerators - unseen by the novice user may often speed up the interaction for the expert user such that the system can cater to both inexperienced and experienced users. These allow users to perform frequent actions faster.

Check items:

  1. Does the task cater to both experienced and inexperienced users?

12. Aesthetic and minimalistic design

Dialogues should not contain information that is irrelevant or rarely needed. Every extra unit of information in a dialogue competes with the relevant units of information and diminishes their relative visibility. Related information is clearly grouped together.

Check items:

  1. Does the system contain information that is relevant and needed? (no irrelevant or rarely needed info)
  2. Is the information/elements properly positioned and aligned?

13. User guidance

Users should be guided throughout the process.

Check items:

  1. Is the user guided throughout the process?

14. Help and documentation

Even though it is better if the system can be used without documentation, it may be necessary to provide help and documentation. Any such information should be easy to search for, be focused on the user's task, list concrete steps to be carried out, and not be too large.

Check items:

  1. Is the necessary documentation provided?
  2. Does the content appear in a structured and logical order?
  3. Is the content up to date and accurate?

15. Clear call to action

Users should be able to clearly define the next action, which needs to be performed.

Check items:

  1. Was the user able to clearly define the next step/action to perform?

16. Restore task progress

Users should be able to resume work from where they left off.

Check items:

  1. Can the user resume work from where they left off?

17. Comply with accessibility standards (optional)

The system should conform to Section 508 Standards or W3C Web Content Accessibility Guidelines (WCAG) 2.0/2.1 Level A/AA/AAA.

18. Cross-browser compatibility (optional)

The system works as expected in selected browsers.

Check items:

  1. Does the system work as expected in selected browsers (IE8–11, CH, FF, SF)?

19. Responsive Design (optional)

The system should be accessible through selected devices.

Check items:

  1. Does the system work as expected in selected devices, e.g. mobile, tab, and desktop?

4. Approach

There are three phases for the approach. Developers need to go through all three phases and testers can start from the second phase (task breakdown) onwards.

4.1 Pre-development research

Developers need to do pre-development research when adapting the framework to develop user-centric software.

Pre-development research includes:

  • Identifying/mapping the high-level epics with the feature
  • Identifying the related personas
  • Creating user stories related to the feature

4.2 Task breakdown

Check whether the items are marked against the tasks that are derived from the user stories. A feature may have one or more tasks associated with it. A task list needs to be created in order to evaluate the feature/scenario.

4.3 Evaluation

According to the template, the developer/tester needs to go through the check items for each task and mark the fields related to each check option.

There are three types of check options:

Irrelevant

- Irrelevant

The checklist item does not map to the task or the task is out of the scope of the system.

Fulfilled

- Fulfilled

The checklist item is fulfilled in the task.

Unfulfilled

- Unfulfilled

The checklist item is not fulfilled in the task.

After filling the check items, developers/testers will get a clear understanding on the missing UX aspects of the feature/scenario. Developers can work on the unfulfilled items before releasing the feature and testers can report issues.

5. Conclusion

The UX framework discussed in this white paper will help developers and testers to identify and bridge the experience gaps that customers and end users may face even before releasing the feature. This saves both the developer’s and customer’s time and effort. It covers the key UX principles and aims at improving customer and end-user experience.

6. References

For more details about our solutions or to discuss a specific requirement contact us.

x

Interested in similar content?