Chapter 8
Agenda/Learning Objectives
Understand why iterative testing is critical to UCD
Define usability and what it measures
1) Understand why iterative testing is critical to UCD
a) Iterative testing: you test the design throughout the products development
b) Iterative testing lets you integrate the knowledge you acquire from frequent testing to
2) Define usability and what it measures
a) Usability testing: analyzes how well the product being tested is used by its intended users
b) When we perform usability testing, we are looking to understand five usability quality
components (MEELS):
i) Memorability: How well does the user remember how to complete a task so they may
use this information to complete new tasks? This may include knowing which menus to
use to find information on web sites or software, or even how to switch between apps
on a smart phone.
e) Your users may want features in your design that are not necessary to use the productthis
is a preference, not a need
f) Remember that you are testing the product, not the user
3) Address how to decide what usability test you should conduct
a) Summative and formative testing
i) Summative testing:
(1) More experimental
ii) Formative testing:
(1) Uses smaller sample sizes
(2) Tests more frequently during the design of a product
(3) Iterative testing
iii) A/B testing and formative testing are the two tests that will be most useful for
comparing and testing products during your design process
b) Comparison or A/B Testing
i) Compares the usability between two competitive products, two prototypes of a product,
or even an earlier versus later product version
ii) Helps answer: Is one product or version used more successfully than another? Why?
What does this say about how you will begin or continue designing your product?
4) Explain ways to plan for your test
a) Need to answer:
i) What? youre testing
b) Test goals: the desired outcomes from the test
i) Not the desired results, but what you want to learn about your product
ii) These goals can include:
(1) learning about design elements in a current prototype
iii) Test goals can be written as statements or as research questions
(1) Ex: EyeGuide Focus Prototype test goals
c) Context for testing: a discussion of preliminary research, or discovery, you have conducted
to understand your product
i) Should include:
(1) user research
(2) product research
ii) This provides your team and client context about the progress of the project so far and
how the upcoming test will move your and their understanding of the design process
forward
iii) Ex: Excerpt from Context for EyeGuide Focus Prototype
d) User profile: isolate the type of representative users needed for your test
(2) Formative, iterative test = at least 5 users
(3) A/B test = at least 20 users
ii) Which user groups (from your user profile) will you be testing?
iii) How will you find participants for testing?
(1) What users are already available to you?
(2) Professional connections?
(3) User recruiting service
f) Test procedure
i) Description of design for testing
(1) Describe what the product is thats being tested
ii) Test environment
(1) Conditions under which testing occurs
g) Data analysis
i) Dont analyze data while testing is in progress
ii) Quantitative analysis
(1) Use top-down or bottom-up coding
(2) Use severity scales to code user errors
5) Explain ways to conduct your test
a) Interacting with users
i) What kind of legal or organizational procedures do you need to follow?
ii) Do you need approval from the human research board?
iii) Does your organization have a participant consent form for testing?
b) Test script
i) Detailed map of everything that should happen from setting up the room, to engaging
with the user, and even saving data
ii) Idea is that if someone is missing, the test can still proceed because the test script has all
of the details and instructions
v) The post-test activity also needs to be covered in the script. This include post-test
interview questions or questionnaires
c) Marking errors/observation notes
i) Plan how you will take notes during the test
ii) Create a system for documenting information
d) Analyzing your data
i) Determine if outliers represent a usability problem or should be thrown out
6) Review: Takeaways, Discussion Questions, UCD Quests
a) Review key points
b) Group discussion and Q&A
