Deck 5: Methods of Evaluation

Question 1

Human Factors professional use methods of evaluation to assess a product, system, or process because

A) It provides job security for Human Factors professionals
B) They help to insure that the product, system, or process meets the needs of the users
C) It is a requirement in the field of Human Factors
D) It is nice to do but most of the time no problems are found

Answer 1

B

Question 2

In the opening vignette, Bennie had trouble registering for classes because:

A) It was a new system
B) There was a lack of communication about the new system
C) Bennie was an atypical user
D) Both a and b

Answer 2

D

Question 3

One method used to gather information about human interactions in systems and processes is:

A) Interviews
B) Literature reviews
C) Expert consultation
D) Reading a textbook

Answer 3

A

Question 4

A method of involving end users throughout the design process is called:

A) User Design
B) Usability Testing
C) User-centered design
D) User requirement analysis

Question 5

User __________ and ____________ are gathered through the use of observation, interviews, and questionnaires.

A) Needs and requirements
B) Opinions and desires
C) Behaviors and actions
D) Testing and evaluation

Question 6

________________________ is a methodology that gathers knowledge from individuals.

A) Usability testing
B) Card sorting
C) Tacit Technique
D) Participatory design

Question 7

Explaining how to ride a bike taps into a person's

A) Tacit knowledge
B) Explicit knowledge
C) Implicit knowledge
D) Active knowledge

Question 8

_________ knowledge is difficult to verbalize while __________ knowledge is easy to verbalize.

A) Tacit; Explicit
B) Explicit; Tacit
C) Implicit; Explicit
D) Obscure; Common

Question 9

All of the following are types of interviews except:

A) Structured interviews
B) Semi-structured interviews
C) Completely-structured interviews
D) Unstructured interviews

Question 10

Data analysis is easiest with which of the following types of interviews?

A) Unstructured
B) Semi-structured
C) Structured
D) None of the above

Question 11

In an unstructured interview with free-form questions, researchers use a process called -------------------------------to identify the common underlying themes in responses.

A) Card sorting
B) Usability testing
C) Participatory design
D) Content analysis

Question 12

Interviews sometimes suffer from intentional or unintentional cues within the environment that suggest how a respondent should answer. These cues are called:

A) Suggestions
B) Demand characteristics
C) Affirmation bias
D) Explicit cues

Question 13

The following are all types of closed-ended questions one might find in a questionnaire except:

A) Likert-type
B) Two-alternative forced choice
C) Multiple-choice
D) Free-form choice

Question 14

"I like cake and icing" is an example of what kind of statement:

A) Leading
B) Loaded
C) Double-barreled
D) Valid

Question 15

A leading question in a questionnaire is:

A) The first question in the questionnaire
B) When two distinct questions are asked in one single question
C) A question that suggests how it should be answered
D) A statement that precedes an actual question

Question 16

A typical Likert-scale has:

A) Two alternative choices
B) An area for the respondent to type a comment
C) At least 10 points (i.e., 1-10)
D) Benchmarks that describe each point on the scale

Question 17

An even number of points can be used with a Likert-scale when:

A) Two choices are possible
B) The researcher does not want to allow a neutral response
C) The questionnaire is being validated
D) It is never appropriate to use an even numbered scale

Question 18

An interview with multiple people, that have something in common, at the same time is called:

A) A group interview
B) Group think
C) Focus group
D) Benchmarking interviews

Question 19

All of the following are possible problems with a focus group except:

A) Participants are able to talk to one another about an issue
B) Participants do not trust the facilitator or others in the group
C) One dominant person in the group does all of the talking
D) The topic is of a sensitive nature and difficult to discuss among strangers

Question 20

Sam is a Human Factors professional and decides to shadow a factory worker, Emma, for a day to learn more about her job. She takes notes on Emma's work flow, tasks, environment, and interactions. Sam is conducting a:

A) Usability study
B) Shadow box plot
C) Contextual inquiry
D) Task analysis

Question 21

The detailed process of identifying a task's goals, sub-goals, and plans is referred to as:

A) Hierarchical task analysis
B) Contextual inquiry
C) Usability study
D) Task domain analysis

Question 22

Cognitive task analysis (CTA) differs from hierarchical task analysis (HTA) in that:

A) CTA builds upon HTA with cognitive and decision-making aspects
B) HTA builds upon CTA
C) HTA and CTA re the same thing
D) CTA can only be done with decision-making experts

Question 23

Human error identification (HEI) is a technique used to identify potential sources of error at each task and subtask of a:

A) CTA
B) HTA
C) ETA
D) None of the above

Question 24

"Monica" is a young female entrepreneur who is interested in technology and innovation. She is an early adopter to the latest tech gadgets, wears a smartwatch, and taught herself computer programming. Such a description of a potential user of a product is called a:

A) User profile
B) User description
C) Monica's world
D) Persona

Question 25

Two methods discussed in the chapter to evaluate the ease of use of a software application interface are:

A) Task analysis and heuristic analysis
B) Contextual inquiry and user profile
C) Heuristic analysis and usability testing
D) Questionnaire and focus group

Question 26

One way to convince corporate executives to conduct usability testing, or other Human Factors methods, is to:

A) Show them the costs of the activity and the resulting benefits
B) Show them a case study demonstrating the value of the HF activity and impact on customer satisfaction
C) Calculate per user how much $$$$ is saved by implementing fixes that resulted from the activity
D) All of the above

Question 27

Usability testing is a method of assessing a user interface by:

A) Surveying users on their likes and dislikes
B) Observing users complete representative tasks individually
C) Discussing the software in a focus group setting
D) Posting a request for feedback on a social network

Question 28

Data collected during a usability study includes:

A) Objective data (time, success, # steps to complete)
B) Subjective data (difficulty, satisfaction)
C) User opinions on likes and dislikes
D) All of the above

Question 29

Variations to traditional usability testing that is done face-to-face with a single user include:

A) Remote testing where the user and facilitator are separated geographically
B) Presenting the application in a large auditorium projected screen and asking for input
C) Two participants work through the tasks together with one facilitator
D) Both A and C

Question 30

Browsing the internet for a visually impaired user:

A) Is sometimes done using a screen reader that speaks the text content of the page aloud
B) Can only be done using extreme magnification
C) Can only be done with the assistance of a sighted person
D) Can only be done at specialized clinics that have expensive equipment and monitors

Question 31

Describe user-centered design. How does it differ from participatory design?

Question 32

Define interview, questionnaire, and focus group.

Question 33

List Nielsen's 10 heuristics along with an example.

Question 34

Describe three types of questions to avoid when constructing a survey and an example of each.

Question 35

Describe two measures that are typically gathered in a usability test.

Question 36

Compare and contrast interview, questionnaire, and focus group techniques of information gathering. Describe the pros and cons of each.

Question 37

Compare a heuristic evaluation to a usability test. What are the benefits of each?

Question 38

You are working for an office supply store on their website. They want to redesign their main navigation menu listing all of their products. Describe how you would create and conducts a card sort to examine this. How would you summarize your results?

Question 39

For the same office supply store in #3, create three usability test tasks that you could use to assess the usability of the new site navigation. List what measures you would gather after each task and at the end of the entire test.