UX Portfolio

UXD ‘Systems’ Usability Test & Redesign

The Brief

The module coursework was to analyse, usability test and then redesign and prototype, a screen-based, work-oriented user experience. I chose Cardiff Bus.com. Please find links to both low and high fidelity prototypes, a video explaining the redesign and the CIF Usability test.

The Prototypes

High fidelity prototype: HTML and CSS with Bootstrap can be viewed here: http://www.crystalhinam.co.uk/cardiff-bus-prototype

Low fidelity prototype: Clickable PDF built using ‘Balsamiq’ software, can be accessed here: Cardiff Bus Prototype

cardiff-bus

Task to complete using prototype:

Find a bus route number.
“Find the bus route to get you from ‘Barry Island Marine Hotel’ bus stop to ‘Central Station Wood Street’ bus stop in Central Cardiff at 08:19 on a Monday. “

Steps to complete task:
Home page: Enter start and end destination, date and time (it doesn’t matter what data input) or leave blank >Click ‘Search’ > Journey part one – suitable journeys list: click ‘select route’ > Journey part two -travel route: > click ‘map’ > Journey part three – selected route >Click ‘Barry Island Marine Hotel’ >Selected bus stop.

Multimedia Presentation: Keynote presentation including stills, ‘Silverback’ video and annotations exported as .mov file available to view on YouTube:

 

CIF Usability Test Report

I selected www.cardiffbus.com to research, test and redesign. My evaluation report can be viewed as a PDF:  K1356636-Cardiff-Bus-Report.

Introduction and Background

Cardiff Bus
CardiffBus.com is a website for bus transport in Cardiff and the surrounding areas for the company ‘Cardiff Bus’. The website provides the public with route, timetable and location information. Cardiff Bus is a well known and established company, originating in 1902 (Cardiff Bus, 2013).

1-cbCurrent Trends / Situation
Survey results by the Welsh Assembly Government (2010) show only 16% of bus users in Cardiff and surrounding areas (SWWITCH) obtain bus routes and timetables information using websites. If a greater volume of Cardiff Bus users obtained information via online sources this could save money in producing printed timetables and answering phone calls, and information would be up to date for users and easy to update for Cardiff Bus staff.

 Overall problem
The key area of focus for the Cardiff Bus usability test was to identify issues experienced by users whilst obtaining the information required to travel via bus:

  • Bus routes
  • Bus times
  • Bus ticket prices
  • Bus stop locations

Current Website Interaction
The Cardiff Bus website has a simplistic site map and each task can be completed in a few steps, however due to inconsistent navigation and unstructured layout, I intend to discover if users will take the optimal path or produce errors and take alternative paths.

Expected Usability Issues
Users may initially locate the correct page of the site for most tasks but will get confused/frustrated when attempting to locate specific information due to poor interface design, information architecture and an inconsistent main navigation. I expect to encounter results of inefficiency and errors with an overall feeling of dissatisfaction for the website.

However, the website may score well on learnability and memorability due to the simplicity of the site map. Users may learn how to locate the information they need quickly; I have created two task scenarios that are similar to test this prediction. I think the website is aesthetically pleasing; it has a simple and bold appearance with an appealing colour palette, I foresee it will score well on visual appearance.

Aims
Using my test scenarios and post-test questionnaire I aim to gather quantitative and qualitative insights in to the information seeking features on the Cardiff Bus website.

My usability test will focus on the following areas:

  • Finding bus routes.
  • Finding bus times.
  • Finding ticket prices.
  • Finding bus stops/station locations.

I will access my aims using the following criteria:

  • Learnability: How easily and quickly users complete tasks on their first visit of the Cardiff Bus website.
  • Memorability: How easily and quickly users can locate the correct bus route after completing a similar task previously in the test.
  • Efficiency: How easily and quickly users can locate bus routes, times and locations after they have learned the design.
  • Errors: The number of errors users make when locating bus routes, times and locations, and how easily users recovered following these errors.

Analysis

Personas
My personas are based on a survey of bus passengers in Wales at the end of 2010 (Welsh Assembly Government (2011) by the Welsh Assembly Government. The survey provides an insight into general bus user groups:

  • More women than men use buses.
  • More teenagers and older people (aged 45 and over) use buses. Proportionally fewer people use the bus in the aged 26 to 44 group.
  • More bus users are students, or are economically inactive (through having full time caring responsibilities, retired and so on).

Persona 1 is a student bus user, according to the Welsh Assembly Government, more bus users are students, or are economically inactive (through having full time caring responsibilities, retired and so on).

Persona 1: Student Bus User

persona-1

Persona 2 is a young professional woman, according to the Welsh Assembly Government study more women than men use buses and more teenagers use buses.

Persona 2: ‘No hassle’ traveller

persona-2

Persona 3 is a retired lady with a free bus pass, the Welsh Assembly Government study stated that more bus users are economically inactive (through having full time caring responsibilities, retired and so on).

Persona 3: Bus Pass User
persona-3

Method
A Common Industry Format (CIF) usability test was carried out on the Cardiff Bus website to evaluate a user’s ability to complete tasks focusing on bus routes, timetables and ticket prices, and was compared with a competitors website.

Usability testing took place at 3 locations:

  • Kingston University, Kingston upon Thames, Surrey KT1 2EE on 06/12/13
  • Flat Palace Garage, Palace Road, KT1 2LG on 06/12/13
  • 84A Port Road East, Barry, CF62 9PU 14/12/13.

on 06/11/2013 using:

  • ‘Silverback’ application
  • Firefox browser version 25.0
  • Apple MacBook Pro 13-inch: 2.9GHz
  • iPhone 4S
  • Stopwatch App on iPhone 4S

Sessions were recorded and analysed to identify potential areas for improvement to the website. The following was captured:

  • navigational choices
  • task completion rates
  • comments
  • errors
  • overall satisfaction ratings
  • questions
  • overall feedback

Participants
Participants were contacted and recruited through verbal contact. Following verbal contact, attendees were contacted via email, telephone and text message requesting their availability for participation on specific dates. Participants responded with their specific times.

Six participants were selected, aged between 26 – 56, four female and two male; all use the internet on a regular basis. Two participants were students (persona 1) who also work part-time, two participants were in full time employment and fall into the low bus user group of ages 26 to 44 (persona 2) and two participants were aged 55 and over, living in the Cardiff area (persona 3). Reason for selection was based on research on from a survey of bus passengers in Wales at the end of 2010 (Welsh Assembly Government (2011) by the Welsh Assembly Government.

Participants

table-participants

The Usability Test

  • Each individual session lasted approximately 50 minutes. The format was as follows:
  • Participants were welcomed into the testing environment.
  • Participants were given a brief overview of the test format and asked to read and sign a consent form.
  • Participants were asked to fill out a brief background questionnaire.
  • The hardware and software used to record the session was shown and described to the participants.
  • Participants were given a list of task scenarios to complete using the Cardiff Bus website and Arriva Trains Wales website as a competitor comparison.
  • Following the tasks, participants were asked to rate the overall website interface and experience using a 7 point ‘Likert Scale’ ranging from selections ‘not at all’ to ‘very’. This form rated usability, attractiveness, efficiency, learnability and memorability.

Evaluation Tasks/Scenarios
Test participants attempted completion of the following tasks:

Task One: Find a bus route number.
“Find the bus route to get you from ‘Barry Island Marine Hotel’ bus stop to ‘Central Station Wood Street’ bus stop in Central Cardiff at 08:19 on a Monday. “

HTA-1

 Task Two: Find a bus ticket price.
“Find the price of a single fare ticket from ‘Barry Island Marine Hotel’ bus stop to ‘Central Station Wood Street’ bus stop in Central Cardiff at 08:19 on a Monday. “

HTA-2

Task Three: Find a group ticket price.
“Yourself and 3 friends are staying in Pentrebane, Cardiff for the weekend. You need to travel from ‘Pentrebane (Beechley Drive)’ bus stop to ‘Central Bus Station’ to go shopping then travel back to ‘Pentrebane (Beechley Drive)’ using the same bus stops. How much will a group ticket cost? .”
HTA-3

 

Task Four: Find bus route letter (location).
“You don’t know Cardiff very well and want to find out where to catch your bus to get to Pentrebane (route 61). Locate the bus station letter for route 61 from Cardiff. “

HTA-4

Task Five: Find a bus route number.
“Find the bus route to get you from ‘Pentrebane (Beechley Drive)‘ bus stop to meet a friend at ‘Central Bus Station’ bus stop at 19:15 on a Tuesday.”

HTA-5

Task Six: Find a train time and price.
“Find the train time to get you from ‘Barry Island’ train station to ‘Cardiff Central’ train station by 08:54 on a Wednesday. “
HTA-6

 

Results

Overall, participants found the Cardiff Bus website to be functional but frustrating and confusing. All tasks were completed, but produced assists and errors.

Qualitative results were collected through comments, actions, errors and assists.

Participants commented on their frustration when using the website, especially on first encounter. Participant comments reflect this:

  • “Every task was frustrating” Participant 6.
  • “This website is atrocious” Participant 6.
  • “I can’t find my way back to the previous page” Participant 6 when maps opened in PDF format.
  • “I would have quit by now and picked up the telephone.” Participant 6 on task 1.

A highlighted issue is the non-logical layout on most pages; participant comments reflect this:
“This wasn’t what I expected” Participant 2.
“This is a bit much, it’s too complicated” Participant 4.
“There is too much to read” Participant 4.
“This page is a bit long, I’m a bit confused” Participant 5.

Another highlighted issue is the lack of input areas to insert locations rather than searching for them:
“I’d expect to find somewhere to put my start and end details.” Participant 1.
“I’m looking for an A to B area.” Participant 2.
“Nowhere to pick a start location, I would have expected that.” Participant 6.

Findings per task
Quantitative results were captured through time on task and post test questionnaire data.

Time on Task
The time on task for each participant was recorded and noted. Some tasks were more difficult to complete than others and this is reflected by the average time on task.

Time on Task (seconds)

table-time

The time on task show interesting results:

  • Task 1 and task 5 ask the participant to do the same task, but to locate different areas. 5 out of 6 participants completed task 5 significantly quicker than task 1, this shows the site’s learnability is high.
  • 5 out of 6 participants located both the time and price for a transport ticket on a competitors website in task 6 in a quicker time than locating the time alone in task 1 on the Cardiff Bus website.

Mean and Median time per task

table-time-mean

 

graph-time

Errors
During each task scenario, the number of errors and assists were recorded. Results show that participants first encounter with the website in task 1 had the highest number of errors (10) and assists (7). Task 5 is a replica of task 1 with a different destination to locate, this task had fewer errors and assists showing the websites high learnability. Task 6 had no assists or errors, this was a task scenario using a competitors website. Participants had not encountered this website previously, suggesting the interface is user friendly and highly useable.

graphk-errors

table-errors

Overall Metrics
After task completion, participants completed a questionnaire rating the website using a scale ranging from ‘completely agree’ to ‘completely disagree’:

table-post-test

graph-post-test

The questionnaire results show that participants:

  • believe the site is not easy to navigate.
  • could not locate the information they needed.
  • could not find the information they needed quickly.
  • were not satisfied with the information displayed in ‘routes and times’.
  • were not satisfied with the information displayed in ‘route maps’.
  • were not satisfied with the information displayed in ‘tickets and fares’.
  • were not satisfied with the layout of webpages.
  • would not recommend the website to others (if relevant)
  • completely agreed that the ‘Arriva Trains Wales’ website easier to use.

Overall Ratings
Participants also rated the Cardiff Bus website on it’s usability, attractiveness, efficiency and learnability; the Mean results are shown in the table and graphs below:

table-usabity

graph-osability

Usability
Participants weren’t satisfied with the Cardiff Bus websites usability; all ratings were 4 and under out of 7.

Attractiveness
The Cardiff Bus website scored average and below average ratings by all participants for attractiveness.

Efficiency
Participants weren’t satisfied with the Cardiff Bus websites efficiency; ratings were all 3 and under out of a possible 7.

Learnability
Participants were relatively satisfied with the websites learnability, when completing similar tasks for a second time, task times were significantly shorter.
Participant likes, dislikes and recommendations
Upon completion of the 6 tasks, participants provided feedback of what they liked most and least about the Cardiff Bus website, and provided recommendations for improving the website interface.

What participants liked the most
Participant comments:

  • “I like the way train trains are displayed in a side scrolling list” Participant 3.
  • “That was easy to find, I like that” Participant 4 when locating a group ticket price.
  • “Once I was familiar it’s easier to use” Participant 2 when completing task 5 which is similar to task 1.

What participants liked the least
Participant comments:

  • “This wasn’t what I expected” Participant 2 when single route page was displayed.
  • “This is a bit much, it’s too complicated” Participant 4 when ‘routes and times’ page was displayed.
  • “There is too much to read” Participant 4 when ‘routes and times’ page was displayed.
  • “This site is not good for foreign visitors, there is a language barrier” Participant 2.
  • “This page is a bit long, I’m a bit confused” Participant 5 when single route page was displayed.
  • “I can’t find my way back to the previous page” Participant 6 when maps opened in PDF format.

Recommendations
Following the usability test results and post-questionnaire results analysis, the following recommendations will address and improve the areas where participants experienced problems and frustrations, or found the website interface and information architecture confusing.

Priority Legend:
High = Task cannot be completed.
Medium = Task completed with significant effort and failed attempts.
Low = Task completed with minor complications and/or annoyance.

change-1

change-2

change-3

change-4

Conclusion
Most participants found the Cardiff Bus website to be functional but felt that it requires considerable improvement to ensure a user friendly experience. If the design recommendations are implemented the user experience will improve significantly for usability and efficiency.

If repeating the usability test, it could be improved by using:
A wider age range of participants. Users to match each persona could provide extra user needs resulting in further ideas for redesign and alternative problems with the website in terms of accessibility.

The evaluation has provided helpful feedback through highlighting problematic task scenarios and website user interface features in these scenarios. The redesign will concentrate on improving the website usability and efficiency, predominantly by linking the main website functions:

  • Journey routes
  • Journey prices
  • Journey times and
  • Journey maps

By making these changes I feel the overall user experience would be greatly improved.

References

WWW Website
Cardiff Bus (n.d) About us website content (Online). Available at <http://www.cardiffbus.com> (Accessed 28/10/2013)

WWW Document
Welsh Assembly Government (2011) Welsh Bus Passenger Survey 2010 (Online). Available at <http://wales.gov.uk/topics/statistics/headlines/transport2011/110311/?lang=en> (Accessed 28/10/2013)

WWW Document
Welsh Assembly Government (2010) Bus passenger survey results for: SWWITCH Regional Authority – Wales (Online). Available at < http://wales.gov.uk/docs/statistics/2011/110311busswwitchen.pdf> (Accessed 28/10/2013)

WWW Webpage
Cardiff Bus (n.d) About us website content (Online). Available at <http://www.cardiffbus.com> (Accessed 28/10/2013)

Image (A)
Welsh Assembly Government (2010) Bus passenger survey results for: SWWITCH Regional Authority – Wales (Online). Available at < http://wales.gov.uk/docs/statistics/2011/110311busswwitchen.pdf> (Accessed 28/10/2013)