RP002 Usability Testing Round 2

TBD: Short paragraph on goal and purpose of this usability testing.

Contents

1. Prerequisites
   1. Test Plan
2. Preparation
   1. Website Overview Page
   2. Target Users
   3. Sample Project Landing Pages
3. Recruitment
   1. End Users
   2. Volunteer UXRs
4. Interview Logistics
   1. Interview Materials
   2. Tool Setup
   3. UXRs Training
5. Results
   1. Interview Records
   2. Data Analysis
   3. Identified Issues
   4. Journey Map
6. Additional Resources
   1. Guide to Usability Testing
   2. Referenced Resources
   3. Contact Us

1 - Prerequisites

i - Test Plan

• Created a test plan which was tailored to the product end users
  • Referenced prior usability testing 1
  • Discussed Recruitment with Stakeholder
  • See test plan (TODO: LINK[Test Plan])

2 - Preparation

i - Website Overview Page

• Reused the overview webpage which presents the LA TDM Calculator project
  • Serves as a landing page for user recruitment in usability testing
  • Introduces LA TDM Calculator Project detailing about:
    • TDM and TDM Calculator
    • TDM Program Update
    • Current status of TDM Calculator
    • Resources from LADOT
  • Becomes future homepage for the HfLA TDM Calculator project
    • Future state of the website will be an overview of the project detailing about HfLA's contributions
    • The design will have to be reworked to make a clear distinction from the TDM calculator and our project which seeks to work on government calculators
      • Old design

ii - Target Users

Reused work from prior usability testing

• Identified and grouped users types into buckets that best represent end users
  • Specified user permission levels and user stories in context of TDM Calculator to later match end users with the appropriate sample project landing pages
  • Referenced survey from EmpowerLA which is another HfLA project

New for this usability test

• Revised (TODO: LINK[user survey questions]) that aimed to gather personal and industrial demographic information to ensure representative samples from all user types

iii - Sample Project Landing Pages

Reused work from prior usability testing

• Worked with LADOT and LACP to create sample projects, which later were used to make landing page assets

  • Reviewed some LACP forms to understand how TDM Calculator's target users are used to providing project information and what pieces of project information might be needed in the planning process. These forms are generally used during the planning process and are not being replaced or required by the TDM calculator and some fields in the forms share similarities to the questions asked by TDM calculator.
    • Planning Application, Transportation Study Assessment, Application for Building Permit or Grading & Certificate of Occupancy, and other applications/forms
  • Reviewed references for sample projects to help determining project levels and applicable strategies
    • This was last edited by Kevin H in Feb 2021. It looks like we intended to use it to define which sample projects and landing pages fit each category so that we had good coverage of all user experiences regardless of the size of developer. However, we did not connect the details in this workbook with any of the landing pages later created.
  • Utilized case study log to determine detailed sample project information
    • We are in the process of migrating our folders from Google MyDrive to Google Shared Drive
      • PDF's landing page assets are in the new drive
      • The Google Site landing pages and TDM Calculator Usability Testing Discussion Guide per interview is currently in the old drive.
        • Old/Current
        • New that we are migrating to [TODO: UPDATE LINKS TO LANDING PAGES]
• Created landing page assets to use during user interviews, including sample projects that best match users' experience, login credentials, and satisfaction questions and corresponding likert scales, and A/B Testing for design preference

  • 1 https://sites.google.com/view/la-tdm-calculator-45364536/
  • 2 https://sites.google.com/view/la-tdm-calculator-55785578/
  • 3 https://sites.google.com/view/la-tdm-calculator-99239923/
  • 4 https://sites.google.com/view/la-tdm-calculator-88778877/
  • 5 https://sites.google.com/view/la-tdm-calculator-66556655/
  • 6 https://sites.google.com/view/la-tdm-calculator-33443344/
  • 7 https://sites.google.com/view/la-tdm-calculator-11231123/
  • 8 https://sites.google.com/view/la-tdm-calculator-66616661/
  • 9 https://sites.google.com/view/la-tdm-calculator-45454545/
  • 10 https://sites.google.com/view/la-tdm-calculator-19851985/
  • 11 https://sites.google.com/view/la-tdm-calculator-19911991/
  • 12 https://sites.google.com/view/la-tdm-calculator-19111911/
  • 13 https://sites.google.com/view/la-tdm-calculator-18551855/
  • 14 https://sites.google.com/view/la-tdm-calculator-12141214/
  • 15 https://sites.google.com/view/la-tdmcalculator-16431643/
  • 16 https://sites.google.com/view/la-tdm-calculator-76547654/
  • 17 https://sites.google.com/view/la-tdm-calculator-11991199/
  • 18 https://sites.google.com/view/la-tdmcalculator-11761176/
  • 19 https://sites.google.com/view/la-tdm-calculator-33453345/
  • 20 https://sites.google.com/view/la-tdmcalculator-22232223/
  • 21 https://sites.google.com/view/la-tdmcalculator-56565656/
  • 22 https://sites.google.com/view/la-tdm-calculator-33243324/
  • 23 https://sites.google.com/view/la-tdmcalculator-92149214/
  • 24 https://sites.google.com/view/la-tdm-calculator-44064406/
  • 25 https://sites.google.com/view/la-tdm-calculator-11771177/
  • 26 https://sites.google.com/view/la-tdm-calculator-88448844/

New for this usability test

• Updated landing pages to have design preference information between two live versions of application (sidebar)
• new [TDM Calculator Usability Testing Discussion Guide] is in the new drive folder

3 - Recruitment

i - End Users

Reused work from prior usability testing

• Recruited 31 (TODO: DOUBLE CHECK NUMBER) interested end users via distributed [user survey] on ArcGIS platform
  • Summarized [demographic information] of all interested end users who responded to the user survey
    • Intended to check if the sample is representative and help assigning interested end users into corresponding groups
• Sent [invitation to interview] to interested end users through MailChimp
  • Coordinated and carbon copied with LADOT and LACP in the process
  • Streamlined [survey results]
    • Need to processe data/result of end users who actually registered for the interview sessions to help determining appropriate sample projects for each registered end users
  • Recorded the [status of reaching out and registering for interviews]

New for this usability test -- CHECK MAILCHIMP

ii - Volunteer UXRs

• Recruited volunteering UXRs from HfLA UI/UX community for training to conduct 1:1 interviews with registered end users by sending recruiting messages in UI/UX community with the link to interest form Didn't end up recruit any volunteer... but prepared the materials (TODO: ADD LINK To TRAINING DECK)

4 - Interview Logistics

i - Interview Materials

• Steps used in finalizing the draft referenced testing 1
• Script Template for Interviews: [Template - TDM Calculator Usability Testing Discussion Guide]: ADD LINK
  • Description: Interview Script and Template for 1:1 interviews including intro, site testing, SUS scores and closing questions, which helps UXR to follow the same procedures for all interview sessions and make notes accordingly as interviews go. A copy of this template is created for each interview and the link placed into the Usability Testing Session Table
• Background Image for interview
  • Description: Zoom background image for UXRs to use for interview sessions
• 02 Usability Testing Session Table ADD LINK, made modifications on top of testing 1 materials
  • Description: Contact management spreadsheet used by the Project team to understand the status of the outreach being done by LACP and LADOT and pairing each interview with the interviewer and all the information they required to conduct the interviews including their name and email, designated interviewee guides/notes, individual link to the websites (sample project info if they don't have one, login credentials, SUS and scale question outline), project type, username and passwords, and interview time

ii - Tool Setup

• Used the [email protected] to setup the following tools
  • Zoom: Used to conduct interviews by UXRs
  • OtterAi (paid): to transcribe the interviews
  • Calendly: Used to allow interviewees to select/add to calendar time slot they are available.
  • Google Calendar: For the Calendly integration
  • Slack (provided by HfLA)
  • ArcGIS
  • MailChimp [LINK Figma file]

iii - UXRs Training [DIDN'T END UP TRAINING, BUT HAVE MATERIALS READY]

Pre-Interview

• Review Other Hack for LA project processes
  • BOP Project WIKI
• Review Slide deck from previous round
• Created Training Presentation outline and determined what style guide to use.
• Slide deck for UXR training
  • Description: Slides used to train volunteer User Experience Researchers to do 1:1 interviews

Post-Interview

• 06 TDM Usability Testing - UX Team Lead Documentation / Responsibilities
  • Description: UXR Team Lead overview of the responsibilities for each team member to ensure follow through with end user interviewees with guidance on the following topics:
    • How UXRs claim interviews as the interviewees sign up for time slots.
    • What UX team lead should do when time slots are not claimed.
    • How Team Lead notes session responsibility in the Usability Testing Session Table
    • How Team Lead handles zoom link not working for UXR
• Usability Testing - Researchers Documentation / Responsibilities
  • Description: Volunteer UXRs overview of the responsibilities to ensure follow through with end user interviewees with guidance on the following topics:
    • Where UXRs find all the resources (included in the Usability Testing Session Table) to prepare and conduct the interviews
    • How UXRs change Zoom background for interview sessions
    • Additional resources to help UXRs understand the interview processes and TDM Calculator project, and get assistance

5 - Results

i - Interview Records

• Usability Test Transcript - Otter.Ai [UPDATE LINK]

ii - Data Analysis

• [TDM - Usability Testing - Problems Identified and User Scores 05/20/2021] Replace link to the stakeholder report
• [ADD miro board]

iii - Identified Issues

• See the Issues on the Project Board [UPDATE LINK]
• See total number of open issues
• See total number of closed issues

iv - Journey Map

• Final journey Map [UPDATE LINK]

6 - Additional Resources

i - Guide to Usability Testing

• A Beginner's Guide to Usability Testing by Maze

ii - Referenced Resources

• Our survey questions were informed by the survey work Hack for LA's Open Community Survey project did with EmpowerLA (see their survey)

iii - Contact Us

• Our Slack channel: #tdm-ux

Return to Contents