So, what's the problem anyway?

In 2023, Ford Motor Company lost $1.9B in warranty costs for vehicles needing repairs, that the assumption is to build an ecosystem of tools and services to improve quality control. My team was responsible for creating a LLM chatbot experience in getting actionable insights more fast, usable, equitable, and flexible to systems engineers.

Walking in their shoes

At my core, I put users at the center of everything I do by conducting thorough research.

  • How long have you been using LLMs and what led you to start using them?

  • What types of tasks or activities do you primarily use LLMs for?

  • Have you experienced any limitations in using them so far?

  • Does the source of data matter to you?

  • What future improvements would you like to see?

  • How “human” would you say your conversations feel? Does it display a distinct personality?

  • What types of ethical concerns, if any, do you have?

After conducting interviews, I synthesized data using affinity and empathy mapping. The topics of safety, the impact of one's work and information reliability came up consistently throughout my user interviews which led to three main insights —

  • Hallucination: LLMs can sometimes generate outputs that seem realistic, but are factually incorrect or nonsensical. They may “hallucinate” information that is not grounded in their training data.

  • Safety and misuse potential: There are risks that LLMs could be misused to generate misinformation, propaganda, or other harmful content at scale. Ensuring safe and responsible development and deployment is crucial.

  • Handling of uncertainty: LLMs can generate believable-sounding content even when uncertain or incorrect. Reliably conveying uncertainty and deferring when unsure is an ongoing challenge.

Based on my findings from my user interviews and data synthesis, I created a primary persona, The Systems Engineer, and his journey map.

Duration 8 weeks

Role Product Designer

Team Product Owner, Product Manager, Product Designer, 4 Software Engineers, Data Scientist

Tools Balsamiq, Figma, Miro

Bringing it to life

I brought my sketches to life by building out my wireframes, and conducted my first round of testing to gain initial feedback on what was and was not working in my design. There were 3 primary takeaways from my first round of user testing —

  • Critique #1 - Unclear Verbiage

  • Critique #2 - Inconsistent UI: Throughout the application, I designed icon elements from searching, asking questions to filtering on data sources — to be clickable. They consist of a white circle, surrounding the grey icon. However, on the chatbot bubbles where individuals can go to download data, copy text and rate the response, the call-to-action elements were blue and clickable. Users expressed that it was confusing as it was inconsistent with the experience they encountered.

  • Critique #3 - Necessary Elements: One of the tasks was for the user to select a data source. After they select a data source, they filter to narrow their focus. The original design did not provide an option to clear their selections and caused confusion. Some users voiced that that they expected one, and some thought that clearing them manually was acceptable.

Once I established my brand's guidelines, I applied my branding to my wireframes, and put together a prototype and began user testing. I conducted my usability testing with 5 participants. I had my testees run through various tasks within the application, and speak their way through the steps of each task. I gained a lot of insight by simply observing each participant go through each task, however, I was open to feedback at the end of each usability test and gained lots of information on both the UX and UI aspects of my design.