Enterprise UI
User Experience
IoT Design
+28%
Improved usability score on the user experience, navigation and gesture usage.
-30%
Reduced task on time required for typing, editing and deleting sentences on-the-go.
Introduction
User Simulation and Experience Research Lab managed by Prof Davide (Dept. Chair at IU) was offered a grant by Google and NSF (National Science Foundation) to explore navigating technology(typing, browsing) for Blind and Visually Impaired (BVI) community as the apps rely more on screens. However BVI users face issues with privacy and safety to use screen-centric apps in public and social settings. We explored efficient ways of auditory navigation by performing exploratory and comparative research studies to identify the context of usage, and compare current apps used, for easy adaption.
MY ROLE
Research, analysis, conceptualization, information architecture, designing high-fidelity prototypes and evaluation.
TEAM
2 Product Designers
2 UX Researchers
1 BVI Researcher
3 Stakeholders
DURATION
2 Years
SOFTWARE
Proof of Concept
We built a beta version of the auditory keyboard- Keyflows using a bluetooth Kit integrated with Android package manager, and coded an algorithm for navigation, three-layer character paradigms, and mapped text entry gestures. This has an ability to move linearly across the characters to select, delete and edit the sentences using gestures then created a video simulations to evaluate its performance. In the following video, I am using Keyflow to type “leaving @ 2 PM” using auto-suggestion feature.
Proposed Screen-less keyboard: keyflows in use, to type on-the-go
Design Process
We performed initial research followed by two studies: usability and comparative testing. We took the advantage of audio simulations and video prototyping to validate the usability and efficiency of the product. We followed a technology driven approach due constraints with unavailability of reference resources and developers in the team to implement design recommendations.
High-level design thinking process
Research
To identify focus points and develop a feature roadmap, we interviewed the Blind individuals
and cleared out initial assumptions.
Literature reviews — We researched related paper works and articles on the topics of accessibility for BVI, partially impaired. This set the base of what to look for in interviews and research studies.
Expert Interview — We interviewed BVI professionals, who are tech savvy and use accessibility product to perform day-to-day activities.
Interaction with the expert during one of the lab sessions
Looking from the lens of “Hierarchy of Needs in Design Terms”, currently, digital interfaces are less shaped to fit-in the contextual usages of the BVI users. According to the World Health Organization (WHO) 285 million people are Blind or Visually Impaired (BVI). They face these challenges on daily basis:
Inhabitant to screens
Replace manual effort & reduce human errors by standards for On-the-Go Inventory Tracking.
Complexity to type-out a text
Help users easily complete their tasks - nozzle manufacturing rather on Miscellaneous tasks.
Cane in one hand and dog-leash in other.
PO Cycles should adapt to variable levels of inventory information density across verticals.
Privacy concerns and social inclusion
PO Cycles should adapt to variable levels of inventory information density across verticals.
Ideation
We created two initial low-fidelity prototypes. After thorough expert evaluations and cognitive walkthroughs, we chose Alphabetical layout based on the pros vs cons list. Then prototype was developed to validate the concept.
Alphabetical layout: User controlled, linear scanning of characters and enabled to switch to edit mode.
Probabilistic suggestion of letters until three “grams” after which it switches to suggesting words. Then advanced prototype had been created using android and MYO’s android SDK. The reason behind the programming the prototype was we wanted to present something to the expert users to spark the conversation about the concept and see what different ideas emerge.
The interaction model with defined gestures on the left
Usability Testing
BOSMA, an organization that provides workforce development and training to the blind and visually impaired.
Our mission was to understand the following when a blind individual behaves and experiences during their first exposure to a screen-free, entirely auditory keyboard:
Investigate the user experience and performance.
Gauge their navigation behavior in controlling keyflows
Understand the limits and potential of this approach to enhance the accessibility of typing
Qual results
There were major themes: Convenience factor, form factor constraints, system adaptions, user control vs system control.
This qualitative data helped us to learn the user behavior and the quantitative data, to validate the performance of the system.
Convenience factor
Phone out of sight
The user felt convenient to type the sentence without having to hold the phone.
User Control vs system control
Customize the auditory speed
Users wanted to control the speed of the text to speech(system response).
New system adaption
Slow but accomplished the tasks
Most participants were able to type at least one word in screen-less mode but typing took an inordinately long amount of time.
Form factor constraints
Breakpoints in the experience
The Myo armband posed barriers to optimal and fluid interaction.
New system adaption
Convenience factor
Novel technology for an interaction model
The user favor the adaption of this new system mainly the convenience it offer irrespective of the context of usage.
Moreover, the navigation experience keyflows offer seemed easy to use with less learning curve.
High Learnability
Easy to learn system layout and interaction components
Low Recall rate
Difficulty in recalling the gestures and context
Low error rates
Limited gestures improved typing performance
High Task on Time
Typing combination chars led to context switching
Low Cognitive load
Linear nav and simple gestures has less cog load
Ideation
The keyflows concept is again modified to incorporate the insights drawn from the usability study as it impeding the task achievement time which is defining the purpose of quick messaging.
We explored nimble ways to model to reduce the cognitive load. To develop the prototype, we performed a few expert evaluations of each layout then we finalized on a hybrid layout of the all the three.
Followed by prototyping with alternative screen-free input device. Along with that we ideated suitable gestures such as taps and mapped the functionalities. To develop the prototype we used android and TapStrap’s android SDK. These are the following features:
The interaction model with defined gestures on the left
Comparative Analysis
To assess how our new design performance, we sought to do a comparative study against an accessibility industry standard, VoiceOver.
We strategies the study plan and recruiting procedure aligning to IRB protocols. We conducted the interview sessions with 12 BVI individuals, who have the used Keyflows functionality before, over the span of 4 weeks and interview session for an hour long.
Which device is preferred and why?
Why Voiceover is faster?
Can Keyflows be a substitute to VoiceOver; if so how?
What are the potentials of screen-less approach to enhance the typing usability?
The comparative representation of complex screen-based VS simple auditory/gestured based keyboard
Qual results
Similar to the first version, the V2 was also been validated for its design criteria through formulated approaches for screen-less typing such as Degree of using a specific gesture to perform a task (typing or editing), User Burden, System Usability. We inferred that the high cognitive loads and learnability was due to the increase in the number of gestures. Yet, this predictable layout of the model and landmarks for quick access led to high efficiency, task on time, and accuracy.
Acquired user preference
Typing got faster and better over time
Though voiceover posed complexity to type, user performed the tasks efficiently. Moreover, there users with partially blind were able to see the screen.
Socially acceptable
Social inclusivity is a pay-off
Keyflows can be only a situation substitute( Social inclusivity) of VoiceOver but not always. Even to type the quick messages on-the-go or while engaged in a social setting.
New system adaption
Convenience factor
Gestures of the new-off-the-shelf device was an overkill
Users preferred Keyflows, but it lacked in the performance due to difficulty caused by device and gestures - having to recall and confusion caused while mapping them to their functionality.
Preference for privacy
90% of users prefer Keyflows
The user felt convenient to type the sentence without having to hold the phone vs the voiceover.
System customization
Customize the layouts
Users can build the sense of ownership when they the system has the ability to manipulated the layouts to match their typing patterns.
Quant Results
Improved Efficiency
Switching layouts and the typing was efficient
High Accuracy
Predictive layout improved typing with no errors
Low Learnability
Of the system layout and interaction components
Low Task on Time
Typing combination chars led to context switching
High Cognitive load
Three paradigms nav and gestures has high cog load
DESIGN GOAL: Personalization
USER SCENARIO
Manual tap to select a character or to navigate across the layout can be tedious in contexts such as traveling on bus, Switching attention between two tasks (hearing an announcement and system audio feedback).
SOLUTION
Users can scroll through the letters or let the system control it gives the user flexibility when their hands are full.
DESIGN GOAL: SIMPLE
USER SCENARIO
Drawing similarities from current keyboards features, not having access to auto typed letter can lead to active attention allocation on the typing task.
SOLUTION
System suggests the user with frequently used words or trigram words for improved efficiency.
DESIGN GOAL: Flexible
USER SCENARIO
Usually BVI disabled users are trained to use technology and are accustomed to operated them at varying speeds, much higher than a normal abled user. When in a meeting or occupied with other tasks, it can be difficult to type combination character sentences.
SOLUTION
Get the flexibility to modify the default keyboard layer based on their typing patterns.
Learning
The project was a innovative solution which pushed my creative abilities. While in the process, I learnt..
Striking a balance between features and complexity. Making it seamless and even thinking about error rectification aspects.
To design better for a Auditory User Interface(AUI) and to improve it we used Amazon Polly, Simulation videos and developed prototypes to gauge the prototypes pros and cons.
Doing pilot tests and documenting results to understand where the system could be improved to get the best out of the actual study.
Objective set out to improve the Keyflows
Intuitive gestures by changing four finger tap to thumb and pinky tap as the latter signifies the start and end point.
Leveraging scrolls over taps to reduce the gestures inputs
Enhance efficiency in terms of audio cues by using affordance and signifiers that have close relation with the functionality and the context of the paradigms
That's me giving demo and briefing on the next steps to the clients
If you have any more questions or want to know more details, please don't hesitate to contact me. For now, please consider checking my other work, my experiments, or learn more about me.
