SLANG LABS USER TESTING AND RESEARCH
Slang is a Voice to Action platform, which can be used to quickly add a multi-lingual voice interface to any mobile app. The voice interface will allow end-users to interact with the app via voice, as well as with touch. The goal of this project was to test out two multi-lingual voice-based applications among users that aren't tech-savvy and use the data derived to identify problems, gain meaningful insights and script a research paper based on the results
MY ROLE
TIMELINE
Research, Affinity Mapping, User testing, transcription, Qualitative and Quantitative Analysis, Report, Research Paper
7 months
THE CHALLENGE
TESTING:- Since the target group for testing were emerging users(people that weren't tech-savvy), it was tough to spot such users that fit the profile. Furthermore, the language barrier meant that is was tough to guide users if the users were stuck at a particular task.
​
RESEARCH PAPER:- Since there were many papers that were based on voice augmentation, something new was needed to capture the attention of readers.
FINAL DELIVERABLES
1. User test data
2. Quantitative and qualitative analysis
3. Research paper
1. Hypothesis before testing, target audience, inclusion criteria and sample size
HYPOTHESIS
-
Voice augmented interface would facilitate overcoming interface navigation challenges
-
Voice augmented interface would facilitate overcoming challenges due vernacular language
-
It would reduce the time taken for task completion
-
Help reducing the number of errors in performing a task
TARGET AUDIENCE
Emergent users are the people, who are just beginning to get access to advanced mobile devices and services. These new mobile natives are also used to dealing with a range of everyday challenges, including economic, geographic, technological and educational constraints.
INCLUSION CRITERIA
Language – Hindi and English
Education – graduates or lesser (unless above 70yrs old)
Has access to a smartphone ( if not own one, having one at home)
SAMPLE SIZE
-
Age – above 20 years
-
Sex ratio – 50/50 female and male
2. Consent forms and questionnaire
CONSENT FORM
Consent forms were scripted in both Hindi and English to provide subjects with the information they need to make a decision to volunteer for the user tests
.png)
.png)
QUESTIONNAIRE
Questionnaires were also scripted in Hindi and English. These were filled after the tasks were completed by the users.
.png)
.png)
3. User testing and data compilation
USER TESTING
A total of 50 interviews were conducted by my partner and me, which mostly consisted of users we found at random public spaces like parks, bus stops and coffee shops. The interviews were recorded in both Hindi and English and a female to male users ratio of 50:50 was achieved.
.png)
.png)
.png)
DATA COMPILATION
The raw quantitative data was jotted down on excel. This data mostly involved time taken as well as the success rate for each task. The comments derived from the recordings were transcripted into the same file. The questionnaire answers that were based on a scale of 1-5 were recorded separately to use to derive the System Usability Scale(SUS) of the application.
4. Quantitative, Qualitative analysis and SUS
Qualitative analysis
Using the comments that were transcripted, an affinity map was drawn out to dive more into the problems that the users faced and take into account the suggestions that users stated to arrive on a concrete design statement that could be used for the improvement of the application. Some of the results are given below the map :-

.png)
.png)
.png)
.png)
Quantitative analysis
Quantitative analysis was done based on the data(mainly timings of tasks) that was recorded based on video recordings. A sample size of 19 was obtained (after removing outliers who didn't complete more than 5 out of the 10 tasks) and the data was tabulated for the users based on time taken for each task. ANOVA tests were done to compare Voice vs Visual and English vs Hindi(for voice tasks). An analysis of variance (ANOVA) is a statistical technique that is used to check if the means of two or more groups are significantly different from each other.ANOVA checks the impact of one or more factors by comparing the means of different samples.
.png)
.png)
.png)
.png)
System Usability Scale(SUS)
A SUS(System Usability Scale) was used to measure the ease of use of the apps. A System Usability Scale (SUS) is a 10-item Likert scale questionnaire that provides an at-a-glance look at the ease of use (or lack thereof) of websites, software, hardware, mobile devices, and other technological applications. The SUS consists of the following 10 questions that can be answered on a five-point Likert scale of "Strongly Disagree" to "Strongly Agree." A total of 48 users filled in the questions given in the SUS. The SUS was calculated based on the data derived.
.png)
5. Research paper
Based on all these insights, a research paper titled "VOICE AUGMENTED EXPERIENCE FOR APPS IN INDIAN LANGUAGES" t was written by me, my partner, Ms.Disha Rathi and the Slang Labs project manager, Mr. Vinayak. This paper was selected and was presented at the ALL INDIA HCI CONFERENCE 2019 held in Hyderabad.
​
Link to paper:-
https://drive.google.com/file/d/0B0ae_WIRzgW7cmRfcjBDWUJsalBUd1B2U3Y4VHhkTnh1OUJB/view?usp=sharing