Project Everest

[PROPOSED EXPERIMENT] USSD User Experience Energy Malawi December 2018

The purpose of this document is to outline the USSD User experience to be undertaken by the December/January Energy Team.

CONTENT:

Lean Phase: Channel testing

Assumption: Customers understand the language and the structure of the menus in the USSD system.

Time Period: 1 week

SUCCESS METRIC:

Success will be measured through the feedback to the validation questions given by local community members after interacting with the USSD system. 25 local people from rural areas with 50 local people tested.

 Validation Questions:

USSD understood - Did you understand all menus?

Easy to use - Was the structure of the menus easy to follow?

Culturally appropriate - Was the Chichewa language appropriate and clear?

 CRITERIA:

Green Light - Proceed to test USSD customer engagement experiment.

Success point - USSD is understood and deemed easy to use and culturally appropriate language that is determined with a ‘yes’ response to the validation questions by more than 95% of the population with a phone and more than 90% of people without phones.

Orange Light - Re-evaluate the language and structure of the USSD system

Orange light point - USSD is understood and deemed easy to use and culturally appropriate language that is determined with a ‘yes’ response to the validation questions by less than 95% but more than 70% of the population and with a phone and less than 90% and more than 70% of people without phones.

Failure Point - USSD is understood and deemed easy to use and culturally appropriate language that is determined with a ‘yes’ response to the validation questions by less than 70% of the population with a phone and 70% of people without phones.

Red Light - Re-evaluate the use of the USSD service and the language used as well as the structure of the menus. Determine if it is the USSD or language or structure of the menus.

 EXPERIMENT BUILD:

USSD Texting menu

Feedback:

  1. This experiment can be run with any local community member

  2. Information to be identified:

  • Location (Nancholi or Blantyre)
  • English competency (Judge yourself)
  • Phone ownership

Important information to convey before the user tests the USSD system:

  • The system is free to use (no credit used)
  • The USSD is Airtel only (for now)
  • We are not requesting that they purchase products or express interest only to test the system
  • No data collected will be used for expressions of interest or orders
  • They are to navigate through the menus without any prompts from ourselves on how to do so.
  • They will be asked questions regarding the experience once they complete the menus.

Feedback questions:

There is a PDF with all the questions attached.

Google Form:

https://docs.google.com/forms/d/e/1FAIpQLScdpssMC2teHV6svRUwJKCaK4ry5A3eqS_s3I2y7TmEwFbgjw/viewform?usp=sf_link

FUTURE STEPS:

  1. USSD feedback will need to be conducted on all new menus added to the system.

  2. A Chichewa menu implementation will need to be tested for types of language to use.

  3. USSD feedback can be done through USSD if the previous menus are successfully validated.

 RESULTS:

https://docs.google.com/spreadsheets/d/1Rq5LLGbn4xWV12nwFJGEME0ISC8Okd4VJg4uRRNzHDo/edit#gid=1011001954

 Results Summary:

  • The results from the Blantyre experiment showed a 100% understanding of the language and structure from 10 surveys within Blantyre.
  • The English menus were the only ones that were tested due to no Chichewa translation, therefore only 2 out of the 3 validation questions were tested.
  • The system at one point did not work for 4/10 users at one time due to a server busy response.
  • 2/10 Users accidentally exited the USSD menu while they entered their name or village due to the many different options required to go through for a keypad phone, it is easy to misclick and exits the USSD.
  • The language menu was not clear that it was asking for input, feedback from 1/10 users
  • Menus don’t always request input but assume that the user will input something feedback from 1/10 users

 Validated learnings - Hit Orange Light

  • The English language was 100% understood
  • The structure of the English menus was 100% understood
  • The language and cultural clarity and appropriateness not validated due to no Chichewa translation
  • Users are not able to complete the survey due to server dropouts and misclicks exiting USSD 6/10 times

 Actions On:

  • Remove the name and village input into the USSD
  • Add a village option (not input)
  • ie. What village are you from? (Blantyre, Nancholi)
  • Identify server issues and if they can be fixed
  • Create an experiment to determine if the village options are sufficient to determine meeting locations for EOI and orders
  • Edit menus to request input for all menus to ensure users are prompted
  • Build SOP for Nancholi USSD testing
  • Translate USSD menus
edited on 20th December 2018, 14:12 by Max Telford

sunnysunny 1 month ago

I enjoyed over read your blog post. This was actually what i was looking for and i am glad to came here!
wuxiaworld

Reply 0

Share