Project Everest

Work Update

[Work Update]: SoCon Malawi: USSD Experiment Design - July 2018

by
Seif Zakri Stacey
+1
Seif Zakri Stacey | Jul 17, 2018 | in Knowledge Base

Here in Malawi we are experimenting with large scale data gathering through the sending of mass texts. We are hoping this will provide a more effective method of gathering information while maintaining its accuracy.


USSD Survey Design


After the workshop with Gabe we identified several key design factors that needed to considered when establishing the USSD survey design in Malawi. We felt the design should be clarified for anyone that doesn’t understand the relatively low adoption rate of technology here in Africa. A lot of people in Malawi still use brick phones (E.g. Old Nokia phones) so we catered the survey design to that level of technology. We feel the information below was important to mention because it shows the specific information we are trying to gather while also keeping the survey scalable, as well as other relevant information like costs and formatting.


Goal


Large-scale Data gathering: This survey was designed to gather a large amount of qualitative data in a short time frame


Validate Assumptions: It was also designed to validate the assumptions we had of our customer segment


Sample


Scope: the survey will be sent out to all over Malawi, which will give us unbiased information on a large scale


Scale:10% response rate, therefore to get a sample size of 30 for example we would need to issue 300 surveys. However our target demographic might only make up part of that 10% so we will need to issue even more surveys to gain a more accurate sample. For example 1000


Target: Our target demographic is women in fringe rural/ urban villages with limited income and limited access to finance. The good thing about the USSD is that though our target demographic might only make up part of the sample size, since it's such a large sample size we actually gain a lot of insight into other potential customer segments


Data points


General demographic information


Occupation(s) and their fluctuation for the people in Malawi


General access to income and capital available to the people


Financial and business education/ literacy


Potential as a customer


Usability/UX


Ease of Use: making it easier to use for the customer makes it more likely they finish the whole survey. E.g limit the amount of questions, completable in 5 minutes, language is in Chichewa etc.


Progress Tracker: so the person answering can track their progress through the survey (E.g. 1/20 for question 1)


Reward: offering a reward might offer a higher completion rate but introduces another fixed cost (E.g. 50kwa = ~$US 0.069)


Cost estimates


Fixed cost: $US 34.4/ month subscription services


Note: these costs might change in the near future after PEV makes changes to its USSD use


Format


Output -> Website -> Database -> Spreadsheet


The answers from our survey are sent to the Africa’s talking website where it is processed and a database is created. From there the database in converted into a spreadsheet that we can access and analyse


Ideas


These are ideas for future expansion, elaboration or collaboration. These can ideas itself can be expanded or elaborated upon.


Energy team: we could use the clients used in the energy teams USSD as potential target customers for future sales


Partnership with Airtel: Since all of our projects implement the use of Airtel at some stage in the project, it may be possible to begin a partnership with them either through cheaper mass texts or by gaining numbers for a specific region (E.g. the villages around Blantyre)


Longitudinal Data: It would be efficient to gather data in the future about things like customer satisfaction, progress in their business/ life, and effectiveness with the delivery model etc.

edited on 5th September 2018, 23:09 by Justin Hakeem

William Lee Jul 17, 2018

Again, really excited for this - how quickly can you guys get this in field?

Reply 1

Seif Zakri Stacey Jul 17, 2018

I’ll have to confirm with Ella but I think the first round of surveys have been sent out for both the socon and energy team

Reply 1

Ella Grier Jul 20, 2018

I have sent all edits to Q (our partner for USSD development in Malawi) and she will make the relevant changes over the weekend. This should be rolled out by the start of Week 4 so we can get to see some initial results before we finish up on July Project. Over the next six months we should be able to see the effectiveness of this method of data collection and will know how we can improve and implement it across all projects over the December-February period.

I have a question on your proposal to incentivise surveys. Have you done a more extensive analysis of the costing for Africa's talking subscription, cost per bulk text, cost per survey taken and cost to direct money into many account. I suspect there would also be an additional cost with setting up an application such that 50MWK is directly transfered into their airtel/mpamba account from a PEV account. From our experience in energy sales and payments for utilities, this is generally 2% of the total transaction. I don't think this option would be viable now but as you were saying, in future, if you move to Airtel as an agent of survey distribution (as opposed to Africa's talking) this cost could be better managed. It would also be interesting to compare the two platforms independently of incentive cost, would people trust an engagement text coming from the pre-saved 'Airtel' number and be more likely to complete the survey or not?

I'm super excited to see how this goes. While you can't limit your target demographic completely many of the ideas coming out of your lean canvas' are heavily based on delivering finance through telecom credit. Given your survey is going direct to phone users, thats already a valuable filter you have on this data collection and that's gold!

Reply 1

View all replies (2)

Henry Baldwin Jul 18, 2018

Personally I'm very excited to see the results of this survey. If USSD proves to be a durable and valid form of data gathering, it's potential capability to offer a relatively simplistic widescale study format could be very useful in the future. Our data at the moment from qualitative measures in surveys and interviews has so far proven to be very useful, so I'm excited to be able to access a large scale quantitative measure. I'm keen to see how far we can narrow the scope of the USSD surveys (targeting a specific region instead of all Malawi) as that would drastically increase the efficiency of our studies.

We'll post an update when we get results.

Reply 0

Ella Grier Jul 20, 2018

It would be really interesting to see if you could obtain a database from any gov'ts or telecom providers that have a list of numbers registered to a specific area/region of Malawi. While you may not be able to refine it entirely, you could look at giving this info to Africa's Talking and send your engagement text to a specific set of numbers.

Reply 0

Seif Zakri Stacey Jul 24, 2018

I definitely think this is a good idea because the limit to questions does inhibit is slightly to validating our customer segment and we use one of the questions to establish their distance to their closest city. Getting the scope of the survey down to a regional area could free a question that we could use to identify another problem/ value experienced and further validate our customer segment. I will include this in the critical actions for the next team in the handover document

Reply 0

View all replies (2)

Mallory Dobner Jul 20, 2018

I have a question regarding the use of Airtel. As TNM is cheaper for calls and texts energy has found that there is a significantly higher prevalence of TNM and therefore Mpamba (TNM's mobile money service) in rural areas. Would it be more effective for a focus on a partnership and tailoring systems towards the more used telecom system at this early stage? From my experience Airtel is easier to use, and in the future it would make sense to be partnered with both, but as a system to start with I think Mpamba is the better system to sort out first as it is closer to your target market.

Reply 0

Henry Baldwin Jul 24, 2018

We definitely should explore further the differences and viability of the two telecoms. Any relationship that we could establish would be immensely useful to all the projects in Malawi. Unfortunately we do not have time to explore these options, and will need to be followed up by a team in the future.

Reply 0

Andrew Vild Aug 14, 2018

Status label added: Work Update

This doesn't outline explicitly how the experiment is being conducted. I would love to see a post that has the results of this. If you could also link that post in this post it will enable ease of navigation! Thanks!

Reply 0