Project Everest

Adopted Experiment

[PROPOSED EXPERIMENT]: Literacy Assessment - FarmEd Timor December 2018

by
Conor Young
Conor Young | 11 months ago | in FarmEd - Timor-Leste

To be completed alongside: The Sales experiment.

 

Lean phase: Key Metrics.

 

Assumption: Farmers within our target market are of a sufficient literacy level to use the FarmEd app.

 

Time Period: Two weeks.

 

Success metric: % of people who:

 

1  Are able to read at a basic level of Tetun and comprehend effectively

 

2  Are able to read at an intermediate level of Tetun and comprehend effectively

 

3  Are able to read at an advanced level of Tetun and comprehend effectively

 

Criteria:

 

Green light: Proceed with implementing app with average level of Tetun literacy comprehension.

 

Success point:

If the amount of basic comprehension is 70% or more.

Signs of Intermediate and Advanced comprehension will be noted as a bonus and provides understanding for future implementation of more difficult concepts. 

 

Orange light: Implement some visual and audio aid in the app to ensure understanding for sections of the app that are considered harder to comprehend.

 

Red light: Assess the written content of the app and implement the use of audio or visual aid.

 

Failure point: If the amount of basic comprehension is 40% or less.

 

Experiment build:

  1. Create a comprehension and reading test** as well as an accompanying survey to assess the different levels of reading comprehension.
  2. Have English translated to Tetun
  3. Test the Comprehension and reading survey on 50 farmers by showing them a printed/electronic copy of the reading questions.
  4. Alternatively if a sale is made reading and comprehension can be assessed through the MOU.
  5. Collate test responses and evaluate level of reading comprehension via the survey. 

**[https://docs.google.com/document/d/1RbgwybvEJ...FGvUH4c8qQ/edit] - link to literacy test.

edited on 11th December 2018, 23:12 by Lucy Preiss

Lucy Preiss 11 months ago

Hi Conor! Can you please attach a copy of the literacy test you are using! Have a think about the metrics you are using to define your success and failure points. Currently if you found that 25% of people had an advanced comprehension and if 35% of people had a basic comprehension then you would have both success and failure. How do your orange and red lights differ? I know you know this stuff, you just need to get it into the post!

Reply 1

Conor Young 11 months ago

Hi Lucy, Thanks for the feedback. I have edited a fix so that results regarding advanced and intermediate comprehension are perceived as a bonus and have indicated that this could be useful in the future when implementing additions to the app of greater comprehension difficulty.

Link to the literacy test: [https://docs.google.com/document/d/1RbgwybvEJ...FGvUH4c8qQ/edit ]

Reply 1

Lucy Preiss 11 months ago

Status label added: Experiment adopted

Reply 1

Marco Stocca 11 months ago

Great job here Conor, loving the experiment!

Reply 2

Conor Young 11 months ago

Thanks for the comment really appreciate the feedback.

Reply 2

Fiona Aaron 11 months ago

Hi Conor, great experiment. How's this experiment coming along? How are people responding to the literacy tests?

Reply 1

Conor Young 11 months ago

Hi Fiona, so far from the sample data of 23 we have found 68.2% have had no problem with comprehension, 9.1% had intermediate/basic level of comprehension and 22.7% had no ability to read/comprehend. However, empirically we have found that a lot of those that had no comprehension had family members that could assist.

Reply 0

Fiona Aaron 11 months ago

That's great! Please make sure to write an experiment post when finished. Also good because hopefully the balance of images and text on the app will be suitable for a larger portion of the market.

Reply 0

View all replies (2)