Proposed Experiment

[PROPOSED EXPERIMENT]: Health Malawi - Validating Accuracy of Symptom Based Triage - [2/3] - User Input Accuracy

 Lean Phase: Solution

Assumption: To ensure that usability of the system is does not affect the results obtained

Time box: 1 week (January week 4)

Success Metric

Percentage of symptom only triages that are the same as face-to-face consultations.

Green light: Continue on to RP stage 3

Success point: The metric is based upon a 95% confidence interval involving both experiment 1 and 2, where 90.25% accuracy from experiment 2 is needed. This is the success point, thus more than 90% is needed to be in the green light.

Orange light: optimise the user interface, remodel and retest.

Failure point: The failure point for this design is then 70%, as a high accuracy is crucial for the 95% confidence interval. Thus anything below 70% is deemed a red light and a redesign of the interface and experiment is necessary.  

Red light: Redesign user interface, including symptom input and results output. Followed by conducting the experiment.

Experiment Build

Brick phones will be used in clinics to trial the systems usability for patients. This will be followed by an interview with questions revolving around clarity, readability and trust . [see attached file]. The interview will prompt patients to answer open ended questions to find flaws and improvements surrounding the usability. The results will then be compared with the doctors diagnosis to see the impact patient usability has on results.

The experiment will be conducted within Gateway and Zingwangwa Clinics where a minimum of 30 data points will be collected. This data will be used to modify and advance the system, moving the health project closer to a viable product.

Proposed learning

To validate that the usability of the system does not affect the results obtained, the program must be clear, readable and trustworthy.

Medical Goals

The medical goal for this experiment is to validate the MVP’s usability. After receiving COMREC approval we will be able to test our current MVP with real patient data to test, validate, and improve on the current system.

Methodology and data collection

Before clinic visit

  1. Organise time to visit  both clinics

  2. Print off patient number template, consent form and interview questions from first experiment for clinician

  3. Finalise and review interview outline

  4. Brief translator on responsibilities

Stratified sampling method

  1. Separate groups into male and female (15 in each group)

  2. Using random number generator to produce 5 random numbers for each group

  3. The random number equates to the order patients walk through the door

  4. If patients do not wish to participate, repeat process

During clinic visit

  1. Introduction pitch – include PE overview, what the experiment is, consent form

  2. Once consented explain how to participate with aid of translator

  3. Give patient identification number and record

  4. Give phone to the patient

  5. Explain how to use program

  6. Prompt patient to use the system

  7. Record all questions asked by the patient when using  the system

  8. Conduct interview after the patient has completed using the system (before they visit the clinician)

  9. Have the clinician fill out the symptom survey form from experiment 1 and collect.

After clinic visit

  1. Collate all the interview results and find main issues faced

  2. Collect data from phone results and compare with clinician symptom form. Fill out the excel sheet with these results


A spreadsheet has been established to determine if symptom only triage is the same as face-to-face triage when used purely by patients. This spreadsheet collects a multitude of different types of data; what the patient inputted, what the patient said to the doctor, the match percentage and how long they have had the symptoms. From this data we will then look at the different trends within the data to identify issues present.

Medical Results

Data will be inputted in an excel spreadsheet which will average out the answers and then convert them into percentages for a simpler read. The recommendation of the medical professionals will be assumed correct, however it is important to note that their response is not going to be 100% accurate. At this stage we will not be differentiating between the results given by the different occupations (nurses, clinicians, doctors) and will be unable to take into consideration the different experience levels of individual clinicians.

If the results reach the green light number of above 90%, this will deem success and will mean that our experiment has worked and we can move to the next stage of the product execution.

Interview Questions

1. On a scale of 1-5 how easy was the program to use?

a. What wasn’t clear

b. Why wasn’t that clear?

c. How would you improve this?

2. On a scale of 1-5 how well did you understand this

a.      What wasn’t clear

b.      Why wasn’t that clear?

c.      How would you improve this?

3. On a scale of 1-5 How accurately does this represent how you are feeling?

4. What improvements would you recommend for the system as a whole?


edited on 15th January 2019, 09:01 by Gabriel Raubenheimer

Gabriel Raubenheimer 1 week ago

Status label added: Proposed Experiment

Reply 0

James Balzer 1 week ago

Really well developed experiment. Logical, in-depth and multivariate considerations are really pertinent here. I'm really interested to see how this experiment is going to go!

Reply 1

Gabriel Raubenheimer 6 days ago

Thank you for your comment Jimmy!

I look forward to discussing it with you.

Reply 0