projecteverest

Experiment adopted
Complete

[Proposed Experiment]: Malawi - Revised Risk Profiling Testing for Existing Defaulting Customers and Non-Defaulting/Proxy Customers - February, 2019

Lean Phase: Customer Segment

Assumption: The previous risk profiling tool was inaccurate (the weightings aren’t accurate or the metrics may not be explanatory), with less than 50% of the defaulting and non-defaulting customers being identified correctly (see data here). This may be due to the scores/weightings not being based on any data, as well as potential irrelevancies in the metrics. Via surveys of the remaining customers, we will be able regress the data to fit a model on which to base our weightings/scores on. The surveys will also enable us to further test the validity and relevance of the metrics. The revised risk profiling tool will be more accurate, with a high percentage of customers being correctly identified as non loan-worthy or loan-worthy.

We will be able to create an accurate risk metric (being the dependent variable), based on repayment rate, pattern, and qualitative questions from the survey.

Time Period: Approximately 2 weeks (or time taken to profile a large sample of the remaining defaulting customers and an equivalent number of non-defaulting/proxy customers)

Success Metric:

1) % of defaulting customers would fail the risk profiling survey
2) % of non-defaulting customers would pass the risk profiling survey.

Criteria:

Success Point: More than 50% of defaulting customers are correctly identified by the risk profiling tool as not being loan worthy. More than 50% of non-defaulting/proxy customers are correctly identified by the risk profiling tool as being loan worthy.

Green Light: Proceed to use the risk profiling tool to determine loan-worthy candidates.

Orange Light Point: Between 30-50% of defaulting and non-defaulting/proxy customers surveyed are correctly identified by risk profiling tool.

Orange Light: Modify the regression model/the weighting of certain answers and potentially remove or add new questions.

Failure Point: Less than 30% of defaulting and non-defaulting/proxy customers surveyed are correctly identified by the risk profiling tool.

Red Light: Assess whether questions asked are accessible and relevant, and re-assess metrics for creditworthiness. Potentially develop new metrics for more reliable profiling based on further research into the reasons behind the high default rate.

Experiment Build:

1) Survey a large sample of the remaining defaulting customers and an equivalent number of non-defaulting/proxy customers under the guidelines of this SOP. Use this survey for existing customers and this survey for any new customers. Since there are 55 remaining defaulting consumers, the feasibility of interviewing all 55, as well as the non-defaulting customers, is low. We will attempt to ensure customers from a range of villages have been surveyed.

2) Survey an equivalent number of non-defaulting/proxy customers. Although non-defaulting customers would be ideal, they may not be as accessible as proxy customers, such as village bank customers and customers who bought the GED Lite upfront.

3) Fill in the data in the online survey, which links to an Excel spreadsheet.

4) Assign a credit-worthiness value to each customer (expanding out from a binary classification of defaulting = non-creditworthy and non-defaulting = creditworthy) based on the current repayment patterns, repayment ratios and qualitative information collected within the surveys.

5) Develop a regression model based on the data collated in Excel.

6) Use the regression model to determine new weightings for each answer in the survey.

7) Assess which customers would pass the risk profiling tool and those who would fail.

8) Determine the accuracy of the risk profiling tool.

9) Reassess the revised weightings/the validity of the metrics if the accuracy is low. A set of independent variables may have too high a covariance to consider both, or may not correlate to credit-worthiness. If this is the case, further revise the metrics to iterate the regressive model to better accuracy.

edited on 13th February 2019, 12:02 by Kyana Chan

Wade Tink 4 weeks ago

Status labels added: Experiment adopted, In Progress- second 50%

Reply 0

Wade Tink 4 weeks ago

This experiment was adopted and conducted over Jan & Feb. The challenge was accessing the individual customers which highlighted issues with the onboarding process (information gathered). It was determined that completing this experiment in it's entirety would be too manually intensive due to the difficulty in finding each customer.
On average it was taking an afternoon to find five customers. Thus it is possible to achieve, but it would consume the time of an entire team.

It was determined that the value was better attained by understanding the qualitative reasons versus the quantitative data.

Users tagged:

Reply 0

Wade Tink 4 weeks ago

Status label added: Complete

Status label removed: In Progress- second 50%

Reply 0

Share