Hunger Safety Net Programme Impact Evaluation 2012, Second Follow-up Round
Kenya, 2012
Get Microdata
Reference ID
ken-opm-hsnpie-2012-v1
Producer(s)
Oxford Policy Management Limited
Created on
Dec 08, 2017
Last modified
Dec 08, 2017
Page views
136461
Identification
Survey ID Number
ken-opm-hsnpie-2012-v1
Title
Hunger Safety Net Programme Impact Evaluation 2012, Second Follow-up Round
Subtitle
Second Follow-up Round
Country
| Name | Country code |
|---|---|
| Kenya | KEN |
Abstract
The Hunger Safety Net Programme (HSNP) is a social protection project being conducted in the Arid and Semi-Arid Lands (ASALs) of northern Kenya. The pilot phase has now concluded and the HSNP is beginning to scale up under Phase 2. The ASALs are extremely food-insecure areas highly prone to drought, which have experienced recurrent food crises and food aid responses for decades. The HSNP is intended to reduce dependency on emergency food aid by sustainably strengthening livelihoods through cash transfers.
Oxford Policy Management was responsible for the monitoring and evaluation (M&E) of the programme under the pilot phase, with the intention of informing programme scale-up as well as the government’s social protection strategy more generally. The M&E involved a large-scale rigorous community-randomised controlled impact evaluation household survey, assessment of targeting performance of three alternative targeting mechanisms (Social Pension; Dependency Ratio; Community-based Targeting), qualitative research (interviews and focus group discussions) to assess targeting and impact issues less easily captured in the quantitative survey, and on-going operational and payments monitoring to ensure the smooth implementation of the programme. Findings were communicated to the HSNP Secretariat, Government of Kenya and the Department for International Development (DFID) on a regular basis to inform and advise on policy revisions and development. The M&E component used the data it produced to advise the design of HSNP Phase 2, including micro-simulations of different programme targeting scenarios and review of the phase 2 targeting approach which combines proxy means testing with community-based targeting.
The impact evaluation study compares the situation of HSNP and control households at the time of their selection into the programme (baseline), with their situation 12 months (year 1 follow-up) and 24 months later (year 2 follow-up). Over this 24-month period most of the HSNP households covered by the evaluation had received 11 or 12 bi-monthly transfers (initially KES 2,150, increased to KES 3,500 by the end of the evaluation period).
The baseline data collection was completed in November 2010, the first round of follow-up data collection finished in November 2011, and the final round of fieldwork - in November 2012.
Oxford Policy Management was responsible for the monitoring and evaluation (M&E) of the programme under the pilot phase, with the intention of informing programme scale-up as well as the government’s social protection strategy more generally. The M&E involved a large-scale rigorous community-randomised controlled impact evaluation household survey, assessment of targeting performance of three alternative targeting mechanisms (Social Pension; Dependency Ratio; Community-based Targeting), qualitative research (interviews and focus group discussions) to assess targeting and impact issues less easily captured in the quantitative survey, and on-going operational and payments monitoring to ensure the smooth implementation of the programme. Findings were communicated to the HSNP Secretariat, Government of Kenya and the Department for International Development (DFID) on a regular basis to inform and advise on policy revisions and development. The M&E component used the data it produced to advise the design of HSNP Phase 2, including micro-simulations of different programme targeting scenarios and review of the phase 2 targeting approach which combines proxy means testing with community-based targeting.
The impact evaluation study compares the situation of HSNP and control households at the time of their selection into the programme (baseline), with their situation 12 months (year 1 follow-up) and 24 months later (year 2 follow-up). Over this 24-month period most of the HSNP households covered by the evaluation had received 11 or 12 bi-monthly transfers (initially KES 2,150, increased to KES 3,500 by the end of the evaluation period).
The baseline data collection was completed in November 2010, the first round of follow-up data collection finished in November 2011, and the final round of fieldwork - in November 2012.
Kind of Data
Sample survey data [ssd]
Unit of Analysis
Households and individuals
Version
Version Description
Version 1
Scope
Notes
The scope of the study includes:
- Household demographic characteristics
- Health
- Individual mobility
- Housing and amenities
- Consumption expenditures - food and non-food consumption
- Income from livelihood activities
- Assets
- Food availability and coping strategies
- Subjective poverty assessment
- Formal/informal transfers, food aid and employment programs
- Saving and borrowing
- Community infrastructure, water, shocks, seasonality, prices
- Community HSNP targeting.
- Household demographic characteristics
- Health
- Individual mobility
- Housing and amenities
- Consumption expenditures - food and non-food consumption
- Income from livelihood activities
- Assets
- Food availability and coping strategies
- Subjective poverty assessment
- Formal/informal transfers, food aid and employment programs
- Saving and borrowing
- Community infrastructure, water, shocks, seasonality, prices
- Community HSNP targeting.
Keywords
| Keyword |
|---|
| Social Protection |
| Cash Transfers |
| Impact Evaluation |
| Randomized Controlled Trial |
| Kenya |
| Sub-Saharan Africa |
| Food Security |
| Pastoralism |
| Pastoralist |
| Disaster Risk Management |
| Drought |
| Arid Lands |
Coverage
Geographic Coverage
Counties of Turkana, Marsabit, Wajir, and Mandera.
Universe
All persons living within "secure" sub-locations across all counties at the time of sampling (2008; due to sporadic insecurity across the four counties, a small portion of sub-locations were deemed to be insecure when the sample was drawn and so excluded from the sample frame).
Producers and sponsors
Primary investigators
| Name |
|---|
| Oxford Policy Management Limited |
Producers
| Name | Role |
|---|---|
| Institute for Development Studies | Consortium member HSNP M&E component |
| Research Solutions |
Funding Agency/Sponsor
| Name | Abbreviation | Role |
|---|---|---|
| Department for International Development | DFID | Programme and Evaluation Funder |
Sampling
Sampling Procedure
At follow-up 2, in addition to attrition, the sample size is further reduced because the follow-up 2 survey covered eight fewer sub-locations, 40 rather than 48. Overall 2,436 households were surveyed (at the baseline, 5,108 households were covered).
The evaluation sub-locations were selected from a sample frame of all secure sub-locations in each district. In each district 12 sub-locations were selected with PPS (Probability Proportional to Size) with implicit stratification by population density such that there is an even number of selected sub-locations per new district.
The evaluation sub-locations were sorted within districts by population density and paired up, with one of the pair being control and one being treatment.
The sampling strategy for the quantitative survey was designed in order to enable a comparison of the relative targeting performance of three different targeting mechanisms. These are:
- Community-based targeting (CBT): The community collectively selected households they consider most in need of transfers, up to a quota of 50% of all households in the community;
- Dependency ratio targeting (DR): Households were selected if individuals under 18 years old, over 55 years old, disabled or chronically ill made up more than a specified proportion of all household members;
- Social pension (SP): All individuals aged 55 or older were selected.
For both the treatment and control sub-locations there are an equal number of CBT, SP and DR sub-locations. Assignment of targeting mechanisms to sub-locations was done randomly across the same pairs that were defined to assign treatment and control status.
In all the evaluation sub-locations, the HSNP Admin component implemented the targeting process. In half the sub-locations the selected recipients started receiving the transfer as soon as they were enrolled on the programme - these are referred to as the treatment sub-locations. In the other half of the evaluation sub-locations, the selected recipients were not to receive the transfer for the first two years after enrolment - these are referred to as the control sub-locations.
The households in the treatment sub-locations that are selected for the programme are referred to as the treatment group. These households are beneficiaries of the programme. In control sub-locations the households that are selected for the programme are referred to as the control group. These households are also beneficiaries of the programme but only begin to receive payments two years after registration. The targeting process was identical in the treatment and control sub-locations.
The following population groups can thus be identified and sampled:
- Group A: Households in the treatment sub-locations selected for inclusion in the programme;
- Group B: Households in control sub-locations selected for inclusion in the programme but with delayed payments;
- Group C: Households in treatment sub-locations that were not selected for inclusion in the programme;
- Group D: Households in control sub-locations that were not selected for inclusion in the programme.
Because targeting was conducted in both treatment and control areas, households were sampled in the same way across treatment and control areas. Selected households (groups A and B) were sampled from HSNP administrative records. Sixty six beneficiary households were sampled using simple random sampling (SRS) in each sub-location (in two of sub-locations this was not possible due to insufficient numbers of beneficiaries in the programme records). In cases of household non-response replacements were randomly drawn from the remaining list of non-sampled households. This process was strictly controlled by the District Team Leaders.
Non-selected households (groups C and D) were sampled from household listings undertaken in a sample of three settlements within each sub-location. These settlements were randomly sampled. The settlement sample was stratified by settlement type, with one settlement of each type being sampled. Settlements were stratified into three different types:
1. Main settlement (the main settlement was defined as the main permanent settlement in the sub-location, often known as the sub-location centre and usually where the sub-location chief was based. As there was always one main settlement by definition, the main settlement was thereby always selected with certainty).
2. Permanent settlements (permanent settlement is defined as a collection of dwellings where at least some households are always resident, and/or there is at least one permanent structure).
3. Non-permanent settlements.
As concern community level data, community questionnaires were conducted in every community for which at least one household interview was attached. A community was defined as a settlement or a sub-section of a settlement if that settlement had been segmented due to its size. Due to missing data, a small proportion of households are not linked to any community data.
The above explanation is taken from "Kenya HSNP Monitoring and Evaluation Component: Impact Evaluation Final Report 2009 to 2012". For more details please refer to this report in Related Materials section.
The evaluation sub-locations were selected from a sample frame of all secure sub-locations in each district. In each district 12 sub-locations were selected with PPS (Probability Proportional to Size) with implicit stratification by population density such that there is an even number of selected sub-locations per new district.
The evaluation sub-locations were sorted within districts by population density and paired up, with one of the pair being control and one being treatment.
The sampling strategy for the quantitative survey was designed in order to enable a comparison of the relative targeting performance of three different targeting mechanisms. These are:
- Community-based targeting (CBT): The community collectively selected households they consider most in need of transfers, up to a quota of 50% of all households in the community;
- Dependency ratio targeting (DR): Households were selected if individuals under 18 years old, over 55 years old, disabled or chronically ill made up more than a specified proportion of all household members;
- Social pension (SP): All individuals aged 55 or older were selected.
For both the treatment and control sub-locations there are an equal number of CBT, SP and DR sub-locations. Assignment of targeting mechanisms to sub-locations was done randomly across the same pairs that were defined to assign treatment and control status.
In all the evaluation sub-locations, the HSNP Admin component implemented the targeting process. In half the sub-locations the selected recipients started receiving the transfer as soon as they were enrolled on the programme - these are referred to as the treatment sub-locations. In the other half of the evaluation sub-locations, the selected recipients were not to receive the transfer for the first two years after enrolment - these are referred to as the control sub-locations.
The households in the treatment sub-locations that are selected for the programme are referred to as the treatment group. These households are beneficiaries of the programme. In control sub-locations the households that are selected for the programme are referred to as the control group. These households are also beneficiaries of the programme but only begin to receive payments two years after registration. The targeting process was identical in the treatment and control sub-locations.
The following population groups can thus be identified and sampled:
- Group A: Households in the treatment sub-locations selected for inclusion in the programme;
- Group B: Households in control sub-locations selected for inclusion in the programme but with delayed payments;
- Group C: Households in treatment sub-locations that were not selected for inclusion in the programme;
- Group D: Households in control sub-locations that were not selected for inclusion in the programme.
Because targeting was conducted in both treatment and control areas, households were sampled in the same way across treatment and control areas. Selected households (groups A and B) were sampled from HSNP administrative records. Sixty six beneficiary households were sampled using simple random sampling (SRS) in each sub-location (in two of sub-locations this was not possible due to insufficient numbers of beneficiaries in the programme records). In cases of household non-response replacements were randomly drawn from the remaining list of non-sampled households. This process was strictly controlled by the District Team Leaders.
Non-selected households (groups C and D) were sampled from household listings undertaken in a sample of three settlements within each sub-location. These settlements were randomly sampled. The settlement sample was stratified by settlement type, with one settlement of each type being sampled. Settlements were stratified into three different types:
1. Main settlement (the main settlement was defined as the main permanent settlement in the sub-location, often known as the sub-location centre and usually where the sub-location chief was based. As there was always one main settlement by definition, the main settlement was thereby always selected with certainty).
2. Permanent settlements (permanent settlement is defined as a collection of dwellings where at least some households are always resident, and/or there is at least one permanent structure).
3. Non-permanent settlements.
As concern community level data, community questionnaires were conducted in every community for which at least one household interview was attached. A community was defined as a settlement or a sub-section of a settlement if that settlement had been segmented due to its size. Due to missing data, a small proportion of households are not linked to any community data.
The above explanation is taken from "Kenya HSNP Monitoring and Evaluation Component: Impact Evaluation Final Report 2009 to 2012". For more details please refer to this report in Related Materials section.
Deviations from the Sample Design
The reduction in the number of sub-locations surveyed at follow-up 2 was the result of decisions made by the programme and its stakeholders, rather than a technical decision by the evaluation team. This reduction in sample size is unfortunate for a number of reasons. Firstly, it undermines the study design to the extent that the smaller sample size reduces the ability to detect impact with statistical significance. Secondly, it affects the balance of the sample, meaning that treatment and control populations are less balanced at baseline than they were with the original sample structure. Lastly, the sample was designed to be seasonally balanced across the whole calendar year, which is no longer the case as sub-locations that would have been surveyed in the latter and early part of the calendar were dropped. Another implication of the reduced sample at follow-up 2 is that the baseline estimates presented in this report differ from those presented in the baseline and follow-up 1 impact reports. This is because the estimates now relate to slightly different populations.
Weighting
Two versions of the sampling weight are provided:
1) hh_wt sampling weights produce estimates for all households living in sub-locations covered by the evaluation (i.e. the study population). They do not provide estimates for any larger population.
Weights are given by the inverse probability of being selected by strata. For selected households, the weights are given by: wi = Ni /ni, where:
- ni is the number of beneficiary households interviewed in the ith sub-location;
- Ni is the number of beneficiaries listed in the HSNP administrative data for that sub-location.
For non-selected households, the weights are given by:wijk = 1 / [ (aijk/Aijk) *(1/bij)*(1/cij) ], where where:
- Aijk is the total number of non-beneficiary households of residency status k in the selected segment of the selected type j settlement in sub-location i;
- aijk is the number of households of residency status k in the selected segment of the selected type j settlement in sub-location i that were interviewed;
- bij is the total number of segments in the selected type j settlement in sub-location i (often bij=1);
- cij is the total number of settlements of type j in sub-location i.
2) hh_wt_original sampling weights produce representative statistics for the entire population of secure sub-locations within each district.
Community-level variables can be weighted using community weights (cmq_wt), which equal the sum of household weights across the households lnked to that community.
The above explanation is taken from "Kenya HSNP Monitoring and Evaluation Component: Impact Evaluation Final Report 2009 to 2012". For more details please refer to this report in Related Materials section.
1) hh_wt sampling weights produce estimates for all households living in sub-locations covered by the evaluation (i.e. the study population). They do not provide estimates for any larger population.
Weights are given by the inverse probability of being selected by strata. For selected households, the weights are given by: wi = Ni /ni, where:
- ni is the number of beneficiary households interviewed in the ith sub-location;
- Ni is the number of beneficiaries listed in the HSNP administrative data for that sub-location.
For non-selected households, the weights are given by:wijk = 1 / [ (aijk/Aijk) *(1/bij)*(1/cij) ], where where:
- Aijk is the total number of non-beneficiary households of residency status k in the selected segment of the selected type j settlement in sub-location i;
- aijk is the number of households of residency status k in the selected segment of the selected type j settlement in sub-location i that were interviewed;
- bij is the total number of segments in the selected type j settlement in sub-location i (often bij=1);
- cij is the total number of settlements of type j in sub-location i.
2) hh_wt_original sampling weights produce representative statistics for the entire population of secure sub-locations within each district.
Community-level variables can be weighted using community weights (cmq_wt), which equal the sum of household weights across the households lnked to that community.
The above explanation is taken from "Kenya HSNP Monitoring and Evaluation Component: Impact Evaluation Final Report 2009 to 2012". For more details please refer to this report in Related Materials section.
Data Collection
Dates of Data Collection
| Start | End |
|---|---|
| 2012-02 | 2012-11 |
Data Collection Mode
Computer Assisted Personal Interview [capi]
Access policy
Contacts
| Name | Affiliation | |
|---|---|---|
| Marta Marzi | Oxford Policy Management Limited | marta.marzi@opml.co.uk |
| Fred Merttens | Oxford Policy Management Limited | fred.merttens@opml.co.uk |
Citation requirements
Oxford Policy Management Limited. Kenya Hunger Safety Net Programme Impact Evaluation 2012, Second Follow-up Round [dataset]. Oxford: Oxford Policy Managment [producers], 2012. Washington: World Bank [distributors], 2017.
Disclaimer and copyrights
Disclaimer
The user of the data acknowledges that the original collector of the data, the authorized distributor of the data, and the relevant funding agency bear no responsibility for use of the data or for interpretations or inferences based upon such uses.
Metadata production
Producers
| Name | Abbreviation | Affiliation | Role |
|---|---|---|---|
| Development Data Group | DECDG | World Bank | Creator of metadata |
| DataFirst | Creator of updated metadata |
Date of Metadata Production
2014-01-07
DDI Document version
v2