Data Collection Notes
WORLD examined constitutional and legal provisions as they set a foundation for rights and are a first step toward improving outcomes. Across countries, having laws on paper does make a difference in practice. Laws and constitutional rights lead to change by shaping public attitudes, encouraging government follow-through with inspections and implementation, and enabling court action for enforcement. Even when local enforcement is inadequate, laws may still have an impact by shaping the terms of political debate and providing levers for civil society advocates. Laws are a mechanism by which power can be democratically redistributed, changes in institutions can be created to ensure greater fairness, and a social floor guaranteeing minimum humane conditions can be established.
DATA SOURCES
In selecting data sources to analyze, WORLD's first priority is to identify sources containing full-text original legislation. To ensure the greatest level of accuracy and comparability across countries, the aim is always to read the original laws (primary sources) rather than secondary summaries or policy descriptions. Primary sources allow for more accurate coding across countries, particularly in complex legal areas. Working with primary sources also allows us to provide excerpts or links to actual legislation and constitutions for those interested in passing new laws or creating reform in their countries. Documents are reviewed in their original language or in a translation into one of the UN's official languages.
Secondary sources are used when information is unclear or insufficient for particular countries. In choosing these secondary sources, those that are comparable across multiple countries are prioritized, such as global or regional sources. When using information sources that cover a limited number of countries, the aim is to ensure that the information they contain can be made consistent with other sources.
For the Protection from Child Marriage Database, legislation to construct the data was located primarily through official country websites, the Lexadin World Law Guide, the Foreign Law Guide, the International Labour Organization (ILO)'s NATLEX database, the Pacific Islands Legal Information Institute, the Asian Legal Information Institute, and JaFBase. In some cases, hard copies and electronic copies of legislation were obtained from libraries such as the Swiss Institute for Comparative Law, the University of California Los Angeles (UCLA) Law Library, the Harvard Law School Library, and the Northwestern University Library. The data captures national-level legislation. In countries where minimum age of marriage laws are set at the state or provincial level coding is based on the lowest minimum age among all of their states or provinces. Given that the scope of the full project includes 193 UN member states, and that the role and strength of case law varies substantially across countries, an analysis of case law relevant to legal provisions for the minimum age of marriage could not be included. Including case law in future analyses will be helpful to better understand the minimum age of marriage permitted by law.
When legislation was not available from these sources, analysts reviewed the most recent reports submitted by countries to the monitoring committees of the Convention on the Rights of the Child (CRC) and the Convention on the Elimination of All Forms of Discrimination against Women (CEDAW), as well as the reports detailing the committees' concluding observations.The foundation for the longitudinal data from 1995 to 2013 was built in collaboration with McGill University's Maternal and Child Health Equity (MACHEquity) team. WORLD's team updated and expanded the data to May 31, 2023.
CODING FRAMEWORKS
In this work, coding refers to the process of translating legislative, policy, or constitutional text into a set of features which can be quantitatively analyzed to provide readily understandable summaries of policy approaches across countries and transformed into data visualizations, such as maps or charts. For example, a researcher reviews many pieces of labor and social security legislation and uses them to answer questions such as: Does a country guarantee paid parental leave? Is it available to all parents, only mothers, or only fathers? How long is paid leave? What is the wage replacement rate? How long do workers need to have been employed to access paid leave?
To answer these questions consistently across countries, the WORLD team first identify the essential policy features to be captured, including intrinsic characteristics, such as coverage; important elements identified in policy research; and minimum standards recognized in global agreements, where they exist. Researchers then read legislative text from 20 to 30 countries to develop an understanding of the approaches countries take in each of these areas. A coding framework consisting of questions and close-ended responses is developed to capture the essential policy features systematically across countries based on the range of approaches identified. Research team members then test whether this coding framework accurately captures approaches on an additional ten to twenty nations.
Once a viable framework is created, feedback is sought from civil society and researchers working in these areas to ensure the questions asked will provide the critical answers needed to inform policy debates. Their feedback can lead to more scoping and test coding to determine which questions are feasible to answer with available legislation, recognizing that some important areas are not always covered by national laws and policies. For example, access to sanitation facilities and safe transportation matters deeply to girls' ability to complete their education but is rarely addressed in a meaningful way in national-level education laws and policies. In other cases, new areas of research might involve going beyond the initial legislation we planned to code, expanding the scale of the project.
Capturing the richness and variety of approaches taken by different countries is our priority throughout the coding process. At times, research teams would have already analyzed 60 to 80 countries before coming across a single country whose approach to a particular problem was different enough in important ways that it could not be adequately captured within the coding scheme. In these cases, the coding scheme was revised to add the elements necessary to capture new features of legislation and policymaking that had presented themselves. All previously coded nations were reviewed to determine whether the revised coding system would alter how they were analyzed. In other words, the new coding system, better adapted to the full variety of approaches nations around the world take, was applied to all countries in the end.
The data sources available contained systematic information on legislation and policies but not on implementation. To ensure consistent approaches across countries, reports that contained comprehensive information on policies but only limited incidental information on implementation were coded only for policies. Obtaining systematic sources of information on implementation should be a pressing priority for global organizations.
CODING PROCESS
Core to ensuring transparency and consistency is developing a codebook that details the rules and examples for coding each question. Researchers rely on this codebook to make decisions on coding policy features. The codebook is designed to be as straightforward as possible, but some questions require judgment calls. To minimize human error, we use a double-coding system where two researchers independently code legal text for each country and then meet to compare their results. When two researchers cannot reach consensus based on the existing codebook, they bring these questions to the full coding team and senior analysts. This team meets regularly to discuss any questions or concerns that arise through the coding process. We record detailed minutes of these meetings and update the codebook to reflect any determinations that impact the coding rules.
ACCURACY, ANALYSIS, AND UPDATING
Upon completion of coding, the WORLD team conducts systematic quality checks. They also carry out targeted checks of countries that appear as outliers globally or for their region or income level.
World uses the most up-to-date sources available for each of their datasets. While this approach is designed to achieve accuracy, it is important to note that when publicly available sources have not been fully updated, the most recent amendments may not be captured in the data. Further, the process of coding legislation inevitably involves important matters of interpretation. For all datasets, the WORLD team welcome receiving feedback and copies of laws from anyone who believes the datab may not be fully up-to-date.