DEBATE ON METHODOLOGY OF NATIONAL SURVEY

Copyright infringement not intended

Context: The Government of India recently created a commission to assess the methodology of the National Statistical Organisation (NSO) chaired by Pronab Sen, former Chief Statistician of India.

Details

Why is there a need for a review?

• According to some experts, the usage of outdated survey methodology by national surveys such as the National Sample Survey (NSS), National Family Health Survey (NFHS) and Periodic Labour Force Survey (PLFS), have systematically underestimated India’s development.
  • According to them, this outdated methodology has failed to capture reality in the recent past as the “Indian economy has been incredibly dynamic in the last 30 years”.
• Some experts believe that there is no need for restructuring and overhauling the survey methodology just because it doesn’t suit certain narratives of development. However, they accept that there may be errors, which should be minimised. They also advocate the usage of appropriate sample weights to make the national sample adequately representative.

Understanding and analysing these claims in the context of the available evidence is crucial since national-level data is a crucial resource for research, policymaking, and development planning.

What next?

• The Pronab Sen Committee has a crucial task of ensuring that the sample used for national-level surveys is representative of the diverse population of India. However, changing the entire survey methodology may not be the best solution, as it may introduce a new bias in favour of urban areas, which are already overrepresented in policymaking, planning and financing.
  • Instead, the committee should focus on addressing the specific concerns raised by various stakeholders about the existing methodology, and make necessary adjustments to improve its accuracy and reliability.
• The aim should be to eliminate any possible rural bias, if it exists at all, without compromising the quality and coverage of the survey data.

National level surveys

About

• National surveys are large-scale studies that collect data from a sample of the population on various topics, such as health, education, economy, and more.
• These surveys provide valuable insights into the current situation and trends of a country, which can inform policymaking, development planning, and research. However, conducting national surveys is not a simple task. It requires careful planning and execution to ensure that the data collected is accurate, representative, and relevant.

Main features of national surveys

Sample Representativeness

• The methodology of national surveys must ensure that the selected sample is representative of the entire population to draw valid and reliable conclusions. This involves using appropriate sampling techniques, such as random sampling or stratified sampling, to select the participants and avoid biases or errors. The sampling frame, which is the list of potential participants, should also be updated and accurate.

Data Collection Techniques

• Surveys may use various data collection methods, such as face-to-face interviews, telephone interviews, questionnaires, or a combination of these approaches. Each method has its advantages and disadvantages in terms of cost, time, response rate, quality, and coverage. For example, face-to-face interviews may provide more in-depth and detailed information but are more expensive and time-consuming than telephone interviews or questionnaires. The choice of data collection technique depends on the research objectives, budget, resources, and availability of the participants.

Sample Size

• The size of the sample is crucial for the survey's accuracy. A larger sample generally provides more precise estimates and allows for more detailed analysis. However, a larger sample also requires more resources and may increase the complexity and difficulty of data collection and management. The optimal sample size depends on the level of confidence and margin of error that the researchers want to achieve.

Frequency

• National surveys can be conducted periodically to capture changes over time and track trends. For example, some national surveys are conducted annually, biannually, or every few years. The frequency of national surveys depends on the research goals, availability of resources, and relevance of the topic. Periodic surveys allow for longitudinal analysis and comparison of results over time.

Data Quality and Validation

• Ensuring data quality and validating the collected information is critical to maintaining the survey's credibility. Data quality refers to the accuracy, completeness, consistency, and reliability of the data. Data validation refers to the process of checking and verifying the data for errors or inconsistencies.

Some of the prominent national-level surveys conducted in India include:

Census of India

• The Census of India is conducted once every ten years by the Registrar General and Census Commissioner of India. It provides comprehensive data on the country's population, including demographic, social, and economic characteristics. The latest census was conducted in 2011.

National Sample Survey (NSS)

• The NSS is conducted by the Ministry of Statistics and Programme Implementation (MoSPI). It gathers data on various socio-economic aspects of households and individuals. The survey is conducted in rounds, and each round covers specific themes like employment, consumption expenditure, health, education, etc.

National Family Health Survey (NFHS)

• NFHS is conducted by the Ministry of Health and Family Welfare in collaboration with various agencies. It gathers data on fertility, maternal and child health, family planning, nutrition, and related indicators. It is conducted periodically to monitor health and family welfare indicators across the country.

National Health and Morbidity Survey (NHMS)

• The NHMS is conducted by the Ministry of Health and Family Welfare and provides information on the health status of the Indian population. It covers a wide range of health-related issues, including non-communicable diseases, infectious diseases, and health behaviours.

National Youth Policy Survey

• This survey is conducted by the Ministry of Youth Affairs and Sports to gather data on the aspirations, concerns, and challenges faced by the youth of India. It helps in formulating policies and programs that cater to the needs of the young population.

These are just a few examples of the various national-level surveys conducted in India. Each survey serves specific purposes and contributes to the overall understanding of the country's socio-economic and demographic landscape. The data collected through these surveys are crucial for evidence-based policymaking and program implementation at the national and state levels.

Significance

Informed Decision-making

• Accurate and up-to-date survey data enable policymakers to make informed decisions and design effective policies to address socioeconomic challenges. For example, national surveys can provide insights into the living conditions, income, education, health, and employment status of the population, as well as their opinions and preferences on various issues. These data can help policymakers identify the needs and priorities of different groups and regions, and tailor policies accordingly.

Resource Allocation

• National surveys help in allocating resources efficiently to areas that require greater attention. For example, national surveys can reveal the gaps and disparities in access to basic services, such as health care, education, water, and sanitation, among different segments of the population. These data can help policymakers allocate funds and resources to improve the quality and coverage of these services and reduce inequalities.

Monitoring Progress

• These surveys are essential for tracking progress toward development goals and evaluating the effectiveness of policies and programs. For example, national surveys can measure the impact of policies and programs on various indicators, such as poverty, malnutrition, literacy, maternal and child mortality, and environmental sustainability. These data can help policymakers assess whether the policies and programs are achieving their intended outcomes, and identify areas for improvement.

Academic Research

• Researchers rely on national survey data to study various social, economic, and health-related issues. For example, national surveys can provide rich and reliable data for conducting descriptive, analytical, and causal studies on topics such as gender equality, social mobility, migration, urbanization, climate change, and infectious diseases. These studies can generate new knowledge and evidence that can inform policy design and implementation.

Challenges that researchers may face when conducting national surveys are:

Rapidly Changing Landscape

• In dynamic economies like India, changes occur at a rapid pace, and traditional survey methodologies may struggle to keep up with these changes. For example, the COVID-19 pandemic has disrupted many aspects of life and livelihoods, and the impact may vary across regions and groups.

Representativeness

• Ensuring a truly representative sample that includes diverse segments of the population, including marginalized communities, can be challenging. India is a vast and diverse country, with multiple languages, religions, castes, ethnicities, and cultures. Moreover, there are significant disparities in access to education, health care, and other basic services.

Data Quality and Integrity

• Maintaining data quality, avoiding biases, and addressing data falsification are significant challenges. Data quality can be affected by various factors, such as survey design, questionnaire design, interviewer training, data collection methods, data entry, data cleaning, and data analysis.
• Biases can arise from various sources, such as sampling errors, non-response errors, measurement errors, and response errors. Data falsification can occur due to various reasons, such as lack of supervision, incentives, or penalties for interviewers or respondents.

Funding and Resources

• Conducting large-scale national surveys requires substantial funding and resources. The cost of conducting a national survey can vary depending on various factors, such as the sample size, the number of questions, the mode of data collection, the duration of the survey, the geographic coverage, the logistics involved, and the personnel required.
• Conducting a national survey requires coordination and collaboration among various stakeholders, such as government agencies, research institutions, funding agencies, civil society organizations, and media outlets.

Coverage

• National surveys may face challenges in reaching and covering the entire population or certain segments of the population, such as remote or rural areas, marginalized or vulnerable groups, or hard-to-reach or hidden populations.

Response

• National surveys may face challenges in obtaining responses from the participants, such as low response rates, non-response bias, or social desirability bias. These challenges may affect the validity and reliability of the results.

Ethics

• National surveys may face ethical challenges in conducting research with human subjects, such as informed consent, privacy and confidentiality, respect and dignity, beneficence and non-maleficence, or justice and equity. These challenges may affect the rights and welfare of the participants and the reputation and credibility of the researchers.

Way Forward

Methodological Updates

• Governments should regularly review and update survey methodologies to keep pace with changing socio-economic dynamics. For example, they should adjust the sampling design, questionnaire content, and weighting procedures to reflect the current population structure, consumption patterns, and price levels.

Incorporate Technology

• Embracing technological advancements can improve data collection methods, enhance accuracy, and reduce costs. For instance, using computer-assisted personal interviewing (CAPI) devices can facilitate data entry, validation, and transmission, as well as enable geo-referencing and tracking of surveyors.

Capacity Building

• Investing in the training and capacity building of surveyors can lead to better data collection and management. Surveyors should be equipped with the necessary skills and knowledge to conduct interviews effectively, handle sensitive questions, deal with non-response and refusal cases, and follow ethical standards.

Collaboration with Experts

• Engaging with experts and researchers can bring valuable insights to the survey design and implementation process. Experts can provide technical assistance and guidance on various aspects of the survey methodology, such as sampling, questionnaire design, data processing, analysis, and dissemination.

Transparency and Open Data

• Governments should promote transparency by making survey data openly accessible for research and analysis, fostering accountability. They should adhere to international standards and best practices for data documentation, anonymization, archiving, and sharing.

Continuous Monitoring and Evaluation

• Regular monitoring and evaluation of the survey process can help identify and address shortcomings promptly. Governments should establish mechanisms for collecting feedback from surveyors, respondents, users, and stakeholders on various aspects of the survey quality, relevance, timeliness, accessibility, and impact.

Conclusion

• The debate surrounding the methodology of national surveys is essential for ensuring the accuracy and relevance of data used for policymaking and development planning. Governments need to address the challenges, implement necessary improvements, and embrace technology to make national surveys more robust and effective tools for informed decision-making.

Must Read Articles:

National Statistical Office (NSO) Data: https://www.iasgyan.in/daily-current-affairs/national-statistical-office-nso-data

PRACTICE QUESTION Q. What is the significance of national surveys in policymaking and development planning, and what are the key challenges they face? How can governments and stakeholders address these challenges and chart a way forward to enhance the effectiveness and relevance of survey data for informed decision-making?

https://epaper.thehindu.com/ccidist-ws/th/th_delhi/issues/45005/OPS/G4QBHD9P6.1.png?cropFromPage=true