Abstract

The official statistics produced by the Government of India are scattered across various metadata, definitions, and terminologies, as they are generated by over 60 central ministries. These datasets are not always machine-readable and often do not interconnect effectively. While the Ministry of Statistics and Programme Implementation has established defined metadata and structures, the absence of a unique identifier creates issues with comparability and interoperability, hindering evidence-based policy-making.

To address these challenges, Niti Aayog, the premier think tank of the Government of India, undertook a significant initiative to integrate and make over 6,000 key datasets from these 60 ministries machine-readable and interoperable. This was achieved through the National Data Analytics Platform (NDAP) (https://ndap.niti.gov.in), which utilizes geographic, temporal, and attribute identifiers. NDAP aims to democratize data delivery by making government datasets easily accessible, implementing rigorous data-sharing standards, and enhancing interoperability across the Indian data landscape. It provides a seamless user interface with user-friendly tools, hosting data in clean, machine-readable formats while ensuring robust documentation for each dataset.

Datasets on NDAP are made interoperable by mapping them to a common set of geographical and temporal identifiers using a Local Government Directory Code. This functionality allows users to merge datasets from various sectors and sources, facilitating easier cross-sectoral analysis. NDAP features an in-built merge tool that enables users to combine up to three datasets for comprehensive analysis. All datasets on the platform must meet a minimum data quality standard defined by NDAP’s in-house 5-star rating framework. This standard ensures that datasets include thorough documentation and have successfully undergone internal quality checks, maintaining fidelity to their original sources. Users of NDAP can create flexible tables and visualizations for exploratory analysis using the platform’s built-in analysis tools. They can generate maps, bar charts, line charts, pie charts, choropleth (heat map) maps, and scatter plots with any dataset and indicator at the most suitable levels of aggregation for their analysis. The dataset identification process at NDAP incorporates use-case inputs from sector experts in academia, policy, journalism, and other fields. This approach ensures the availability of datasets that cater to the needs of real-life data users. NDAP also features a natural-language-enabled search function, allowing users to locate specific datasets, indicators, or collections. Search results can be filtered by state/district, ministry, sector, time period, and geographical granularity.

NDAP accelerates data analysis and enables users to derive meaningful insights with a single click. Some example use cases where NDAP proves efficient include: 1. The Health Team wants to compare population density within districts with the availability of health facilities to recommend areas in India where new health facilities are most needed. 2. The trade office requires data on the import and export of various commodities to perform time series analyses of trade statistics. While Open Government Data is available on data.gov.in, it does not integrate datasets, serving primarily as a data dump. NDAP addresses these issues and provides API access to its data.