This article, which first appeared on Ideas for India, marks the occasion of National Statistics Day today (June 29). Instituted by the Government of India to honour the legacy of professor Prasanta Chandra Mahalanobis, whose pioneering work in economic planning and statistical development shaped India’s post-independence trajectory, the day commemorates his birth anniversary and recognises the crucial role of statistics in nation-building.
Statistics is a discipline deeply embedded in almost every other field – be it natural sciences, social sciences, medicine, or policy – especially when one aims to empirically test hypotheses. Researchers, scholars, and policymakers routinely rely on statistical reasoning to ensure their findings are scientifically sound and methodologically robust. For instance, the work of statisticians like Austin Bradford Hill and Richard Peto, alongside physician Richard Doll, was pivotal in conclusively establishing the causal link between smoking and lung cancer, which dates back to as early as 1950. Their contribution went far beyond merely handling data; they applied rigorous statistical thinking to support a landmark public health discovery.
But what exactly is statistics?
Anyone who has taken a statistics course, whether in high school or at the undergraduate level, has likely encountered two common definitions of statistics. On the one hand, statistics refers to data, that is, numerical information obtained by collecting, organising, and summarising observations. On the other hand, statistics is also a scientific discipline comprising a set of methods used to collect, classify, analyse, interpret, present, and visualise data and draw inferences. In everyday usage, the former definition tends to dominate. Statistics is often reduced to its data-handling aspect, while the methodological and inferential core of the discipline is overlooked.
As a result, anyone who works with data is frequently labelled a ‘statistician’, even if they lack a basic understanding of survey design, hypothesis testing, predictive modelling, or the measurement and management of biases and uncertainties. If you have ever tried hiring a statistician in the social and development sector, you have likely encountered this challenge: it is extraordinarily difficult to find candidates who truly understand statistics in its deeper, methodological sense; even among those who hold degrees in the subject. Economists, demographers, sociologists, and data analysts are more readily available. But where are the statisticians? Why are they so hard to find? And why does their absence matter for data-driven, scientifically grounded decision-making?
Demand-supply conundrum
Unlike other quantitative disciplines such as economics, statistics faces a clear and persistent demand-supply gap. In the early 2000s, only a limited number of colleges in India even offered the subject as a major or honours, while economics and related fields were more broadly available. The situation has not improved much since then. While the number of institutions may have increased over the years, the foundational issue of quality remains unresolved, in part because the initial supply base of trained statisticians was small to begin with.
Those who do receive rigorous statistical training and grasp the subject typically pursue one of three broad paths: (i) Higher education in statistics, either in India or, more commonly, abroad; (ii) Private sector corporate employment, particularly in finance, banking, or consulting roles that offer lucrative compensation; or (iii)Government service, often through the Indian Statistical Service (ISS) cadre.
Those who pursue higher education often remain in academia or later join international pharmaceutical or tech companies, especially outside India. Their training and capabilities contribute to cutting-edge research or corporate decision-making, but seldom find their way back into the Indian statistical ecosystem. Those opting for the second path help private firms optimise operations and maximise profits; applications that, while important, rarely intersect with effective governance, public policy or national development. The third group, which joins the ISS, enters secure government jobs but is not always equipped or supported to contribute meaningfully.
Gaps in institutional and curricular support
A sustainable statistical infrastructure hinges on a steady stream of well-trained and motivated professionals. Our universities remain overly focused on teaching statistical techniques in abstraction, often disconnected from the real-world issues confronting society. An obvious example is the lack of practical training on integrating diverse data sources – a skill essential in today’s complex data environment. Borrowing strength from multiple data sources, including survey data, administrative and programmatic data from government systems, remote-sensing data from satellites, and other alternate sources, is crucial for improving inference by overcoming the limitations of individual data sources. This ‘triangulation’ of data sources not only strengthens inference but also surfaces inconsistencies and biases, prompting improvements in data quality and design.
At the same time, the persistent neglect of foundational modules, lack of curriculum reform, and inadequate faculty development continue to erode the pipeline of competent statisticians. For instance, the current M.Sc. Statistics syllabus at Calcutta University allots a mere 20 marks (out of a total of 1000 across four semesters) for sample surveys, taught with outdated material and without any emphasis on real-life survey designs or approaches to minimise the total survey error to produce near-unbiased estimates of population characteristics. The curriculum remains fixated on measuring sampling error without any recognition of various types of non-sampling errors, such as coverage bias, non-response bias, and measurement error, and ways to minimise them while designing and implementing surveys.
When asked about modernising the syllabus, the faculty response was telling: “But who will teach it?” This dilemma reflects a deeper structural gap. Curriculum reform cannot succeed without parallel investments in higher education reform, faculty training, institutional partnerships, a broader vision for statisticians’ roles in public service, and a willingness to embrace interdisciplinary approaches to problem-solving.
Reforming the Indian Statistical Service: Unlocking untapped potential
Revitalising the ISS is quite crucial. One critical opportunity lies in reimagining the current recruitment procedure of the ISS. At present, the structure and content of the ISS examination are heavily skewed in favour of candidates with a master’s in statistics. This creates an uneven playing field and results in a recruitment pool that is narrow in both academic diversity and practical expertise. As a consequence, some of the brightest minds in the country are effectively excluded from the selection process, not due to a lack of ability, but because the exam is overly tailored to a specific academic background.
To make the ISS cadre more inclusive and better equipped for the demands of a data-driven world, the examination must be redesigned to evaluate a broader set of competencies. These should include applied statistical reasoning, computational skills, real-world data analysis, and an understanding of policy contexts. Emphasising practical skills will open doors to candidates from diverse academic backgrounds, including data science, computer science, economics, engineering, and other allied disciplines. Such diversification would enhance the ISS’s capacity to respond to complex societal challenges through data-informed decision-making.
Equally important is the modernisation of the ISS training curriculum. The Ministry of Statistics and Programme Implementation (MoSPI) in collaboration with the Capacity Building Commission (CBC) and the National Statistical Systems Training Academy (NSSTA), developed the Annual Capacity Building Plan (ACBP), which focuses on aligning training programmes with identified competency needs around domain-specific knowledge, functional skills, and behavioural competencies. However, a contemporary training programme could also include modules on data mining, machine learning, Big Data algorithms, data privacy and security, artificial intelligence (AI), and AI ethics. These additions will better equip officers to set up an automated data quality assurance mechanism for sample surveys, work on predictive modelling using large and complex datasets, and contribute meaningfully to evidence-based governance and policymaking in the digital age.
Finally, ISS reform must go beyond technical upskilling. It should encourage a culture of interdisciplinary collaboration, field engagement, and partnerships with academic and research institutions. A shift in organisational mindset is essential; one that values innovation, critical thinking, and a proactive approach to addressing India’s developmental priorities.
Policy disconnect and the role of the diaspora
A striking policy disconnect underpins the current state of statistical education and practice in India. In contrast, economists are often better positioned in policy conversations – not necessarily because of superior technical skill, but because they engage directly with real-world problems.
In his recent book Accelerating India’s Development: A State-Led Roadmap for Effective Governance, Karthik Muralidharan outlines a compelling vision for leveraging data and outcome-based frameworks in policymaking. Ideally, such a data-driven governance roadmap should emerge from within the statistical community itself. Yet, many statisticians shy away from applied, high-impact policy questions – perhaps due to a narrow view of what constitutes “pure” statistics or a reluctance to engage with messy, real-world data. This self-imposed detachment has long-term consequences: it sidelines statisticians from shaping key national conversations where their expertise is sorely needed.
The Indian statistical diaspora, particularly those in academia abroad, represents a valuable yet underutilised resource. While many colleagues in the US and other countries possess exceptional academic credentials, their contributions to India’s statistical development often remain limited to biennial conferences or ceremonial collaborations. These events, though useful for networking and scholarly exchange, frequently fall short of addressing on-the-ground challenges or fostering sustained institutional capacity. The focus must shift from episodic engagement to long-term partnerships – built on mentorship, co-developed research agendas, and active involvement in curriculum reform, training, and policy work.
Bridging this policy disconnect requires statisticians – both within India and across the diaspora – to move beyond the comfort of theoretical rigour and embrace a more engaged, problem-solving role. The tools of our trade must not only serve methodological advancement, but also help shape a more equitable, informed, and accountable society.
Bridging statistics and data science: From identity crisis to synergy
The perceived rivalry between statistics and machine learning is, in truth, a false dichotomy. Statisticians must recognise that there is no fundamental conflict between the two, only a gap in mindset and adaptability. The machine learning community has been both agile and pragmatic, drawing deeply from statistical theory and building upon it with remarkable innovation. Consider how the concept of bootstrap sampling, rooted in classical statistics, evolved into the random forest algorithm. Or how the L1-penalty, a central feature of Lasso regression, was repurposed to tackle over-parameterisation in neural networks. These are not examples of opposition; they are illustrations of synergy.
The real challenge lies within the academic walls of traditional statistics departments. Curricula have not kept pace with the evolving demands of modern data science, and reforms should not begin only once statisticians reach ISS – these methods should be incorporated into academic curricula as well. To drive this transformation, statistics departments must actively collaborate with experienced industry professionals. Practitioners must go beyond occasional guest lectures or advisory roles. They must co-create syllabi, mentor students, and serve as a bridge between theoretical rigour and real-world application. For statisticians in the field, this is a call to stewardship; an opportunity to shape the next generation of professionals.
Faculty members, too, must embrace this moment of change. Without a proactive response, the discipline risks irrelevance. We are already seeing early warning signs: several undergraduate mathematics programmes in India have recently been discontinued due to dwindling enrolment. A similar future may await statistics departments that continue to resist reform.
Importantly, the demand for data expertise is not in decline; it is growing. But if academic institutions do not evolve, that demand will increasingly be met by engineers or professionals trained through short-term certification programmes. The tragedy would not be the rise of these new practitioners, but the self-imposed marginalisation of statisticians from a domain they helped build.
Rather than competing for territory, statisticians and computer scientists must work together to define the new frontiers of data science – where statistical rigour meets computational power.
Conclusion
In a data-driven era, the role of statisticians in shaping national development is more vital than ever. Yet, their presence in key policy domains remains limited. Critical sectors like health, education, nutrition, and governance continue to suffer from a lack of robust statistical insight.
To change this, we must modernise academic curricula, reform professional training, and promote meaningful collaboration between statisticians, computer scientists, and development practitioners. This transformation demands a collective effort from educators, institutions, professionals, and the Indian statistical diaspora.
The question is no longer whether statisticians can contribute to nation-building, but how we are preparing them to lead. The future of the discipline and its impact on India’s growth, depends on the urgency and clarity of our response.
What we need now is innovative leadership – of the kind that Prof. Mahalanobis exemplified. But leadership of that calibre does not emerge in a vacuum. It is fuelled by purpose, passion, and a deep engagement with the problems of the time – qualities that once defined the Indian Statistical Institute and must define our efforts again today. For this revival to take root, support must come from the highest levels of government. Only then can statistics reclaim its rightful place in India’s development journey.
The authors gratefully acknowledge the discussions and comments they received from various people based on the earlier drafts of this article. In particular, they extend their heartfelt thanks to Radhabinod Barman, Alok Arunam, Arpita Ghosh, Rupam Pal, Manasi Narasimhan, Shivani Raturi, Aparna Dwivedi.
The views expressed in this post are solely those of the authors, and do not necessarily reflect those of the I4I Editorial Board.
The Wire is now on WhatsApp. Follow our channel for sharp analysis and opinions on the latest developments.
