INTRODUCTION
Artificial Intelligence (AI) is transforming services of the government from traffic regulation, law enforcement, welfare distribution, to healthcare at a rapid and unprecedented rate in India.[1] AI allows for the automation, and therefore, efficiency and timeliness of the tasks of governance.[2] Analysts note AI is revolutionising public administration in India through improvements in efficiency and delivery of citizen engagement. Yet, there is a dark side to this speed of innovation, in both lawful and unlawful ways. AI systems may encode hidden discrimination, and unfortunately, bias is typically reflected. In the words of India’s Chief Justice, “AI [or algorithms] can perpetuate and amplify discrimination because [they are] opaque, complex and biased.”[3] As a point of law, an algorithm for e-governance purposes has the potential to yield unjust outcomes that contradict the constitutionally protected values and rights to equality and fairness.[4]
WHAT IS ALGORITHMIC BIAS?
To put it simply, algorithmic bias refers to systematic unfairness exhibited by artificial intelligence (AI) systems toward individuals from certain groups.[5] This may happen when training data used to build the AI models indicates or embodies societal bias, or if the design of the algorithm is based on faulty assumptions or stereotyping. For example, a job‑recruitment AI would systematically favour male candidates if its training data contained far fewer representations of women.[6] In India, significant examples of these biases are seen. Studies have shown that people with darker skin were significantly less accurately identified by facial-recognition tools, and that automated hiring identified female and minority candidates more poorly than male candidates. Experts warn that biased models “may exacerbate socio-economic inequalities in India and put already marginalised populations at further risk.”[7]
REAL LIFE OUTCOMES FROM INDIAN E-GOVERNANCE
Biased algorithms have harmed people already in the course of Indian e‑governance projects. A marked example is Telangana’s Samagra Vedika, a data-integration platform designed to reduce welfare fraud. While the goal was to make administration more efficient, it left many eligible families ineligible.[8] Investigation showed that Samagra Vedika cancelled 1.86 million ration cards and rejected over 140,000 applications with little or no notice.
In one instance, a poor widow was ineligible for rations after the system confused her household with another that owned a motor vehicle. People had little or no possibility for redress: even government officials denied Right to Information (RTI) applications for the source code and for criteria, justifying their denial through “commercial confidentiality.”[9]
Similar issues were mirrored in other Indian states. In Odisha in 2015, an IBM data tool used for a food-security scheme rejected benefits for more than five million people in a single day.[10] Activists had to file RTI applications to sort through the mass demunition of benefits. The department’s initial response to activists was completely dismissive, and they provided no useful account of the disqualification rationale.
Similarly, in the case of the rural employment scheme (MGNREGA), mandatory Aadhaar verification caused countless wrongful exclusions.
Biometric discrepancies and data inaccuracies caused thousands of employees to forfeit wages and benefits.[11] The Supreme Court determined that these exclusions “impact and have adverse consequences on the fundamental right to livelihood and social security.”[12] These examples demonstrate how errors in governance driven by AI can result in an erosion of fundamental rights directly.
LEGAL CONTEXT: CONSTITUTION AND ADMINISTRATIVE LAW
The Constitution of India provides strong protections. Article 14 prohibits arbitrary, unreasonable action by the State; it provides that equality before law guarantees equality. Courts have a long-standing view that every statutory power of government must be exercised fairly and reasonably.[13] In administrative law, state action can be found invalid for 3 reasons: unlawfulness, unreasonableness, and procedural irregularity. None of these doctrines comes into question concerning algorithmic decision-making processes.
If, for example, a person is denied welfare benefits through an opaque data filter, then they may be able to challenge that denial using Article 14 as a basis. Article 21, which protects the right to life and liberties, extends to the right to a fair procedure, as established by the Supreme Court in Maneka Gandhi v. Union of India.[14] Accordingly, every AI-based decision must be judged against the same standard of reasonable and fair actions as the law imposes on human decision makers.[15]
The Function of RTI and Transparency
India’s Right to Information Act (2005) has become an effective mechanism for demanding accountability of algorithmic governance. Journalists and citizens have used RTI to learn about how AI grounds are being used to deny entitlements.[16]
For example, in the Odisha food-security case, RTI appeals led the department to disclose that an IBM tool had been used to screen beneficiaries, something which had not been disclosed. In a different case, the government acknowledged in an RTI response that a citizen’s entitlements were denied because of “algorithmic decision-making,” without verifying its accuracy.[17] This type of transparency is vital because it allows aggrieved citizens to challenge AI-based exclusions through the courts or an administrative avenue.
WHAT DOES INDIA’S NEW DATA PROTECTION LAW MEAN?
The Digital Personal Data Protection Act (DPDPA), 2023, affords some individual rights over personal data, including consent, limited use, and data security.[18] However, it does not address algorithmic decision-making and AI bias directly.
Unlike the European Union’s GDPR, which better protects citizens and grants them a right to explanation in the case of automated decisions, there is no equivalent in India’s DPDPA.[19] There is a void here; there is no statutory right for individuals impacted by AI rules and regulations to receive an explanation or to appeal.
Therefore, while DPDPA enhances privacy protections for consumers, it provides little or nothing to mitigate or remedy algorithmic discrimination.
POLICY SOLUTIONS: WHAT NEEDS TO CHANGE
Experts have suggested a layered strategy that includes the following approaches to remediate AI bias in the Indian government;
- Mandatory Transparency
Government agencies using AI must make it clear that they are using AI. The citizenry should know that an AI tool is being used and how it works. At the very least, it should be explained in clear may what criteria, inputs, and logic are being used to make automated decisions.
- Bias Audits and Accountability
Both bias audits and shared accountability must be undertaken for every government AI tool deemed high-risk, especially for welfare, policing, and credit purposes. Audits should be independent and continuous, and their findings must be made publicly available. If certain communities are found to be disproportionately affected, then corrective measures should be taken.[20]
- Grievance Redressal Mechanisms
There should also be a clear appeals process stating how a citizen can appeal adverse algorithmic outcomes. ‘Grievance redressal officers’ or “AI grievance cells” can be put in place to establish accountability. In addition, people must be able to insist that their automated decisions will be confirmed by human review if an automated decision is adverse to their rights.[21]
- Keep Humans in the Process
Public employees should not automatically accept algorithm outputs. Officers need the training to ensure that the application of an AI is a tool and not a final decision-maker. If AI makes a recommendation that is just not believable, the human element must overrule. When we relinquish decision-making to machines wholesale, constitutional protections fall away.
- Dedicated AI Laws
India’s planned Digital India Act will provide an opportunity to help frame algorithmic fairness, transparency, and explainability in legislation. Standalone rules for public-sector AI and similar to the EU’s AI Act — will help protect citizen rights.[22]
JUDICIAL OVERSIGHT: A REQUIRED MECHANISM
Until new legislation is developed, the courts are the only body that can meaningfully intervene in algorithmic abuse in government. Citizens and civil society must incorporate Article 32 (Supreme Court) and Article 226 (High Courts) to contest an AI-based decision that is unfair.[23]
Legal experts have pointed out that technology must comply with the Constitution, rather than requiring the Constitution to comply with technology. If an AI system acts in an unreasonable or discriminatory way, it is subject to judicial review under the same criteria as other state action.[24] In the view of a former Supreme Court judge, “Technology should never be at the expense of the progress and welfare of the people and their fundamental rights.”[25]
CONCLUSION
AI could reform governance in India, but only if done in the right way. In the absence of protections, algorithmic systems can unobtrusively encode and amplify biases that would erode justice. India can build an e-governance system that is more inclusive, constitutional, and humane by ensuring transparency, fairness, and recourse. Until then, all we have is vigilance, legal, civic, and journalistic, as our strongest algorithm for accountability.
Author(s) Name: Tanisha (Vivekananda Institute of Professional Studies- Technical Campus)
References:
[1] Vidushi Marda, ‘Artificial Intelligence Policy in India: A Framework for Engaging the Limits of Datafication’ (2018) < https://royalsocietypublishing.org/doi/10.1098/rsta.2018.0087> 25 July 2025.
[2] Government of India, National Strategy for Artificial Intelligence (NITI Aayog, June 2018) https://www.niti.gov.in/sites/default/files/2023-03/National-Strategy-for-Artificial-Intelligence.pdf accessed 25 July 2025.
[3] Chief Justice DY Chandrachud, 60th Convocation Address, IIT Madras (Chennai, 22 July 2023), reported in Times of India (‘AI has potential to perpetuate and amplify discrimination, CJI Chandrachud says at IIT-Madras convocation’)https://timesofindia.indiatimes.com/city/chennai/ai-has-potential-to-perpetuate-and-amplify-discrimination-cji-chandrachud-says-at-iit-madras-convocation/articleshow/102041753.cms accessed 25 July 2025
[4] Constitution of India 1950, art 14
[5] Solon Barocas and Andrew D Selbst, ‘Big Data’s Disparate Impact’ (2016) 104(3) Cal L Rev 671.
[6] Joy Buolamwini and Timnit Gebru, ‘Gender Shades: Intersectional Accuracy Disparities in Commercial Gender Classification’ (2018) 81 Proceedings of Machine Learning Research 1
[7] Arvind Narayanan and Sayash Kapoor, ‘AI as Normal Technology’ (Knight First Amendment Institute, 15 April 2025) https://kfai-documents.s3.amazonaws.com/documents/c3cac5a2a7/AI-as-Normal-Technology—Narayanan—Kapoor.pdf accessed 25 July 2025
[8] Al Jazeera, How an algorithm denied food to thousands of poor in India’s Telangana (24 January 2024) https://www.aljazeera.com/economy/2024/1/24/how-an-algorithm-denied-food-to-thousands-of-poor-in-indias-telangana accessed 25July 2025
[9] Al Jazeera (n 8)
[10] Tapasya Grantee and Kumar Sambhav Shrivastava, How We Investigated Welfare Algorithms in India (Part II) (Pulitzer Center, 30 May 2024) https://pulitzercenter.org/how-we-investigated-welfare-algorithms-india-part-ii accessed 25 July 2025
[11] Reetika Khera, ‘Exclusion by Design: Aadhaar and the Welfare State’ (2019) 54(13) Economic and Political Weekly 13
[12] Justice KS Puttaswamy (Retd) v Union of India (2018) 10 SCC 1
[13] EP Royappa v State of Tamil Nadu (1974) 4 SCC 3
[14] Maneka Gandhi v Union of India (1978) 1 SCC 248
[15] Amlan Mohanty and Shatakratu Sahu, India’s Advance on AI Regulation (Carnegie India/Carnegie Endowment for International Peace, 21 November 2024) https://carnegieendowment.org/research/2024/11/indias-advance-on-ai-regulation?lang=en accessed 26 July 2025
[16] Amnesty International, ‘Here’s What a Recent Amnesty Report Says About Algorithmic Decision-Making in India; What it Missed’ (Harro, 3 May 2024) https://harro.com/2024/05/03/heres-what-a-recent-amnesty-report-says-about-algorithmic-decision-making-in-india-what-it-missed/ accessed 26 July 2025
[17] Sandra Wachter, Brent Mittelstadt and Luciano Floridi, ‘Why a Right to Explanation of Automated Decision-Making Does Not Exist in the General Data Protection Regulation’ (2017) 7 International Data Privacy Law 76.
[18] Digital Personal Data Protection Act 2023
[19] Amber Sinha, The slow march to a data protection law in India (blog post, 26 Oct 2022) https://www.ambersinha.com/blog/the-slow-march-to-a-data-protection-law-in-india accessed 26 July 2025
[20] OECD, State of Implementation of the OECD AI Principles: Insights from National AI Policies (OECD Digital Economy Papers No 311, 18 June 2021) https://doi.org/10.1787/1cd40c44-en accessed 26 July 2025
[21] 21 European Union, ‘Artificial Intelligence Act – Proposal’ COM(2021) 206 final https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX%3A52021PC0206 accessed 26 July 2025
[22] European Commission, ‘Proposal for a Regulation Laying Down Harmonised Rules on Artificial Intelligence (Artificial Intelligence Act)’ COM(2021) 206 final https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX%3A52021PC0206 accessed 26 July 2025
[23] Constitution of India 1950, arts 32 and 226
[24] Shyam Divan, The constitution, liberty and privacy (Journal of Indian Law & Society, V-11(1) Monsoon 2020) https://jilsblognujs.wordpress.com/wp-content/uploads/2025/01/shyam-divan_111.pdf accessed 26 July 2025
[25] B. N. Srikrishna Committee on Data Protection and Artificial Intelligence in India, ‘The Srikrishna Committee Data Protection Bill and Artificial Intelligence in India’ (Centre for Internet and Society, 3 September 2018) https://cis-india.org/internet-governance/blog/the-srikrishna-committee-data-protection-bill-and-artificial-intelligence-in-india accessed 26 July 2025
