Pakistan’s National AI Policy Must Build A Legal Duty Of Care For Algorithms

We have long acknowledged that there is a duty of care owed to anyone who carelessly puts others in danger. Tort law incorporates the ethical and legal requirement that employers, manufacturers, and physicians act responsibly. However, what happens if the ‘actor’ we rely on is a line of code rather than a real person? The response is now urgent in Pakistan.

Cheap, always-on AI may seem to fill the void left by the widespread use of smartphones and severe gaps in mental health services. Tens of millions of individuals in Pakistan are estimated to require psychiatric assistance, but there are very few psychiatrists per capita; extensive surveys and World Health Organization reports indicate that only a small percentage of people have access to professional care. An app that ‘listens’ in that space may feel preferable to nothing, but it is not a human.

The fundamental issue is the mismatch. Families suing businesses following fatal events, legal tests of responsibility when chatbots malfunction, and courts demanding accountability rather than allowing platforms to hide behind ‘we’re a tech company’ defences are already occurring overseas. Regulators and standards organisations are moving quickly: the European Union’s liability proposals have pushed the idea of making it simpler for victims to establish harm from AI, while the National Institute of Standards and Technology AI Risk Management Framework provides a structure for risk-based tasks. Pakistan cannot disregard these blueprints.

Pakistan does not have to start from scratch. The National AI Policy, which addresses ethics, skills, and governance, was approved by the federal government earlier in 2025. Meanwhile, the Personal Data Protection Bill (2023) is still in the bill and draft stages, and enforcement organisations are still being set up. This policy moment is an essential opportunity. Without quick, clear regulations, the same technology that could advance agriculture, healthcare, and education could instead put the most vulnerable people at risk: the young, the underprivileged, and those without legal recourse.

The benefits of the technology, such as faster triage, improved access to therapy, and automated legal research, are real. However, it is the law, not product promotion, that determines how risk and benefit are balanced

What would a duty of care for AI look like in Pakistan? Here are practical, defensible steps:

1. Make explicit disclaimers and human hand-off mandatory for health-facing AI. It should be legally mandatory for every chatbot or assistant that diagnoses, recommends therapies, or responds to suicidality to publish quantifiable safety testing and escalate cases to a human physician or contact centre. This type of sectoral limitation protects users while maintaining innovation.
2. Enact and implement data protection regulations immediately. Any AI duty of care rests on a Personal Data Protection framework that includes enforcement, breach reporting, and explicit limitations on profiling and the processing of sensitive data. Implementation institutions and regulations are essential; pending legislation and drafts are insufficient.
3. Establish a ‘standard of care’ for high-risk systems that is specific to AI. Take a cue from the National Institute of Standards and Technology and the European Union’s risk-based approach and mandate third-party testing, audits, and incident reporting for systems whose failure could result in bodily or psychological harm.
4. Establish transparency guidelines and a no-fault compensation fund. A rapid route to relief is necessary for victims of demonstrable AI harm. While courts develop case law, a small levy on significant AI deployments could support a recovery fund.
5. Require legal and judicial filings to be transparent and provenance-based. The issue of ‘hallucinations’ is not merely theoretical; fraudulent, artificial intelligence-generated citations have already flooded courts across the globe. Courts and law firms in Pakistan need to be cautioned and provided with technical guidance to treat AI-sourced content as unverified until it is validated.
6. Crisis lines and public literacy. To ensure that people in danger have more options than simply talking to an algorithm, the government and civil society should support literacy campaigns explaining what chatbots can and cannot do, and expand the availability of crisis hotlines. Given the sensitivity of mental health data, this cannot be negotiable.

If left unchecked, this is a surefire way to produce uneven harm. In Pakistan, the damage is likely to cluster in areas with weak safety nets: young people who injure themselves, low-income women who receive ‘teletherapy’ in difficult personal circumstances, and litigants who use artificial intelligence to draft legal pleadings and unintentionally contaminate court records. The benefits of the technology, such as faster triage, improved access to therapy, and automated legal research, are real. However, it is the law, not product promotion, that determines how risk and benefit are balanced.

Duty of care has always been a collective agreement. That compact must be rebuilt for systems that scale faster than our laws in the age of algorithms. Pakistan’s National AI Policy is a good place to begin, but it will leave people exposed if it does not include legal obligations and compensation mechanisms. If we value dignity, it must be written into the code.