The integration of Artificial Intelligence (AI) into healthcare systems represents a paradigm shift in medical practice and patient care, especially in a nation like India with its sizable population and varied healthcare concerns. AI technologies are starting to become increasingly important in improving patient outcomes, diagnosis, and disease management as the healthcare landscape changes.
AI is being used to fill in a number of important holes in the present healthcare system. Even though AI has an opportunity ahead of it to revolutionize healthcare delivery, implementing these technologies is full with moral, legal and practical difficulties and lot of other complexities such as data privacy and security, healthcare infrastructure, workforce implication, public trust and acceptance, funding and investment. It remains unsettled. These issues are further complicated by India’s distinct demographic and socioeconomic diversity, which makes it essential to customize AI solutions to the particular requirements of the country’s citizens.
The need for a balanced approach is highlighted by the dual character of AI in healthcare, which is both a transformational instrument and a topic of intense scrutiny. In order to maximize the advantages of AI while lowering the hazards involved, this study promotes cooperation between stakeholders. By doing this, it hopes to strengthen the framework that encourages the moral and successful incorporation of AI technology into India’s healthcare system, which would ultimately improve healthcare outcomes and delivery for the country’s diverse population.
Intersection of AI in healthcare in India
The application of artificial intelligence (AI) in healthcare is transforming the field, especially in India, where AI technology adoption is expanding quickly. AI is being used in healthcare to improve patient management, diagnosis, and therapy, among other areas. Large-scale medical data processing enables AI systems to spot patterns and trends that human clinicians might miss, resulting in more precise diagnoses and individualized treatment regimens. Predictive analytics, for example, is using AI algorithms more and more to evaluate patient risk factors for disorders like diabetes and heart disease, allowing for early intervention.
AI simplifies administrative procedures in healthcare systems while also enhancing clinical results. AI can automate processes like data input, claims processing, and appointment scheduling, freeing up healthcare workers to concentrate more on patient care. AI has a wide range of possible uses in healthcare, from telemedicine platforms that enable remote consultations to robotically assisted procedures.
But this quick adoption of AI also brings up serious issues with algorithmic bias, data privacy, and the morality of machine-driven decision-making. Strong accountability frameworks must be established as India continues to use AI in healthcare in order to handle these issues and guarantee that the advantages of AI are realized while protecting patient rights and safety. Due to its distinct demographics and technical environment, India is quickly becoming a major participant in the integration of artificial intelligence (AI) in healthcare. Given the size and diversity of the population, the nation’s healthcare system offers substantial potential for AI applications that improve overall healthcare delivery, expedite patient management, and increase diagnostic accuracy. From AI-assisted diagnostics that enhance early illness detection of conditions like cancer to predictive analytics that identify high-risk patients, AI technologies are being used in many areas of healthcare.
Initiatives like AI-powered chatbots and virtual assistants, for example, are transforming administrative procedures and patient interactions, freeing up healthcare professionals to concentrate more on providing direct patient care. Furthermore, AI-enabled telemedicine developments have greatly increased access to healthcare services, especially in rural areas with limited medical resources.
The private sector and the Indian government are working together more and more to create an atmosphere that encourages AI innovation. In order to further establish India as a leader in AI-driven healthcare transformation, initiatives such as the Ayushman Bharat Digital Mission seek to integrate digital health solutions throughout the nation.
To guarantee that the advantages of AI are distributed fairly among the diverse Indian population, however, issues like algorithmic bias and data privacy concerns must be resolved. Indian stakeholders have taken a number of actions to establish a regulatory environment that encourages innovation while giving patient safety and ethical issues top priority, realizing the pressing need for a strong accountability structure. Standardized protocols, better data governance, and fostering public confidence in AI-driven healthcare solutions are some of the important concerns that these initiatives aim to address.
Recent developments in AI ethics, machine learning, and natural language processing offer a solid background for this investigation, highlighting the significance of firmly establishing AI development in scientific inquiry.
Cybersecurity Dangers
Cybercrimes have significantly increased in tandem with the adoption of AI in Indian healthcare, presenting major risks to the security and integrity of private patient information. Some of the major cybercrimes are Ransomware Attacks: Ransomware attacks, in which fraudsters encrypt important data and demand a ransom to unlock it, are increasingly targeting healthcare facilities. Data Breaches: Sensitive patient information has been made public due to major data breaches caused by unauthorized access to healthcare systems.
Concerns over data privacy and security procedures in healthcare institutions were raised. Phishing and Business Email Compromise: Phishing techniques are used by cybercriminals to fool healthcare workers into disclosing private information or sending money. Insider Threats: Workers or other trusted individuals may use their access to private data maliciously, which could result in insider threats. Distributed Denial of Service (DDoS) assaults: By flooding systems with traffic, DDoS assaults can render healthcare services inoperable and deny authorized users access to necessary services.
Accountability to address ethical and operational challenges for the use AI in healthcare in India
Al’s application in the healthcare industry calls into question the ideas of transparency and performance responsibility. Al’s decision-making liberty calls into question healthcare accountability. Deep learning algorithms are barely controlled by humans. Humans have little control over how they combine and compare data as long as learning algorithms are operating Individual healthcare providers, including physicians, nurses, and community health workers, as well as institutions or buildings that offer services, including hospitals and family group practices, are the parties that bear responsibility for healthcare.
However, these parties feel less responsible for their choices and behaviours when Al is used. Healthcare providers become less accountable as they realize they have no control over the AI’s decision-making process. Ultimately, this may have an impact on the overall standard of healthcare services. Furthermore, the inability of healthcare practitioners to comprehend and communicate to patients the decisions made by Al results in a lack of internal and external transparency, which impacts the responsibility of all parties involved.[1]
The use of AI technologies gives some optimism as India deals with a number of healthcare issues, such as a lack of qualified healthcare workers, an increasing load of diseases, and inadequate healthcare facilities. AI’s capacity to process enormous volumes of data and produce insightful results at previously unheard-of rates offers a chance to speed up the diagnostic procedure, allowing for early disease detection and, as a result, prompt therapies.
Millions of people’s lives could be saved and their quality of life enhanced by such developments. In this regard, the use of AI in Indian healthcare is based on diagnostic accuracy. Despite their value, traditional diagnostic techniques can occasionally be constrained by human error, laborious procedures, and access issues in remote locations. In many medical areas, from radiology and pathology to cardiology and oncology, AI-powered diagnostic systems have shown impressive accuracy. More accurate and trustworthy diagnoses could result from the combination of machine learning algorithms and deep learning techniques, which have demonstrated great promise in spotting minute patterns and anomalies in patient data and medical imaging.[2]
Legislative Framework
AI as a medical device under EU law
The late 1990s featured the establishment of EU regulations pertaining to the functionality and safety of medical devices. These regulations were significantly updated in 2017 to harmonize the disparities in regional legal interpretations around Europe and to bring EU laws into compliance with technological advancements, shifts in medical science, and advancements in legislative process. The previously in effect directives were superseded in May 2017 by the new Medical Devices Regulation (EU) 2017/745 (MDR) and the In-vitro Diagnostic Medical Devices Regulation (EU) 2017/746 (IVDR). Due to the transitional periods, the IVDR will take effect five years after its adoption (beginning in May 2022) and the MDR three years after its adoption (beginning in May 2020). Al can be considered software, according to a study of pertinent principles and EU legislation.
The MEDDEV’s “Guidelines on the Qualification and Classification of Stand-Alone Software used in Healthcare within the regulatory framework of medical devices” 2.1/6 as of July 2016 offer a definition of software that is extremely pertinent to Al, despite the MDR itself lacking one. The standards describe “software” as a collection of instructions that converts input data into output data. The current type of Al, narrow Al, is always restricted to the set of instructions that its developer originally described, even though Al is self-learning and may thus modify its algorithms to the real-world environment.
To put it another way, a software engineer builds a particular framework for Al that converts inputs into the required outputs. The ‘framework’ itself remains constant during learning; algorithms adjust to this constrained framework. Al therefore qualifies as software according to the MEDDEV’s criteria. It is anticipated that the guidelines, which were given about the framework for medical devices that had previously been used and are not legally binding, will be adhered to, even with regard to the new regulation.[3]
Proactive measures taken by India
India has taken proactive measures to address these challenges, such as the establishment of the National Digital Health Mission (NDHM) and the introduction of guidelines for AI in healthcare by the NITI Aayog. A major advancement in India’s healthcare system is the National Digital Health Mission (NDHM), which seeks to establish an all-encompassing digital health ecosystem that incorporates artificial intelligence (AI) technologies. The NDHM, which was started by the Indian government, aims to improve access to healthcare services nationwide and facilitate the smooth sharing of health data by building a strong digital infrastructure.
The creation of a Health ID for each citizen, which will act as a unique identification and allow people to access their medical records and receive individualized care, is the cornerstone of the NDHM. Because it enables the collection and analysis of massive datasets that can fuel predictive analytics and enhance diagnostic precision, this endeavour is essential for integrating AI technologies in healthcare. AI systems, for example, can examine medical data to spot patterns and forecast health hazards, resulting in prompt treatments and improved patient outcomes.
In order to effectively use AI, the NDHM also stresses interoperability across different health systems. Healthcare practitioners can use AI-driven solutions for patient monitoring, treatment suggestions, and early disease detection by dismantling data silos.
However, while the NDHM lays a strong foundation for AI in healthcare, it must address critical challenges such as data privacy, cybersecurity threats, and ethical considerations surrounding AI deployment. The Digital Personal Data Protection Act (DPDP) 2023 aims to safeguard personal health information, but its successful implementation will be vital in fostering public trust in digital health initiatives.
The National Digital Health Mission (NDHM) in India aims to create a comprehensive digital health ecosystem, yet it has notable loopholes concerning cybersecurity that could expose sensitive health data to cybercrimes.
- Inadequate Cybersecurity Framework: While the NDHM outlines the establishment of a Security Operations Centre (SOC) and a security policy, it lacks detailed procedures for responding to privacy breaches. This absence raises concerns about how effectively the system can protect against cyberattacks, especially given the increasing frequency of such incidents in the healthcare sector.
- Vulnerability to Data Breaches: The NDHM’s reliance on cloud-based services and open-source Application Programming Interfaces (APIs) creates potential vulnerabilities. Each service provider develops its own security measures, leading to inconsistencies that could be exploited by cybercriminals. Reports indicate that hospitals have already faced ransomware attacks, highlighting the urgent need for standardized security protocols across all digital health platforms.
- Linkage with Aadhaar: The integration of health data with the Aadhaar biometric identity system poses significant risks. Previous breaches of the Aadhaar system have raised alarms about data security, and there is no clear framework detailing how health data will be safeguarded when linked to this controversial project. Concerns about unauthorized access and misuse of sensitive health information are exacerbated by this linkage.
- Insufficient Training and Awareness: Many healthcare professionals may lack adequate training in cybersecurity practices, making them susceptible to social engineering attacks and other cyber threats. The NDHM does not address the need for comprehensive training programs to equip healthcare workers with the skills necessary to recognize and mitigate cyber risks.
In 2018, the Government of India’s policy think tank, “NITI Aayog,” was given permission to develop the country’s strategy for artificial intelligence and other cutting-edge technology. NITI Aayog concentrated on five industries, including healthcare, that are thought to gain the most from AI in addressing societal issues. In this endeavour, NITI Aayog has chosen the motto “AI for all” (#AIforAll). Additionally, NITI Aayog aims to guarantee sufficient data security and privacy while striking a balance between morality and innovation. The following suggestions are being implemented to quicken the rate of advancements in healthcare related to AI.
- Creating a multistakeholder marketplace—The National Artificial Intelligence Marketplace. The development stage requiring a multitude of specialized processes. In order to encourage the development of sustainable AI solutions in healthcare it is crucial to address information asymmetry and promote effective collaboration. This may be possible by creating a ‘marketplace’ which could:
- Enable access to the required AI component, be it data or business models, and services, such as data annotation, and enable rating of these assets;
- Serve as a platform for execution and verification of transactions. Ensure data today which is being collected at the individual hospital level but not analysed be collected to build the big dataset envisaged to accelerate AI work. Unravelling new sources of data and facilitating more efficient use of computational and human resources. It is estimated that only 1% of data today is analysed due to lack of awareness and availability of AI experts. For instance, several medical imaging centres are collecting valuable data; however, these databases are not analysed since AI models cannot be created without computational infrastructure and trained personnel. In the presence of a formal marketplace, diagnostic centres would have an incentive to collect these data and provide access to the data in the market with requisite security measures in place.
- Provide an opportunity to address ethical concerns regarding data sharing: creation of a formal marketplace for data transactions would ensure the development of data security measures to prevent misuse of valuable information. The development of a data protection law in India is currently underway. This, along with the promotion of local innovators and leaders in AI technologies, are crucial steps to ensure that the big data generated in India is used to empower local populations and provide them with services, rather than to exploit them for commercial gains.[4]
Although the NITI Aayog 2018 report on AI in healthcare lays out ambitious ambitions to incorporate AI technologies into India’s healthcare system, it has a number of shortcomings, especially with regard to data privacy and cybersecurity.
- Insufficient Cybersecurity Protocols: It does not, however, contain specific cybersecurity rules to guard against breaches and hacks of private health information. This supervision puts patient privacy and data integrity at great danger, especially in light of the substantial cyber risks that have been encountered by the Indian healthcare sector, including data breaches that have affected millions of people.
- Absence of Accountability Frameworks: The lack of explicit liability frameworks may cause misunderstandings about who is responsible for mistakes made by AI systems, which might erode public confidence in these technologies as they become more and more integrated into healthcare decision-making.
- Fragmented Data Ecosystem: This fragmentation raises the possibility of biased or erroneous results because of inadequate datasets, in addition to making the deployment of AI solutions more difficult.
- Regulatory Uncertainties: It does not include particular regulations that address the cybersecurity threats and ethical issues related to AI uses in healthcare. Lack of such laws could stifle creativity and result in uneven industry practices.
The NITI Aayog guidelines 2021 enshrines the principles of responsible AI. It mentions the 7 broad principles for responsible management of AI such as
- Principle of Safety and Reliability
- Principle of Equality
- Principle of Inclusivity and Non-discrimination
- Principle of Privacy and security
- Principle of Transparency
- Principle of Accountability
- Principle of protection and reinforcement of positive human values
It has mentioned the legal and regulatory approaches for managing AI systems stating that in certain high-risk sectors such as health and finance there can be existing protection and guidelines but new legal protections may be needed. There can be some Some relevant legislation for protection from AI related concerns exist in certain cases but would need to be adapted to cater to challenges posed by AI.
Some sectors have unique considerations that may require sector specific laws for AI. The strategy for NDHM identifies the need for creation of guidance and standards to ensure reliability of AI systems. So, it can be concluded that it has not answered the required questions. It was general and not specified[5].
The Digital Personal Data Protection Act (DPDP) 2023
The Digital Personal Data Protection Act (DPDP),2023 establishes a comprehensive framework for protecting personal data in India, addressing various aspects of data governance and individual rights. Section 2 defines essential terms such as “personal data,” ” and “data fiduciary,” providing clarity on the roles and responsibilities of different entities involved in data processing.
The lawful data processing outlined in Section 3, which include requirements for explicit consent, purpose limitation, data minimization, and accuracy. Section 4, a special provision for sensitive data, it states the obligations of data fiduciary. It provides for Processing Sensitive Personal Data but does not adequately address specific needs for healthcare data, which involves extremely sensitive information, such as genetic and biometric data.( Mental health information).Section 7 states about certain legitimate uses. Security Safeguards requires data fiduciaries to adopt reasonable security practices but lacks industry-specific guidance, which is critical for healthcare data given its sensitivity.
Provide industry-specific security guidelines within Section 7, including encryption standards, secure data access protocols, and regular vulnerability testing, particularly for AI in healthcare. Security standards and cybersecurity has to be addressed. S.10 of the act states additional obligations of significant data fiduciary for enhancement of fairness and transparency but it does not extend to algorithmic decisions made by AI systems, which could directly impact healthcare outcomes. So, there can be a bias. Algorithm transparency and bias has to be addressed. Additionally, Section 21 requires data fiduciaries to conduct data protection impact assessments for processing activities that pose significant risks, particularly when dealing with sensitive data.
To ensure compliance, the DPDP outlines penalties for violations in Sections 19-20 and establishes a grievance redressal mechanism in Section 23 for individuals to lodge complaints regarding data processing practices. Furthermore, Sections 24-26 specify the creation of a Data Protection Board, tasked with overseeing enforcement, handling disputes, and ensuring accountability in data handling practices across the sector.[6] Collectively, these provisions aim to foster a secure and transparent environment for personal data protection in India.
Scope
The paper will explore integration of AI applications in healthcare, including accountability, legal framework in India. It aims to highlight how improvement can be made when there is increasing integration of AI in healthcare for future emerging situations ensuring patient safety and operational efficiency.
Limitation
This study only focused on doctrinal view point. The author only relied on secondary sources. As the research was completed in a short time it was not able to be further expanded. So future researchers can expand their scope.
Research Findings
- Legal framework- India has taken proactive measures to address these challenges, such as the establishment of the National Digital Health Mission (NDHM) and the introduction of guidelines for AI in healthcare by the NITI Aayog. The absence of standardized processes and supervision systems, however, still presents difficulties.
- Adoption of AI- Increasing openness and social engagement via educational initiatives can contribute to a rise in public acceptability and trust.
Suggestion
Research suggests that developing a legal framework that requires for the improvement and integration of AI in healthcare in India for future emerging circumstances and also the author suggests for a cooperation between legislators, technologists, and healthcare professionals.
Conclusion
In Conclusion, without first building a strong foundation, India cannot immediately jump into the application of AI in healthcare. Although AI has enormous promise to enhance healthcare delivery, there are still several obstacles that need to be overcome. The technological resources required for successful AI application are frequently lacking in the current infrastructure, especially in rural areas.
Although the Digital Personal Data Protection (DPDP) Act of 2023 offers a framework for data protection, it needs to be sector specific as it is strongly stated in the NITI Aayog 2021 report. India can create the framework required to successfully incorporate AI into its healthcare system by giving infrastructure development top priority, improving data quality, raising public awareness, and making sure regulations are followed.
Since there is an emerging AI healthcare around global and ours is a developing country and democratic country, amendments to existing guidelines can be passed and existing lacunae can be cleared.
References
- Kempton, Alexander Moltubakk, and Vassilakopoulou, Polyxeni (2021): Accountability,Transparency & Explainability in AI for Healthcare. 8th International Conference on Infrastructures in Healthcare. DOI: 10.18420/ihc2021_018
- Smith, H. (2021). Clinical AI: opacity, accountability, responsibility and liability. Ai & Society, 36(2), 535-545.
- Azzali, E. (2020). Accountability in AI as global issue. Management, 20, 22.
- Procter, R., Tolmie, P., & Rouncefield, M. (2023). Holding AI to account: challenges for the delivery of trustworthy AI in healthcare. ACM Transactions on Computer-Human Interaction, 30(2), 1-34.
- Pradhan, K., John, P., & Sandhu, N. (2021). Use of artificial intelligence in healthcare delivery in India. Journal of Hospital Management and Health Policy, 5.
- Kiseleva, Anastasiya. (2020). Ai as medical device: is it enough to ensureperformance transparency and accountability?. European Pharmaceutical Law Review(EPLR), 4(1), 5-16.
- https://www.meity.gov.in/writereaddata/files/Digital%20Personal%20Data%20Protection%20Act%202023
- https://www.niti.gov.in/sites/default/files/2021-02/Responsible-AI-22022021
- https://www.niti.gov.in/sites/default/files/2023-03/National-Strategy-for-Artificial-Intelligence.
- Bajwa, J., Munir, U., Nori, A., & Williams, B. (2021). Artificial intelligence in healthcare: transforming the practice of medicine. Future healthcare journal, 8(2), e188-e194.
- Olawade, D. B., David-Olawade, A. C., Wada, O. Z., Asaolu, A. J., Adereni, T., & Ling, J. (2024). Artificial intelligence in healthcare delivery: Prospects and pitfalls. Journal of Medicine, Surgery, and Public Health, 100108.
[1] Kiseleva, Anastasiya. (2020). Ai as medical device: is it enough to ensure
performance transparency and accountability?. European Pharmaceutical Law Review
(EPLR), 4(1), 5-16.
[2] Ibid
[3] Ibid
[4] Ibid
[5] https://www.niti.gov.in/sites/default/files/2021-02/Responsible-AI-22022021.
[6] The Digital Data Protection Act,2023, Section 2 (t) “personal data” means any data about an individual who is identifiable by or in relation to such data;
s.2(i) “Data Fiduciary” means any person who alone or in conjunction with other persons determines the purpose and means of processing of personal data;
S.3. Subject to the provisions of this Act, it shall— (a) apply to the processing of digital personal data within the territory of India where the personal data is collected–– (i) in digital form; or (ii) in non-digital form and digitised subsequently; (b) also apply to processing of digital personal data outside the territory of India, if such processing is in connection with any activity related to offering of goods or services to Data Principals within the territory of India; (c) not apply to— (i) personal data processed by an individual for any personal or domestic purpose; and (ii) personal data that is made or caused to be made publicly available by— (A) the Data Principal to whom such personal data relates; or (B) any other person who is under an obligation under any law for the time being in force in India to make such personal data publicly available.
Sections 4,7,10,19,20,21,23,24,25,26 of the Digital Personal Data Protection Act,2023 https://www.meity.gov.in/writereaddata/files/Digital%20Personal%20Data%20Protection%20Act%202023.pdf
Author: Vahitha Parveen A, Student, School of Law, Sastra Deemed University, Thanjavur.
