Emotional AI and Data Protection: Relevant Implications for Older Adults and Other Vulnerable Subjects
DOI:
https://doi.org/10.30827/dlf.2.2024.30946Keywords:
Artificial Intelligence, Emotion Recognition Systems, Vulnerable Subjects, Data Protection, Emotion Data, Transparency, DiscriminationAbstract
By examining the legal framework that has recently emerged in Europe, the work focuses on the main problems caused by the proliferation of AI technologies used for the care and well-being of vulnerable subjects (especially older people) and aims to identify tools of protection that can be used by the subject-user in the event of discrimination and violations of fundamental rights.
One of the areas which has recently been most affected by AI is that concerning the use of emotion recognition systems for the health-care and well-being of these individuals, who are not offered secure protection for self-determination and privacy.
In light of an analysis of the rapid advancement of these systems, the article also examines, through a comparative perspective, the regulatory challenges and the issues faced by Europe and the US.
References
A. ALSLAITY and R. ORJI, “Machine Learning Techniques for Emotion Detection and Sentiment Analysis: Current State, Challenges, and Future Directions”, Behaviour & Information Technology, 43, 1, 2024, pp. 139-164.
A. BRUNZINI, M. CARAGIULI, C. MASSERA and M. MANDOLINI, “Healthy Aging: A Decision-Support Algorithm for the Patient-Specific Assignment of ICT Devices and Services”, Sensors 2023, 23, https://doi.org/10.3390/s23041836.
A. MANTELERO and V. TIANI, “Norma UE su AI ‘appello urgente per una solida valutazione d’impatto sui diritti fondamentali’”, AgendaDigitale, 2023, https://www.agendadigitale.eu/sicurezza/privacy/norma-ue-su-ai-appello-urgente-per-una-solida-valutazione-dimpatto-sui-diritti-fondamentali/.
A. PALIOTTA, “Successful, Active and Healthy Aging: differenze e similarità nell’approccio al tema dell’invecchiamento”, Vita e Pensiero, 2022.
A. PIPITONE, “Empatia upmo-robot: il complesso rapporto tra l’AI e le emozioni”, AgendaDigitale, 2023, https://www.agendadigitale.eu/cultura-digitale/empatia-uomo-robot-il-complesso-rapporto-tra-lai-e-le-emozioni/.
A. PISAPIA, “What can we expect from European regulation on artificial intelligence?”, Cyberspace and Law, 24, 1, 2023.
A. ROTA, “Invecchiamento attivo e solidarietà tra le generazioni nel dialogo sociale europeo”, Rivista del Diritto della Sicurezza Sociale, 3, 2023.
A. SETHUMADHAVAN, G. BAIK, L. D’AMBROSIO et al. “World Economic Forum, Designing Artificial Intelligence Technologies for Older Adults” - Insight Report, 2021, pp. 2-10.
A. ZINZUWADIA, J.P. SINGH, “Wearable devices-Addressing Bias and Inequity”, The Lancet – Digital Health, 2022, https://www.thelancet.com/journals/landig/article/PIIS2589-7500(22)00194-7/fulltext;
B. HERRMANN, “The Perception of Artificial-Intelligence (AI) Based Synthesized Speech in Younger and Older Adults”, International Journal of Speech Technology, 26, 2023.
C. BURR, N. CRISTIANINI, J. LADYMAN, “An Analysis of the Interaction Between Intelligent Software Agents and Human Users”, Minds and Machines, 28, 2018.
C. IRTI, “L’uso delle tecnologie mobili applicate alla salute: riflessioni al confine tra la forza del progresso e la vulnerabilità del soggetto anziano”, Persona e Mercato, 1, 2023, pp. 32-33.
C. JEE, “Emotion Recognition Technology Should Be Banned, Says an AI Research Institute”, MIT Technolgy Review, 2019, https://www.technologyreview.com/2019/12/13/131585/emotion-recognition-technology-should-be-banned-says-ai-research-institute/. Article19, “Emotion Recognition Technology: A Threat to Free Speech, Equality and Privacy”, 2021, https://www.article19.org/resources/emotion-recognition-technology/.
C.M. CASCIONE, Il lato grigio del diritto, Torino, Giappichelli, 2022.
D. AMRAM, “La transizione digitale delle vulnerabilità e il sistema delle responsabilità”, Rivista italiana di medicina legale, 1, 2023, pp. 2-6.
D. CASTRO, (director of the Center for Data Innovation and Vice President of the Information Technology and Innovation Foundation), “The EU’s AI Act Is Premature, Says ITIF”, 2023, https://itif.org/publications/2023/12/08/the-eu-ai-act-is-premature/.
D. CHANG, “AI Regulation for the AI Revolution”, Singapore Comparative Law Review, 2023.
D. DEVAULT, R. ARTSTEIN, G. BENN, T. DEY et al, “SimSensei Kiosk: A Virtual Human Interviewer for Healthcare Decision Support”, Conference: Proceedings of the 2014 International Conference on Autonomous Agents and Multi-agent Systems, 2014.
D. RUGGIU, “L’emozione, nuovo territorio di conquista dell’intelligenza artificiale: applicazioni e rischi”, AgendaDigitale, 2021, https://www.agendadigitale.eu/cultura-digitale/lemozione-nuovo-territorio-di-conquista-dellintelligenza-artificiale-applicazioni-e-rischi/.
D.U. GALETTA, “Human-stupidity-in-the-loop? Riflessioni (di un giurista) sulle potenzialità e i rischi dell’Intelligenza Artificiale”, federalismi.it, 5, 2023, p. 4, https://www.federalismi.it/nv14/editoriale.cfm?eid=665.
E. CALZOLAIO, “‘I Dispositivi medici ‘intelligenti’: spunti di comparazione giuridica”, Il Foro Italiano, 2, 2022, pp. 75-83.
E. STEINDL, “Does the European Data Protection Framework Adequately Protect Our Emotions? Emotion Tech in Light of the Draft AI Act and Its Interplay with the GDPR”, European Data Prtoection Law Review, 8, 2, 2022.
E.M. INCUTTI, “Sistemi di riconoscimento delle emozioni e ruolo dell’autonomia privata: linee evolutive di un umanesimo digitale”, Giustiziacivile, 2022.
EHDEN (European Health Data & Evidence Network), “Protecting People while Using their Health Data for Research, a Framework for Understanding Relative Responsibilities and Roles”, 2022, https://www.ehden.eu/protecting-people/.
European Data Protection Supervisor (EDPS), “TechDispatch on Facial Emotion Recognition”, 1, 2021.
G. ÅGREN K., BERENSSON, “Healthy Ageing – A Challenge for Europe”, 2006, p. 9, chrome-extension://efaidnbmnnnibpcajpcglclefindmkaj/https://ec.europa.eu/health/ph_projects/2003/action1/docs/2003_1_26_frep_en.pdf.
G. CERRINA FERONI, “Intelligenza artificiale e ruolo della protezione dei dati personali”, Garante per la protezione dei dati personali, 2023, https://www.garanteprivacy.it/home/docweb/-/docweb-display/docweb/9855742.
G. DE GREGORIO and F. PAOLUCCI, “Dati personali e AI Act”, MediaLaws, 2022, https://www.medialaws.eu/dati-personali-e-ai-act/.
G. MOSCA, “Deep learning: cos’è, come funziona e applicazioni”, AgendaDigitale, 2023, https://www.agendadigitale.eu/cultura-digitale/deep-learning-cose-come-funziona-e-applicazioni/.
G. SCORZA, “Scorza: AI Act è a rischio, ecco le regole che servono – Intervento di Guido Scorza”, Garante per la protezione della privacy, 2023, https://www.garanteprivacy.it/home/docweb/-/docweb-display/docweb/9960565.
G.M. RICCIO and G. GIANNONE CODIGLIONE, “La rilevanza delle basi giuridiche per il trattamento di dati personali mediante sistemi di intelligenza artificiale”, in A. PAJNO, F. DONATI and A. PERRUCCI (eds.), Intelligenza Artificiale e Diritto: una rivoluzione?, Bologna, Il Mulino, 2022, pp. 281-311.
H. DEVLIN, “AI Systems Claiming to ‘Read’ Emotions Pose Discrimination Risks - Expert Says Technology Deployed is Based on Outdated Science and therefore is Unreliable”, The Guardian, 2020, https://www.theguardian.com/technology/2020/feb/16/ai-systems-claiming-to-read-emotions-pose-discrimination-risks.
H. HYDEN, “AI, Norms, Big Data, and the Law”, Asian Journal of Law and Society, 7, 3, 2020.
H. STEEGE, “Algorithm-Based Discrimination by Using Artificial Intelligence: Comparative Legal Consideration and Relevant Areas of Application”, European Journal of Privacy Law & Technology, 1, 2021.
J. STYPINSKA, “AI Ageism: A Critical Roadmap for Studying Age Discrimination and Exclusion in Digitalized Societies”, AI & Society, 38, 2023.
J.R. FLAHAUX, B.P. GREEN and A.G. SKEET, “Ethics in the Age of Disruptive Technologies: An Operational Roadmap”, Santa Clara University, 2023, https://www.scu.edu/institute-for-technology-ethics-and-culture/itec-handbook/.
K.A. CHAGAL-FEFERKORN, “The Reasonable Algorithm”, University of Illinois Journal of Law, Technology & Policy, 2018, 1.
L. COLONNA, “Artificial Intelligence in the Internet of Health Things: Is the Solution to AI Privacy More AI?”, Boston University Journal of Science and Technology Law, 27, 2, 2021, pp. 312-344.
L. COLONNA, “Artificial Intelligence in the Internet of Health Things: Is the Solution to AI Privacy More AI?”, Boston University Journal of Science and Technology Law, 27, 2, 2021, pp. 312-344.
L. MONTALBANO, “Brain-Machine Interfaces and Ethics: A Transition from Wearable to Implantable”, Journal of Business and Technology Law, 16, 2, 2021, pp. 191-222.
L. TIRABENI, “I dispositivi indossabili per il benessere”, Il Mulino, 3, 2022, pp. 119-120.
M. DI SALVO, “IA ed emozioni umane: definizione, regolamentazione e possibili implicazioni”, Diritto.it, 2023.
M. DUROVIC and J. WATSON, “Nothing to Be Happy About: Consumer Emotions and AI”, Multidisciplinary Scientific Journal, 2021, 4.
M. MARTORANA, “IA e riconoscimento delle emozioni: rischi e possibili vantaggi”, AgendaDigitale, 2023, https://www.agendadigitale.eu/sicurezza/privacy/ia-e-riconoscimento-delle-emozioni-rischi-e-possibili-vantaggi/.
M. MCTEAR, K. JOKINEN, M.M. ALAM, Q. SALEEM, G. NAPOLITANO et al, “Interaction with a Virtual Coach for Active and Healthy Ageing”, Sensors, 2023, 23, https://doi.org/10.3390/s23052748.
M. MENASSA, K. STRONKS, F. KHATAMI, Z. M. ROA DÍAZ, O. PANO ESPINOLA et al., “Concepts and Definitions of Healthy Ageing: A Systematic Review and Synthesis of Theoretical Models”, eClinicalMedicine, 2023, https://doi.org/10. 1016/j.eclinm, 2022.
M. PURDY, J. ZEALLEY and O. MASELI, “The Risks of Using AI to Interpret Human Emotions,” Harvard Business Review, 2019, https://hbr.org/2019/11/the-risks-of-using-ai-to-interpret-human-emotions;
M.C. CARROZZA, C. ODDO, S, ORVIETO, A. di MININ and G. MONTEMAGNI, “AI: profili tecnologici. Automazione e Autonomia: dalla definizione alle possibili applicazioni dell’Intelligenza Artificiale”, BioLaw Journal, 3, 2019, pp. 237-254.
N. NI LOIDEAIN, R. ADAMS and D. CLIFFORD, “Gender as Emotional AI and the Case of ‘Nadia’: Regulation and Ethical Implications”, SSRN, 2021, https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3858431.
N. SHEN, “AI Regulation in Health Care: How Washington State can Conquer the New Territory of AI Regulation”, Seattle Journal of Technology, Environmental & Innovation Law (SJTEIL), 13, 1, 2023, pp. 1-4
P. EKMAN and W.V. FRIESEN, “The Reportoire of Nonverbal Behaviour: Categories, Origins, Usage and Coding”, Semiotica, 1, 1, 1969, pp. 49-98.
P. EKMAN, “Constans Across Cultures in the Face and Emotion”, Journal of Personality and Social Psychology, 17, 2, 1971, pp. 124-125.
P. EKMAN, J.R. DAVIDSON, The Nature of Emotion: Fundamental Questions, USA, Oxford University Press, 1994, passim; P. EKMAN, “Basic Emotion”, in T. DALGLEISH and M.J. POWER (eds.), Handbook of Cognition and Emotion, England, John Wiley & Sons, 2000, pp. 45-47.
P. KROOT TUPAY, M. EBERS, J. JUKSAAR and K. KOHY, “Is European Data Protection Toxic for Innovative AI? An Estonia Perspective”, Juridica International, 30, 2021, pp. 99-110.
P. KULURKAR, C.K. DIXIT, V.C. BHARATHI et al, “AI Based Elderly Fall Prediction System Using Wearable Sensors: A Smart Home-Care Technology with IOT”, Sensors, 25, 2023.
P. OTTOLINA, “AI Act, controllo biometrico a distanza e riconoscimento delle emozioni: i nodi dell’accordo Ue sull’intelligenza artificiale”, Corriere della Sera, 2023, https://www.corriere.it/tecnologia/23_dicembre_08/ai-act-controllo-biometrico-a-distanza-e-riconoscimento-delle-emozioni-dove-si-e-incagliato-l-accordo-ue-2ee5a84c-a930-45ea-8046-b595e368cxlk.shtml.
P. STANZIONE, “‘Dispositivi indossabili: rischi per la privacy. Che fine fanno le informazioni raccolte?’ – Intervista a Pasquale Stanzione”, Garante per la protezione dei dati digitali, 2021, https://www.garanteprivacy.it/home/docweb/-/docweb-display/docweb/9552323;
R. CALO, “Privacy, Vulnerability and Affordance”, DePaul Law Review, 66, 2017.
R. CARLEO, “Il trattamento dei dati sanitari digitalizzati tra tutele individuali e interessi comuni”, in U. RUFFOLO and M. GABRIELLI (eds.), Intelligenza artificiale, dispositivi medici e diritto, Torino, Giappichelli, 2023.
R. PICARD, “Affective Computing”, MIT Media Laburatory Percetual Computing Section Technical Report, 1995, passim.
S. WACHTER, “The Theory of Artificial Immutability: Protecting Algorithmic Groups Under Anti-Discrimination”, Tulane Law Review, 2022, 97.
S.B. DAILY, M.T. JAMES, D. CHERRY, J.J. III PORTER, S.S. DARNELL, J. ISAAC and T. ROY, “Affective Computing Historical Foundations, Current Applications and Future Trends,” in M. JEON (ed), Emotions and Affect in Human Factors and Human-Computer Interaction, Elsevier Academic Press, 2017, pp. 213-231.
T. TELFORD, “‘Emotion Detection’ AI is a $20 Billion Industry. New Research Says it Can’t Do What it Claims - Artificial Intelligence Advanced by Such Companies as IBM and Microsoft is Still no Match for Humans”, The Washington Post, 2019. https://www.washingtonpost.com/business/2019/07/31/emotion-detection-ai-is-billion-industry-new-research-says-it-cant-do-what-it-claims/.
T.R. MOSLEY, “AI Isn’t Great at Decoding Human Emotions. So Why are Regulators Targeting the Tech?”, MIT Technology Review, 2023, https://www.technologyreview.com/2023/08/14/1077788/ai-decoding-human-emotions-target-for-regulators/.
V. SALVATORE, “L’Unione europea disciplina l’impiego dell’intelligenza artificiale e dei processi di digitalizzazione anche al fine di promuovere la tutela della salute”, in V. SALVATORE (ed), Digitalizzazione, intelligenza artificiale e tutela della salute nell’Unione europea, Torino, Giappichelli, 2023.
W. NICHOLSON II PRICE, “Problematic Interactions between AI and Health Privacy”, Utah Law Review, 4, 2021.
Y. CAI, X. LI and J. LI, “Emotion Recognition Using Different Sensors, Emotion Models, Methods and Datasets: A Comprehensive Review”, Sensors, 23, 2023. efaidnbmnnnibpcajpcglclefindmkaj/https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10007272/pdf/sensors-23-02455.pdf; J.S. BARD, “Developing Legal Framework for Regulating Emotion AI”, Boston University Journal of Science and Technology Law, 27, 2, 2021.
Downloads
Published
Issue
Section
License
Copyright (c) 2024 De Lege Ferenda
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
