Counterfactual Assessment and Valuation for Awareness Architecture

Counterfactual Assessment and Valuation for Awareness Architecture (CAVAA)

Portrait colour photograph of Alberto Giubilini and Cristina Voinea

Oxford's CAVAA team members, Dr Alberto Giubilini and Dr Cristina Voinea

Funding

European Commission, EIC 101071178; CO-PI: Julian Savulescu

Project Dates

October 2022 - October 2026

Oxford Project Manager

Alberto Giubilini

Oxford Researcher

Cristina Voinea

Full CAVAA Project

https://cavaa.eu/

 

Project Description

Robotics and artificial intelligence technologies are becoming increasingly advanced, and some researchers hope to build robots or AI systems that are aware of the world around them. The team at the Uehiro Oxford Institute will examine ethical issues that arise due to AI awareness, including questions of privacy and value alignment. In collaboration with CAVAA collaborators at Uppsala University and Sorbonne University, the Oxford team will investigate human judgments about privacy and other values, which may then inform policy recommendations for the design, construction, and regulation of AI systems.

Examining the ethical issues raised by aware AI involves several components. In some of our work, we examine normative and philosophical questions about what it means for an AI to be aware, whether it’s possible for AI systems to infringe our privacy, and what it might take for AI to be aligned with our norms and values. In other work, we investigate the human relational psychology of interacting with AI, as well as human preferences about how AI should treat the information they may learn about humans. Finally, in order to make reasonable assessments of the risks posed by AI, as well as reasonable suggestions for how to design ethical AI, we learn from researchers who are designing and building current generations of AI architectures and social robots.

Full details of the project and our collaborating partners can be found on the CAVAA website.

Outputs

UOI Publications

Earp, B. D., Porsdam Mann, S., Allen, J., Salloch, S., Suren, V., Jongsma, K., Braun, M., Wilkinson, D., Sinnott-Armstrong, W., Rid, A., Wendler, D. and Savulescu, J., (2024), 'A Personalized Patient Preference Predictor for Substituted Judgments in Healthcare: Technically Feasible and Ethically Desirable', American Journal of Bioethics, Vol: 24(7): 13-26 [PMC11248995]

Earp, B. D., Porsdam Mann, S., Liu, P., Hannikainen, I., Khan, M. A., Chu, Y. and Savulescu, J., (2024), 'Credit and blame for AI-generated content: Effects of personalization in four countries', Annals of the New York Academy of Sciences, Vol: 1542(1): 51-57 [PMC11668494]

Giubilini, A., (2025), 'It is not about AI, it is about humans. Responsibility gaps and medical AI', Journal of Bioethical Inquiry, Vol: 22: 527–537 [PMC7618118]

Giubilini, A., Voinea, C., Porsdam Mann, S., Earp, B. D. and Savulescu, J., (2024), 'Know Thyself, Improve Thyself: Personalized LLMs for Self-Knowledge and Moral Enhancement', Science and Engineering Ethics, Vol: 30(6): 54 [PMC11582191]

Iglesias, S., Earp, B. D., Voinea, C., Mann, S. P., Zahiu, A., Jecker, N. S. and Savulescu, J., (2025), 'Digital Doppelgängers and Lifespan Extension: What Matters?', The American Journal of Bioethics Vol: 25(2): 95-110 [PMC11804783]

Kagan, B., Loeffler, A., Boyd, J. and Savulescu, J., (2024), 'Embodied Neural Systems Can Enable Iterative Investigations of Morally Relevant States', Journal of Neuroscience, Vol: 44(15): e0431242024

Kataoka, M., Gyngell, C., Savulescu, J. and Sawai, T., (2023), 'The Ethics of Human Brain Organoid Transplantation in Animals', Neuroethics Vol: 16(27)

Levy, N., (2024), 'Consciousness ain't all that', Neuroethics, Vol: 17(21)

Lyreskog, D., Zohny, H., Porsdam Mann, S., Singh, I. and Savulesu, J., (2024), 'Decentralising the Self – Ethical Considerations in Utilizing Decentralised Web Technology for Direct Brain Interfaces', Sci Eng Ethics, Vol: 30(28)

Lyreskog, D., Zohny, H., Savulescu, J. and Singh, I., (2023), 'Merging Minds: the conceptual and ethical impacts of technologies for collective minds', Neuroethics, Vol: 16(1): 12

Register, C., (2025), 'Individuating artificial moral patients', Philosophical Studies, Vol: 182: 3225–3246 [Oxford Research Archive]

Rueda, J., Pugh, J. and Savulescu, J., (2023), 'Rethinking techno-moral disruption in bioethics, society, and justice', Trends in Biotechnology, Vol: 41(6): 743-744 

Vazirani, A., Mccaffrey, T., Savulescu, J. and Porsdam Mann, S., (2024), 'BrainSwarming, blockchain, and bioethics: applying Innovation Enhancing Techniques to healthcare and research', Sci Rep, Vol: 14(862)

Zohny, H., Lyreskog, D., Singh, I. and Savulescu, J., (2023), 'The mystery of mental integrity: Clarifying its relevance to neurotechnologies', Neuroethics, Vol: 16(20)

UOI blogs/media

Blog post 'Friend AI: Personal Enhancement or Uninvited Company?' by Chris Register (8 October 2024)

Workshops and talks

In the spring of 2025, we hosted a workshop on 'Privacy, Awareness, and Alignment in AI', with participants from Oxford, Google Deep Mind, Sorbonne, Uppsala, Cambridge, and Sheffield.