A manifesto for Intelligent Experiences
By Drew Hemment, Elena Simperl, Mark Sandler, Helen Kennedy, Steve Benford, Sunil Manghani
This is our manifesto for intelligent experiences, along with some ideas on how AI design needs to change...
Let's look beyond the impressive capabilities of current AI tools, to envision 'intelligent experiences' that can surprise and delight us, are culturally enriching, and are inclusive, fair and environmentally sound.
Intelligent experiences are encounters with creative works infused with new and emerging technologies. They encompass sensory, emotional and intellectual engagement with various interfaces and artifacts powered by machine learning models.
These experiences are where humans meaningfully engage with machine learning's predictive capabilities. They manifest the outputs of machine logic in forms that humans can understand and interact with.
Crucially, intelligent experiences foster serendipitous, unexpected interactions rather than trying to replicate interactions learned from the past. This goes beyond efficiency, or mimicking the works of human artists. Intelligent experiences are celebrations of diverse interpretations and interactions, and mediators of constructive discussions rather than heated exchange. They provoke us to search for meaning and come to new interpretations of cultural works and ultimately of ourselves.
The arts, like any other field, have a responsibility to use technologies responsibly, so we call for intelligent experiences to be not only delightful and enriching, but also socially and environmentally just. They need to express a multiplicity of data rights and ownership, and acknowledge and reward effortful human creative practice.
For this to work, we need to ask questions of AI itself, and consider how the technology needs to evolve to make a lasting impact on human creativity. This vision for intelligent experiences will require AI to become more subjective, playful, able to deal with ambiguity, open to failure, improvised, negotiated, and being open to critique from those who engage with it. We believe intelligent experiences can help to radically change how we think about AI design, moving beyond the current paradigm of learning patterns from large amounts of data, to embrace human traits such as bias, disagreement, and uncertainty as a signal with creative potential rather than noise that needs to be removed.
This needs to work on three levels – creative, technical and social
Until the explosion of generative AI in 2023 and 2023, AI was typically viewed as a means to: (i) automate and scale creative ‘products’ and processes; (ii) enable micro-targeted, hyper-personalised experiences, often in exchange for consumer data; and (iii) streamline and amplify how people input within a creative process. We now see powerful new tools, such as DALL-E 2, Midjourney and ChatGPT powered by recent advances in diffusion models and large language models. We are also beginning to see AI being used in combination with other emerging technologies, from extended reality and robotics to new quantum technologies.
And yet, the new tools are black boxed and offer limited creative agency to human users. There are societal consequences that are well documented. They can amplify harmful bias, they are energy intensive, human labour is exploited, privacy can be violated, and value and power are centralised in the hands of a few.
Our manifesto for intelligent experiences anticipates a whole new context for making, sharing, learning, connecting and consuming. To deliver our manifesto, it is necessary to consider creative, technical and social factors:
Creativity – Empowering creative practice
For the creative sector, the step change is towards experiences infused with AI and other emergent technologies, that foster serendipitous and surprising interactions, that go beyond replicating interactions learned from the past. The arts demonstrate the value of human traits such as intuition, spontaneity, provocation and risk taking. In the arts, bias can be a virtue: it is talent and taste. More legible and configurable tools can enable artists to more meaningfully interact with AI models, and build on sophisticated machine learning predictions derived from up to tens of billions of parameters. Inturn, to achieve our vision for intelligent experiences, we hope to see the work of artists and techniques in creative domains drive innovations and make new demands on the technologies. Creative disciplines can create more balanced datasets for AI; make AI systems more interpretable; examine failure as a productive feature in AI design; and accommodate singular truths being displaced by a multiplicity of ideas and interpretations. The arts offer a site of creative experimentation where the most recent technology ideas can be connected to applications and impacts in the real world, uncovering challenges and opportunities.
Technical – Augmenting AI
For science and engineering, the step change is towards AI and related technologies that are more human, changing how machine learning and other automation techniques work, through a focus on discovery of new features that are not in the data and on valuing human skills and expertise. Machine learning, the most dominant form of AI today, generates outputs based on ingestion of vast quantities of historic data. Consequently, it amplifies existing features in the data, and, to that extent, it produces predictable outcomes, similar in some high-dimensional way to the examples it has learned from. The capabilities of the new generative AI tools are hugely impressive. Of high interest are the so-called emergent properties of large language models which are new and unexpected. We see something similar in the new game strategies computers find in chess, Go and beyond. This shows that AI is capable of unexpected discoveries, goes beyond merely uncovering a pattern in the data that was not obvious to the naked (human) eye. Nonetheless, fundamentally AI can only discover things that were already in the data. To discover something genuinely creative, AI needs to radically change, through new approaches to design, evaluation, and use. This requires a significant shift in AI research across different academic communities, towards participatory AI design, and more holistic notions of system evaluation and impact. It requires AI systems that work with and for people in adapting, evolving, responsive ways. The ingenuity of the creative sector can shape emerging AI, and inform the next generation of transformative, yet human-centric technologies.
Social – Improving fairness, diversity and sustainability
For society, the step change is developing new participatory, interdisciplinary formats to understand and assess when products and services do not amplify harmful bias, or squander planetary resources, and suggest new ways to think about social, economic, and environmental justice that resonate with diverse demographics. Through open and inclusive engagement with the international communities of critical arts, we can place a premium on research that is socially and environmentally reflexive, that informs policy and advances the betterment of society. The creative sector can help society navigate the profound transformations brought about by new and emerging technologies, and find new, progressive ways to engage with them. Artists, adept at surfacing critical societal issues, can contribute novel and more holistic perspectives on human-AI interaction, and improve understanding of how people perceive, think and behave in response to automated decision making. The creative community so recently shaken by the Covid-19 crisis can help us to identify radical new forms of inclusive value and anticipate the implications for future disruption.
Towards Intelligent Experiences
We hope these ideas can help creators to inspire the next generation of globally connected audiences, while at the same time avoiding, and perhaps even helping us to tackle, some of the negative societal consequences of current AI.
In our own research, we are taking steps towards this vision for intelligent experiences through our work on Experiential AI, which aims to open up the AI field to greater transparency and collaboration between human and machine (first proposed in our 2019 article in AI Matters, and recent results presented at Creativity & Cognition in 2023). We have also discussed how artists can inform the public conversation on AI (see AI in the Public Eye at FAccT 2023), or challenge us to think deeply about technology by asking difficult questions (see ‘Five Provocations for a More Creative TAS’). This is aligned with new research directions in other domains, such as data-centric AI where bias, disagreement, uncertainty are recorded with the dataset because that's the only way to minimise the risk that the AI is harmful.
We hope this manifesto can inspire new forms of creative practice and experiences for audiences, across the full range of creative disciplines. This can include new artistic forms in which AI is facilitates experiences infused with intelligence; use of autonomous systems to support or enhance creativity; and intelligent systems as an autonomous creator and creative partner. It also includes novel ways to present and experience creative works, by the approach and principles in this manifesto being applied to discover, curate, distribute, and consume creative works; to drive conversational approaches to co-interpretation; and to manage content authenticity, provenance, and intellectual property rights.
We can envision practical applications of intelligent experiences in many settings, from new forms of content discovery to live events, visitor experience, and also non-art applications:
Live & Immersive Events
Intelligent experiences opens up a range of opportunities and questions for immersive, live interactions, in gaming, and for the events industry. Intelligent experiences that are not only immersive, but also intuitive, uncontrived, and open to change, can enhance the intimate connection between a live music audience and an act on stage, or a gamer and a character in an open world game.
Museum Curation & Visitor Experiences
The contemporary museum is not so much driven by the need to acquire more exhibits as it is to find diverse and inclusive ways for visitors to interpret existing exhibits and discover relevance. Intelligent curation can engage audiences in co-interpreting content with and through AI, reflecting debates about identity, and facilitating personal meaning making and serendipitous encounters. This can lead to emotional and personally meaningful conversations, and edifying, beneficial and equitable experiences.
Information & Public Displays
Beyond the arts, as new computational methods become suffused in urban environments, and informational interfaces across a wide range of settings, from hospital wards to train stations, they will directly shape how we co-interpret and make meaning of daily experiences or critical information. It becomes possible to foster more contextual understanding, and enhance the capacity for the decisions and actions of technical systems to be made more legible to humans.
–
Our work in developing this manifesto has been community driven. The community first convened in 2019 around The New Real group and our Experiential AI theme, and then through the AI & Arts interest group at the Alan Turing Institute. It has brought together the research community, professional artists, cultural organisations, technology companies, policymakers, and the public. This work continues to this day, and we invite you to join us in this journey.
