Microsoft Research Explores 'AI as a Living Medium' to Treat AI as Malleable Material

Microsoft Research's 'AI as a Living Medium' is a research project exploring interaction paradigms that shift generative AI from a passive question-answering box into a flexible, manipulable expressive material. According to an October 23, 2026 note.com article by つるとーのひとりごと, the project emphasizes alternatives to conventional prompt-and-response workflows, including directly shaping and refining computational outputs during the creative thinking process.

Image source: https://www.microsoft.com/en-us/research/project/ai-as-a-living-medium/

Reference: https://www.microsoft.com/en-us/research/project/ai-as-a-living-medium/

Project reference: https://www.microsoft.com/en-us/research/project/ai-as-a-living-medium/

Project page: https://www.microsoft.com/en-us/research/project/ai-as-a-living-medium/

Concept reference: https://www.microsoft.com/en-us/research/project/ai-as-a-living-medium/

Related article: note.com/turuto (https://note.com/turuto/n/ne2c11988dce0?hl=ko)

According to the note.com article, current conversational AI experiences predominantly rely on a turn-based prompt format: users provide natural language instructions, and the model returns a result. Microsoft Research emphasizes shifting beyond this static exchange by enabling users to directly mold, edit, arrange, demonstrate, and explore outputs as dynamic materials.

Shifting from Prompting to Direct Manipulation of Living Media

According to the note.com article, the term 'Living Medium' does not denote biological vitality, but rather a computational material that continually transforms, adapts, and grows in response to user manipulation and situational context.

• Direct Manipulation Over Conversational Commands: Rather than relying only on broad natural-language requests, the framework emphasizes selecting, moving, mixing, transforming, and restructuring text or images.
• Focusing on Ideation and Process: Instead of treating AI merely as an oracle delivering finished answers, the medium supports uncovering possibilities, steering direction, and shaping evolving thoughts.

Prototype Studies: Texterial, Live Artifacts, and MagicCopy

The note.com article covers three related studies spanning text editing, dynamic canvas design, and cross-application workflows.

• Texterial (CHI 2026 Honorable Mention Award): Proposes a 'text-as-material' writing paradigm. Its implementations include 'Text as Clay,' where users sculpt sentences through physical gestures, and 'Text as Plants,' allowing concepts to branch, grow, and be pruned interactively.
• Live Artifacts (LiveCanvas): Positioned between static documents and interactive software, this approach embeds generative specifications within the artifact itself, allowing text, imagery, and audio to adapt to time, location, or live data. Through LiveCanvas, authors visually define which elements remain fixed and which adapt.
• MagicCopy: A transfer mechanism that uses source and destination application contexts plus user instructions to extract, interpret, transform, and restructure data across applications. In a related study, 16 participants used a prototype system to move data across three tool pairs; reducing manual reconstruction was the stated goal.

Material Flexibility and Human Editorial Responsibility

The note.com author argues that even when AI serves as a flexible drafting and restructuring medium, users should retain final intent and editorial control.

In dynamic environments such as LiveCanvas, the author explicitly configures which components vary and which stay invariant. The note.com author says AI can assist with research, organization, rewriting, and format conversion, while users set communication goals, verify facts, and decide what to retain.

Sources

• Microsoft Research: AI as a Living Medium
• note.com/turuto: AI from Answering Tool to Thinking Material: Reading Microsoft Research's Living Medium