DeepMind and animators use AI in the short Dear Upstairs Neighbors | Keryc
A little while ago at Sundance the short 'Dear Upstairs Neighbors' premiered, born from a collaboration between veteran animators and the Google DeepMind research team. What was the goal? To explore how generative tools can be woven into auteur filmmaking without stealing the artist's voice.
De la experiencia personal al lenguaje visual
Director Connie He started from a personal, everyday moment: not being able to sleep because of noisy neighbors. That seed grew into Ada, a protagonist whose hallucinations escalate from the absurd to the epic.
The design team, led by Yingzong Xin, set a clear aesthetic: a bedroom in cool tones as a refuge, hallucinations in neon palettes, and a visual language that shifts with emotional state. Ada has a very strict silhouette: a lock of hair and a bun that must never obscure the face. That 2D rule was an artistic requirement that guided all the technical work.
Herramientas y retos técnicos
The biggest challenge was reproducing pictorial styles and very specific visual rules consistently across animated sequences. The researchers did more than apply off-the-shelf models: they developed new capabilities so models could learn artistic concepts from just a few images.
Fine-tuning of Imagen and Veo with few examples: they trained fine versions of the models using Xin's paintings and designs. The result wasn't just color or texture; the networks learned deeper concepts, like two-point perspective and prioritizing silhouette.
Important: the model didn't just imitate styles, it adapted shapes to respect 2D rules impossible to represent with a rigid 3D sculpture.
Video-to-video: show instead of describe
To control rhythm, timing and framing, the team avoided relying solely on text prompts. They developed video-to-video workflows that let a raw animation (hand-drawn or 3D) serve as a guide. The model transforms that sketch into the final look, preserving motion and offering an adjustable balance between strict control and creative improvisation.
Practical examples:
Ben Knight made a 3D animation in Maya; the researchers used the fine-tuned version of Veo to generate the final appearance.
Mattias Breitholtz worked in TV Paint; the team used a fine-tuned Imagen inside a flow with ComfyUI to turn frames into the desired aesthetic.
The coordination allowed each artist to work in their comfort zone and still get coherent results.
Iteración localizada y refinamiento
No shot was perfect on the first try. They integrated local refinement tools to edit specific regions without regenerating the whole clip. One concrete example: improving Ada's hair silhouette by adding a mask and asking the model to improvise a lock that respects the design rule.
For scenes like a charming howling dog, they used Veo in image-to-video mode; the unfine-tuned version tended toward photorealism, so the fine-tuned version pushed the image toward the sought pictorial texture.
Producción, escala y entregables
To bring the short to cinemas, they used upscaling to 4K with Veo. The team tuned the model behavior to add detail without breaking the pictorial texture.
Production capacity included integration with Flow and plans for availability in Google AI Studio and Vertex AI, thinking about real filmmakers' needs for control, scalability, and compatibility with professional pipelines.
Lecciones técnicas y humanas
The project shows something often forgotten: AI is a tool that amplifies human decisions, not a replacement. Daily collaboration between animators and researchers was key: artists guided the intent, and researchers translated that intent into technical capabilities.
Fine-tuning models with few examples can capture deep artistic concepts.
video-to-video workflows are essential when movement and timing matter more than static appearance.
Local refinement tools make agile iteration viable in a film production environment.
Mirada final
'Dear Upstairs Neighbors' isn't just a technical demo; it's a map of how AI research can integrate with the animator's craft to create complex visual narratives. Worried that AI will displace the artist? Here's the practical answer: when tools are designed alongside creators, the result usually empowers creativity.
