The Conversational Director

Traditional AI video workflows create two distinct friction points: submit-and-wait latency that interrupts creative momentum, and text-based input that fragments visual attention. We present research on conversational directing — combining real-time streaming with voice commands to create a unified creative dialogue where directors see results immediately as they speak. This paradigm treats AI generation not as a batch process but as a continuous, responsive creative partner.

Current AI video generation operates on a submit-and-wait model: the creator describes their intent, submits a request, waits seconds to minutes for results, evaluates, and iterates. Each cycle breaks the creative flow state that Csikszentmihalyi (1990) identified as essential for optimal creative performance.

This latency problem compounds with text-based input. Directors must first translate their visual intuition into language (cognitive overhead), submit the description (wait), evaluate the mismatch between intent and output (disappointment), and rephrase (more cognitive overhead). Each full cycle can take 30–90 seconds — destroying any possibility of sustained creative flow.

Real-time streaming generation inverts the traditional pipeline. Instead of generating complete outputs on submission, the system produces a continuous visual stream that responds to directorial input in real-time. The director sees the generation evolve as they provide guidance — creating a feedback loop measured in milliseconds rather than minutes.

This approach aligns with Shneiderman's (1983) direct manipulation principles: continuous representation of the object of interest, physical actions instead of complex syntax, and rapid, incremental, reversible operations. The video stream becomes the "object of interest" that the director manipulates through natural input modalities.

Voice input addresses a specific limitation of text-based prompting: the attentional split between the creative output and the input mechanism. When a director types a prompt, their visual attention leaves the frame. When they speak, their eyes remain on the composition — maintaining the visual-spatial awareness that defines directorial craft.

• Eyes-on directing: Voice commands allow directors to maintain visual focus on the frame while issuing creative direction.
• Natural language for creative direction: "Push in slowly," "warm up the highlights," "hold that" — the language directors already use on set.
• Temporal precision: Voice provides real-time timing cues that text cannot express — the cadence and urgency of creative direction.

Conversational directing combines these two innovations into a single creative workflow. The director speaks to the AI as they would to a camera operator or gaffer, seeing the results materialize in real-time. The traditional generate → evaluate → re-prompt cycle collapses into a continuous creative conversation.

Dietrich's (2004) research on transient hypofrontality demonstrates that the neural state underlying creative flow is fragile — interruptions of even a few seconds can collapse it, requiring minutes to re-establish. Real-time streaming with voice input minimizes these interruptions by maintaining continuous creative engagement.

Our Live Director mode implements the conversational directing paradigm within the AIM Previz platform. Directors can engage a real-time streaming session where the AI responds to both voice and visual controls simultaneously. Camera movements, lighting adjustments, and compositional changes happen in real-time as the director guides the generation through natural creative language.

This mode complements our existing Manual and Auto director modes, providing a third paradigm optimized for exploratory creative work where the director wants to "discover" the shot rather than engineer it.

Conversational directing represents a fundamental shift in human-AI creative collaboration. By eliminating submit-and-wait latency and replacing text input with voice, we create conditions where sustained creative flow becomes possible with AI tools for the first time. The director directs — the AI responds — in real-time.