Selection over Generation

Abstract

1. Introduction

2026 marks an inflection point. Diffusion models now synthesize photorealistic imagery in seconds. Video generation systems like Sora, Runway Gen-3, and Kling demonstrate that AI-generated animation is technically feasible.

Yet a fundamental barrier prevents professional adoption: the absence of deterministic control.

Consider the task facing a production studio: animate a character turning their head 23° left while shifting gaze 15° right with a subtle asymmetric smile. Text prompts cannot express this. ControlNet guidance cannot guarantee it. And even if the input frame is perfect, how do you verify the generated video actually follows the intended motion?

We introduce the Parametric Control Layer—infrastructure positioned at both ends of the video generation pipeline. For input, it provides a searchable index of 153K+ frames. For output, it extracts parameters from every frame of generated video, enabling automated verification that head pose, eye gaze, and expressions match expected values.

1.1 Research Contributions

This proof of concept investigates:

• Parametric indexing: Representing extracted character features in a coordinate space for retrieval.
• Repeatable selection: Finding indexed frames nearest to a requested state under the same data and search configuration.
• Output analysis: Investigating whether related feature measurements can help evaluate generated video.
• A control-loop concept: Select → Generate → Verify → Iterate, with each stage requiring separate evaluation.
• Internal performance measurements: Query timings on the indexed corpus under the conditions described in this report.

2. The Control Problem in Generative Animation

2.1 Why Text Prompts Fail

Natural language is semantically rich but geometrically imprecise. The prompt "character looking slightly left" defines an infinite set of valid outputs. Even detailed prompts like "head rotated 20 degrees left, eyes looking forward, neutral expression" cannot constrain a generator to a single deterministic result.

This creates three critical failures for professional workflows:

• Non-reproducibility: The same prompt can yield different results across runs, complicating iterative refinement unless model, seed and settings are controlled.
• Imprecision: A semantic description alone does not specify an exact measured head angle.
• Entanglement: Changing one requested attribute, such as head turn, can affect expression or gaze.

2.2 The Limitations of Conditional Control

ControlNet and similar approaches improve precision through structural guidance (depth maps, pose skeletons, edge detection). However, they remain fundamentally generative—each inference produces an output whose exact pose and gaze require measurement. This report does not establish a reproducibility percentage for these external systems.

A body pose skeleton alone does not encode every facial control. Independent gaze and head orientation need additional conditioning or evaluation.

2.3 The Control Loop Problem

Modern video generation architectures (image-to-video, frame interpolation, motion transfer) share a common requirement: a deterministic seed frame. This frame establishes character identity, initial pose, and stylistic parameters that propagate through generated sequences.

This project investigates two tasks: programmatic selection of a recorded seed frame and measurement of the generated output—did the character turn to the requested angle, and did the gaze track correctly? The latter remains a research direction in this report.

3. Methodology

3.1 Parametric Space Definition

We define a 50-dimensional parametric space P ⊂ ℝ⁵⁰ that captures the essential degrees of freedom in character facial performance. This space decomposes into three independent subspaces:

The subspaces represent:

• Head Pose Subspace H: A novel "joystick-style" parameterization that maps 3D rotation to an intuitive 2D control surface plus roll and depth channels.
• Gaze Direction Subspace G: Independent eye tracking with horizontal/vertical components, blink state, and validity indicators for robust handling of edge cases.
• Expression Subspace E: High-dimensional blendshape representation compatible with industry-standard facial animation pipelines, enabling direct integration with VFX workflows.

3.2 Proprietary Feature Extraction

Our extraction pipeline transforms raw imagery into parametric coordinates through a multi-stage process optimized for both accuracy and throughput:

3.3 Joystick Parameterization

Traditional Euler angle representations suffer from gimbal lock and unintuitive interaction. We introduce a "joystick-style" mapping that projects 3D head rotation onto a bounded 2D surface:

This parameterization exhibits several desirable properties: bounded range [-1, 1], intuitive directional semantics, smooth interpolation, and natural correspondence to physical joystick input devices used in animation production.

Domain-Specific Calibration: Anime Facial Geometry

A critical insight from our anime-domain analysis: stylistic conventions in character illustration introduce systematic biases in landmark detection. Specifically, anime noses are typically drawn approximately 0.10 units left of the true facial midline—a consistent artistic convention across the genre.

Without correction, frontal poses would be misclassified as "looking right" because our landmark detector correctly identifies the nose position, which is stylistically offset. We introduce a domain-specific correction factor:

This correction is applied bidirectionally during both indexing and retrieval, ensuring that head_jx = 0 returns perceptually frontal poses despite the underlying data showing a leftward statistical bias (mean = -0.236). This represents the correct encoding of anime-style facial geometry rather than an error in extraction.

3.4 Perceptually-Weighted Distance Metric

Retrieval employs a weighted metric that reflects perceptual salience rather than raw geometric distance:

Weights are empirically tuned to match human perceptual judgments, prioritizing head pose (most salient), followed by gaze direction, then expression details. This ensures retrieved frames match human intuition about "closest match."

4. System Architecture

4.1 Index Structure

Each frame maps to a dense parametric embedding capturing the full 50-dimensional state:

FrameEmbedding {
    head_pose:    [h_x, h_y, h_roll, h_depth]     // 4D pose vector
    gaze:         [g_x, g_y, blink, valid]        // 4D gaze vector
    expression:   [e_1, e_2, ..., e_42]           // 42D blendshape
    metadata:     {character, timestamp, quality}  // Auxiliary data
}

The index maintains constant-time lookup properties while supporting complex multi-parameter queries with configurable tolerance bounds on each dimension.

4.2 Query Processing Pipeline

Queries execute through a staged pipeline optimized for both precision and speed:

1. Constraint Filtering: Apply hard constraints (character identity, validity requirements)
2. Tolerance Bounding: Reject candidates outside specified tolerance on active parameters
3. Distance Ranking: Sort remaining candidates by perceptually-weighted distance
4. Result Assembly: Return top-K matches with distance scores and confidence metrics

This architecture achieves real-time performance (~3ms) on the full corpus without requiring GPU acceleration or approximate methods—critical for interactive applications and high-throughput automated pipelines.

4.3 API Design Philosophy

The API exposes parametric queries as first-class operations, enabling both direct human interaction and programmatic access for AI agents:

5. Experiments & Results

5.1 Dataset Characteristics

The January 2026 study used the following indexed corpus:

The public corpus was generated for this research study. The corpus spans diverse poses, expressions, and gaze configurations, providing dense coverage across the parametric manifold.

5.2 Performance Benchmarks

We measure query performance on high-end consumer hardware (Intel i9-13980HX, 16GB RAM, RTX 4080 available but not required for search) to establish baseline performance:

*Retrieval precision against extracted parameters, not ground truth. See limitations section.

Real-time latency (~3ms) enables interactive exploration while supporting high-throughput batch processing for automated pipelines.

5.3 Comparative Evaluation

We conducted controlled studies comparing parametric selection against existing approaches for the task of finding a specific character pose:

*Note: Compares search (instant) to generation (slow). Retrieval precision is relative to extraction, not ground truth.

6. Research Applications

The study suggests possible uses for parametric retrieval and visual measurement in character workflows: selecting reference frames, comparing intended and extracted motion, and evaluating changes in a generation pipeline.

These applications require further validation. The public demonstration does not establish an end-to-end production system, independent verification of every output, or a commercial deployment offering.

6.5 Limitations & Honest Assessment

We believe in transparent research. Here are the current limitations of our approach:

Precision Claims

Our ±0.5° head angle precision claim requires context:

• This is retrieval precision, not ground truth: We return frames whose extracted parameters match within ±0.5°. The extraction itself has error margins.
• No ground truth validation: We don't have 3D-scanned ground truth for our generated frames. Precision is measured against our own extraction pipeline, not absolute reality.
• Depends on extraction quality: Our parametric extraction uses standard computer vision techniques (MediaPipe, learned models). These have their own error bounds (~2-5° typical for 2D-to-3D pose estimation).

Identical requests use the same retrieval procedure when the index and configuration are unchanged. This does not establish a reproducibility percentage. Absolute precision depends on extraction quality.

Design Decisions & Scope

• Identity variance as feature: Our current index deliberately includes design variants of base characters—subtle variations in facial structure, proportions, and style that represent the natural output distribution of generative models. Rather than filtering for perfect identity lock, we capture how the same "character concept" manifests across generation runs. This enables studios to evaluate parametric consistency independent of identity drift, and to select preferred design variants for downstream refinement. Identity-locked retrieval remains a research direction.
• Style/lighting consistency: Our parameters capture geometry, not appearance. Two frames can match parametrically but look different stylistically—this is intentional, as it isolates pose control from style control.
• Temporal smoothness via parametric interpolation: While we select individual frames, our parametric space enables trajectory planning: given start/end parameters, we compute intermediate waypoints along the manifold and retrieve the nearest indexed frames. For production animation, this provides keyframe suggestions that animators can refine, or that can seed frame interpolation models with geometrically-correct targets.

Comparison Methodology

Our "15× faster / 30× more precise" comparisons are against text prompting for finding a specific pose. This is a favorable framing—we're comparing search (instant) against generation (slow). A fairer comparison would note that generators create novel content; we only retrieve from a finite library.

7. Research Directions

The public study motivates further investigation in three areas:

These are research directions rather than commitments to a commercial service, deployment option or release date. Our first product remains in private development.

8. Conclusion

This proof of concept explores parametric character-frame retrieval using a precomputed index. It connects feature extraction with repeatable selection and investigates how related measurements can inform visual verification.

The 153,000+ frame study is limited to its indexed character imagery, extraction method and internal test conditions. Further work is needed to establish independent accuracy, broader generalization and production suitability.