Live UE Scene Bridge

Problem

The client side was prepared with diverse cameras, actions, and characters for launch, but controls for directing environments — light spawning, weather, and time-of-day — were lacking.

Solution

An AI layer (Claude/Gemini) converts natural language into commands, and a UE C++ plugin executes TOD, light spawning, and weather control in the editor in real time. Cesium requires raw coordinates, so the system translates city names and context into coordinates — a single line like "Show me downtown Tokyo" or "6 PM Seoul" sets up the entire background. Nano Banana and Veo-3 are also integrated to quickly edit, refine, and extend final video output.

3-Layer Architecture

Web UI / CLI → HTTP:34115 → FastAPI + NLP (Claude/Gemini)
                                    ↓
              ┌──────────────┬──────────────┬──────────────┐
              │ UE C++ Plugin│ Nano Banana  │ Veo-3        │
              │ TCP:55557    │ Image Gen    │ Video Gen    │
              └──────────────┴──────────────┴──────────────┘

Core — Cesium + UDS Integration

The most invested part. By combining Cesium terrain data with Ultra Dynamic Sky, a single natural language line sets up an outdoor background with location + time of day + weather + mood. Adding Nano Banana (i2i) on top transforms it into the desired concept or feel, so you go from a text prompt to a styled concept image without leaving the tool.

Style transformation result — "Give me a post-apocalyptic mood"

i2i pipeline in Web UI

Roblox Integration — Business Demo

Fetched FBX via Roblox API, imported into UE, placed characters on generated backgrounds, then produced images/videos with Nano Banana/Veo-3. Demo content targeting CEO, planners, and team leads.

Roblox character placed on generated background

Result