Houyi Shoots
the Sun

academic course, individual

1 week + 2 weeks, continuous, 2024

#virtual production, #world building, #show control, #style transfer, #LoRA

Unreal Engine, Max, ComfyUI, Pinokio

Houyi Shoots the Sun is part of a collaborative project that combines world-building in Unreal Engine with AI-driven style transfer. The concept is inspired by the ancient Chinese myth of the same name. I created Scene 1, which depicts the moment when ten suns rose into the sky simultaneously. The unbearable heat left people in despair. In the world-building part, I constructed the scene in Unreal Engine and used Max to orchestrate the show controls. In the style transfer part, I trained with LoRA and applied ComfyUI to perform style transfer on both the environment and the character design.




Storyline
-
(My Part) Ten suns appeared in the sky at the same time, scorching the land and drying up the crops. People were overwhelmed by the heat, collapsing to the ground and losing consciousness.
-
Some fierce beasts and monstrous birds fled from the dried-up rivers and lakes and the fiery forests, causing the villagers to run away in fear.
-
Hou Yi descended from the heavenly palace under the orders of the Heavenly Emperor.
-
Hou Yi shot down nine of the suns with his bow, leaving only one. The villagers cheered in celebration.

illustration of the myth by Chinese Learning
Scene Construction
Mountain in the back where Houyi will descend on, forest on the right where beasts will run out of, and village on the left. Show control triggers fire and people falling on the ground.

draft 1

event graph of fire blueprint

viewport of fire blueprint

draft 2

event graph of villager blueprint

viewport of villager blueprint

Max patcher of show controls
Final Scene

Style Transfer with LoRA
Character Training in Pinokio/Fluxgym​




Style Training in Pinokio/Fluxgym​




ComfyUI Setup with LoRA


Final Results

reference

character transfer

style transfer
Style Transfer and VFX Breakdown

character transfer with Viggle

background transfer with EbSynth

clothing transfer with Adobe Firefly

VFX breakdown with Adobe After Effects
References
-
Legolas (The Hobbit/Lord of the Rings)
-
Tsuki’s Odyssey
On this page or go back?
On this page
-
Storyline
-
Scene Construction
-
Final Scene
-
Style Transfer with LoRA
-
Style Transfer and VFX Breakdown
-
References
Other Archived Projects
-
Digital Heritage Exhibition
-
Charity: Water Marketing Campaign
-
Tianzige NGO’s Official Website
-
Walligraphy
-
Jackson Square Fountain
-
Houyi Shoots the Sun
-
New York Series
-
Character Introduction
-
Fishing Day
-
Shanghai Series
-
Stone of the Sun
-
Quiet Time Room
-
Electromagnetic Gun
-
Acrylic Paintings
-
Modular Font