Tracked Billboard
In this section, you'll discover how to use the Tracked Billboard pipeline with Reality 5.4. For our demonstration, we will employ TRAXIS Talent Tracking as a provider.
What Is Tracked Billboard?
Tracked Billboard enables you to seamlessly integrate your keyed talent into the Reality 5.4 scene, achieving a photorealistic appearance. Furthermore, it can be utilized in a variety of creative scenarios, including Teleportation and Fly Cam.
Process
Login to Reality Hub.
Activate the Launcher module and launch your project.
Go to the Engines section, drag and drop the
UE5
process into Nodegraph canvas.
Go to the Node Details Panel.
Click on the Spawn Reality Camera and Spawn Reality Tracked Billboard function buttons.
As soon as you click on abovementioned function buttons, following changes occurs as illustrated above:
Reality Actors folder has been created.
New Properties has been added.
Create
AJAIn
andAJAOut
nodes, then designate your reference by right-clicking on each node. In our example, we have selectedSingleLink 1
forAJAIn
andSingleLink 4
forAJAOut
.Establish your corresponding camera tracking node. In our example, we utilized
Xync
.Connect the
SingleLink 1
output pin of theAJAIn
node to theSync
input pin to theXync
node.
Breaking Track
Breaking Track is a procedure for extracting specific output from a Track
node. For instance, you might prefer to use only the Location
data from your talent tracking software/hardware or on a focus property of Lens Distortion
data for your pipeline. This approach also allows you to conduct advanced compositing operations.
Now:
Create a
Break Track
node.Connect the
Track
output of theXync
node to theInput
input pin of theBreak Track
node.
Cyclorama Connections
Now that we have AJA nodes for video I/O, talent tracking, camera tracking, and Reality 5.2 nodes, as illustrated in the previous image, it's time to incorporate Cyclorama.
Add a
Cyclorama
node to the canvas.
Distorting and Undistorting
Most camera lenses exhibit some level of optical distortion, and as a result, you often need to perform undistortion and then distortion operations on your video source to address this issue.
For instance, when dealing with the image from your studio and the 3D Cyclorama mesh you've created inside Reality 5.2, they may not align perfectly in terms of lens distortion. To ensure proper alignment between the Cyclorama and camera image, it's essential to employ the undistortion process.
Create two
Distort
nodes and oneUndistort
node.
Go to the Properties section of each node you created.
Right-click the
Distortion
property for each node and select "Show as Input".
Connect the
Lens Distortion
output of theBreak Track
node to theDistortion
input pin of theDistort
andUndistort
nodes.Connect the
SingleLink 1
output pin of theAJAIn
node to theIn
input pin to theUndistort
node.Connect the
AJAIn
node'sSingleLink 1
output toTrack
input of theCyclorama
node.Connect the
Undistort
node'sOut
output pin toCyclorama
node'sVideo
input.Connect the
Cyclorama
node'sRender
output to firstDistort
node'sIn
input pin.Connect the
Cyclorama
node'sMask
output pin to the secondDistort
node'sIn
input pin.
Adding RealityKeyer
Tracked Billboard pipeline requires keyed image of your talent; therefore, you have to use Reality Keyer.
Create a
RealityKeyer
node.Connect the
Out
output pin of the firstDistort
node to theClean Plate
input of theRealityKeyer
node.Connect the
Out
output pin of the secondDistort
node to theClean Plate Mask
the input of theRealityKeyer
node. Connect theSingleLink 1
output pin to theInput
input pin of theRealityKeyer
node.Create a second
Undistort
node, right-click on itsDistortion
property and select "Show as Input".Connect the
Output
output pin of theRealityKeyer
node to theIn
input of the secondUndistort
node.
Talent Location
In order to place your talent inside the Reality 5.4 scene, you have to provide talent location data via a tracking system. In our example we utilized TRAXIS Talent Tracker.
Create a corresponding talent tracking data node. In our example we use
FreeD
node.Add the second
Break Track
node.Create a
Cast vec3 to vec3d
node.Connect the
Location
output pin of the secondBreak Track
node tovec3
input pin of theCast vec3 to vec3d
node.
Engine Connections
Connect the
vec3d
output pin of theCast vec3 to vec3d
node toUE5
node'sReality Tracked Billboard Talent Location
input pin.Connect the second
Undistort
node'sOut
output pin toReality Track Billboard Video
input of theUE5
node.Connect the
Xync
node'sTrack
output pin to theReality Camera Track
input pin of theUE5
node.Create a
Composite Passes
node.Connect
UE5
node'sReality Camera Scene
output pin toComposite Passes
node'sRender
input.Connect the
Xync
node'sTrack
output to Track input of the Composite Passes node.Connect the
Output
output pin of theRealityKeyer
to theVideo
input of theComposite Passes
node.Connect the
Output
pin of theComposite Passes
node toSingleLink 4
input pin of theAJAOut
node.