Tracked Billboard
Last updated
Last updated
In this section, you'll discover how to use the Tracked Billboard pipeline with Reality 5.4. For our demonstration, we will employ TRAXIS Talent Tracking as a provider.
Tracked Billboard enables you to seamlessly integrate your keyed talent into the Reality 5.4 scene, achieving a photorealistic appearance. Furthermore, it can be utilized in a variety of creative scenarios, including Teleportation and Fly Cam.
Login to Reality Hub.
Activate the Launcher module and launch your project.
Go to the Engines section, drag and drop the UE5
process into Nodegraph canvas.
Go to the Node Details Panel.
Click on the Spawn Reality Camera and Spawn Reality Tracked Billboard function buttons.
As soon as you click on abovementioned function buttons, following changes occurs as illustrated above:
Reality Actors folder has been created.
New Properties has been added.
Create AJAIn
and AJAOut
nodes, then designate your reference by right-clicking on each node. In our example, we have selected SingleLink 1
for AJAIn
and SingleLink 4
for AJAOut
.
Establish your corresponding camera tracking node. In our example, we utilized Xync
.
Connect the SingleLink 1
output pin of the AJAIn
node to the Sync
input pin to the Xync
node.
Breaking Track is a procedure for extracting specific output from a Track
node. For instance, you might prefer to use only the Location
data from your talent tracking software/hardware or on a focus property of Lens Distortion
data for your pipeline. This approach also allows you to conduct advanced compositing operations.
Now:
Create a Break Track
node.
Connect the Track
output of the Xync
node to the Input
input pin of the Break Track
node.
Now that we have AJA nodes for video I/O, talent tracking, camera tracking, and Reality 5.2 nodes, as illustrated in the previous image, it's time to incorporate Cyclorama.
Add a Cyclorama
node to the canvas.
Most camera lenses exhibit some level of optical distortion, and as a result, you often need to perform undistortion and then distortion operations on your video source to address this issue.
For instance, when dealing with the image from your studio and the 3D Cyclorama mesh you've created inside Reality 5.2, they may not align perfectly in terms of lens distortion. To ensure proper alignment between the Cyclorama and camera image, it's essential to employ the undistortion process.
Create two Distort
nodes and one Undistort
node.
Go to the Properties section of each node you created.
Right-click the Distortion
property for each node and select "Show as Input".
Connect the Lens Distortion
output of the Break Track
node to the Distortion
input pin of the Distort
and Undistort
nodes.
Connect the SingleLink 1
output pin of the AJAIn
node to the In
input pin to the Undistort
node.
Connect the AJAIn
node's SingleLink 1
output to Track
input of the Cyclorama
node.
Connect the Undistort
node's Out
output pin to Cyclorama
node's Video
input.
Connect the Cyclorama
node's Render
output to first Distort
node's In
input pin.
Connect the Cyclorama
node's Mask
output pin to the second Distort
node's In
input pin.
Tracked Billboard pipeline requires keyed image of your talent; therefore, you have to use Reality Keyer.
Create a RealityKeyer
node.
Connect the Out
output pin of the first Distort
node to the Clean Plate
input of the RealityKeyer
node.
Connect the Out
output pin of the second Distort
node to the Clean Plate Mask
the input of the RealityKeyer
node. Connect the SingleLink 1
output pin to the Input
input pin of the RealityKeyer
node.
Create a second Undistort
node, right-click on its Distortion
property and select "Show as Input".
Connect the Output
output pin of the RealityKeyer
node to the In
input of the second Undistort
node.
In order to place your talent inside the Reality 5.4 scene, you have to provide talent location data via a tracking system. In our example we utilized TRAXIS Talent Tracker.
Create a corresponding talent tracking data node. In our example we use FreeD
node.
Add the second Break Track
node.
Create a Cast vec3 to vec3d
node.
Connect the Location
output pin of the second Break Track
node to vec3
input pin of the Cast vec3 to vec3d
node.
Connect the vec3d
output pin of the Cast vec3 to vec3d
node to UE5
node's Reality Tracked Billboard Talent Location
input pin.
Connect the second Undistort
node's Out
output pin to Reality Track Billboard Video
input of the UE5
node.
Connect the Xync
node's Track
output pin to the Reality Camera Track
input pin of the UE5
node.
Create a Composite Passes
node.
Connect UE5
node's Reality Camera Scene
output pin to Composite Passes
node's Render
input.
Connect the Xync
node's Track
output to Track input of the Composite Passes node.
Connect the Output
output pin of the RealityKeyer
to the Video
input of the Composite Passes
node.
Connect the Output
pin of the Composite Passes
node to SingleLink 4
input pin of the AJAOut
node.