Green Screen Virtual Studio

This page is under construction and will be updated soon.

Real-time Green Screen Virtual Studio is a specialized production setup used in film, television, and other media industries. It involves shooting a subject or scene against a cyclorama, which is replaced with your real-time rendered image by Reality 5.3 SP2 using RealityKeyer.

Process

Select the Example Project from the Renderers section inside the Reality Hub Launcher module.

Select Reality 5 in the Launch With section as illustrated above.
Click on the Launch button.

Go to the Engines tab, drag and drop the UE5 node into the Nodegraph canvas.

Select the UE5 in the nodegraph canvas.
Click on the UE5 in the Nodes section.
Click on the Spawn Reality Camera and Spawn Reality Projection Cube buttons as illustrated above.

Add AJAIn and AJAOut nodes to the Nodegraph canvas.
Right-click on the AJAIn node and select an input channel from Node Context Menu.
Right-click on the AJAOut node and select an output channel.
Connect the corresponding SingleLink output pin of the AJAIn node to SingleLink input of the AJAOut node. (In our example we utilized SingleLink 2 and SingleLink 3)

Right click on the SingleLink 2 and utilize APM.
Add a Tracking Node (in our example it is the Xync node) into canvas.

Go to the Xync node property, click on the Enable checkbox.

Changing the Transform Mapping

Depending on your studio setup, you may be required to change the Transform Mapping property group settings.

In our example, we have to change the NegateX, NegateZ, CoordinateSystem, and NegatePan properties, as illustrated above.

Breaking Track

In order to get Lens Distortion data, you need to use the Break Track node.

Create a Break Track node.
Connect the Xync node's Track Output pin to the Break Track node's Input pin.

Create an Undistort node.

Undistorting the image taken by your studio camera is necessary because almost every camera lens has some degree of optical distortion. On the other hand, the Cyclorama node creates a 3D shape that isn't affected by lens distortions. To make sure the Cyclorama and camera image match up correctly, it's crucial to use the undistortion process.

Go to the Properties of the Undistort node.
Right-click on the Distortion property and select the "Show as Input" option.
Connect the Lens Distortion output of the Break Track node to the Distortion input pin of the Undistort node.
Connect the SingeLink output pin of the AJAIn node to the In input pin of the Undistort node.
Connect the Out output pin of the Undistort node to Video input of the Cyclorama node.
Connect the Track output pin of the Xync node to Track input of the Cyclorama node.

Create a RealityKeyer node and two Distort nodes.
Right-click on each Distort node's Distortion property and select the "Show as Input" option.
Connect the Render output pin of the Cyclorama node to the In input pin of the Distort node.
Connect the Mask output pin of the Cyclorama node to the In input pin of the second Distort node.
Connect the Lens Distortion output pin of the Break Track node to the Distortion input pin of the first Distort node.
Connect the Lens Distortion output pin of the Break Track node to the Distortion input pin of the second Distort node.
Connect the Out output pin of the first Distort node to the Clean Plate input of the RealityKeyer node.
Connect the Out output pin of the second Distort node to the Clean Plate Mask the input of the RealityKeyer node.
Connect the SingleLink 2 output pin of the AJAIn node to the Input input pin of the RealityKeyer node.

Utilizing Portal Ping and Routing Features

At this stage, by its nature, the nodegraph gets complicated and hard to read as illustrated above. To overcome such issues, you can utilize a Portal Pin and Reroute feature.

Right-click on the SingleLink 2 output pin of the AJAIn node and select the "Copy Pin" option
Right-click on an empty space inside the Nodegraph Canvas and select & paste the previously copied pin as a portal pin as illustrated above.

The AJAIn node's SingleLink 1 output pin becomes a Portal as illustrated above and can be used anywhere inside the nodegraph with the same functionality. Portal Pin also inherits any changes in the main node.

Connect the SingleLink 2 portal pin to In input pin of the Undistort node.
Right-click on an empty space in the nodegraph canvas and paste the SingleLink 2 portal pin again.
Connect the second SingleLink 2 portal pin to the Sync input pin of the Xync node.
By following the previous behavior, convert the Lens Distortion output pin of the Break Track node into the portal pin.
Paste and connect the Lens Distortion portal pin to the corresponding input pins of nodes.
Add Rerouting between node connections you desire.

Add a second Undistort node to nodegraph.
Right-click on the Distortion property and select the "Show as Input" option.
Connect the Lens Distortion output pin of the Break Track node to the Distortion input pin of the Undistort node or create another Lens Distortion portal pin and connect it to the Distortion input pin of the Undistort node.
Connect the Output pin of the RealityKeyer node to the In input pin of the Undistort node.
Connect the Out output pin of the Undistort node to the Projection Cube Video Input input of the UE5 node.
Create another Xync Track portal pin and connect it to the Reality Camera Track input of the UE5 node.
Create a Composite Passes node.
Connect the RealityKeyer node's Output pin to Video input pin of the Composite Passes node.
Create another Xync Track portal pin and connect it to the Track input pin of the Composite Passes node.
Connect the RealityCamera Scene output pin of the RE5 node to the Render input of the Composite Passes node.
Connect the Output output pin of the Composite Passes node to SingleLink input pin of the AJAOut node.

PreviousTracked Billboard NextShadow and Ambient Occlusion in Augmented Reality

Last updated 1 year ago