Lightact media server works seamlessly with Unreal Engine 4 (UE4). This means that it can send and receive textures from UE4 and it can also send variables to UE4. This allows you to use Lightact’s media server functionalities with Unreal Engine’s rendering capabilities. In this tutorial, we’ll go through the steps of setting up a basic project which will use bi-directional texture sharing (via spout) and one-directional variable sharing.
If you prefer to watch video tutorials, there’s a whole playlist on Unreal Engine and Lightact Media Server Integration on YouTube.
1 – Lightact setup
First, we’ll create a Lightact project from start to finish and then we’ll switch to UE4 and finish that part.
Spout sender in Lightact
We’ll start by opening the Devices window by clicking on Window and then Devices in the Top menu or pressing F7 on your keyboard. When the window opens, right-click somewhere in the empty area on the left, hover above Spout and click on Spout client.
When the Spout client node appears, select it and change Spout name property on the right to LA2UE.
Click Apply and make sure the Streaming checkbox is checked. We’ve created a Spout sender that will be sending a texture from Lightact to UE.
Spout receiver in Lightact
Now, right-click on an empty area again, hover above Spout and click on Spout server. This will be the node that will receive a spout texture from UE. In the Spout name property on the right insert UE2LA.
Click Apply and make sure the Listening checkbox is checked.
JSON sharer in Lightact
The only thing that’s left for us to do in the Devices window is to set up a JSON sharer node. It writes Lightact variables to a shared memory from where Unreal Engine can read them. So let’s get straight to it.
Right-click on an empty area again, hover above JSON sharer and click on Add new.
When the node appears we just change the Handle name property on the right to vars2UE and make sure the Streaming checkbox is checked.
By the end of this, you should have 3 nodes in the Devices window as shown below.
Setting up layer layout
We’ve done everything that needs to be done in the Devices window, so you can close it if you want. Now, we have to insert a new layer and set up its layout so that it sends and receives Spout textures from the Spout nodes you’ve inserted, and that it sends variables to JSON memory sharer.
So let’s right-click in the Sequencer and insert a new layer.
After that, double-click on it so that its layout opens. When it opens, insert the following nodes:
- Render to canvas
- Spout sender
- Spout receiver
- Shared memory sender (you can click on Add float button immediately because that’s what we’ll be adding)
- Perlin noise
- Sine wave
Connect it as shown in the screenshot above and make sure the transport is on the layer so that the connections fade up and down.
What we have to do next is the following:
- Select Spout reader node and in its properties on the right select Spout server 1
- In the Perlin noise node type in 1920×1080 in the Resolution textboxes
- Select Spout sender node and in its properties on the right select Spout client 1
- In the Sine wave node type in 45 and 0.1 in the Amplitude and Frequency respectively
- In the Shared memory sender node under the FName property type in floatAngle.
The final layout should look like this:
That’s all we needed to do in Lightact. Now it’s time to set up Unreal Engine.
2 – Setting up Unreal Engine
Creating a C++ project
The first thing you’ll want to do is to create a new C++ project. To keep things as simple as possible, let’s open the Epic Games Launcher, head to the Library tab and click on Launch button under the latest version of Unreal Engine.
When Unreal Editor opens, click on New Project tab, then open C++ tab, then click on First Person template, then select No Starter Content, type in the location and name of your project and click on the green Create Project button.
When the project opens, you can close the Unreal Editor.
Installing UE4 plugins
To make Unreal Engine work with Lightact we first need to install 2 Unreal Engine plugins. We are making them available for free on Lightact Github page. The names of the plugins you should install are Spout-UE4 and Lightact-UE4.
Open their pages and click on Clone or download and then on Download ZIP.
After you’ve downloaded and extracted both ZIP files please follow the instructions on plugins’ Github pages in order to install them into your project. The installation process is the same as with all C++ UE4 plugins. When you’ve successfully installed them they will both appear in the Plugins window of Unreal Editor and have their Enabled checkboxes checked.
Great! Now we can move on to creating the project. There are 3 parts to it: send a texture from UE4 to Lightact, receiving a texture from Lightact in UE4 and receiving a floatAngle variable from Lightact in UE.
Spout sender in Unreal Engine
The first step is to create a Render target in the Content Browser. You do that by right-clicking and in the empty area of the Content Browser find Render target in the Materials & Textures submenu.
To keep things neat, rename it to UE2LARenderTarget and open it by double-clicking on it. In the window that opens, in the Size X and Size Y enter 1920 and 1080. This will be the resolution of the Spout sender stream and it has to match to the Spout receiver resolution in Lightact.
Next step is to create a Scene Capture Component 2D which will render to our UE2LARenderTarget. You do that by typing scene in the Search Classes textbox in the main window of Unreal Editor.
When Scene Capture 2D appears simply drag it to the viewport.
Then, select it and in its properties on the right, in the Scene Capture section, under Texture Target, select our UE2LARenderTarget, and under Capture Source select Final Color (LDR) in RGB.
Great, now we can move on to creating the blueprint.
Creating the blueprint
We’ll work in the Level Blueprint to keep things simple. You open by going to the main window of the Editor and clicking on Blueprints and then on Open Level Blueprint.
When it opens, insert the following nodes:
- Spout Sender
- Close Sender
- Event End Play
If you can’t find these nodes, it means Spout plugin isn’t installed and enabled correctly.
In the Spout Sender node:
- in the Spout Name text box type in UE2LA. This has to match the name of Spout receiver on Lightact’s side,
- in the Send Texture From drop-down select Texture Render Target 2D and
- in the Texture Render Target 2D drop-down select our UE2LARenderTarget.
Then, in the Close Sender node type in UE2LA in the Spout Name text box.
Now, connect the 3 nodes you created as shown below
This is it. Congratulations! To summarize what the blueprint above does: it sends the texture from our Render Target to UE2LA spout every frame (that’s why its connected to Event Tick node). The Close Sender node closes this spout when the gameplay stops.
In the next step, we’ll create a Spout Receiver.
Spout receiver in Unreal Engine
The first step in creating a spout receiver is to create an object in the viewport that will display the texture coming from Lightact. So, let’s create a cube. We do that by dragging Cube from the Classes library on the left to the viewport.
For the purposes of clarity let’s name it LA2UECube.
Now, open the Level Blueprint and create the following nodes:
- Spout Receiver
- Close Sender
In the Spout Name text box of both the Spout Receiver and Close Sender nodes type in LA2UE. This is the name of the spout that has to match the name of the spout sender in Lightact.
Then make sure the LA2UECube cube you created in the main viewport is still selected and right click somewhere in the empty area of the level blueprint. Click on Create a Reference to LA2UECube.
A new node will appear that represents a reference to our cube.
Drag off of the blue pin and in the menu that appears when you let go of the left mouse button type in Set material. Click on Set Material (StaticMeshComponent).
By now you should see 3 nodes as shown below.
The only thing that’s left for us to do is to connect the whole thing. Connect all the nodes as shown below.
Part 1 shows the Spout Sender part, which we did in the previous chapter and part 2 shows the Spout Receiver part.
Congratulations! That’s all we had to do in this chapter. Now if you press play in Unreal Editor you should see a cube with a static Perlin noise texture and in Lightact you should see the texture from Unreal Engine.
A quick tip, if you want the Perlin noise texture to be dynamic you should connect Application runtime to the Runtime pin of the Perlin noise node back in Lightact.
Using Texture output instead of Material
As you can see, the Spout Receiver node in UE4 has 2 outputs, one is the Material, which we are using here, and the other is Texture. Using Texture pin instead of Material gives you many more options but the explanation exceeds the scope of this quick start guide.
However, if you are interested in that you can watch a dedicated video tutorial on Webcam and dynamic Materials in Unreal Engine and Lightact.
Spout framework has 3 texture sharing modes: GPU texture sharing, CPU texture sharing and Memory sharing. The one you should use, and the one that has by far the best performance is GPU texture sharing. You can see which one you are using if you run Spout demo receiver program (available as part of Spout SDK) and when it receives a Spout stream you should see Textureshare label on it, as shown below:
If you see anything other than that, chances are Spout won’t work in UE4.
You can do change that by running Spout DX mode program(also part of Spout SDK).
In there you should make sure that in Share mode section, Texture radio button is selected.
Receiving variables in Unreal Engine
Remember that JSON sharer node we created at the beginning of this guide? It’s used to send variables from Lightact to UE. Why would you possibly need that? Well, this allows you to use any input Lightact has, to control UE. For example, with this you can control UE4 using TCP, UDP, OSC, Art-Net, Lightact WebUIs, online RSS feeds or any online or local text file and so on. Luckily for us, this is really easy to do.
In the Level Blueprint create the following nodes:
- Shared memory JSON to string map
- 2 Branch nodes
- another reference to LA2UECube cube
- Make Rotator
In the Shared memory JSON to string map node type in vars2UE in the Handle Name text box and 1024 in the Handle Size text box.
These values have to match the values you typed into the JSON memory sharer 1 node in Lightact.
Then drag off of the Result Map pin and, in the menu that appears, type in Find. Click on the Find node in the menu. When the node appears, type in floatAngle in the text box of the Find node.
We’ll use this variable to rotate our LA2UECube a bit, so drag off of the blue pin and start typing in set actor relative rotation. Click on SetActorRelativeRotation to insert the node.
Now connect all the nodes like this:
Then connect the pink (string) output on the Find node with the Y green (float) input on the Make Rotator node.
You should see something similar to this:
Then connect the white (execution) input pin on the Shared memory JSON to string map node to the white (execution) output pin on the Set Material node you created in the previous chapter. By now your blueprint should look somewhat similar to this.
That’s it! Congratulations. If you press play you should see the LA2UECube slowly rotate back and forth.