<aside> 💡 Usage guidance of UnrealCV+.

</aside>

UnrealCV+ is built upon UnrealCV and aims to help researchers build and use virtual worlds based on Unreal Engine (UE4/UE5). It extends the Unreal Engine project with a plugin by providing:

A set of commands to interact with the virtual world.
Communication between the UE environment and an external program, such as Python.

UnrealCV+ can be used in two ways:

The first one is using a compiled game binary with UnrealCV+ embedded. This is as simple as running a game, no knowledge of Unreal Engine is required.
The second is installing the UnrealCV+ plugin to Unreal Engine (UE4/UE5) and using the editor of UE to build a new virtual world.

📦 Install UnrealCV Client

We recommend you use Anaconda to install and manage your Python environment. CV2 is used for image processing, like extracting object masks and bounding boxes. Matplotlib is used for visualization.

Run the command in terminal to install UnrealCV+ client:

pip install unrealcv

🐍 Python APIs

Launch the UE Binary

from unrealcv.launcher import RunUnreal

ue_binary = RunUnreal(ENV_BIN=env_bin, ENV_MAP=env_map)

 env_ip, env_port = ue_binary.start(args.use_docker, parse_resolution(args.resolution), args.display, args.use_opengl, args.offscreen, args.nullrhi, str(args.gpu_id))

Connect to UE Environment

from unrealcv.api import UnrealCv_API

# connect to the launched UE environment
# mode: 'tcp' or 'unix', 'unix' is only for local machine in Linux
unrealcv = UnrealCv_API(port=9000, ip='127.0.0.1', resolution=(640,480), 'tcp')

# examples of calling functions by Python APIs
# get the number of cameras
print(unrealcv.get_camera_num())

# get a color image
img = unrealcv.get_image(cam_id=0, mode='lit', show=True)

# disconnect to UE environment
unrealcv.client.disconnect()

🆕 Improvement in UnrealCV+

<aside> 💡 We have enhanced the FPS (frames per second) in multi-agent settings and image rendering by incorporating parallel processes into the UnrealCV Server. Note that FPS can be further improved by reducing the image resolution, lowering the rendering quality, and decreasing the number of agents in the environment.

</aside>