Description
The camera payload can be used to capture and save or export images from the simulation environment. Building on Unreal Engine’s camera architecture as a base, Zendir has created the following camera payloads:
- Optical Camera: An RGB optical camera by default that can also become Monochromatic.
- Infrared (IR) Camera: A thermal camera designed to visualize the temperatures of any simulation physical components within its Field of View.
Both of these cameras can also be configured as an Event Camera which is inspired by Neuromorphic camera systems and can generate Event Data by comparing sequential frames.
Example Use Cases
- Optical and Thermal Imaging: Examples of objects that the camera can image include the spacecraft’s orbital body, planets, or other orbital objects such as spacecraft and debris.
- Event Detection: The Event Camera can be used to detect events within its Field of View that can then trigger other systems. For example, the Event Camera can instruct an Optical Camera to capture an image.
Module Implementation
The Camera Payload accepts a Device Status Message that allows the Instrument Manager Software to enable or disable the camera during the simulation. The camera payload also features several configuration variables such as Field of View, Sample Rate, File Name, and Exported File Type. the rest of the implemented module is built around Unreal Engines Camera capabilities.
The Camera Payload has a single output message, Camera Image Message, that contains the resolution, file name, and type as well as the raw file data. The message can be used to transmit camera payload data over the TT&C system.
The Event Camera Implementation works by comparing the Current Intensity value of a Pixel to its Previous Intensity. An Event is detect if the delta between both Intensities is greater than a configured Polarity Threshold Factor. The Polarity Threshold and Intensity values are clamped to a 0 → 1 range. For the following Polarity Thresholds, the following will occur:
- Polarity Threshold of 1= No Events will ever be detected.
- Polarity Threshold of 0.5 = Will only detect Events where a Pixel’s intensity increased or decreased by 0.5
- Polarity Threshold of 0 = Any change in Pixel Intensity will be detected as an Event.
The Event Camera is capable of generating both a visual representation of the Event Data as well as String arrays of the following format. The BrightnessChange is represented by a 0 or a 1 and is calculated from the above Polarity Threshold and Pixel Intensities. The brightness change of 0 represents a decrease event for a pixel with an increase represented by a 1. The format also includes a configurable Timestamp for the event and the position of the pixel that detected the event.
<Timestamp> <Horizontal Pixel> <Vertical Pixel> <BrightnessChange>
Assumptions/Limitations
- The Camera system can only capture an image of simulation objects that have an assigned mesh.
- The IR Camera variant requires the thermal properties of other objects to be configured to generate the required heatmap.
- The Event Camera can’t generate data on the first frame as it relies on comparing frames to detect a change. Also, it can only detect events based on every Unreal update, therefore it’s best to only perform one simulation step per Unreal update.