Camera

SWITCH TO API

Description

The camera payload can be used to capture and save or export images from the simulation environment. Building on Unreal Engine’s camera architecture as a base, Zendir has created the following camera payloads:

• Optical Camera: An RGB optical camera by default that can also become Monochromatic.
• Infrared (IR) Camera: A thermal camera designed to visualize the temperatures of any simulation physical components within its Field of View.

Both of these cameras can also be configured as an Event Camera which is inspired by Neuromorphic camera systems and can generate Event Data by comparing sequential frames.

---

Example Use Cases

• Optical and Thermal Imaging: Examples of objects that the camera can image include the spacecraft’s orbital body, planets, or other orbital objects such as spacecraft and debris.
• Event Detection: The Event Camera can be used to detect events within its Field of View that can then trigger other systems. For example, the Event Camera can instruct an Optical Camera to capture an image.

---

Module Implementation

The camera is a Sensor that captures images at a configurable sample rate. When the sensor is operational and the sample interval has elapsed, the camera sets a ready flag that triggers image capture through the visualization engine.

Optical Parameters

The camera models realistic optical behavior through interdependent parameters. Changing one parameter automatically updates related values to maintain physical consistency.

Field of View

The field of view is calculated from the sensor geometry and focal length:

$${\theta }_{\text{FOV}}=2\arctan \left(\frac{d}{2f}\right)$$

where $d$ is the sensor diagonal length and $f$ is the focal length. The diagonal is computed from the resolution and pixel pitch:

$$d=\sqrt{(W\cdot p{)}^2+(H\cdot p{)}^2}$$

where $W$ and $H$ are the horizontal and vertical resolution in pixels, and $p$ is the pixel pitch.

Sensor Size

The physical sensor dimensions are derived from resolution and pixel pitch:

$$S_x=W\cdot p,S_y=H\cdot p$$

Hyperfocal Distance

The hyperfocal distance is the focusing distance beyond which all objects appear acceptably sharp:

$$H=\frac{f^2}{N\cdot c}+f$$

where $f$ is the focal length, $N$ is the f-number (focal length divided by aperture), and $c$ is the circle of confusion.

Depth of Field

The near and far depth of field limits define the range of acceptable focus:

$$D_{\text{near}}=\frac{s(H-f)}{H+s-2f},D_{\text{far}}=\frac{s(H-f)}{H-s}$$

where $s$ is the focusing distance and $H$ is the hyperfocal distance. The total depth of field is:

$$\text{DoF}=D_{\text{far}}-D_{\text{near}}$$

Event Camera Mode

When configured as an event camera, the sensor detects pixel intensity changes between frames. An event is triggered when the intensity change exceeds the polarity threshold:

$$|\Delta I|>{\tau }_p$$

where $\Delta I$ is the intensity change and ${\tau }_p$ is the polarity threshold (0 to 1). Events are recorded with timestamp, pixel position, and polarity (increase or decrease).

---

Assumptions/Limitations

• The camera can only capture images of simulation objects that have an assigned mesh.
• The sensor must be in operational state to capture; standby or shutdown states prevent image capture.
• Event camera mode cannot generate data on the first frame as it requires a previous frame for comparison.
• The IR camera variant requires thermal properties to be configured on target objects.