Camera

SWITCH TO API

Description

The camera payload can be used to capture and save or export images from the simulation environment. Building on Unreal Engine’s camera architecture as a base, Zendir has created the following camera payloads:

• Optical Camera: An RGB optical camera by default that can also become monochromatic.
• Infrared (IR) Camera: A thermal camera designed to visualize the temperatures of any simulation physical components within its field of view.

Both of these cameras can also be configured as an Event Camera which is inspired by neuromorphic camera systems and can generate event data by comparing sequential frames.

---

Example Use Cases

• Optical and Thermal Imaging: Examples of objects that the camera can image include the spacecraft’s orbital body, planets, or other orbital objects such as spacecraft and debris.
• Event Detection: The event camera can be used to detect events within its field of view that can then trigger other systems. For example, the event camera can instruct an optical camera to capture an image.

---

Module Implementation

The camera is a Remote Sensor that captures images at a configurable sample rate. When the sensor is operational and the sample interval has elapsed, the camera sets a ready flag that triggers image capture through the visualization engine. If the sample rate is set to zero or a negative value, the camera captures continuously.

Optical Parameters

The camera models realistic optical behavior through interdependent parameters. Changing one parameter automatically updates related values to maintain physical consistency. The field of view can be constrained between configurable minimum and maximum bounds.

Field of View

The field of view is calculated from the sensor geometry and focal length:

$${\theta }_{\text{FOV}}=2\arctan \left(\frac{d}{2f}\right)$$

where $d$ is the sensor diagonal length and $f$ is the focal length. The diagonal is computed from the resolution and pixel pitch:

$$d=\sqrt{(W\cdot p{)}^2+(H\cdot p{)}^2}$$

where $W$ and $H$ are the horizontal and vertical resolution in pixels, and $p$ is the pixel pitch.

Sensor Size

The physical sensor dimensions are derived from resolution and pixel pitch:

$$S_x=W\cdot p,S_y=H\cdot p$$

A smaller pixel pitch combined with resolution reduces the physical size of the sensor, resulting in a more cropped image.

Aperture and F-Number

The aperture defines the diameter of the lens opening. A larger aperture allows more light to reach the sensor. The f-number (or f-stop) relates the focal length to the aperture diameter:

$$N=\frac{f}{A}$$

where $f$ is the focal length and $A$ is the aperture diameter.

Circle of Confusion

The circle of confusion defines the acceptable level of blur on the sensor. Light from an object converging before or after the sensor plane will be out of focus. If the blur exceeds the circle of confusion, the object lies outside the depth of field planes.

Hyperfocal Distance

The hyperfocal distance is the closest focusing distance at which everything beyond half that distance appears acceptably sharp:

$$H=\frac{f^2}{N\cdot c}+f$$

where $f$ is the focal length, $N$ is the f-number, and $c$ is the circle of confusion.

Depth of Field

The near and far depth of field limits define the range of acceptable focus:

$$D_{\text{near}}=\frac{s(H-f)}{H+s-2f},D_{\text{far}}=\frac{s(H-f)}{H-s}$$

where $s$ is the focusing distance and $H$ is the hyperfocal distance. The total depth of field is:

$$\text{DoF}=D_{\text{far}}-D_{\text{near}}$$

Event Camera Mode

When configured as an event camera, the sensor operates as a neuromorphic camera that detects pixel intensity changes between frames. An event is triggered when the intensity change exceeds the polarity threshold:

$$|\Delta I|>{\tau }_p$$

where $\Delta I$ is the intensity change and ${\tau }_p$ is the polarity threshold (0 to 1). Events are recorded with timestamp, pixel position, and polarity (increase or decrease). The camera tracks both the cumulative event count since enabling event mode and the event count from the most recent capture.

---

Assumptions/Limitations

• The camera can only capture images of simulation objects that have an assigned mesh with Zendir Editor and Zendir Studio.
• The sensor must be in operational state to capture; standby or shutdown states prevent image capture.
• Event camera mode cannot generate data on the first frame as it requires a previous frame for comparison.
• The IR camera variant requires thermal properties to be configured on target objects.