Data Storage Message Writer

Description

The Data Storage Message Writer is a behavior that enables periodic logging of simulation messages to a partitioned data storage system. This component attaches to a Partitioned Data Storage and records registered messages at configurable intervals, creating a timestamped archive of simulation data. The stored messages can be retrieved by time, exported to CSV files, or popped sequentially for transmission. This functionality is essential for telemetry recording, science data archival, and store-and-forward communication architectures.

Example Use Cases

Telemetry Logging: Record spacecraft state messages at regular intervals for post-mission analysis.
Science Data Archival: Store payload data messages for later downlink during ground station passes.
Message Buffering: Buffer messages for transmission when communication links become available.
Time-Based Retrieval: Access historical message values closest to specific mission events.

Module Implementation

The Data Storage Message Writer is a Universe Behaviour that attaches to a Partitioned Data Storage component and manages the registration, storage, and retrieval of simulation messages.

Message Registration

Messages must be explicitly registered before they are logged. Registered messages are tracked in an internal dictionary with an active flag:

M = {(m_{i}, a_{i}) : m_{i} \in Message, a_{i} \in {true, false}}

where $a_{i}$ indicates whether message $m_{i}$ is actively being logged.

Write Timing

Messages are written to storage at fixed intervals controlled by the WriteInterval parameter. The write condition is:

write = {true false if t \geq t_{next} otherwise

After each write cycle, the next write time is updated:

t_{next} \leftarrow t + T_{interval}

where $T_{interval}$ is the configured write interval.

Data Format

Each stored message is serialized to JSON with a timestamp wrapper that includes:

Field	Description
Time	Simulation time at which the message was recorded
Data	Serialized message fields

Pointer Management

Each write operation returns a data pointer that references the storage location. Pointers are organized by message, enabling efficient retrieval:

N_{saved} = m \in M \sum ∣ P_{m} ∣

where $P_{m}$ is the set of pointers for message $m$ .

Retrieval Methods

The component provides multiple retrieval mechanisms:

Method	Description
`ReadLatest`	Retrieves the most recently stored value for a message
`ReadClosest`	Finds the stored value closest to a specified time
`PopData`	Removes and returns the oldest stored data across all messages

Time-Based Retrieval

The ReadClosest method searches all stored instances of a message to find the one with timestamp closest to the target:

k^{*} = i ar g min ∣ t_{i} - t_{target} ∣

In case of ties, the earlier message is preferred.

Data Export

The Export method writes all stored data to CSV files organized by message type. Export options include:

Option	Description
`formatTime`	Convert timestamps to human-readable DateTime format using the epoch message
`delete`	Remove data from storage after export

CSV files use a pipe delimiter to avoid conflicts with comma-containing data fields.

Persistence

The component implements save and load functionality to preserve pointer metadata across simulation sessions. The pointer mapping is serialized to JSON format during save operations and restored during load operations.

Assumptions/Limitations

A valid Partitioned Data Storage parent is required for the component to function.
Messages must be registered before the first write interval elapses to be captured.
Deregistering a message stops future logging but does not delete previously stored data.
The ReadClosest method reads all stored instances to find the closest time, which may be slow for large datasets.
CSV export uses a pipe delimiter; data containing pipes may cause parsing issues.
Pointer metadata is stored separately from message data; storage corruption may orphan pointers.
The write interval timer starts at zero; the first write occurs immediately after interval