Digital twin

The digital twin is the layer that turns Lupin's stream of per-tag observations into a single queryable model of the greenhouse. It is one rclpy node, lupin_twin, sitting over two pure-Python modules, a per-tag observation store (lupin_twin/state.py) and an inverse-distance-weighted field reconstruction (lupin_twin/idw.py), and it exposes exactly two read-only contracts to everything downstream.

This page is the reference for those two contracts and the map of how observations flow through the store, how the IDW heatmap stays honest, and why the HMI is a pure consumer that never writes back.

Inputs, internals, and pure-consumer outputs

The twin has two inputs, not one, and that detail is load-bearing for the "map fills in as the robot explores" feel. The orchestrator feeds it /floranova/observations (the scan path, sensor readings plus flower classification), and the perception aggregator feeds it /perception/discovered_tags, which pins a tag's map pose the instant exploration finds it, before any bridge scan has produced a reading. The first stream carries data, the second carries position, and the store reconciles them.

Bridge-agnostic by design

Neither input cares where its data was born. Whether the mission orchestrator filled /floranova/observations from the mdp-greenhouse sensor simulator or from real perception plus on-board sensor modules is invisible here, and whether /perception/discovered_tags came from a simulated AprilTag or a live apriltag_ros detection is equally invisible.

That is the whole point, the same node code runs in simulation and on hardware with no change. Swapping the greenhouse bridge is a bridge-implementation concern, never a twin-contract change. The twin has no dependency on Nav2, Gazebo, or the perception internals; it starts immediately and waits for observations to arrive.

The concurrency model, single-threaded on purpose

The node runs on a SingleThreadedExecutor with one MutuallyExclusiveCallbackGroup shared by all four callbacks, the observations subscription, the discovered-tags subscription, the /twin/get_field service, and the publish timer. Because rclpy serialises every callback in that group, only one of them ever touches the store dict or the cache OrderedDict at a time, so neither needs a lock. The store is thread-unsafe by design, and the design is correct precisely because nothing concurrent can reach it.

The code still pays a tiny premium to keep a future MultiThreadedExecutor cheap to adopt. Each cached field is a frozen _CachedField dataclass, and _fill_field_response copies its fields into a fresh GetField.Response on every call rather than handing back the rclpy Response object the executor passed in. Caching the Response would alias one allocation across calls, fine under the current executor and a footgun the moment anyone goes multi-threaded, so the fresh-copy costs about one tuple-unpack and removes the hazard ahead of time.

ROS interfaces

Observation ingestion routing

_on_observation and _on_discovered_tags are the two front doors, and each one drops anything it cannot place rather than guessing. The decision tree below is the actual branch order in node.py.

A few of these branches are deliberate product decisions, not edge-case defensiveness:

• Non-OK observations are still recorded, just unpinned. A STATUS_UNREACHABLE or scan-failure observation has has_pose false, so the store keeps the tag in the table with no map pin. The operator wants to see that a tag was attempted and failed, so the twin lists it rather than hiding it.
• A wrong frame_id is dropped loudly, not silently re-projected. An orchestrator or bridge accidentally publishing in /odom would otherwise pin tags at meaningless map coordinates, the kind of bug that costs an afternoon, so the twin warns once and drops the message.
• KIND_ANOMALY and any unmodelled kind are dropped with a throttled warning. The pest flag rides the flower path's anomaly boolean instead, so a future producer that wires KIND_ANOMALY upstream sees why its observations vanish rather than debugging a silent dead end.

Who owns what, the discovery and scan handshake

The two inputs race for the same tag, and the store arbitrates by first-write-wins. _on_discovered_tags skips any tag that has_pose() already, so once the scan path has pinned a tag the discovery feed never touches it; conversely, if discovery lands first, it pins the position pose-only and the later KIND_TAG_READING scan fills the sensor readings without re-stamping the pose. The scan path owns readings and species, the discovery path owns nothing but the first position, and record() keeps whichever pose arrived first.

Contract 1, the latched /twin/state snapshot

_publish_state walks every tag in the store and emits a TwinState (a std_msgs/Header plus a TwinTagState[] tags array). One TwinTagState per observed tag carries:

• string tag_id, the detector-space tag identifier.
• geometry_msgs/Pose pose, the map-frame tag pose. orientation.w == 0 means never observed, and the HMI places no pin in that case.
• SensorReading[] readings, the latest reading per sensor name/value.
• builtin_interfaces/Time last_observed, the durable absolute timestamp of the most recent observation, the field off-line consumers should treat as the source of truth.
• float32 stale_seconds, a publish-time (now - last_observed) convenience for the HMI's staleness fade.
• string species, float32 species_confidence, bool anomaly, flower classification co-located with the tag (see below).

Because the snapshot is latched at depth 1, a freshly-connected HMI sees the current world immediately, with no bootstrap round-trip.

Contract 2, the on-demand /twin/get_field IDW service

GetField reconstructs a sensor heatmap over a map-frame 2D bounding box. The request carries sensor_type (one of temperature, humidity, co2, light, soil_moisture), a resolution (cell size in metres), and bbox_min_x / bbox_min_y / bbox_max_x / bbox_max_y. The response carries ok, error_message, a row-major float32[] values grid (width x height, NaN for cells with no nearby data), width, height, origin_x, origin_y, resolution_used, value_min / value_max (over non-NaN cells only, both 0 if all NaN), and sample_count.

_handle_get_field rejects an empty sensor_type, a non-positive resolution, and an inverted bbox. On any of those it returns the failure contract the HMI must branch on, ok = false with a populated error_message and every numeric field zeroed, values empty, width/height/sample_count 0. The HMI keys off ok and never reads a garbage grid.

The interpolation

Each grid cell c evaluates to:

field(c) = sum_i (w_i * v_i) / sum_i w_i        w_i = 1 / d_i ^ power

where the sum runs over observed tags within idw_falloff_radius_m of the cell, and d_i is the Euclidean distance from the cell centre to tag i. power defaults to 2.0, the textbook IDW exponent, giving a soft, visually pleasant gradient. Lower is too smooth, 4+ becomes Voronoi-like, and the operator-tunable range that stays sane in practice is [1, 4].

The per-cell decision, three guards in order

The three correctness guards are not a flat list, they are a single branch order, and the order is the mechanism. The nearest-sample pass runs first and does double duty, it gates the honesty test and captures the value the fallback branches reuse.

1. Honesty gate (NaN beyond max distance). If the nearest tag to a cell is farther than idw_max_distance_m (default 1.5 m), the cell is left NaN. The HMI renders NaN as transparent, so unexplored greenhouse keeps the colour of the SLAM map underneath, Lupin paints colour only where the data supports it.

2. Coincident-cell pin (1 mm). Cells closer than COINCIDENT_EPS_M (1e-3 m) to a tag are pinned to that tag's exact value rather than dividing by a near-zero distance in the weighted sum. Operators expect the heatmap at a tag location to read exactly that tag's reading.

3. The fallback, and why the donut. When a cell passes the honesty gate but no sample falls within idw_falloff_radius_m, the weighted-sum denominator is 0 and the cell falls back to nearest_value instead of NaN. That fallback is the donut-causer, an isolated tag whose falloff radius is smaller than its max distance grows a hard constant ring of nearest-value cells in the gap between the two radii. The defaults set the two radii equal (both 1.5 m) precisely so that gap never exists, so the per-cell outcome is exactly one of three, NaN, pinned-coincident, or weighted-sum, with the nearest-value ring designed out.

lupin_twin:
  ros__parameters:
    idw_power: 2.0
    idw_falloff_radius_m: 1.5
    idw_max_distance_m: 1.5

Performance, why no KD-tree

compute_idw_field is O(cells x samples) with two passes per cell, the nearest-finder and then the weighted sum. For a 200x200 grid (40k cells) at the <= 30-sample ceiling that is roughly 2.4 million distance evaluations in pure Python, which lands inside the sub-100 ms budget. A KD-tree would shave the nearest-finder, but at 30 samples the constant-factor and build cost dwarf the win, so the code stays flat-array simple and the comment at the nearest-finder says exactly that.

The explored_mask, shipped but dormant

compute_idw_field fully supports an explored_mask, a row-major bool array that forces unexplored cells to NaN regardless of sample distance, and it is unit-tested. The node passes explored_mask=None today, so the honesty gate's second intended layer, clipping the heatmap to the SLAM-explored area off /map, is a tested, ready hook rather than live behaviour. It is a clean follow-up, not a missing feature.

Edge semantics the consumer must handle

A few degenerate cases are part of the contract and are unit-tested. An empty sample set returns an all-NaN grid with value_min = value_max = 0 and sample_count = 0, which is the legitimate "nothing observed yet" response, not an error. grid_dimensions clamps any degenerate bbox to at least 1x1 so a zero-area request still returns a well-formed tiny grid. Consumers branch on sample_count and ok, never on grid emptiness alone.

Free repeat queries, the LRU cache

Field results are memoised in a bounded OrderedDict LRU (cap 64, _FIELD_CACHE_CAP in node.py) keyed by (sensor, observation_count, resolution, bbox). The store's observation_count is a single monotonically-increasing integer bumped on every accepted record() or record_flower(), and because it is part of the key, cache invalidation is one integer changing, any new observation produces a fresh key, the next request misses, and the field recomputes naturally. There is no eager clear, no TTL, and no scan of stale entries.

That is strictly better than the naive "clear the cache on every write" approach, because writes do zero cache work, correctness comes for free from the key, and staleness is bounded to never. The one subtlety the cap guards against is a misbehaving consumer, an HMI bug that calls GetField with a sliding bbox every frame would balloon the keyspace with distinct keys, which is exactly why the 64-entry LRU exists.

How observations land in the store

Pose stability over AMCL drift

Each tag's map pose is cached from the first write that supplies one, and never re-stamped on later visits. Tags do not move, so re-stamping each visit would only inject AMCL localisation jitter into the HMI's tag marker. This is first-write-wins at the store level, and the discovery feed competes with the scan feed for that first write, whoever lands first pins the position. It ties directly to the presence-only AMCL gate the mission runs during EXPLORING and MONITORING, the twin deliberately trades a little staleness for a stable marker rather than a jittering one.

NaN/inf filtering at the store boundary

samples_for_sensor() returns (x, y, value) tuples only for tags that have both a pose and a finite reading for the requested sensor. Non-finite values, a sensor stub returning float('nan') or a divide-by-zero in a future bridge, are filtered out here, before the IDW math ever sees them. This matters because a single NaN in a weighted-sum numerator would poison every cell within the falloff radius and never recover, so the filter lives in the store, not the renderer, and is covered by a unit test.

Flower classification (KIND_FLOWER)

The perception aggregator co-locates the YOLO tulip class with each AprilTag and publishes a KIND_FLOWER observation on the same /floranova/observations topic. _ingest_flower merges species, species_confidence, and the anomaly (the bug pest) flag onto the tag's buffer, keyed by flower.tag_id, with latest-wins semantics, so a clean re-scan clears a prior pest flag. If neither the sensor-reading path nor the discovery feed has pinned the tag, the flower's own pose pins it.

The HMI is a pure consumer

The web HMI reads the two contracts and writes nothing back. lib/twin.ts subscribes the latched TwinState, lib/heatmap.ts renders a GetField grid, and MapCanvas.tsx draws the box rectangle, the per-species bloom dots, and the diamond tag marker on top of the SLAM map. Two consumer behaviours are concrete enough to surface here:

• The opacity ramp. heatmap.ts maps each cell from fully transparent at value_min to about 60% opacity at value_max, so a cold cell barely tints the floor plan and a hot one reads clearly while the SLAM map stays legible underneath. NaN pixels are left untouched, which is what makes the honesty gate visible to the operator.
• The row flip. ROS grids are bottom-left origin and the HTML canvas is top-left, so heatmap.ts writes destination row height - 1 - j for source row j. Get this wrong and the heatmap renders mirrored against the map, which is exactly the sort of silent geometry bug that looks "almost right".

Health thresholds live HMI-side, on purpose

The twin stays a pure data plane. It carries species / species_confidence / anomaly out on each TwinTagState, but it never decides whether a reading is healthy. There is a reference copy of the ideal ranges in the twin package, lupin_twin/lupin_twin/config/ideal_ranges.yaml, yet the node never reads it, a grep finds zero references in node.py, state.py, or idw.py. The real consumer is the HMI's tulip-health.ts, which hardcodes the same {min, max} thresholds client-side.

Running it standalone

twin.launch.py brings up the node alone (executable twin_node, node name lupin_twin). It assumes the orchestrator is already publishing observations; normally it is composed from a sim or hardware bring-up.

ros2 launch lupin_twin twin.launch.py use_sim_time:=true
ros2 service call /twin/get_field lupin_msgs/srv/GetField \
  "{sensor_type: temperature, resolution: 0.1, bbox_min_x: 0.0, bbox_min_y: 0.0, bbox_max_x: 5.0, bbox_max_y: 5.0}"

ros2 launch lupin_twin twin.launch.py frame_id:=map
ros2 topic echo /twin/state --qos-durability transient_local   # latched; VOLATILE shows nothing

See also