Interfaces

Every custom message and service in lupin_msgs, the typed contracts the mission orchestrator, perception aggregator, digital twin, bridge, and arm servers publish, subscribe, serve, and call across packages, with QoS, the polymorphic Observation envelope, the orientation.w sentinel, the mission-event severity classifier, and the producer/consumer map.

lupin_msgs is the interface-definition package: a single ament_cmake package that holds every custom ROS 2 message and service in the Lupin stack. It ships no nodes and no launch files, only the typed contracts that the mission orchestrator, perception aggregator, digital twin, greenhouse bridge, and arm servers link against. Eleven messages and eleven services are generated over std_msgs, geometry_msgs, sensor_msgs, and builtin_interfaces, and they are the seams where one package's output becomes another's input.

This page is the reference for all of them, and the map of who produces and who consumes each one. Every field is taken verbatim from the .msg / .srv source, then annotated with the QoS it actually ships at, the node that fills it, and the node that reads it. Where the wire contract is subtle, a latched backlog, a polymorphic envelope, a sentinel pose, a severity classifier, the prose explains why it is shaped that way.

The lupin_msgs/package.xml maintainer is still TODO@student.tudelft.nl, the placeholder copied from older packages. The correct contact is o.a.e.devos@student.tudelft.nl; an open-source reader chasing the maintainer should use that.

How the types nest

The message set is deliberately shallow in topic count and deep in composition. One envelope, Observation, carries every world event; a handful of leaf types (SensorReading, FlowerPoint, Polygon) appear on both the perception side and the twin side so the HMI renders the same shapes regardless of which producer emitted them.

The TS layer doubles the surface: every interface here is hand-mirrored in lupin_web/web/src/types/ros.ts as a MSG_TYPES / SRV_TYPES map plus a per-type interface, because roslib.js has no code generation. That mirror is a real second consumer, and a maintenance coupling, when a .msg field changes the TS interface must change with it or the HMI silently reads undefined.

The producer / consumer map

The stated purpose of this page is to say, per interface, who fills it and who reads it. Here is the whole graph; the per-message sections below drill into each edge.

Three edges that surprise readers: DiscoveredTags fans out to both the orchestrator (which counts tags toward the discovery goal) and the twin (which pins them the instant they appear); MissionState is consumed not only by the HMI but by the aggregator, which uses the orchestrator's current_target for temporal co-location of flowers; and the bridge is the only place a real sensor reading enters the graph, every TagReading originates from GetTagReading.

Cross-cutting design patterns

Five conventions recur. Read these first, they explain why the individual definitions look the way they do.

The polymorphic Observation envelope

Everything the mission emits about the world travels in one message type, Observation, rather than a topic per observation kind. A uint8 kind field with KIND_* constants selects at most one of three typed sub-messages (TagReading, FlowerObservation, AnomalyReport); the others are default-zeroed.

The extension contract is concrete. To add a KIND_ANOMALY producer you append a constant, add a typed field, and add a matching branch in lupin_twin. Until that twin branch exists, the twin logs and drops any unmodelled kind rather than crashing, so a new producer is safe to ship ahead of its consumer.

Consumers switch on kind and ignore unknown kinds rather than failing, which is what lets the stack grow without breaking subscribers compiled against an older build.

The latched-snapshot pattern, and the one exception

TwinState, DiscoveredTags, and MissionState ship RELIABLE + TRANSIENT_LOCAL, depth 1 (latched). A late-connecting HMI, or an orchestrator that subscribes only after a mission starts, immediately receives the full current world state with no bootstrap round-trip. Each is a periodic full-table snapshot, not a delta stream, and an empty array is a legitimate startup state, not an error.

Observation is the exception. /floranova/observations ships RELIABLE + TRANSIENT_LOCAL depth 50, not depth 1, a replayable backlog of the mission so far. The depth 1 latched topics hand a late subscriber only the current state; the depth-50 observation feed hands a late twin or web subscriber the whole history, so it can reconstruct every reading that has landed, not just the latest snapshot.

Every HMI subscription is effectively VOLATILE, because roslib.js exposes no QoS API, while several publishers are TRANSIENT_LOCAL. This is benign only because the latched topics also republish at ~1 Hz, so a reloaded HMI fills within a cycle. Anything that needs the current value immediately must pull it through a service call, never lean on web subscription durability. This is the single most important consumer-side gotcha, see docs/CONTRACTS.md #3.

The `orientation.w == 0` "never observed" sentinel

Across Observation.tag_pose_in_map, TwinTagState.pose, and ConfirmTag.tag_pose_in_map, a geometry_msgs/Pose whose orientation.w == 0 means "this pose was never populated", there is no separate validity flag. The twin and HMI treat such a pose as missing and place no map pin.

The subtlety the contract makes load-bearing: this is an invariant the producer actively maintains, not just a convention consumers honour. make_tag_observation (in observations.py) normalizes an all-zero quaternion to identity (w = 1.0) whenever the position is valid, precisely so a valid position still renders a pin. The round-trip is the mechanism, the producer forces w = 1 on real data, therefore the consumer can trust that w == 0 means missing.

Sim vs hardware gates

CalibrateArm and ConfirmTag are hardware-only. In simulation the arm shim has no Hiwonder offset services and refuses to calibrate, and the bridge already returns readings keyed by tag id so visual confirmation is unnecessary. The sensor-bearing messages, by contrast, are simulator-driven. One interface set spans both targets, only the gates differ.

Wire-forward-compatible status codes

Service status enums grow at the end, existing values never change, so a consumer compiled against an older build still decodes the codes it knows. This applies to GetTagReading.status, Observation.status, and the Observation.kind constants alike.

Mission

Observation

Polymorphic envelope for anything the mission emits about the world. Carried on /floranova/observations (RELIABLE + TRANSIENT_LOCAL depth 50). Produced by the orchestrator (node.py, on every SCANNING / UNREACHABLE / SKIPPED outcome) and by the perception aggregator. Consumed by lupin_twin and the web Mission tab.

std_msgs/Header header
string mission_id
string source                    # e.g. "InspectionMission", future: "MappingMission"

uint8 KIND_TAG_READING=0
uint8 KIND_FLOWER=1
uint8 KIND_ANOMALY=2
uint8 kind

uint8 STATUS_OK=0
uint8 STATUS_UNREACHABLE=1
uint8 STATUS_SCAN_FAILED=2
uint8 STATUS_SKIPPED=3
uint8 status
string status_detail             # free-text reason on non-OK; empty on OK

# At most one of these is populated per message, selected by `kind`.
TagReading tag_reading
FlowerObservation flower
AnomalyReport anomaly

geometry_msgs/Pose tag_pose_in_map

tag_pose_in_map is the map-frame pose at observation time, populated only for STATUS_OK. The orchestrator prefers the tag's own map pose (from the discovered-tags feed, which the aggregator resolves via TF) so the operator map pins the tag where it physically is, not where the robot stood. It falls back to the latest AMCL snapshot, the robot's standoff pose offset from the tag, only when the tag's own pose is unknown at scan time. Per the sentinel rule, orientation.w == 0 means neither source had a pose and the value is missing.

This is the corrected behaviour. An earlier draft of these docs claimed the pose came from the AMCL snapshot alone; the code was changed to prefer the tag's own pose (audit commit ae6c926), and the same fix applies to TwinTagState.pose.

The sentinel round-trip, and the AMCL-vs-tag pose preference, both visible in one place:

MissionState

Periodic snapshot of the mission orchestrator, published at a fixed rate (default 5 Hz) whenever the node is alive, fields go empty when no mission runs. Latched (RELIABLE + TRANSIENT_LOCAL depth 1) on /mission/state. Consumed by the web HMI and by the perception aggregator, which reads current_target to co-locate flowers with the tag the orchestrator is actively scanning.

std_msgs/Header header
string mission_id                # empty when no mission active
string mission_type              # "InspectionMission" | "ExplorationMission" | ""
string lifecycle_state
string mission_phase
string current_target            # tag_id being processed, "" otherwise
uint16 targets_total
uint16 targets_completed         # status=OK
uint16 targets_failed            # status=SCAN_FAILED
uint16 targets_unreachable       # status=UNREACHABLE
uint16 targets_skipped           # status=SKIPPED (abort + skip_current)
string last_error                # raw last-event code (battery_low, nav_status_6, ...)
string last_event                # friendly label, "" when no event
uint8  SEVERITY_INFO=0
uint8  SEVERITY_WARN=1
uint8  SEVERITY_ERROR=2
uint8  last_event_severity
uint16 tags_discovered
uint16 discovery_goal
bool   estop_engaged
bool   paused
float32 battery_percentage
bool    battery_low
builtin_interfaces/Time started_at  # zero stamp when no mission has started yet

mission_phase is the sub-state, and the field is split out of a compound internal state at publish time. The orchestrator's live state is a string like INSPECTING_SCANNING; _publish_state strips the lifecycle prefix into lifecycle_state and the remainder into mission_phase. The phases are NAVIGATING, SCANNING, PUBLISHING, LOCALIZING, plus a synthetic CONFIRMING, injected when SCANNING is waiting on an in-flight ConfirmTag call so the operator sees what the orchestrator is actually blocked on. EXPLORING has no sub-machine, so its phase is empty.

The last_event / last_event_severity pair is the fix for the HMI war-story where every notice rendered as a red orchestrator error. last_error is a raw code (battery_low, nav_status_6, dependency_timeout), and a pure classifier, events.py classify_event, maps it to a friendly label and a severity so the HMI shows the right tone.

So battery_low surfaces as a WARN reading "battery low", a transient nav_status_* retry is a WARN, and only a genuine blocking fault (localization_failed, dependency_timeout) escalates to ERROR. The classifier is ROS-free and unit-tested in isolation. battery_percentage / battery_low carry the orchestrator's own battery view, which drives the low-battery dock divert; the HMI also shows the raw /battery BatteryState separately.

StartMission

Request the orchestrator to begin a mission. Called by the web HMI, served by the orchestrator.

string mission_type              # "InspectionMission" | "ExplorationMission"
string[] tag_sequence            # InspectionMission: empty = all tags (tag_locations.json, numeric order)
                                 # ExplorationMission: ignored (targets are discovered)
uint16 discovery_goal            # ExplorationMission only; 0 = use orchestrator param default
---
bool accepted
string mission_id                # populated on accept (mission_id_prefix + uuid hex)
string error_message             # populated on reject; empty on accept

The request is rejected when lifecycle_state is FAULT, when a mission is already running (lifecycle_state in {PREPARE, EXPLORING, INSPECTING, MONITORING, RETURNING}), or when mission_type is unknown to the build. Otherwise the orchestrator transitions READY → PREPARE.

Perception

TagReading

Bundled sensor readings observed at a single greenhouse tag. The payload of an Observation with kind == KIND_TAG_READING, and the response payload of GetTagReading.

string tag_id                              # matches mdp-greenhouse tag IDs
builtin_interfaces/Time stamp              # wall-clock time when the bridge polled the simulator
float64 sim_time_of_day_seconds            # sim-internal time of day in [0, 86400)
SensorReading[] readings                   # subset of the 5 default types; varies per tag

readings is a per-tag subset, not all five sensor types on every tag. The greenhouse defines which sensors each tag exposes, so one tag may carry only temperature and humidity while another carries the full climate set. This is the root of the "only one tag had complete climate data" observation, that is by design, not a dropped reading.

SensorReading

A single sensor measurement at a tag. The element type of TagReading.readings and TwinTagState.readings, and one of the shared leaf types that crosses the perception/twin boundary unchanged.

string name      # e.g. "temperature", "humidity", "co2", "light", "soil_moisture"
float64 value    # value in the sensor's natural units (sim is unit-agnostic)

FlowerObservation

Flower-detection observation, produced by the aggregator when it co-locates a YOLO flower classification with a discovered AprilTag, carried inside an Observation with kind == KIND_FLOWER.

string tag_id                              # the co-located AprilTag; twin keys per-tag state by this id
geometry_msgs/PoseStamped pose             # map-frame pose of the co-located tag (frame_id == "map")
string species                             # "tulip_red" | "tulip_white" | "tulip_pink"; "" if none
float32 confidence                         # classification confidence [0, 1] for `species`
bool anomaly                               # true when the YOLO "bug" class co-located with this tag
FlowerPoint[] flowers                      # per-bloom dots scattered inside the planter box (see FlowerPoint)
geometry_msgs/Polygon box_footprint        # tag-anchored planter-box rectangle, map frame

The anomaly bool rides the flower path (not AnomalyReport) because it is keyed by the same tag_id the twin merges on; standalone anomalies still use KIND_ANOMALY. pose is the tag anchor; the per-bloom positions live in flowers.

FlowerPoint

One localized bloom inside a planter box. Carried as the repeated flowers field on both FlowerObservation and TwinTagState so the HMI scatters species-coloured dots inside the box rectangle instead of stacking every bloom on the tag pin.

geometry_msgs/Point position   # map frame (z is 0 in v1, 2-D placement)
string species                 # "tulip_red" | "tulip_white" | "tulip_pink"; "" if only the "bug" class
float32 confidence             # [0, 1] for `species`
bool anomaly                   # the "bug" class co-located in this column

The producer mechanism is worth stating plainly, because the inputs explain the v1 limitations. box_geometry.py turns a tag's map TF into a rigid box frame (box_from_tag: the tag's local +Z projected onto map XY is the box normal, rotated 90 degrees is the lateral axis), then bin_detections bins each YOLO detection into one of seven lateral columns across the box and places one bloom per occupied column. The column index comes from a lateral_fraction fusing the arm's shoulder_pan angle with the detection's normalized image-x.

Depth is not measured. The gripper camera is uncalibrated (intrinsics K = 0, no depth stream), so position.z = 0 and the per-bloom depth across the box is a deterministic golden-ratio jitter, not a range reading. The lateral cue is a real bearing; the depth is cosmetic scatter so blooms do not draw as a straight line. This is the temporal-co-location + lateral-cue v1 limitation, the dots are placed, not triangulated.

AnomalyReport

Future anomaly observation (e.g. fallen plant, unexpected obstruction). Stubbed to lock the contract; nothing emits these yet. Payload of an Observation with kind == KIND_ANOMALY.

geometry_msgs/PoseStamped pose
string description
uint8 severity                   # 0=info, increasing → more urgent

This is deliberate dead wiring, documented in docs/CONTRACTS.md #1. Pest detection today rides the flower path's anomaly boolean during SCANNING; standalone anomalies have no producer. To wire one up later: emit Observation(kind=KIND_ANOMALY, anomaly=AnomalyReport(...)) and add a KIND_ANOMALY branch in lupin_twin, which currently logs-and-drops the unmodelled kind.

DiscoveredTag

One AprilTag discovered live by perception during an ExplorationMission, produced by the aggregator, which projects the detector's per-tag TF into the map frame and fuses the YOLO flower classifier. Unlike tag_locations.json (a-priori), these are found at runtime.

string tag_id                    # matches the detector + greenhouse-bridge id space
geometry_msgs/Pose pose_in_map   # tag's actual location (solvePnP + TF lookup)
string species                   # YOLO dominant flower class, or "" if none yet
float32 species_confidence       # [0, 1]; 0.0 when species is empty
bool anomaly                     # true when the YOLO "bug" class was seen at this tag
uint32 sightings                 # confident detections fused into this entry

sightings lets a consumer require >= K confident detections before trusting the pose, debouncing against a single noisy frame.

species, species_confidence, and anomaly are producer-set but consumer-unread for logic (docs/CONTRACTS.md #2). The aggregator fills them on every tag, but no mission decision routes or filters on them, they exist only to be mirrored into TwinTagState for HMI display. A reader hunting for "where does species change the mission" will find nothing, by design.

DiscoveredTags

Latched snapshot of every AprilTag discovered so far during exploration, published by the aggregator on /perception/discovered_tags (RELIABLE + TRANSIENT_LOCAL depth 1). Consumed by both the orchestrator, which counts distinct tags toward the discovery goal and reads pose_in_map for the monitoring approach poses, and the twin, which pins each tag the instant it appears.

std_msgs/Header header
DiscoveredTag[] tags

GetTagReading

Request a one-shot sensor reading for the specified greenhouse tag. Served by lupin_greenhouse_bridge at the private name ~/get_tag_reading, called by the orchestrator during SCANNING. This is the single entry point for real sensor data into the whole graph.

string tag_id
---
uint8 STATUS_OK=0
uint8 STATUS_UNKNOWN_TAG=1
uint8 status
string error_message    # empty when status == STATUS_OK
TagReading reading      # populated when status == STATUS_OK; default-zeroed otherwise

ConfirmTag

Visually confirm a specific AprilTag is in frame from the robot's current pose. Served by the perception aggregator, called by the orchestrator at the start of SCANNING, before the bridge call, when require_visual_confirmation is true.

Hardware-only. Not implemented in sim, the simulator's bridge already returns readings keyed by tag id, so visual confirmation is a hardware gate. When the gate is enabled the orchestrator surfaces this call as the synthetic CONFIRMING phase in MissionState.

string expected_tag_id
---
bool detected
float64 detection_confidence    # [0, 1]; implementation-specific scale
geometry_msgs/Pose tag_pose_in_map  # zero when detected=false
string error_message            # empty when detected=true

When the expected tag is never detected, the implementation sets detected=false and populates error_message; the orchestrator surfaces that string in MissionState.last_error and the HMI mission log. tag_pose_in_map is a future hook for visual servoing, the current orchestrator only inspects detected.

Twin

TwinState

Periodic snapshot of every tag the digital-twin node has observed so far, published at ~1 Hz on /twin/state (RELIABLE + TRANSIENT_LOCAL depth 1), consumed by the HMI. Empty tags is the legitimate startup state.

std_msgs/Header header
TwinTagState[] tags

TwinTagState

Snapshot of one greenhouse tag from the twin's perspective, emitted as a repeated field inside TwinState.

string tag_id
geometry_msgs/Pose pose                    # snapshotted on first scan; orientation.w == 0 = "never observed"
SensorReading[] readings                   # latest reading per sensor type; index by name
builtin_interfaces/Time last_observed      # absolute ROS time of most recent observation
float32 stale_seconds                      # (now - last_observed) at publish time
string species                             # YOLO dominant class; "" until classified
float32 species_confidence                 # [0, 1]; 0.0 when species is empty
bool anomaly                               # YOLO "bug" class seen at this tag (latest-wins)
FlowerPoint[] flowers                      # per-bloom dots scattered inside the planter box (see FlowerPoint)
geometry_msgs/Polygon box_footprint        # tag-anchored planter-box rectangle, map frame

The twin caches pose from the first OK observation, sourcing the tag's own map pose first, the AMCL standoff pose as fallback, exactly as Observation.tag_pose_in_map does, and does not re-stamp it, so the HMI marker does not jitter on AMCL drift. last_observed is the durable freshness field that persists into FloraNova exports; stale_seconds is a publish-time convenience for the HMI's staleness fade, recomputed only at the 1 Hz tick.

GetField

On-demand IDW (inverse-distance-weighted) interpolation of a sensor field over a 2D map-frame bbox. Served by lupin_twin at /twin/get_field, called by the HMI to paint the heatmap. Computed lazily from per-tag observation buffers and cached by (sensor_type, observation_count, bbox, resolution).

string sensor_type            # "temperature" | "humidity" | "co2" | "light" | "soil_moisture"
float32 resolution            # cell size, metres
float32 bbox_min_x            # map-frame
float32 bbox_min_y
float32 bbox_max_x
float32 bbox_max_y
---
bool ok                       # false on bad sensor_type / degenerate bbox
string error_message
float32[] values              # row-major, width*height; NaN for "no data"
uint32 width                  # cells along x
uint32 height                 # cells along y
float32 origin_x              # map-frame x of cell (0, 0)
float32 origin_y              # map-frame y of cell (0, 0)
float32 resolution_used       # echoed back so the consumer doesn't re-derive it
float32 value_min             # over the non-NaN cells; both 0 if all NaN
float32 value_max
uint32 sample_count           # how many observations contributed

observation_count is in the cache key because it is the cheap invalidation signal, a new observation bumps the count, which busts the cache without the twin diffing observation buffers cell by cell, so repeat calls with no new data return the cached grid for free. Cells beyond max_distance_to_nearest_tag (default 2.0 m) from any observed tag, or in unexplored map area when the twin clips to /map, are encoded as NaN, and the HMI renders NaN as transparent, so the field paints only where data exists.

Arm

SetArmPreset

Move the 4-DOF arm to a named preset pose. The request carries only the name.

string name
---
bool success
string message

The same .srv type is served by two different nodes, with different state ownership:

/lupin/arm/preset, served by arm_preset_server running onboard, drives the built-in hard-coded presets (travel, inspect, home) on /mirte_master_arm_controller/joint_trajectory. This is the path the orchestrator's arm patrol calls.
/lupin/arm/library/goto_pose, served by arm_library_server running laptop-side, replays an operator's saved poses out of ~/.config/lupin/arm_library.json.

The SetArmPreset.srv source comment says to drive the gripper via /io/servo/hiwonder/gripper/set_angle_with_speed. That in-tree comment is outdated, the raw Hiwonder service fights the GripperActionController on hardware. Command the jaw through the bridge at /lupin/gripper/set_angle_with_speed instead, see the gripper command path.

CalibrateArm

Multi-step Hiwonder zero-offset calibration for the arm + gripper servos, driven as a small start / commit / cancel / status FSM.

Hardware-only. In sim, arm_sim_shim has no Hiwonder offset services and the server refuses to start the procedure.

string action
---
bool success
string state                # "IDLE" | "AWAITING_POSE"
string message
string[] joint_names
int32[] offsets_applied     # centidegrees actually written via _set_offset
int32[] diffs_observed      # raw-tick diff (position - home + curr_offset)

Arm pose & sequence library

Five services back the Arm tab's Pose Library and Sequence Recorder cards (and the voice agent's library tools). They are served by arm_library_server, which runs laptop-side and owns ~/.config/lupin/arm_library.json, operator state deliberately kept off the robot. The built-in presets stay in the onboard arm_preset_server.

GetArmLibrary

List saved poses and sequences, names + light metadata only, never the waypoint data.

---
bool success
string json     # {"poses":[{name,note,has_gripper}], "sequences":[{name,mode,duration_s,n_waypoints,include_gripper}]}

SaveArmPose

Save a named arm pose. from_current=true snapshots the latest /joint_states (the UI/voice default); otherwise arm_rad (length 4, ARM_JOINTS order) and, when has_gripper, gripper_rad are used. Radians throughout.

string name
bool from_current
float64[] arm_rad
float64 gripper_rad
bool has_gripper
bool overwrite
---
bool success
string message

ArmRecord

Control a sequence recording. action is start | save | cancel; mode (start only) selects kinesthetic (torque OFF, hand-pose the arm) or teleop (torque ON).

string action
string name
string mode
bool include_gripper
bool overwrite
---
bool success
string message
float64 duration_s
int32 n_waypoints

PlayArmSequence

Replay a saved sequence. speed scales playback time, clamped to [0.25, 2.0].

string name
float64 speed
---
bool success
string message

ArmLibraryEdit

Delete or rename a library entry. kind is pose | sequence; an empty new_name deletes, a non-empty one renames.

string kind
string name
string new_name
---
bool success
string message

action values and their effects

Mission orchestrator

Perception

Digital twin

Web HMI

On this page