Shippy answered live to a query about Ghana's Exclusive Economic Zone and showed exactly how it worked: the source of the boundaries, the data slicing, the timestamp of the query, and a deep link to Skylight so an analyst can verify every number. Why does this matter to you? Because in high-risk operational domains like ocean protection, trust and verifiability are the priority — not the model's magic.
What Shippy is: soul, skills and config
Shippy is designed as three clear layers: soul, skills and config.
-
The soul is the system prompt that defines the agent's personality and operational limits. There you set, for example, that Shippy won't make legal determinations or speculate beyond what the data supports.
-
The skills are concrete modules that follow the same agent-skills spec used by open-source tools: markdown files with structured frontmatter. Each skill is versioned and human-readable. Among Shippy's skills are:
- Queries to the Skylight API for Events (e.g., fishing activity, transshipment).
- Resolving EEZ and MPA boundaries.
- Interpreting vessel tracks and classifying activity.
- Generating deep links to the Skylight map to see the exact context.
-
The config covers the rest: which agent harness to use (Shippy uses
OpenClaw), which LLM to run (todayClaude Opus 4.6), and runtime settings. Secrets like API keys are injected at runtime. Swapping a model or harness is a config change, not a rebuild.
A key decision: bundle the soul and the skills into a versioned
Dockerimage. That makes exactly "what Shippy is" reproducible for a given version.
Practical architecture: typed API, deterministic CLI, and why that reduces mistakes
The biggest problem wasn't the LLM: it was making the agent predictable when talking to a complex API. Skylight has nested inputs, pagination, geometries and filters. Letting the model build HTTP calls produced subtle bugs: malformed pagination that silenced results, mis-encoded geometries, and misunderstood filters.
The fix was a simple, robust middle layer:
- A typed API with resources like Events, vessels, regions, imagery, tracks, and common operations
searchandaggregatewith field-by-field schemas and descriptions. - A CLI (command-line interface) that collapses complexity into typed flags: for example
skylight events search --area POLYGON --from DATE --to DATE. The CLI handles authentication, pagination and structured output. - The skills call the CLI instead of building raw requests. That turns the model's uncertainty into predictable, testable commands.
Small operational detail: the CLI always writes output to a local JSON file instead of piping through the shell. This avoids buffer limits and failures with downstream tools like jq.
Security, isolation and Mothership
Each user talks to Shippy inside an ephemeral, isolated session. The platform that makes it possible is called Mothership: an agent hosting that provisions a dedicated Kubernetes deployment per session.
- When you open a conversation, pods are created that package the agent runtime, its skills and the Skylight CLI.
- The user's JWT is injected when provisioning the session, so all API calls run with the user's data scope.
- Files the agent generates exist only in that session and are never shared between users.
- The sandbox allows running code, installing dependencies and doing multi-step analysis, but with network restrictions that limit access to only necessary services.
This lets, say, a fisheries office in the Philippines see only their Areas of Interest and watchlists, with no risk of context leaking between clients.
Practical evaluation: Harbor, expert scenarios and the LLM judge
Traditional benchmarks fail for agents that act on live data. Shippy needed an evaluation that measured how it selects tools, queries live data and knows when to stop.
So they built an evaluation pipeline with these parts:
- Scenarios and rubrics written by domain experts. Each task has criteria and weights (e.g., for event queries data accuracy weighs more than response style).
- A sandbox that runs the prompt in a real Shippy session.
- A judge LLM that scores each criterion from 0 to 1, with written reasoning, and computes a weighted aggregate to pass or fail.
- Harbor, the evaluation framework, launches a real session of the tagged version of Shippy and produces timestamped results and comparisons with previous runs.
Operational result: if a version regresses on key criteria, it doesn't get deployed to users.
Real mistakes and lessons learned
Working with a system like this isn't theoretical. Skylight logs concrete problems and how they were fixed:
- Letting the agent build API calls produced subtle bugs. Fix: wrap the API with a deterministic CLI.
- Geometry-sensitive queries failed because boundaries were oversimplified. Fix: resolve polygons with the regions API instead of hardcoding coordinates.
- The agent sometimes overstepped by offering tactical recommendations; system prompts must explicitly state limits (support decision-making, not patrol tactics).
- There was a case where Shippy "invented" a CLI command that didn't exist. Lesson: skills must reference verified commands and the CLI must include extensive help and detailed errors.
Practical takeaways for your own agent:
- Version the
Dockerimage that contains prompt + skills. That makes reproducibility easier. - Define APIs with schemas and field descriptions; avoid letting the model craft free-form strings for complex calls.
- Implement a CLI or SDK that encapsulates authentication, pagination and formatting.
- Isolate each session, inject credentials at runtime and delete artifacts when finished.
- Evaluate end-to-end with experts and use an automated judge that explains failures.
What's next for Shippy
Shippy is already in early adoption and the visible roadmap includes:
- Agent-driven UI control: moving from map links to the agent manipulating filters, region and time range in the Skylight map.
- Model routing: delegate simple searches to lightweight models and reserve frontier models for complex investigations.
- Cross-thread memory: persist relevant analyst facts (jurisdiction, preferred sources) so you don't have to re-specify context in every thread.
These improvements aim to reduce latency, cost and friction in real workflows.
Shippy's work isn't just a maritime use case. Mothership and the practices developed already influence other Ai2 projects like EarthRanger and OlmoEarth. The central lesson is clear: in environments where lives, resources and reputations are at stake, engineering around the agent matters as much or more than the model.