Shippy is a maritime agent designed for decisions where a mistake has real consequences. What did we learn when we put AI into a real-time operation with satellite data and vessels changing every minute? Here I'll explain the architecture, the challenges, and the technical lessons Ai2 brings to other environmental platforms.
Anatomy of the agent: soul, skills and config
We think of an agent as three things: soul, skills and config.
- The soul is the
system promptthat defines Shippy's persona and its behavioral limits. It's explicit, auditable and easy to review when you need to adjust restrictions. - The skills are specific instructions packaged as Markdown files with frontmatter (the same spec code tools like Claude Code and Codex use). Each skill is versionable and human-readable: for example, the skill that queries the Skylight API solves a request like "show fishing activity in Panama's EEZ last month" by translating "Panama EEZ" to a polygon via the regions API, querying fishing events in that geometry, and returning deep links to the map.
- The config includes the agent harness (
OpenClaw), the model (Claude Opus 4.6) and execution parameters. The soul and the skills are packed into a versioned Docker image; changing the model or the harness is a configuration change, not a rebuild.
This separation lets you audit policy (soul), iterate skills, and swap infra without breaking what an analyst expects.
Deterministic tools for a non-deterministic agent
Models are inherently non-deterministic. So how do you control the parts you can? With deterministic tools.
Shippy doesn't make raw API calls. It uses a purpose-built CLI that exposes simple commands like skylight events search with typed flags. The CLI handles authentication, pagination and structured output. Result: fewer subtle bugs (malformed pagination, geometry encoding, misinterpreted filters). Also, the CLI always writes output to a local JSON file, avoiding pipe buffer limits and making it easier for downstream steps to process results.
Under the CLI there's a typed API: resources like Events, vessels, regions and tracks, with search and aggregate operations. Those signatures with field-by-field descriptions enable automated tests and documentation the agent can consult.
The pattern is clear: typed API -> deterministic CLI -> skills that call the CLI. Each layer reduces the error surface of the next.
Isolation and hosting: Mothership
Skylight serves hundreds of agencies and NGOs; one user's data must not leak. For this they built Mothership, a platform that provisions an ephemeral Kubernetes deployment per user session.
When you open a conversation, the system spins up pods with the agent runtime, its skills and the CLI. The user's JWT is injected at provision time, so calls return data scoped correctly. Everything the agent writes during the session is isolated and destroyed when the session ends.
Inside the sandbox the agent can write and run code, install dependencies and process datasets, but the network is restricted to only what's necessary. This allows secure multi-step workflows without exposing information between users.
Evaluating an agent, not just a model
Evaluation was designed for the full system too. Shippy is evaluated with live data: scenarios and rubrics created by domain experts, weighting criteria according to the task (for example, a fishing events query prioritizes data accuracy and boundary resolution).
The pipeline: a prompt runs in the sandbox, a judge LLM scores each criterion from 0 to 1 explaining why, and a weighted aggregate is compared against an approval threshold. They run this via Harbor, with a plugin that spins up a real Shippy session against the candidate version. Every change in skills, model or data triggers the evaluation; if there's regression, the version never reaches users.
The results revealed useful patterns: Shippy reliably retrieves data and respects guardrails (for example, it rejects military intelligence queries), but it failed in tactical planning tasks (it tended to give operational recommendations), in geometry-sensitive queries due to simplifications, and once invented a non-existent CLI command. Those failures fed targeted fixes in the skills and the CLI.
Practical lessons and technical recommendations
If you're going to build an agent for a critical domain, note what actually works:
- Define explicit limits in the
system prompt. Don't let the model infer prohibitions through opaque fine-tuning. - Encapsulate complex APIs behind a deterministic layer (CLI or typed SDK). Reduce the model's error surface.
- Use typed schemas and field-by-field documentation so both humans and agents have clear expectations about inputs and outputs.
- Version skills as independent artifacts (Markdown with frontmatter) for traceability and simple deployment.
- Isolate each user session with ephemeral environments (Kubernetes pods) and inject credentials at provision time.
- Evaluate the whole agent against real data with weighted rubrics and expert judgment. Don't stop at static benchmarks.
- Design for model routing: send simple queries to small models and reserve the big model for complex research.
- Consider selective memory to persist analyst-relevant facts (jurisdiction, preferred sources) without exposing privacy.
Where they're going and why it matters
Ai2 opens Shippy to early adopters so they can break schemas and find questions the agent answers poorly. Next pieces: UI control by the agent (manipulating filters and time range on the map), model routing for cost/latency, and cross-thread memory to persist useful context.
Mothership is designed to be general. What works in maritime already influences EarthRanger and OlmoEarth. This isn't just a technical story; it's a practical way to bring trustworthy agents into real operations that protect ecosystems and the people working in the field.
Shippy isn't the finish line—it's a living lab: a mix of careful prompts, infrastructure engineering and expert evaluation that turns AI promises into useful, verifiable tools.