Working with AI agents
Reports are plain code in a git repository, which is exactly the shape a coding assistant works well in: it can read what exists, write something new, and open a pull request you review. This page covers how to set that up so the first draft is usually right.
Write AGENTS.md first
Most bad output comes from missing context rather than a weak model. AGENTS.md
is where you put what a new colleague would need on day one:
# Working in this repository
## Data
- The warehouse is authoritative. Never query the replica.
- Revenue is defined in context/metrics.md. Do not derive it another way.
- `orders_legacy` is deprecated — use `orders`.
## Reports
- Keep database SQL in `queries.py`; use source-specific Python readers where
needed, and keep transformations and report composition in the generator.
- One report per directory under reports/.
- Every report declares its data sources in report.yaml.
## Review
- Open a pull request. Nothing is merged unstested.
- If a metric definition is unclear, ask rather than guess.
Be specific and be blunt. "Do not guess a definition — ask" prevents more bad reports than any amount of prompting.
Refine a local preview
When you ask your coding agent to create or update a report, Trellum's project guidance tells it to build, validate and open the report for browser review automatically. You do not need a separate launch prompt or a personal agent rule. The agent should provide a clickable link and keep listening for feedback.
In review mode you can select a chart or table, or send a general change request. The agent edits the report source in your repository, rebuilds, and the preview reloads. See Refine reports in your browser for the review controls and command loop. Ask explicitly for build-only work if you do not want an interactive preview.
This local source-editing loop is separate from the portal assistant. The assistant answers questions about portal reports and never edits your Git repository. Viewers can ask questions and retrieve reports and data they are allowed to access. Creating or editing reports requires a developer or administrator with repository access; viewers cannot start builds or publish, even with a write-scoped API key.
Give it real data access
For database-backed reports, an assistant that can check a column writes SQL that runs. One that cannot will invent column names that look plausible and fail at build time — or worse, silently return the wrong grain. Other sources can be read with their built-in reader or a project-specific Python adapter.
Two ways to provide access, and you do not need both:
Configured data sources — the default
A source in data-sources/config.yaml is queryable directly, through the same connection a built report uses.
df = query("warehouse", "select ...")
Works in CI or a notebook. Local sources work offline; remote sources still need network access and credentials.
An MCP server — optional
For letting an assistant explore: browsing a schema it has never seen, or reaching a system that is not a configured source at all.
Complements the above; never a prerequisite for it.
Configured sources are usually enough — see Ad-hoc analysis.
Point agents at Trellum, check it, take it out
The startup overview (python -m trellum) checks for a newer stable release
at most once per local calendar day and reuses that day's result. Claude's
session-start hook runs this overview; generated project instructions tell
other coding agents to run it when a new chat starts. The agent should mention
an available update and its upgrade command. Trellum does not upgrade itself.
Offline checks stay quiet, as do CI runs and projects with update_check: false
in config.yaml or FW_UPDATE_CHECK=0 in the environment.
trellum init does this for a new project; for an existing one run it
yourself, from the repository root:
trellum setup project # add the guidance and the hooks
trellum doctor # check that an agent session will actually see them
trellum setup project --remove
Setup tells you what it wrote and the exact doctor and --remove commands
for this project:
- Guidance in
CLAUDE.mdandAGENTS.md: a file that does not exist gets a short pointer; a file you already have keeps every word and gains a fenced<!-- trellum:start -->…<!-- trellum:end -->block at the end. - Claude Code hooks in
.claude/settings.json: aSessionStarthook and aPostToolUsehook, merged next to any hooks you already have.
Running setup again refreshes these in place; it never adds a second copy.
--remove takes back only that, and only what it can prove setup wrote:
- the fenced block, or a whole pointer file that is still exactly what setup created. A pointer you have edited is kept and reported, because it now also holds your words;
- hook entries identical to the two setup writes. Your own hooks, including
ones that merely run
-m trellum, stay; - the
trellumserver entry thatsetup portalwrote to.mcp.jsonand.cursor/mcp.json. Other servers, and atrellumentry setup did not write, stay.
A JSON file left with nothing in it is deleted, along with its .claude or .cursor folder if that
is now empty. Files that cannot be parsed are not touched. .env is never
modified: if it holds TRELLUM_API_KEY, the command says so, and you delete the
line and revoke the key yourself. The command exits non-zero when it leaves
Trellum content behind. Your files come back as they were, line endings and
trailing newlines included for blocks installed by this version. Older blocks
may lose a trailing blank line. JSON files are re-indented with two spaces;
a pre-existing empty JSON file may be deleted once its integration is removed.
Connect it to the portal
The portal is also an MCP server, so the agent that writes reports can read what the portal has built and, with the right key, operate it: check a report's last build and validation, query its published numbers, build it, test a data source, publish pending repository changes. The agent brings its own model — the portal holds no model key for this, and it works on a network with no route out.
Two things to have at hand:
- The server URL —
https://<portal>/s/<org>/<studio>/mcp: the studio's own URL plus/mcp, one server per studio. - A personal API key, sent as a bearer token. The agent acts as you, in that organization, under your studio role.
One command
With the key in hand, run this in the repository root — the API-keys page shows it ready-made, with the URL filled in, for each studio you can reach:
trellum setup portal --url https://<portal>/s/<org>/<studio> --key trellum_pk_…
It writes the two files below, puts the key in .env as TRELLUM_API_KEY,
and adds a short "Portal" section to CLAUDE.md and AGENTS.md so the
agent knows the server exists and what it is for. It refuses to finish while
.env is not ignored by git, and re-running it with a new URL or key
updates the entry in place without touching any other server in either
file. trellum doctor warns about an entry whose URL is malformed or whose
key is missing from .env. What it writes, if you would rather write it
yourself:
Claude Code
.mcp.json in the repository root:
{
"mcpServers": {
"trellum": {
"type": "http",
"url": "https://<portal>/s/<org>/<studio>/mcp",
"headers": { "Authorization": "Bearer ${TRELLUM_API_KEY}" }
}
}
}
Cursor
.cursor/mcp.json, the same shape:
{
"mcpServers": {
"trellum": {
"url": "https://<portal>/s/<org>/<studio>/mcp",
"headers": { "Authorization": "Bearer ${env:TRELLUM_API_KEY}" }
}
}
}
Both files reference the key through an environment variable and are safe to
commit. The key itself lives in your shell environment or the .env the
repository already keeps out of git — never in the file.
read or write
The key's scope decides what the agent is offered:
| Scope | The agent gets |
|---|---|
read |
The four read tools. It can look at anything you can look at and change nothing. |
read,write |
The read tools plus the actions — and they run immediately. There is no approval card here: your coding agent asks you before every tool call, and that is the approval. |
Actions are offered only when an organization admin has turned on Let the assistant propose actions (the same switch the in-portal assistant uses) and your studio role allows them; a viewer gets the read tools whatever the key's scope. Every action is recorded in the audit log against you, with the key's id.
The tools
| Tool | What it does | Needs |
|---|---|---|
list_reports |
The studio's reports: slug, name, category, last build, link | read |
get_report_details |
One report's charts, datasets and columns, validation state, last run | read |
query_report_data |
Filter and aggregate a report's already-built data — published numbers, no warehouse query | read |
read_doc |
A report's report.yaml or queries.py, assistant.md, project docs (allowlisted paths) |
read |
check_repo_changes |
What the portal would publish: the remote head it last saw and when, reports and data-source declarations added, changed or removed since the last publish, the last error | read, developer |
test_data_source |
Re-run a data source's connection test | write, developer |
run_report |
Queue a build now | write, developer |
publish_repo_changes |
Publish fetched-but-unapplied repository changes (manual publish mode) | write, developer |
configure_data_source |
Point you at the page where a declared source's credentials are entered — see below | write, studio admin |
Secrets never travel over MCP
configure_data_source takes no password, key or token. It answers with the
link to the studio's Configure page for that source and the fields it still
needs; you type them there, in the browser, and they go straight to the
encrypted store. A secret passed as a tool argument is refused and nothing is
stored — so a credential never lands in your agent's context or transcript.
Publish and configure from your agent
For developers and administrators with repository access, the agent that edits
reports can also take them to the portal and back. trellum guide portal
prints this loop to the agent; here it is for you.
- Edit and build locally.
python -m trellum.run reports/<slug> --no-serve. - Validate.
python -m trellum validate reports/<slug>— a report is finished when there are no FAILs. - Commit and push. The portal reads the branch it is configured for, never a working tree.
- Check.
check_repo_changessays what the portal saw at its last fetch: the remote head and when, the reports added, modified and removed since the last publish, data-source declarations that changed, and the last error. In manual publish mode this is the review step; in auto mode the fetch already published. - Publish.
publish_repo_changespublishes the pending changes at the head the agent just reviewed. - Build and read the result.
run_reportqueues a build; a moment laterget_report_detailsshows the last run, its status and the validator summary the portal recorded. A FAIL here is the same FAIL the local validator shows: the fix is in the repository. - Fix a data source. A build that fails in the driver usually means a
credential:
test_data_sourcere-runs the connection test and reports the detail line;configure_data_sourcereturns the studio's Configure link, where you enter the secret in the browser. The agent never sees it. - Add an alert.
create_alertturns "tell me if APAC DAU drops sharply" into a rule on a report — evaluated after each build or on a schedule by the portal's own agent, see Alerts;update_alertchanges it.
Checking repository changes needs developer access and a read key. Reading
build results needs only permitted read access; starting the build is a
developer action. Publishing, starting builds, testing sources and managing
alerts need developer access or higher, read,write and the organization's
actions switch. Configuring source credentials needs a studio administrator.
Your harness asks before each action, and every call is in the audit log
under your key.
A good first prompt
Point at the context explicitly rather than assuming it will be found:
Read
AGENTS.mdandcontext/metrics.md. Add a report underreports/churn-cohorts/showing monthly churn by signup cohort for the last twelve months, using the churn definition in the metrics file. Follow the structure ofreports/weekly-revenue/. Runpython -m trellum.run reports/churn-cohorts --no-serveand fix anything that fails before you finish.
Naming an existing report as the pattern to follow is the single highest-value part of that prompt.
What to check before merging
The diff is the review surface, so review the diff:
- Does the calculation match the definition? Code can run perfectly and compute the wrong thing, whatever source it reads.
- Is the grain right? A join that duplicates rows inflates a total without any error appearing.
- Are filters where they belong? A filter applied at the wrong step quietly changes what a number means.
- Did it invent a column? If the build succeeded, it did not — which is why it should build before you review.
- Is SQL in
queries.py? Inline SQL is harder to review and harder to reuse.
Warning
An assistant is fast at producing plausible analysis. The reviewer supplies the judgement about whether it is correct — that responsibility does not move. The workflow is built so review is possible: a diff you can read rather than a chart you have to trust.
Why this beats asking a chatbot for a number
A chat answer is produced once and cannot be re-derived — you cannot audit it, re-run it next quarter, or explain in a meeting how the number was reached. Here the assistant's output is a program: reviewed, committed, and re-run on a schedule. Everyone who opens the dashboard next quarter sees a number produced by that same approved code.