# Haystack - Complete Documentation > Let your agent see what it really changed. Haystack gives your coding agent hundreds of copies of your app, each set up like one of your customers. ## Haystack: let your agent see what it really changed URL: https://haystack.sh/ # Haystack: let your agent see what it really changed Haystack gives your coding agent hundreds of copies of your app, each set up like one of your customers. ## How it works 1. **Blast radius.** Haystack reads the diff and works out every case the change can affect. The cases are your customers' real journeys, learned from your production traffic, narrowed to the ones the change can reach — not scenarios somebody invented for a test. In one worked example a permissions refactor touching two files reaches 3 services and 27 code paths, producing 14 cases worth running, mapped in about five seconds. You do not write the cases. 2. **Fork your infrastructure.** Haystack snapshots your services and the data behind them, from Postgres and Redis to queues and object storage, then forks that snapshot once per case. Fourteen copies of prod, ready in 26 seconds. These are real environments, not mocks, and nothing that happens inside one reaches prod. 3. **Run every case at once.** Each environment is seeded into a different state: a workspace over its seat limit, an invite nobody accepted, a member whose role just changed. Every environment runs twice, with and without your change, so any difference is caused by the change. 4. **See what actually broke.** Failures come with the screen before and after, the network calls that changed, and the events that stopped firing. The environment stays live, so you can open it. 5. **Your agent runs the loop.** The failing case goes back to your coding agent in a form it can act on. It fixes, re-runs, and keeps going until nothing fails. ## Why it catches what tests miss The failures that reach production are usually quiet. In the worked example a frontend stopped sending a member's role and the access API treated a missing role as editor, so anyone with a pending invite could edit and delete anything in the workspace. Nothing threw, every request returned 200, and the test suite passed. ## Using it The CLI is haystack verify. See [haystack verify](/docs) to set it up and run it, [Working with runs](/docs/runs) for reading results and onboarding, and [Agents](/docs/agents) for setting it up with your coding agent and the agent loop. ## Getting started Haystack is early and taking on a small number of teams. Sign up for early access at https://haystack.sh/ or book a demo. --- ## Set up Haystack URL: https://haystack.sh/agents # Set up Haystack You are a coding agent, and your user asked you to set up Haystack in this repository. Follow these steps in order. When you are done, `haystack verify` works here: after a code change it runs the app with and without the change, explores both side by side, and reports where the change shows up and what broke. There is no configuration file to keep in the repository. Haystack works out how to run the app from what the repository already keeps working: lockfiles, Dockerfiles, compose files, package scripts. What you already know about running it saves most of that work, so step 3 has you write it down as notes. ## Rules - Ask your user only for what you cannot do yourself. Setup needs them for two things: entering a sign-in code on GitHub, and installing the Haystack GitHub App if it is not installed yet. Telemetry (step 7) needs their answers and a few actions only they take. - Never print, log or commit a secret. Setup needs none from you. Never run `haystack telemetry token` or touch the token it writes: that is your user's. - Keep changes inside this repository. Ask before changing anything global, such as installing the CLI globally (step 2). - Show your user what `haystack init` will change before you let it change anything. - Check each step with the command given, and report only what the commands printed. - If a step fails in a way you cannot fix, or the instructions are wrong, tell the Haystack team in a sentence: `haystack feedback ""`. They see it at once. ## 1. Check the repository ```bash git remote get-url origin ``` It must name a repository on github.com. If it does not, stop and tell your user that Haystack verifies GitHub repositories. ## 2. Get the CLI ```bash haystack feedback --help >/dev/null 2>&1 && echo "use: haystack" || npx -y -p @haystackeditor/cli@latest haystack --version ``` If it printed `use: haystack`, the installed CLI is new enough: use `haystack` below. Otherwise it printed the version npx fetched: use `npx -y -p @haystackeditor/cli@latest haystack` in place of `haystack` in every command below (the `-p` and `haystack` are needed: the package has two commands, and npx cannot pick one without them). Do not install anything globally on your own. A global install only matters with Claude Code: without one, `haystack init` leaves out Claude Code's Stop hook and says so in its `notices`. When that notice appears, offer `npm install -g @haystackeditor/cli@latest` in the same message as the changes from step 3. If your user agrees to both, install it before step 5, and step 5 adds the hook as well. It needs Node.js 22.18 or newer. The package is `@haystackeditor/cli`; the unscoped `haystack` package on npm is unrelated. If `haystack` rejects a command below as unknown, it is too old: use the npx form. ## 3. Preview the setup First write what you know about running this app as notes, in a file outside the repository. Onboarding plans from them instead of guessing (a wrong guess at the start command costs it up to half an hour): ```bash cat > "${TMPDIR:-/tmp}/haystack-notes.json" <<'EOF' { "run": "...", "signIn": "...", "workflow": "..." } EOF ``` - `run`: how production installs, builds and starts the app: the commands and the directory each runs in, the port, the databases and services it needs and their versions, migrations and seed data, and the environment variables it reads (their names and what they are for, never secret values). - `signIn`: how a person gets an account and signs in (a sign-up page, a first-start admin, an invitation), and the kinds of users the app has. - `workflow`: what a signed-in user mainly does, step by step through the screens. Write only what the repository or your own work in it shows you, in plain sentences, and leave out a note you do not know. Onboarding still proves everything it takes from them. ```bash haystack init --json --notes "${TMPDIR:-/tmp}/haystack-notes.json" ``` Without `--yes` it changes nothing. Act on its exit code (the JSON `status` says the same): | Exit | `status` | What to do | |---|---|---| | 4 | `login-required` | Sign in (step 4). The output already lists the `changes`: show them in the same message as the sign-in code and ask your user to approve them too, so they answer once. | | 2 | `planned` | Show your user each entry of `changes` (its `path`, `reason` and `diff`) and ask whether to apply them. If they agree, step 5. | | 5 | `app-required` | Ask your user to install the Haystack GitHub App on the repository's owner (it needs an owner admin) at https://github.com/apps/haystack-code-reviewer-pr-hook/installations/new, then run this step again. | | 0 or 3 | `done` | Nothing needed changing, and onboarding has started (3: it is blocked). Go to step 6. | | 6 | `done` | Onboarding stopped before finishing. Run this step again. | | 1 | (the error envelope) | Show your user `error`. | Show your user any `notices` as well: things init left alone, and why. ## 4. Sign in (only when step 3 said `login-required`) Skip this step otherwise: you are already signed in. ```bash haystack login --no-wait --json ``` Tell your user: "Open `verificationUri` and enter the code `userCode`" (the values it printed), together with the changes from step 3 if they have not approved them yet. Then run: ```bash haystack login ``` It returns once they have entered the code. If your shell times out first, run it again: it keeps waiting on the same code until GitHub expires it, about 15 minutes after step 4 began. ## 5. Apply the setup ```bash haystack init --yes --json --notes "${TMPDIR:-/tmp}/haystack-notes.json" ``` It makes the changes it showed, gives Haystack your notes (`notes` in its output says `written`, or `unchanged` when it already had them) and starts onboarding the app on Haystack. At most it changes: - `.claude/settings.local.json`: a Claude Code Stop hook, `haystack verify precompute --hook`, that prepares each change for `haystack verify` when a session stops. Only when Claude Code is in use; the file is per developer, and init keeps it out of commits. - `AGENTS.md` (and `CLAUDE.md` when the repository has one): a short marked note telling coding agents to run `haystack verify` after a change. Its exit codes are those of step 3. Do not commit or push unless your user asks. ## 6. Check onboarding ```bash haystack verify onboarding --json ``` Onboarding reads the repository, builds the app, starts it with its databases and services, prepares its data and test accounts, and proves one workflow works. The app runs on Haystack's isolated machines with no internet access; nothing touches your user's production systems. The first onboarding can take half an hour. You do not need to wait for it. - `onboarding.state` is `onboarding` or `ready`: setup is done. - `blocked`, and `block.ask.question` is present: show your user `block.ask.message` and the `choices`, file their pick with `haystack verify answer ""`, then run `haystack init --json` to continue. - `blocked` otherwise: show your user `block.reason`, and `block.ask.message` when there is one. When the reason is something you know better (how the app starts, a service it needs, how people sign in), correct your notes and run step 5 again: new notes start a new onboarding. - `incomplete`, `cancelled` or `not-started`: run `haystack init --json` to start it again. `dataAbsences` and `versionChoices` are notes for your user, for example that the app ran with an empty database, or which version of a service was picked because the repository does not pin one. ## 7. Set up telemetry Telemetry shows Haystack how real people use the app: their journeys through its pages (a tag in the browser, once visitors consent), the shapes and settings of the data its server handles, and how often a value is missing. Haystack then explores what real users do. Every `haystack init` also plans telemetry. Until it has your user's answers, only telemetry waits: in the output, `telemetry` parts that are `manual` list in their `steps` what to ask. Ask your user, in one message, for what they list (never guess an answer): - which app serves production, when the repository has several; - the app's production origins, for example `https://app.example.com`; - whether the site asks visitors for consent, and with which consent platform, or that they have decided consent is not required. Preview it with their answers (repeat `--origin` for each origin; `--app` only when the repository has several apps): ```bash haystack init --json --app --origin --consent ``` Show your user the new `changes` and any `notices`, as in step 3. If they agree, apply them: ```bash haystack init --yes --json --app --origin --consent ``` Then go through `telemetry` and `telemetrySteps` in its output with your user: - `.haystack/telemetry-settings.proposed.json` is a proposal of the settings whose values telemetry would record; nothing is recorded until your user approves it. Show it to them. They approve it themselves with `haystack telemetry settings --approve`, which shows the diff and writes `.haystack/telemetry-settings.json`; then commit that file. Never approve it yourself. - A part that is `manual` lists the exact `steps` to take; `unsupported` says why. - Ask your user to run `haystack telemetry token --repo ` themselves, in their own terminal, and to set the three variables it prints in production's server environment. Never run it yourself, and never ask for the token. - When `dbProfileCommand` is set, ask your user to run it themselves with read access to production's database, then the `show` and `upload` commands `telemetrySteps` lists. - Tell your user the off switches: `HAYSTACK_TELEMETRY=0` turns server telemetry off, and removing the tag turns the browser tag off. Once your user has deployed, run the same `haystack init --json` command again. `telemetry.capture.received` and `telemetry.server.received` show the newest telemetry Haystack holds; null means nothing has arrived yet, and `stale` means nothing in the last day. ## 8. Report Tell your user, using only what the commands printed: which files `haystack init` changed, the onboarding state (and why, if it is blocked), any notes from step 6, which telemetry parts are set up and what is left for them to do, and that from now on you will run `haystack verify` after changing the app's code. ## After setup: the loop After you change the app's code, or when your user asks you to check a change, two steps: ```bash haystack verify # the handshake: what the change touches, what production and real users do there, and Haystack's ideas of what could break haystack verify --intent "" --idea "" ``` The first call is the handshake (it exits 4): read it, keep the ideas worth trying, add your own (repeat `--idea`), and say what you were asked to do. The second explores your ideas first. `haystack verify flow ` drills into one flow the handshake lists: its code, the pages it shows on, and what its code saw in production. All three change nothing, so run them without asking. Run it inside the checkout; nothing needs committing or pushing. It waits for onboarding if that is still running. The first call builds and prepares the change if the stop hook has not, and never crawls; the second crawls the app with and without the change for 3 minutes (`--minutes` sets 1 to 30). With building and preparing both versions, the answer often comes 10 to 15 minutes after you start: give the command your longest timeout or run it in the background, and do not pipe it through `tail` or `head`, which hide its progress until it exits. If it times out, run it again: it picks up the same crawl and never starts over. The handshake also says what data the app copy starts from. Keep the app's seed in `.haystack.json` as `"data": { "seed": "", "files": [""] }`: Haystack runs it before each check (from the repository root, with the app's own environment plus `HAYSTACK_APP_URL`, `HAYSTACK_TEST_ACCOUNT_EMAIL` and `HAYSTACK_TEST_ACCOUNT_USERNAME`) and the handshake shows what it made, table by table, or its error with the exit code and the last lines of its output; with no seed declared, `haystack init` and the handshake show the exact JSON for the seed the repository already keeps. It ends with a short report: the files it checked (confirm your latest edits are in), where the change shows up in the app, each finding with its verdict (bug, not a bug, can't tell), why, and the steps that show it, and any changed code the crawl never ran. Fix each bug your change caused, then run it again. `haystack verify --json` prints the same report as one JSON document; `--raw` prints the crawl's whole record, which is large, for scripts. Exit codes: 0 the crawl answered, 1 the command failed, 2 the crawl or onboarding stopped before finishing (run it again), 3 onboarding is blocked (step 6), 4 the handshake (run it again with `--intent`), 5 the app copy is broken and the change could not be tested. More: [haystack verify](/docs), [Working with runs](/docs/runs), [Agents](/docs/agents). Every page here is also markdown at its URL plus `.md`, and [/llms.txt](/llms.txt) lists them all. --- ## Haystack Pricing URL: https://haystack.sh/pricing # Haystack Pricing One price per team. No per-seat maths, and no bill that grows because you shipped more. ## Team - $1,000 / month For one product team shipping to prod. Everything included, however much you ship. | | | |---|---| | Changes verified | Unlimited | | Cases per change | As many as it takes, chosen per change | | Environments | Run and torn down for you | | Seats | Everyone on the team | | Support | Shared Slack channel with our engineers | Also included: the CLI and agent loop, a live environment kept for every failure, and runs on every pull request or on demand. ## Enterprise - custom For orgs with more than one team, their own infrastructure, or procurement. Everything in Team, plus unlimited teams, your cloud or ours, an engineer rather than a support queue, annual invoicing, SSO and audit logging, and security review and DPA. [Contact sales](https://haystack.sh/pricing) ## Common questions **Do you touch our prod systems?** No. Haystack forks a replica of your stack from a snapshot, app and database and queues together, and every run happens inside that copy. Writes, deletes and outbound calls stay in the fork. **How is the state realistic, then?** Each case gets its own fork of the same snapshot, seeded into the state that case describes. That is how a run can put a user mid-trial or a queue mid-drain without touching anything real, and why two cases never contaminate each other. --- ## Haystack Changelog URL: https://haystack.sh/changelog # Haystack Changelog Recent updates and improvements to Haystack. ## July 2026 ### Jul 5 - Built for Coding Agents - Agent setup skill at /skill.md — a coding agent can install and configure Haystack on a repo end-to-end - llms.txt and markdown for agent fetches — Claude, ChatGPT, and Perplexity agents get real content instead of an app shell - CLI 0.15.14 — `haystack init` detects your dev server and names configs after the repo - `haystack-cli` and `haystack-review` on npm now resolve to the real CLI ## June 2026 ### Jun 28 - Sharper Analysis, Bigger PRs - Blast-radius verdicts grounded in named callers and callsites - Materiality filter drops nitpick findings before they reach you - Huge PRs with generated files no longer stall analysis; 4× analysis capacity - New home page built around risk-based QA: one risky change, caught, tested, escalated ### Jun 22 - GitHub Access, Hardened - Every GitHub read and write runs on GitHub App installation tokens — OAuth fallbacks removed - Private-repo analysis results authorized by signed grants instead of live GitHub calls ## May 2026 ### May 31 - The CLI Becomes a Platform - `haystack setup --repo owner/name --yes` runs fully non-interactive; `--json` speaks NDJSON for coding agents - MCP server (`haystack mcp`) exposes triage results and PR state to any MCP client - Webhooks, headless tokens, and versioned JSON schemas for every `--json` output ### May 20 - Onboarding That Survives Real Repos - Protected default branch? Setup opens a bootstrap PR through the Git Data API — works with rulesets and CODEOWNERS - Repo scans clone and search your code directly, replacing the GitHub Search API - Extracted rules arrive in plain language with per-card toggles - Existing open PRs are backfilled for analysis when your config PR merges ### May 18 - Review Chat, Persistent and Polished - Chat turns persist and survive reconnects - Syntax-highlighted before/after diffs, themed code blocks and tables - Resizable inbox/chat split; Drafts bucket with Undraft; live review-comment Threads tab ## April 2026 ### Apr 12 - The Inbox & Feed - Haystack now opens to the Inbox — a unified home for every PR that needs your attention - The Feed surfaces one actionable PR at a time — Issues Found, Review Requested, Needs Assignment, and more - Chat with any PR directly from a feed card - Tabbed sections — Good to Merge, Snoozed, Analyzing, Recently Merged ### Apr 8 - Auto-Fixer Agent (Alpha) - Haystack can attempt to auto-fix straightforward issues in your PRs with a sandbox agent - Smart dismissals remember what you've already reviewed ## February 2026 ### Feb 12 - Cloud Sandbox and In-Review Editing - Pro users can now preview frontend and backend changes in a cloud sandbox - Edit code directly in the review and see live results ### Feb 11 - Documentation Page - Added a /docs page with step-by-step verification flow setup guide - Learn how to set up automated PR verification with /prepare-haystack and /setup-haystack ### Feb 5 - Mobile Support - Haystack now works on mobile devices with a fully responsive design - Floating chat input and swipeable Code/Browser/Terminal tabs for easy navigation - Long-press on any symbol for go-to-definition on touch devices ### Feb 1 - Slash Commands & Dashboard Improvements - New slash commands: /approve, /merge, /comment, and /request-changes to take action directly from chat - Quick approve and merge buttons added to the dashboard for faster PR workflows - Stacked PR navigator for Graphite and ghstack-style PR stacks - Collapsible dashboard sections and ability to dismiss PRs you don't need to review ## January 2026 ### Jan 31 - React File Chunking Fix - Fixed chunking of React files so components are properly grouped in PR sections ### Jan 29 - Agent-First UI and Bug Detection - Redesigned the UI to prioritize Haystack Agent — the agent is now front and center in your review workflow - Made chat a lot faster, especially suggestion pills which now load and respond near-instantly - Haystack now automatically detects possible bugs and issues in your PR so you can catch problems before they ship ### Jan 23 - Edit Code Right in Your Review - Make changes and run them in one click — no local setup required - Pro subscribers get cloud sandboxes with full frontend/backend preview ### Jan 15 - Haystack Agent - Chat with an AI that understands your entire PR context - Ask questions about the code, get explanations, and request changes ## Links - [Home](/) - Learn more about Haystack - [Pricing](/pricing) - Plans and pricing --- ## Install Haystack GitHub App URL: https://haystack.sh/github-app # Install Haystack GitHub App Get AI-powered code review on your repositories in minutes. ## Installation Steps 1. **Click Install**: Visit [github.com/apps/haystack-code-reviewer-pr-hook](https://github.com/apps/haystack-code-reviewer-pr-hook/installations/new) 2. **Select Repositories**: Choose which repos to enable Haystack on 3. **Authorize**: Grant read access to your code for analysis 4. **Start Reviewing**: Open any PR and visit haystackeditor.com/review/{owner}/{repo}/{pr} ## Permissions Required Haystack requests the following permissions: - **Read access to code**: To analyze your PR diffs - **Read access to pull requests**: To fetch PR metadata - **Read access to issues**: To understand linked issues We never write to your repository without explicit action from you. ## Pricing Haystack is priced per team. [Contact sales](/enterprise-contact) for pricing and to get your organization set up. Enterprise adds a self-hosted option, SSO, and dedicated support. [View Pricing Details](/pricing) ## Security - Your code is analyzed in real-time and not stored permanently - We use encrypted connections for all data transfer - SOC 2 compliance in progress ## Support Having trouble? Contact us at support@haystackeditor.com ## Links - [Home](/) - Learn more about Haystack - [Pricing](/pricing) - Plans and pricing - [haystack verify](/docs) - Verify a change against copies of prod, no GitHub App required --- ## Haystack Enterprise URL: https://haystack.sh/enterprise-contact # Haystack Enterprise For teams that need advanced security, compliance, and support. ## Enterprise Features - **Self-hosted deployment**: Run Haystack in your own infrastructure - **SSO/SAML**: Integrate with your identity provider - **SOC 2 compliance**: Enterprise-grade security - **Dedicated support**: Private Slack channel with our team - **Custom integrations**: Connect to your existing tools - **Volume pricing**: Discounts for large teams ## Contact Sales Fill out the form at [haystackeditor.com/enterprise-contact](https://haystack.sh/enterprise-contact) or email enterprise@haystackeditor.com. ## What to Include - Company name and size - Number of developers - Key requirements (self-hosting, SSO, compliance) - Timeline for evaluation ## Links - [Home](/) - Learn more about Haystack - [Pricing](/pricing) - Standard plans --- ## haystack verify URL: https://haystack.sh/docs # haystack verify Run your app with and without a change, explore both side by side, and see what the change broke. ## Set up ```bash npm install -g @haystackeditor/cli haystack login # GitHub sign-in: open the URL it prints and enter the code haystack init # shows what it will change, then starts onboarding your app ``` There is no configuration file. Haystack works out how to run your app from what the repository already keeps working: lockfiles, Dockerfiles, compose files, package scripts. `haystack init` shows each change it will make as a diff and makes them when you agree: a Claude Code Stop hook that prepares each change for `haystack verify` (per developer, kept out of commits), and a short note in AGENTS.md telling coding agents to run `haystack verify` after a change. Then it starts onboarding: Haystack reads the repository, builds the app, starts it with its databases and services, prepares its data and test accounts, and proves one workflow works. The first onboarding can take half an hour, and runs in the background. Or let your coding agent do all of it. Paste this into Claude Code, Codex or Cursor: ```text Set up Haystack for me: fetch https://haystack.sh/agents.md and follow it ``` ## How a crawl works A crawl builds your app twice, once at the base (the merge base of your HEAD and origin's default branch) and once with your change, starts from the code you changed, and explores outward with both side by side. It double-checks and judges every difference it finds. The only difference between the two copies is your change, so a difference in behaviour is caused by it. Both copies run on Haystack's isolated machines with no internet access, with stand-in data and test accounts made during onboarding. Nothing touches your production systems. ## Running it ```bash haystack verify # your change, as it is in your checkout right now haystack verify --minutes 10 # give the crawl longer haystack verify --json # the same report as one JSON document ``` Run it inside your checkout; nothing needs committing or pushing. It names the changed files it checks, waits for the crawl, and prints each step and each bug as the crawl finds it. The answer often comes 10 to 15 minutes after you start, most of it spent building and preparing both versions of the app. Ctrl-C stops waiting, never the crawl: run it again to pick up the same crawl. | Option | Purpose | |---|---| | --minutes | How long the crawl may take, 1 to 30. Default 3 | | --no-wait | Print the crawl's current state and return | | --json | The report as one JSON document (see `haystack schema verify`) | | --raw | The crawl's whole record as one JSON document, for scripts (large) | | --repo | GitHub repository, when origin is not it | | --account | Use a specific saved Haystack account | | Exit code | Meaning | |---|---| | 0 | The crawl answered; the bugs it found are in the output | | 1 | The command failed | | 2 | The crawl or onboarding stopped before finishing; run it again | | 3 | Onboarding is blocked; the output says what would unblock it | | 4 | The handshake: run it again with --intent (and any --idea) | | 5 | The app copy is broken; the change could not be tested | ## Next - [Working with runs](/docs/runs) - [Agents](/docs/agents) --- ## Working with runs URL: https://haystack.sh/docs/runs # Working with runs Read what a crawl found, follow onboarding, and answer its questions. ## Reading the results When the crawl answers, `haystack verify` prints three things: - where your change shows up in the app: each changed spot, how far the crawl got with it, and the steps that reached it; - what the crawl found, bugs first, each judged `bug`, `not a bug` or `can't tell`, with the steps to see it; - the changed code the crawl never ran. Differences the crawl had no time to check are listed as not finished in time; `--minutes` gives the next crawl longer. With `--json`, the same results are in `crawl.manifest` (or `crawl.answer` while the crawl is still sealing them). ## Onboarding ```bash haystack verify onboarding # where onboarding of this checkout's base is haystack verify onboarding --wait # follow it until it is ready or blocked ``` `haystack init` starts onboarding, and so does the first `haystack verify`. When onboarding cannot go on it says why and what would unblock it. When it needs a fact about your app that the repository does not say, it asks a question with fixed choices: ```bash haystack verify answer "" # then run haystack init ``` Every result says when Haystack had to choose for you, for example that the app ran with an empty database, or which version of a service it picked because the repository does not pin one. ## The stop hook With Claude Code, `haystack init` adds a Stop hook, `haystack verify precompute --hook`, to the per-developer `.claude/settings.local.json`. When a session stops it captures the checkout, and Haystack builds the app with and without the change and works out what the change touches, without crawling. `haystack verify` then crawls from that prepared app instead of building it again. The hook takes at most five seconds and never fails the session. ## Hosted runs of pushed commits ```bash haystack verify hosted start owner/repo --base --head haystack verify hosted status --wait haystack verify history owner/repo ``` A hosted run verifies two exact pushed commits on the production Cloud Verifier. `haystack verify cases ` lists its saved cases, and `haystack verify explore ` opens your own copy of one for 24 hours after the run. --- ## Agents URL: https://haystack.sh/docs/agents # Agents Set Haystack up with your coding agent, and let it check its own changes. ## Set up with your agent Paste this into Claude Code, Codex, Cursor or any agent that can run commands: ```text Set up Haystack for me: fetch https://haystack.sh/agents.md and follow it ``` The playbook at [/agents.md](/agents.md) walks the agent through installing the CLI, signing in, previewing `haystack init` and applying it once you agree, checking onboarding, and setting up telemetry. Setup asks you for two things at most: a sign-in code to enter on GitHub, and installing the Haystack GitHub App if it is not installed yet. Telemetry asks for your production origins and consent choice, your review of the settings it may record, and three things only you do: minting the ingest token (`haystack telemetry token`), profiling your database, and deploying. It never asks for a secret. ## The loop After an agent changes the app's code, it runs two commands: ```bash haystack verify # the handshake: what the change touches, and Haystack's ideas of what could break haystack verify --intent "" --idea "" # explore, with the agent's steering first ``` The note `haystack init` adds to AGENTS.md tells every agent working in the repository to do this, without asking. The agent reads the short report it ends with: each finding's verdict (bug, not a bug, can't tell), why and steps, where the change shows up, and the changed code that never ran. It fixes each bug its change caused and runs the command again. Exit code 4 is the handshake asking for `--intent`; exit code 3 means onboarding is blocked, and the report says what would unblock it. Exit code 5 means the app copy is broken and the change could not be tested. `--json` prints the same report as data; `--raw` prints the crawl's whole record, for scripts. ## Onboarding questions When onboarding needs a fact the repository does not say, it asks a question with fixed choices. The agent shows it to you and files your choice: ```bash haystack verify answer "" ``` ## Feedback When Haystack fails, confuses the agent or gets in its way, the agent says so in a sentence and keeps going: ```bash haystack feedback "" ``` The Haystack team sees it at once. ## HTTP API Everything the CLI does is plain REST under `https://haystackeditor.com/v1`, with the repository in the path. Send `Authorization: Bearer `; `haystack login --headless` gives a long-lived token for CI and agents. Errors are JSON, `{ "error": "..." }`, with a status that says what kind. | Method and path | What it does | |---|---| | `POST /v1/repos/{owner}/{repo}/checks` | Check a captured change (the body `haystack verify` sends) | | `GET /v1/repos/{owner}/{repo}/checks?commit={sha}` | Your newest check of that commit | | `GET /v1/repos/{owner}/{repo}/checks/{id}` | One check; `?waitAfter={updatedAt}` holds the read until it changes | | `DELETE /v1/repos/{owner}/{repo}/checks/{id}` | Cancel it | | `POST /v1/repos/{owner}/{repo}/checks/{id}/verdicts` | Your verdict on one of its findings: `{findingId, verdict: bug\|intended\|noise, why, decidedBy: agent\|user}` (what `haystack verify verdict` sends) | | `GET /v1/repos/{owner}/{repo}/checks/{id}/verdicts` | The verdicts given on it, oldest first | | `PUT /v1/repos/{owner}/{repo}/patches/{sha256}` | Upload the captured change's patch | | `GET /v1/repos/{owner}/{repo}/onboarding?base={sha}` | Where onboarding of that base is; `POST` starts it | | `POST /v1/repos/{owner}/{repo}/onboarding/answers` | Answer an onboarding question | | `GET /v1/repos/{owner}/{repo}/runs` | Recent finished hosted runs | | `POST /v1/feedback` | Tell the Haystack team what went wrong | Ask a check read for `Accept: text/markdown` and the answer is the report as `haystack verify` prints it: headline, findings with their steps, where the change shows up, and what never ran. ## MCP server ```bash haystack verify hosted mcp # stdio MCP server for hosted runs ``` It exposes `verify_start`, `verify_status` and `verify_wait` for hosted runs of pushed commits. No credential, repository source or sandbox is reachable through it.