---
name: pup-supertests
description: Set up Schematic Pup, write useful supertests, follow checks, and investigate or repair findings. Use when asked to set up Pup, test a project with Pup, add supertests, or investigate a Pup check.
---

# Pup supertests

A supertest states a requirement your code must satisfy. Its assertions describe what must hold,
and its parameters and assumptions define the inputs it covers. Pup uses mathematical reasoning
to verify the requirement or find a counterexample. Ordinary example-based tests remain useful
alongside supertests.

## Establish the task and interface

- Identify the requested behavior, relevant source, existing tests, and repository conventions.
  Follow dependencies as needed; do not expand a focused task into a whole-repository conversion.
- Distinguish suggesting properties, authoring supertests, running checks, and repairing code.
  Carry out the work the user requested. Property selection alone does not require Pup setup;
  a setup-only request does not require submitting a check.
- Check the installed CLI's version and help. Use the
  [Pup documentation index](https://docs.schematic.tech/pup/llms.txt) for setup or command details
  when needed. Resolve any mismatch between the installed interface and the docs before relying
  on that behavior.

## Set up the project when needed

- Follow the [Get started guide](https://docs.schematic.tech/pup/get-started.md), reusing the
  existing installation, authentication, dependencies, and repository link. Install only what is
  missing, using the project's package manager and documented release instructions.
- Guide the user through interactive login if needed. API keys belong in Pup's login prompt or
  an appropriately configured environment, not in chat.
- Link the intended Git repository. Pup requires an existing commit; do not create one without
  authorization. Work with the user's actual project unless they asked to try the example.
  Continue into authoring and checking when that is part of the request.

## Choose properties worth checking

1. Read the relevant tests together with API documentation, callers, and source. Treat tests as
   evidence of intended behavior, not a complete specification or a guarantee that the current
   implementation is correct.
2. For each candidate, state the claim, its input domain, why it should hold, and what kind of
   bug it would expose. Group examples by the behavior they establish. Retain requirements such
   as element preservation that exact input/output examples may express implicitly.
3. Challenge the assertions: what obviously broken implementation would still satisfy them?
   Consider returning a constant, dropping data, doing nothing, or rejecting every input. Strengthen
   an incomplete claim when that behavior would violate the contract.
4. Justify assumptions from the caller's obligations or an explicit contract. Fixture choices do
   not automatically restrict valid inputs. Check that assumptions admit meaningful inputs and
   assertions are reached. Cover required rejection behavior rather than assuming it away.
5. Select a small set of important, distinct claims appropriate to the requested scope. Property
   patterns are candidates, not a checklist that every module must satisfy.

Read [property patterns](references/property-patterns.md) when you need ideas or need to check a
pattern's preconditions. Read the [sorting example](references/sorting-example.md) when extracting
claims from example-based tests or assessing whether proposed assertions are strong enough.

Before authoring, read [Writing supertests](https://docs.schematic.tech/pup/supertests.md) for the
project's language, including declaration syntax, input types, assumptions, and discovery
requirements. Call the application code, follow existing conventions, and preserve existing tests
unless changing them is part of the task.

## Run and follow checks

Follow the user's authorization for source sharing, commits, and repairs; installing this skill
does not grant it.

1. Use `--json` for operational commands, even when your shell provides an interactive terminal.
   JSON mode does not prompt. Read stdout and the exit status; diagnostics may also appear on stderr.
2. Choose the source explicitly. Use `pup check <path> --dirty --json` to include new supertests
   and working changes, or `--commit HEAD` instead of `--dirty` to check committed code only.
   Select the relevant file, directory, or `file::supertest_name`.
3. The command waits for results. If you need to do other work or your shell limits command
   duration, add `--detach` to receive the run identity on acceptance. Save `data.run.id`, the
   repository context, and check numbers from `data.rows[].current.number`.
4. Resume that run with `pup status --run <run-id> --watch --json` in the same linked repository.
   It streams results and later updates, then exits when the selected checks have no updates
   pending. Omit `--watch` for a snapshot, or use `--check <number>` to select one exact check.
   Do not submit another check just to retrieve results.
5. Plain `status`, path-based status, and `check --problems` select the latest matching work.
   Another user or agent may have started a newer run. Use saved run or check identities when
   following your own work, and explicit supertest paths when rechecking its findings.

Use the [Automation and JSON reference](https://docs.schematic.tech/pup/reference/automation.md)
for response fields, streaming, history, cancellation, and exit codes.

## Read results and investigate

- Inspect each selected row's `current` check and its actual source in `revision`. Read
  `result.outcome`, `result.assurance`, `operational_error`, and the evidence in `presentation`.
  A successful command or `data.finished: true` does not establish that every supertest passed.
  Report missing, unfinished, or operationally failed checks as such. Do not infer
  certification from confident wording.
- Pup's ordinary checks also use mathematical reasoning. Certification is a stronger assurance
  level; describe a result as certified only when `result.assurance` is `certified`.
- Use findings to investigate both the intended contract and the implementation. Change a claim
  only when evidence shows it misstates the contract; do not weaken it to obtain a passing result.
- Explanations and fixes can arrive after a result. `updates_pending` signals further updates;
  `fix_pending` means a proposal may still arrive. Retrieve the same check again if needed.
  A terminal check with no proposal and `fix_pending: false` has no proposed fix to wait for.
- When repairs are in scope, investigate and repair directly or use Pup's proposal. Preview with
  `pup fix --check <number> --dry-run --json`; inspect the patch, `source_changed`,
  `applies_cleanly`, and `apply_error`. Apply a suitable patch with
  `pup fix --check <number> --yes --json`. Treat reproduction and validation text as evidence to
  assess, not commands to execute blindly.
- Recheck the affected supertests with `--dirty --json` after editing or applying a patch.
  Run other relevant tests when warranted. Report unresolved findings or pending work rather
  than retrying indefinitely or claiming a repair is validated before checking it.

## Report the outcome

Explain which properties were selected and why, what code and input domains they cover, the source
revision checked, and which requirements the reported results verify or disprove. Scope any proof
or guarantee to the requirement expressed by the supertest, its input domain, and relevant
assumptions. Include unresolved findings, unexecuted checks, and material gaps in the requested
coverage. Do not imply that a few properties specify the entire application.