Skip to main content
The backtest tools family contains a single tool — backtest_run — that runs a deterministic backtest over a stored StrategySpec and a date range. The engine is trusted, audited numpy code; the agent never provides executable code. Same inputs always produce identical outputs, down to the content hash.

​
backtest_run

Run a deterministic backtest over a StrategySpec and date range. Returns a run_id, the equity curve, full statistics, final portfolio weights, and a risk attestation. The run is stored as an 8-file artifact bundle (see Artifact tools).

​
Arguments

ArgumentTypeRequiredDescription
specobjectYesA StrategySpec dict (same shape accepted by propose_strategy).
rangeobjectYesDate range with start and end (ISO 8601 dates).

​
What it does

  1. Validates the spec against the frozen StrategySpec contract (same validation as propose_strategy).
  2. Resolves the universe to concrete symbols and fetches the required OHLCV data through the data layer, recording provenance.
  3. Locks the environment — the numpy version, data hashes, and engine version are captured in environment_lock.json so the run is reproducible.
  4. Runs the backtest engine: signal → weights → risk → execution, rebalanced at the spec’s horizon.
  5. Computes statistics and a risk attestation.
  6. Stores the full result as an 8-file artifact bundle with SHA-256 content hashes.
  7. Returns the run_id and a summary.

​
Return value

FieldTypeDescription
run_idstring (UUID)The backtest run identifier.
strategy_idstring (UUID)The strategy this run tested (if proposed first).
equity_curvearrayArray of { date, equity } points.
statsobjectFull statistics block (see below).
final_weightsobjectFinal portfolio weights keyed by symbol.
risk_attestationobjectRisk constraints checked against the run.
artifact_bundleobjectPointer to the 8-file artifact bundle.
content_hashstringSHA-256 hash of the canonicalized result.

​
Statistics block

The stats object contains a comprehensive set of performance and risk metrics:
MetricTypeDescription
sharpefloatAnnualized Sharpe ratio (risk-free = 0).
sortinofloatAnnualized Sortino ratio (downside-deviation denominator).
cagrfloatCompound annual growth rate.
max_drawdownfloatMaximum peak-to-trough drawdown (as a fraction, e.g. -0.32).
win_ratefloatFraction of profitable periods.
turnoverfloatAverage daily turnover (fraction of portfolio traded per day).
var_95floatValue-at-Risk at 95% confidence (daily, as a fraction).
cvar_95floatConditional Value-at-Risk (expected shortfall) at 95% confidence.
VaR and CVaR are computed from the daily return distribution using historical simulation. They are not parametric assumptions — they reflect what actually happened in the backtest window.

​
Example call

{
  "spec": {
    "name": "NVDA momentum (50/200 SMA cross)",
    "universe": { "type": "single", "symbol": "NVDA" },
    "signal": { "type": "sma_cross", "fast": 50, "slow": 200, "direction": "long_only" },
    "weights": { "type": "fixed_fraction", "fraction": 1.0 },
    "risk": { "max_weight": 1.0, "stop_loss_pct": null, "max_positions": 1 },
    "execution": { "slippage_bps": 5, "fee_bps": 1 },
    "horizon": "daily"
  },
  "range": { "start": "2022-01-01", "end": "2024-12-31" }
}

​
Example response

{
  "run_id": "run_01HQKX2J3K4M5N6P7R8S9T0V2A",
  "strategy_id": "strat_01HQKX2J3K4M5N6P7R8S9T0V1Y",
  "equity_curve": [
    { "date": "2022-01-03", "equity": 100000.00 },
    { "date": "2022-01-04", "equity": 100120.50 },
    { "date": "2024-12-31", "equity": 312450.75 }
  ],
  "stats": {
    "sharpe": 1.84,
    "sortino": 2.41,
    "cagr": 0.46,
    "max_drawdown": -0.28,
    "win_rate": 0.54,
    "turnover": 0.012,
    "var_95": -0.031,
    "cvar_95": -0.048
  },
  "final_weights": {
    "NVDA": 1.0
  },
  "risk_attestation": {
    "max_weight_constraint": 1.0,
    "max_weight_observed": 1.0,
    "max_weight_breached": false,
    "max_positions_constraint": 1,
    "max_positions_observed": 1,
    "max_positions_breached": false,
    "attested_at": "2025-01-15T14:34:55.412Z"
  },
  "artifact_bundle": {
    "bundle_id": "art_01HQKX2J3K4M5N6P7R8S9T0V2B",
    "files": 8,
    "content_hash": "sha256:b2c3d4e5f6a7b8c9d0e1f2a3b4c5d6e7f8a9b0c1d2e3f4a5b6c7d8e9f0a1b2c3"
  },
  "content_hash": "sha256:c3d4e5f6a7b8c9d0e1f2a3b4c5d6e7f8a9b0c1d2e3f4a5b6c7d8e9f0a1b2c3d4"
}

​
Determinism

Determinism is a hard guarantee, not a best-effort property. Two backtests with the same spec, range, engine version, and data hashes must produce identical output. If they don’t, it’s a bug.
Determinism is achieved through three controls:
  1. Fixed random seed. The engine seeds numpy’s RNG with a deterministic value derived from the spec content hash and range. No call to np.random uses OS entropy.
  2. Pinned numpy. The engine runs against a pinned numpy version recorded in environment_lock.json. Floating-point behavior is stable within a numpy major.minor release.
  3. Fixed data window. The OHLCV data is fetched once, hashed, and locked. The data_manifest.json artifact records every symbol, date, and content hash used. Re-running with the same manifest guarantees identical inputs.
The result is that content_hash is a function of (spec, range, engine_version, data_hashes). If all four match, the hash matches. This lets you verify reproducibility across runs, machines, and time.
  • Different data. If the upstream provider restates a bar (e.g. a split adjustment), the data hash changes and the output changes. This is correct behavior — the data_manifest.json makes it visible.
  • Different numpy version. Floating-point summation order can change between numpy versions. The environment_lock.json pins the version so this doesn’t happen silently.
  • Different engine version. A new engine release may change the order of operations. The engine version is part of the hash input, so a version change produces a different hash by design.

​
The risk attestation

The risk_attestation object records whether the run respected every constraint in the spec’s risk block. It is computed after the backtest, against the realized path — not just the final weights. This catches intra-run breaches (e.g. a position that briefly exceeded max_weight before a rebalance). If any constraint was breached, the corresponding *_breached flag is true and the run is flagged in the web UI. The agent should report breaches to the human before requesting paper-trading promotion.