A cache keyed by the inputs, for the kit's CI: a design

Status: stage 0 is built (owner's decision of 2026-10-27); nothing is skipped. The rest is a design awaiting the owner's decisions (section 7). What stage 0 is: Stage 0, as built, at the end. It answers the roadmap entry "A cache keyed by a hash of the inputs, for CI" (roadmap.md, "After 0.1.0"), which asked first for a note on which jobs are deterministic. Every number below is quoted from ci.md, the workflows or the tests, with the file named; where the repository has no number, the page says so.

The idea: hash exactly what a self-test.yml job reads, plus the versions it runs on; if a job with the same hash already passed, show it as skipped and say which run proved it.

1. What it solves that Select-Jobs.ps1 does not

.github/scripts/Select-Jobs.ps1 answers "which jobs does this change touch?" from the files that differ between the pull request and its base (ci.md, "What runs when"). It has no memory: it does not know that a job already passed on the same inputs. Four cases it cannot see:

How much that could save, from what the repository records:

• A full run is 35 jobs and 513 runner-minutes, 84% of them (about 431) in the 20 jobs of the game-programme matrix (C# pairs 27 minutes on average, GDScript 16) (ci.md, "Measured"). A pull request costs about 14 minutes (documentation only) to 100 (the staged programme's own code) after Select-Jobs, estimated in the same table.
• So on a pull request the ceiling for a repeated job is small in absolute terms (14 to 100 minutes), and the largest single prizes are the field-test job (1031 s, about 17 minutes, ci.md) and the pairs (16 to 27 minutes each).
• On main and the nightly run the bill is 513 minutes per run. That is where the money is, and it is exactly where this design, as asked ("main always runs everything as now"), saves nothing. Letting main reuse a green result is a separate, bigger decision (section 6, stage 3, and the questions at the end).

What the repository does not record, so the real saving is unknown:

• how many runs were repeats of a commit already green ("run all again" shows as run_attempt above 1; ci.md's sample of 100 runs lists events only: 82 push, 17 pull_request, 1 workflow_dispatch, and 47 cancelled);
• for the 47 cancelled runs, which jobs had already finished green before the cancel;
• how often two successive commits of one pull request have the same inputs for a job.

ci.md says the same of its history ("gh run list and gh run view --json jobs, which this change could not reach"). Stage 0 below exists to fill these three gaps before anything is skipped.

2. The key

For a job J on a commit C:

key(J, C) = sha256 of
    1. the git object ids of J's input files at C        (git ls-tree -r C, filtered to J's file set)
    2. the workflow definition: self-test.yml, godot.yml, .github/actions/**, .github/scripts/**
    3. the tools: Godot version, kit version, PowerShell version, runner image
    4. the job id and the event kind where it changes behaviour (pull_request or not)

Git object ids as the content hash. git ls-tree -r <commit> -- <paths> prints one blob id per file; hashing that listing (sorted) is a hash of the files' content and modes, costs milliseconds on a shallow checkout, and needs no file reads. A tree id of one directory would do for a job that reads a whole folder (git rev-parse C:benchmark/games/arena/gdscript), but a job's set is rarely one folder, so the listing is the general form. Git LFS files are pointers in the tree, which is the content id LFS itself uses.

The tools. The kit pins Godot 4.7.2 mono, .NET 8, PowerShell 7.6+, gdUnit4, GUT and coverlet in files that are part of the tree (AGENTS.md "Targets", Update-Versions.ps1), so a version change is already a change of input files. What is not in the tree is the runner: the apt action already keys on $env:ImageOS and $env:ImageVersion (.github/actions/apt/action.yml), RUNNER_OS and RUNNER_ARCH are in the Godot cache keys, and pwsh --version can be read in the step. These go into the key, so a new runner image is a miss. ubuntu-latest and windows-latest are labels that move; the version is what is recorded.

The workflow's own definition. Select-Jobs.ps1 already treats the workflow, its actions and its scripts as the "shared core" that runs every job ($core). The key does the same: any change to those files changes every key. A narrower rule (hash only the job's own block of self-test.yml) is possible, because Select-Jobs already cuts the file into job blocks, but it adds a way to be wrong for little gain; start with the whole file.

How a job's file set is found

Two ways, and the page recommends deciding between them only after stage 0:

A. Derived from Select-Jobs.ps1 (no second list). A file is an input of job J exactly when Select-Jobs.ps1, given that file as the only change, selects J. The script already holds the three sources of that rule: the pattern table ($patterns, $core), the tests each job runs (read from the workflow by Get-TestJobs) and the "a file no test names runs every job" fallback. The key function would run that predicate over git ls-files. Advantages: the set cannot drift from the selection, and a missing path is a bug that exists today in Select-Jobs (tests/Test-SelectJobs.ps1 already holds it), not a new class of risk. Cost: the script's decision has to be callable as a function, which is a change to the script (not made here). Limit: the set is a selection rule, tuned to be safe to over-select, so for most jobs it is large (a lib change selects init and the reusable jobs; .github/ selects everything). The hit rate is bounded by how coarse Select-Jobs is.

B. A declared list per job in one data file (proposed: ci/godot/job-inputs.json, glob lists keyed by job id). More precise, and readable by a person. It needs a test that fails when a job reads a file outside its list. Options, cheapest first:

1. Consistency with Select-Jobs: the list for J must contain every path pattern that selects J. A self-test (Test-JobInputs.ps1, proposed) runs the selection over every tracked file and fails if a file selects J but J's list does not match it. This catches the common error (a new test added to a job) because the selection already reads the workflow.
2. Tracing, run in the nightly build only: wrap one job's Godot and PowerShell runs in strace -f -e trace=openat,execve (or fatrace), collect the paths opened under the workspace, and fail if one is outside the list. Cost is real (the staged programme runs Godot "hundreds of times", self-test.yml), unmeasured, and it can only prove a job did read a file, not that it never would. Use it as an audit, not as the definition.
3. Convention: a job reads only its own folders plus the shared core, written down in AGENTS.md. Cheap, unverified.

Neither way is built. A is the smaller change; B is the more precise one. Both need the consistency test.

What a tree hash of the selected paths gives

Two commits with different ids but the same hash for J are indistinguishable to J. That is the point: a docs-only commit leaves measure's hash unchanged (docs/json.md aside, which measure does read for the error-code test). It also makes the hash a precise statement of what was tested, which a commit id is not.

The squash-merge case

A squash merge creates a new commit on main whose tree is the merge of the pull request into main. The pull request's pull_request run checked out GitHub's merge commit (refs/pull/N/merge), whose tree is also the merge of the pull request into main. If main has not moved since that merge commit was made, the two trees are equal, so every job's hash is equal and every PR-green result is valid for the merge commit. If main moved, the trees differ in exactly the files the other merge changed, and J's hash differs only if one of those is in J's set.

This case only pays if main is allowed to skip, and it exposes a storage problem (next section): a result saved by a pull request run cannot be read by a run on main with the Actions cache.

3. Where results live

Recommendation: the Actions cache marker for stages 0 and 1 (pull-request re-runs, the repository's existing pattern, no new permission), and a decision about a second store only if the owner wants main to skip (stage 3).

Reading a hit. A step in the existing changes job, after Select-Jobs.ps1: one script (proposed .github/scripts/Get-InputHash.ps1, not ci/godot/lib, because it is CI-only and not part of what games run) prints <job>=<hash>; one actions/cache/restore with lookup-only per gated job (ten steps; the key is dynamic, so a matrix does not help); then <job>-hit=true|false outputs. Each job's if: becomes needs.changes.outputs.<job> == 'true' && needs.changes.outputs.<job>-hit != 'true'. The reusable-workflow jobs (zero-config, csharp, explicit-runners) need no edit inside godot.yml: the marker is saved by a final record job (needs: all, if: always(), saving a marker for each job whose needs.<job>.result == 'success'), the way the coverage job already collects from all of them.

Reporting. A skipped-by-cache job shows as skipped, which satisfies a required check, as skipped-by-selection already does. The changes job writes a table to $GITHUB_STEP_SUMMARY: job, hash (first 12 characters), hit or miss, the run id that proved it, and the minutes that run took. A skipped job with no explanation would be indistinguishable from a broken if:.

Telemetry. The telemetry command (roadmap.md) exists in a pending pull request and reads .godot/hyperdot-history.jsonl, a per-game file. CI hit rates are a CI fact, not a game's, so the record job should upload a small cache-observations artifact (one JSON line per job per run: job, hash, hit, event, run_attempt, minutes), the same way jobs upload coverage-* today (retention-days: 14). telemetry would read that and add the hit rate as a field, as the roadmap says it will; this page does not design that field.

4. Which jobs may be skipped

Read from self-test.yml and godot.yml as they are. "Deterministic" means: the same inputs give the same verdict, so a green result for the same hash is a green result for this one.

Other workflows:

5. Risks and their mitigations

6. A staged plan

7. Decision for the owner

1. Is stage 0 enough for now? It costs a little CI time, changes no behaviour, and replaces three missing numbers (repeat rate, cancelled-but-green jobs, identical-input pushes) with measured ones. The alternative is to decide on the estimates of section 1, which are only ceilings.
2. Which jobs may a green be reused for, and is the false-skip risk accepted for them? The table proposes lint, zero-config, explicit-runners, action, init, field-test first; measure never; the pairs only after their races are fixed. Say if the list should be shorter, or if csharp (which posts the pull request comment) should wait.
3. Is it worth the complexity at 14 to 100 minutes per pull request? On a pull request the ceiling is small, and Select-Jobs already took the bill from 513. The case for the cache is stronger if main may use it (stage 3), which is a separate risk decision, or if re-runs and incremental pushes turn out to be common (stage 0 will say). If neither, "not now" is a sound answer, and this page stays as the record.
4. How should a job's file set be defined: derived from Select-Jobs (smaller change, coarser) or a declared list with a self-test (more precise, a second list to maintain)?
5. Opt-in for the model workflows (evals.yml, token-benchmark.yml): never, or an input that is off by default?
6. Where would main read results from, if stage 3 is wanted: a check run (needs checks: write) or a results branch (needs contents: write)?

Stage 0, as built

The owner chose stage 0 on 2026-10-27, with the file set derived from Select-Jobs.ps1 (way A of section 2). It measures; no job's if: changed and nothing is skipped.

Not in stage 0: the game-programme pairs (a matrix job; keyed later, per game folder), coverage, and every if: that would skip.

Reading the result after two weeks. For each cache-observations artifact (gh run download), a line with hit: true and selected: true is a job that would have been skipped. Group by job, event and runAttempt; the minutes are the jobs' own durations (gh api repos/<repo>/actions/runs/<id>/jobs, joined on runId). That answers the three gaps of section 1 and the cache's scope and eviction rules. Markers are Actions cache entries, so a pull request run reads its own earlier runs' and the default branch's, not another branch's, which is the scope stage 1 would have.

A row with runAttempt above 1 does not count for a hit rate: "Re-run failed jobs" does not run changes again, so its hit-* outputs keep the first attempt's lookups and the second attempt's rows understate the hits. Those rows say so with hitStale: true. A job on another runner than changes (local-windows, the Windows and macOS exports in csharp) is keyed with a label of its runner (-RunnerByJob, set in self-test.yml), not an image version, which changes cannot read.