fix(build): speed up build preparation before workers start

[alexey-igrychev]

Summary

Most of the time before the first build worker starts went into re-preparing git repositories and re-hashing stapel stage checksums once per image and platform. On a Deckhouse configuration (1122 content-based tag stages, 188 remote repositories) that phase dropped from 26:33 to 4:58 with warm git mirrors; with an empty git cache, remote prefetching takes it from 22:26 to 13:09, and 8:22 together with werf#7930 and werf#7931. Stage digests are unchanged.

What

• A remote commit without submodules is read from the repository objects instead of a checked-out worktree: on the Deckhouse workload 85 worktree switches (489 s) become 0. A commit with submodules still gets a worktree, and an existing worktree cache is reused, not reset.
• A shallow local clone is unshallowed once per graph calculation instead of once per image: 551 Fetching origin blocks become 1. gitWorktree.forceShallowClone, gitWorktree.allowUnshallow and the shallow git clone is not allowed error are unchanged.
• Each stapel shell stage checksum is hashed once per builder; repeated reads across images and platforms return the cached value. The builder snapshots the shell configuration it was constructed with, so a later mutation of that configuration cannot make execution and the cached checksum diverge.
• With --parallel, remote repositories are prefetched before graph calculation with at most 4 concurrent tasks, bounded further by --parallel-tasks-limit; --parallel=false or a limit of 1 keeps the previous serial order. VERIFIED: git trace2 shows a peak of 4 concurrent fetch/ls-remote processes over 316 started.
• Remote repositories sharing a mirror, including ones reached through a URL alias, are prefetched once rather than once per configured name.
• Every ref of a repository is prefetched, not only the first one: refs that share a mirror are prepared in one task in configuration order, and mirrors are handed to workers dynamically, so a worker that finishes early takes the next mirror instead of idling.
• A slow git ls-remote --tags against one remote no longer blocks tag resolution for other remotes; cache refresh behavior is unchanged.
• VERIFIED: all 1122 anchor stage digests are identical between the unmodified and the changed binary across the warm, cold and prefetching runs of that configuration.
• No new flags, no werf.yaml surface, and nothing changes after workers start.

Why

Preparation was scoped per image and per platform instead of per build: every image re-checked and re-unshallowed the same local repository, remote repositories were cloned and fetched serially at the moment their first mapping was generated, and every remote commit read went through a full worktree checkout that only submodule resolution actually needs. On a configuration with over a thousand stages this made preparation longer than the builds it precedes, and the cost grows with the number of images, not with the amount of work to build. A persistent on-disk cache of prepared state was rejected: it would hide the duplicated work behind an invalidation problem instead of removing it.

[alexey-igrychev]

Verification

• Cold and warm runs of a large Deckhouse configuration up to scheduler start, comparing all 1122 anchor digests between the unmodified and the changed binary; source and binary checksums were pinned so the code could not change mid-measurement. Cold to workers: 22:26 -> 13:09, of which stage determination 17:02 -> 7:24. Warm to workers: 26:33 -> 4:58, worktree switches 85 -> 0, Fetching origin 551 -> 1.
• Mutation: removing the no-submodules shortcut -> Remote object checksums failed; keeping the per-image unshallow -> Local Git preparation failed; dropping the checksum snapshot -> Shell stage checksums failed; resolving a mirror alias as a distinct repository, bypassing the worker limit, or skipping the cache publication -> Remote Git preparation failed; unwiring the limit from --parallel-tasks-limit -> Remote preparation limit failed; restoring the single global tag-cache lock -> Remote tag concurrency timed out. For the per-ref prefetch: prefetching only the first ref of a mirror, deduplicating refs by repository name instead of cache key, and handing out no tasks at all each failed Remote Git preparation.
• Race detector run over pkg/build/image.

Review focus

• initRepoHandleBackedByWorkTree: whether any caller relies on the worktree existing as a side effect for a submodule-free commit.
• prepareRemoteGitRepos: mirror identity comes from GetClonePath() evaluated before fetching; confirm this covers every aliasing case that matters here, and that grouping refs of one mirror into a single task cannot deadlock against withMirrorKindLock.
• Prefetch happens before graph calculation, so a repository referenced only by an image excluded from processing is not prefetched — confirm the selection matches the images actually built.

[alexey-igrychev]

The same change is open upstream as werf#7928, where all checks are green (unit, lint, e2e_simple/extra/complex, integration_git/main, all build matrix entries). This branch carries two fork-only differences: the shell config snapshot also clones Packages, and the new test fixtures keep build.sbom.enable: false.

[alexey-igrychev]

Upstream twin werf#7928 is green with the same commits plus a cleanup that drops an unused context argument and tightens a comment. Measured together with #368 and #369: cold run to the first build worker 13:09 -> 8:22, all 1122 anchor digests identical to the unmodified binary.