Skip to content

Scala safe mode with core capabilities - #301

Open
adamw wants to merge 5 commits into
masterfrom
scala-safe-capabilities-6601c28e
Open

adamw wants to merge 5 commits into
masterfrom
scala-safe-capabilities-6601c28e

Conversation

@adamw

@adamw adamw commented Oct 21, 2026 •

Copy link
Copy Markdown
Member

Moves all side-effecting code into a new core module and compiles the other modules in Scala 3.10 safe mode (-language:experimental.safe).

What changed

  • core (not safe, trusted): the only module allowed to use @assumeSafe. It has:
    • orca.effect: three capabilities, FileSystem, Processes and Network, and their implementations.
    • orca.safe: the prelude (small helpers over os-lib paths, Ox's either and the standard library) and Logger (over slf4j).
    • orca.concurrency: wrappers over Ox (scopes, forks, channels, retry), plus Ref for shared state.
    • orca.mcp: a facade over chimp, for the MCP servers.
    • orca.json: JsonCodec / JsonData / JsonSchemas, with our own schema derivation (tapir's Schema.derived fails capture checking). Parity tests compare it with tapir.
  • Safe modules: tools, claude, codex, gemini, pi, opencode, flow, and a new lifecycle module (setup, recovery and git steps moved out of runner).
  • Not safe: runner (wiring, terminal, logging, all *Agents objects, now moved here) and shell.
  • Service traits (GitTool, CliRunner, Agent, …) stay plain. Implementations take capabilities as constructor parameters; only wiring and tests create real capabilities.
  • Env, clock, randomness and logging are not tracked: safe mode can't enforce them.
  • Tests check that @assumeSafe stays inside an allowlist in core, that safe code can't reach effect implementations directly, and that a Ref update can't capture a capability.
  • Flow scripts are not compiled in safe mode. They only need the //> using scala 3.10.0 bump.

Things to know

  • Scala 3.10.0 has a race when several safe-mode modules compile in parallel in one JVM, which gives false "Cannot refer to …" errors. The build now compiles one module at a time (clean build: ~34s instead of ~19s). If the errors appear in a long-running sbt server, restart it.
  • scalafmt is upgraded to 3.11.5 so it can parse capture syntax. This reformats 61 files that are otherwise unchanged.
  • Behaviour changes:
    • MCP integer inputs are stricter ("7" is rejected).
    • Run-log events sent after finish are dropped with a warning.
    • The MCP host stops in milliseconds instead of ~4s.
  • Shared mutable state was removed where possible, or replaced with Ref.

Testing

A forced clean rebuild compiles with no warnings, scalafmtCheckAll passes, testFull passes in all modules, and the built-in flows compile with scala-cli on 3.10.

🤖 Generated with Claude Code

adamw added 3 commits October 21, 2026 21:36
New `core` module holds the only side-effecting code and three
capabilities: FileSystem, Processes, Network. tools, the backends,
flow and a new lifecycle module (extracted from runner) compile
with -language:experimental.safe. runner and shell stay non-safe.
…ties-6601c28e

# Conflicts:
#	README.md
#	docs/authoring/tutorial.md
#	flow/src/main/scala/orca/AgentSet.scala
#	flow/src/main/scala/orca/settings/AgentSpec.scala
#	flows/epics.sc
#	flows/implement.sc
#	flows/quick.sc
#	flows/resolve.sc
#	flows/review.sc
#	lifecycle/src/main/scala/orca/lifecycle/FlowLifecycle.scala
#	opencode/src/main/scala/orca/backend/opencode/OpencodeApi.scala
#	runner/src/main/scala/orca/flowEntry.scala
#	shell/src/main/scala/orca/shell/actions/SettingsEditAction.scala
@adamw

adamw commented Oct 22, 2026 •

Copy link
Copy Markdown
Member Author

Design decisions and cost per module

Main code in tools, the five backends, flow and the new lifecycle module now compiles with -language:experimental.safe. Tests stay plain. runner and shell stay plain.

Module graph: core ← tools ← flow ← lifecycle ← runner ← shell.

Decisions

Capabilities

  • Three capabilities: FileSystem, Processes, Network. These are the effects safe code cannot reach any other way.
  • No Terminal capability. No safe module needs one.
  • Env, clock, random, logging and Ox are not capabilities. Safe mode does not block Properties.envOrNone, Random or Instant.now, so a capability would enforce nothing. Where these were already passed in as values, they still are.
  • One FileSystem, no read/write split. Every member has a production caller and its own meaning. Members are added only with a caller.
  • Processes covers both call and spawnPiped. OsProcCliRunner is a class over it, so there is one way to start a process.
  • Capabilities are created only by private[orca] def ambient. It is called only from wiring and tests.
  • The cookie scan lives on the spawned process (PipedCliProcess.survivors()). CliRunner and Processes need no extra member for it.
  • Returned handles (PipedCliProcess, HttpClient) are plain types. This is a known trade-off: code holding one is not tracked.

What stays plain

  • Service traits (FsTool, GitTool, CliRunner, FlowContext, Agent, …) are plain traits, not capabilities. Implementations take capabilities as constructor params. Construction moves to wiring.
  • Every *Agents object (and its test) moves to runner, same package. Safe code cannot turn a backend that holds fs/net into a plain AgentBackend.
  • A backend has one way to start processes. For example, ClaudeBackend takes gitAt: os.Path -> GitTool instead of procs.
  • Session probes are constructor params, built in wiring. A safe factory returns ... ->{fs} Boolean. Where the probe needs a directory, the backend passes its own workDir, so the probe and the spawn read the same field. Gemini and opencode keep their probes inside the backend: those probes touch no capability.
  • Helper objects with file effects (OrcaDir, JsonFile, PiSessionStore, RunEventReader, …) become classes that take fs. They are built in wiring, not given (using FileSystem). This keeps fs in a few constructors instead of spreading it across many signatures.
  • runner stays plain (wiring, terminal, logging, FlowLock). shell is out of scope: it has 320 errors in 41 files, and a Terminal capability would touch nearly every file without confining anything.

Core exposure rule

  • @assumeSafe is allowed only in core. It goes on capability classes and pure facades: prelude, Logger, JSON, the orca.mcp facade over chimp, and orca.concurrency (the Ox facade and Ref). No effect implementation carries it.
  • Enforced by two tests. AssumeSafeScopeTest checks that @assumeSafe appears only in core. SafeModeExposureTest checks that safe code using QuietProc, OsPipedSubProcess, CookieScan, JdkHttpClient, LocalServer, os-lib or X.ambient does not compile, and that a Ref update cannot capture a capability.
  • Facades cover only what migrated code uses. The prelude either loses Ox's nested-either check, which we accept.
  • Scoped APIs take the narrowest Ox type (ResourceScope, not Ox) when they only register cleanup.

JSON

  • derives JsonCodec replaces derives ConfiguredJsonValueCodec, a one-word change at each site.
  • JsonData uses a Mirror-based schema derivation in core. Singleton enum cases keep their own names.
  • Permanent schema-parity tests compare it with tapir's Schema.derived.
  • MCP inputs use jsoniter instead of circe. tools main no longer uses circe. Integer inputs are now stricter ("7" and 1.0 are rejected). We accept this: the agent gets an Invalid arguments error and retries.

Mutable state

  • Shared mutable state is removed first, if possible. Removed: the DecodedTurn cancel flag (the channel's closed state is the signal), the AskUserBridge latch, and the ObservedTurn buffer var.
  • What remains uses Ref. Safe mode rejects var fields. No java.util.concurrent types are left in safe main code.
  • Actor- or thread-confined state also uses Ref, only because var fields are rejected. The actor or owner-thread check is what keeps it correct. Each Ref records the alternatives that were considered.
  • RunEventLog's appender actor is now a channel plus one writer fork. An event that arrives after finish is dropped with a warning. This is documented and tested.
  • The PiStdin lock stays as it is. It compiles in safe mode, and changing it would change behaviour.

Module layout

  • New core module (orca-core), below tools. It is not safe-mode, and it is the only place @assumeSafe may appear.
  • New safe lifecycle module (orca-lifecycle, package orca.lifecycle). It holds the logic of FlowLifecycle, SetupPreflight, StackDiscovery, WorktreeRun and the related files, moved out of runner.
  • TtyProbe moves to runner.
  • Logback is now a direct dependency of runner only. It used to reach runner through claude, so the published orca-claude no longer pulls it in.

Build

  • Scala 3.10.0 with a zero-warning build. Scripts, the flow-authoring template and the docs are pinned to 3.10.0.
  • scalafmt 3.11.5 with runner.dialect = scala3future, so capture syntax can be formatted. This upgrade reformatted 61 files the migration did not otherwise touch.
  • Compiles run one at a time (Tags.limit(Tags.Compile, 1)) because of the compiler race below.
  • To verify zero warnings, force a real rebuild. sbt 2 restores clean; compile from its cache, so use set ThisBuild / scalacOptions += "-Xmax-inlines:32"; clean; compile; Test/compile.
Cost per module

"Errors" means the number of errors on the first safe-mode compile.

Module Capability signatures added Main cost Found / gained
core provides FileSystem, Processes, Network, the facades and Ref the whole trusted surface lives here one place to review what is trusted, enforced by tests
tools OsFsTool, OsGitTool, Worktrees, OrcaDir, OrcaFile, JsonFile take fs/procs 302 errors in 45 of 92 files; ~40 test files rewritten removed a cast (Chat.withAgent), 3 pieces of mutable state (3 more moved to Ref), circe in main, and os.pwd/sys.env defaults; no bugs
codex CodexBackend(net, fs, …, sessionProbe), rolloutProbe(fs, …) 35 errors in 4 of 6 files, mostly JSON codec renames tests no longer probe the real ~/.codex/sessions
gemini GeminiBackend +fs, GeminiSettings.register(fs, …) 46 errors in 3 of 7 files 7 tests used the cwd without saying so; found the inline-proxy bug
claude ClaudeBackend: procs replaced by gitAt, plus a transcriptExists probe 58 errors in 6 of 10 files, 0 capture errors tests no longer probe the real ~/.claude/projects; fixed the logback dependency
pi PiSessionStore(fs) is now a class; PiBackend takes a probe; create returns PiBackend^{fs} 3 errors + 4 capture errors (counted after the mechanical edits) PiSessionStore created its own fs inside a helper
opencode Network.httpClient (new), OpencodeBackend +net 0 errors, because the HTTP layer was designed first; the JDK client code (129 lines) was replaced by 81 lines a test confirmed that both close and interrupt unblock the SSE reader
flow RunEventLog.start, ProgressStore.default, ReviewerCatalog.discover take fs; RunEventReader, RunPruning, ProgressScan became classes; OsGateShell(procs, fs) 43 files of mechanical edits, ~60 test call sites; state in StageFrames, RunEventLog, CostTracker, DeniedToolTracker, EventDispatcher moved to Ref or a channel found the parallel-compile race; GateShell gets its workDir from one place
lifecycle WorktreeRun(procs, fs, lock) is now a class; setup takes fs, onPath, tty, ask small diffs (largest: FlowLifecycle +44/−33) setup hid a git call and terminal I/O behind defaults; RoleAgents no longer depends on all five wirings
runner none (plain) receives the *Agents objects and their tests; wiring calls .ambient —
shell none (out of scope) a few .ambient calls (SessionAction, ResumeDetector, ContinuableAttempts) —

Across all modules, the mechanical edits are mostly import changes (NonFatal, Ox, slf4j → facades), os.* → fs.*, and JSON codec renames. Safe mode found no behaviour bugs in the existing code.

Compiler issues (Scala 3.10.0)
  • Race when compiling in parallel. Safe-mode compiles running at once in one JVM report false "Cannot refer to … from safe code" errors. The error can stay stuck in a long-running sbt server. Workaround: serialize compiles, and restart sbt if it gets stuck. A minimized report is ready to file upstream.
  • Crash in CaptureOps.boxDeeply with an inline by-name wrapper (ox.par, useInScope, releaseAfterScope, retry). Workaround: make the wrappers non-inline.
  • Inline proxy loses the capture type (Found AutoCloseable^{c} Required AutoCloseable). Workaround: make discard non-inline. For pipe/tap, use a named val.
  • String.split returns Array[String]^{any.rd}, but only when it is compiled after a capture-checked file. A repro exists.
  • Zero-field cases of a generic enum in a pattern alternative fail to type-check. Workaround: a type pattern, case _: (A[?] | B[?]).
  • IArray.from / IArray.unsafeFromArray return ^{any}. Workaround: core helpers (utf8Bytes, utf8String, readPrefix).
  • Plain code that re-exports @assumeSafe or derives a schema for it fails with "class assumeSafe is marked @experimental". Workaround: a captureChecking import in main code; @experimental on the test class.
  • List.maxByOption takes a pure function. Workaround: map to (value, key) pairs first.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant