Repository navigation
Initial go_modules table and mcp_servers refactor/fixes - #132
Merged
Merged
Conversation
Addresses macadmins#129 : a go.mod/go.sum inventory per recorded project. Moves project discovery, credential redaction and constraint handling into pkg/devclients, pkg/redact and pkg/utils; adds uid and the item-first column order to both tables; documents every warning code; runs the tests natively on Ubuntu, macOS and Windows; stops mcp_servers from putting a panic value in the error osquery logs.
…ed in macadmins#129, still open but syncing for windows local test running
# Conflicts: # go.mod
… warning counts, further mcp_server and testing tweaks
go_modules table and mcp_servers refactor/fixes
grahamgilbert
approved these changes
Oct 29, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a
go_modulestable, the Go module inventory (without resolving actually built versions or traversing/recursing the cache) proposed in #129, built to the schema, discovery, limits and acceptance criteria agreed there. The new table andmcp_serversshare project discovery, credential redaction and constraint handling through packages underpkg/.mcp_serversalso gets fixes that decoupling that code turned up, and the same 'completeness' model when it comes to communicating the state of how successful an inventory run was, and both get a new workflow that runs their tests on GitHub's Ubuntu, macOS and Windows runners.Deferred/not in this pull request
go.workworkspaces are intentionally left out as discussed. Other future 'supply chain' tables with better user guide docs for query writers/admins/etc in this series may necessitate or increase our confidence that this can be done strategically and responsibly, but they don't need to be in the v1.osquery-gobump. It fixes the transport each client opens and adds the column options, and will follow as its own pull request (which will need some fixes adapted to the other older tables, signing myself up for that work as well).mcp_serverstable fix detailsSchema. Changes are additive:
uidcolumn - mostly for parity when writing queries as used in core with the patternLEFT JOIN users USING (uid), but specifically for Windows to also better differentiate from the pre-existinguser_id(which remains holding the SID and remains the same across Mac/Linux) to instead just be the normal shortened RID user account identificationgo_modules, with the meat of what we're looking for/pertinent details/'item' columns first, and the correlation/account and 'completeness' columns lastBehavior.
scan_completedescribes a row's own files, and a problem finding a user's configuration is a warning row that changes no 'server' row. The README lists every warning code in the array with its sentence, its details and its conditional effect onscan_complete.go_modulesuses (per-table, not pooled) with thedevclients.MaxProjectsconstant. Multi-root VS Code workspaces are now read. A configuration file or project list that is a symbolic link is followed one hop on macOS and Linux when its target is in the same home (an exception considered worthwhile since this is how dotfile managers tend to create them). The Windows implementation declines links and refuses a junctioned parent directory altogether, also one project directory that two tools recorded with upper/lower case variation are read once because duh. Projects on a network filesystem that the home is not on are counted and not read. A home on a network filesystem has its projects read, and on Linux 9p is not treated as a network filesystem, because WSL2 mounts the Windows drives over it. A Windows project path in extended-length (\\?\C:\…) or device-namespace (\\.\C:\…) form, which Codex records, is read as the local drive's path;\\?\UNC\…is still counted as another host. (Yes these are progressively very excessively esoteric details.)npx -y @acme/mcp-server@1.2.3downloads, the table has to skip the launcher's options, and some options take a value. Each launcher (npm/npx, uv, pipx, bun, Docker, podman) now has its own list of options, taken from that tool's source or CLI. An option the table doesn't recognize leaves the package columns empty with anunknown_launcher_optionwarning rather than a guess, except for npx, which itself treats an unknown option as taking the next word."mcpServers": nullcounts as no servers, but a file that is onlynullis reported as unreadable. If Codex's config lists one project twice with different trust, it counts as untrusted.utils.RecoverPanics, so a panic reaches osquery as fixed text and never as the panic value.Approach
There is one row for each module version that a project on this host requires in
go.mod, records a hash for ingo.sum, or both. The table inventories declarations and checksum records. It does not compute the build list or verify downloaded/referenced content independently. A requirement is a minimum under minimal version selection, andgo.sumcan keep versions that are no longer needed, so a 'match' would still e.g. require being confirmed with thegocommand in that project.Discovery. Golang code repos/projects come from the lists that IDE's/software clients/agents record, as in
mcp_servers, and the tables to-date do not walk directory trees. The sources are:workspaceStoragerecords, including multi-root.code-workspacefilesrecentProjects.xml~/.claude.json~/.codex/config.tomlOnly the project root is read. A root
go.workis reported asgo_workspace_unsupported, and modules in subdirectories are not read.Parsing. It uses
golang.org/x/modv0.37.0, a new dependency ingo.mod,go.sumanddeps.bzl. The strictgo.modparser runs first, and the lenient parser runs only to recover requirements, reported asgo_mod_partial. Module paths and versions are checked against their formats before they are published. A value that contains a credential format with a distinctive prefix (carefully considered to avoid common/well-known-enough false-positives) is dropped.Limits. Each
go.modandgo.sumis capped at 2 MiB and 5,000 entries. A scan follows at most 1,000 recorded projects per account. A query returns at most 50,000 module rows and 32 MiB, within one MACADMINS_EXTENSION_WALK_TIMEOUT budget for all accounts, which is checked before each file. Each limit that is reached produces a warning row.Completeness.
scan_complete = 1marks a row whose project'sgo.modandgo.sumwere read in full. A problem finding a user's projects is a warning row for that user, chosen from a fixed list, and it changes no module row. Projects in cloud storage, outside the home, or on a network filesystem the home is not on are counted and not read.The table README has the column reference, every warning code, the limits, and what the project lists do not cover.
Operating system support
Actually exercised/confirmed working (~once, on my machinelol, please let me know if I should include a ci-local.sh I'm using for the ubuntu container and macOS testing setup):
ubuntu-latest)Testing/Validation
.github/workflows/native-tests.ymlruns the tests forpkg/...,go_modulesandmcp_serversonubuntu-latest,macos-latestandwindows-latest. It runs on push and pull request, astest.ymlandlint.ymldo. On Windows it doesn't use the race detector, and it also skips the not-applicablepkg/utilsTestExecCmd*tests there, which runecho,cat,sleep,sh, etc.As mentioned above, ci-local.sh harnesses support local/native testing, with Ubuntu/macOS handled from a host with Docker-compatible tooling (like OrbStack) installed, and has been manually QA'd thorough-ish including in
osqueryion Win11. On macOSosqueryi, when run as an unprivileged user (2026-10-15), returned 55 module rows for that user and three diagnostic rows: root's unreadable home, an account whose home does not exist, and four recorded projects outside the home.lint,testandbuildchecks and the three native jobs pass on the fork/my branch.osquerydas root on any platform, and osqueryi on a Linux host.Checklist