RFC: IND AS XBRL support via shared xbrl engine + ind_as adapter (upload → graph v1) #60

Activity

1. MHa55anY commented on Aug 10, 2026

   

   MHa55anY

   on Aug 10, 2026

   Author

   More actions

   Note for maintainers: Please set the issue type to RFC if possible — gh could not set issue type with read-only permission on this repo (UpdateIssueIssueType). Labeling also requires write access; feel free to add area:adapters.

   Also closed accidental duplicate: RoboFinSystems/robosystems#1133.
2. jfrench9

   added the

   RFC

   issue type on Aug 10, 2026
3. jfrench9 commented on Aug 10, 2026

   

   jfrench9

   on Aug 10, 2026

   Member

   More actions

   Thanks for this — it's a genuinely well-researched RFC, and the fact that you read the code before proposing shows. Setting the type to RFC and adding area:adapters.

   Where you're right

   Your core diagnosis holds up. adapters/sec/processors/xbrl_graph.py is ~2,200 lines that import SECClient/SEC_BASE_URL, take sec_filer/sec_report kwargs, normalize CIK, and mint http://www.sec.gov/CIK#{cik} as the canonical entity URI. Alternative A (extend sec/) would drag EDGAR assumptions into a second jurisdiction, and we agree it's the wrong default. Non-goals are scoped the way we'd want them — no crawling, no shared repository, no MCP in v1.

   Two corrections and one question that we think changes the shape of the work.

   1. The taxonomy-neutral Arelle layer already exists — one level up from adapters/

   robosystems/arelle/ is already there and already deliberately taxonomy-neutral. Its entry points are extract_taxonomy(model_xbrl) -> rdflib.Graph and serialize_jsonld(graph, standard, version), and it feeds robosystems/taxonomy/ (discovery.py, loader.py, pins.py, writer.py) which seeds the frameworks/ library.

   It lives outside adapters/ on purpose. Per robosystems/adapters/README.md, an adapter is an external service integration — a client plus transformation, optionally with a shared-repository manifest. A taxonomy-neutral XBRL engine isn't that; it's platform-scoped machinery. So we'd rather not create adapters/xbrl/: it would put neutral code in the layer reserved for service integrations, and split the neutral Arelle surface across two homes.

   The extraction candidate you're looking for is real, though — it's just a different file. adapters/sec/client/arelle.py (ArelleClient, ~700 lines) is cache directory selection, bundled-schema population, offline mode, and controller lifecycle. Almost all of that is jurisdiction-neutral; only the EFM validation configuration is SEC-specific. Hoisting that into robosystems/arelle/ behind a SEC-compatible wrapper is exactly the kind of no-behavior-change foundational PR your Phase 1 describes, and we'd take it on its own merits.

   So: robosystems/arelle/ grows an instance-side companion to its existing taxonomy-side extractor; adapters/ind_as/ (or equivalent) stays thin. Same separation of concerns you argued for, one layer up.

   2. There are two destinations for a non-US taxonomy here, and the RFC only addresses one

   This is the substantive question, and it's your open question RoboFinSystems/robosystems#2 and RoboFinSystems/robosystems#4 combined.

   The corpus path — what SEC does. Filings → Arelle → intermediate tables → a graph of raw facts in the source vocabulary, served as a platform-managed public repository. This is the path your v1 flow describes.

   The framework path — how a reporting taxonomy becomes something tenants actually report under. A framework is a versioned bundle in frameworks/ pinning (package, version) and (bridge, version) tuples, loaded via robosystems/taxonomy/, with per-tenant machinery in robosystems/operations/taxonomy_block/ (reporting_standard.py, reporting_extension.py, chart_of_accounts.py). The data model is already built for jurisdictional variants: Taxonomy.extension_type accepts 'jurisdiction' (robosystems/models/extensions/taxonomy.py), and Element.source already accepts 'ifrs' (robosystems/models/extensions/element.py). Bridges map an external vocabulary into our canonical reporting taxonomy so a report can render in both.

   Your v1 target — "materialize into a normal user/demo graph" — sits between these. That's the ambiguity we'd want resolved before any code, because the two have almost nothing in common downstream:

   • If you mean Indian filings as a queryable corpus (the SEC analogue: many filers, raw facts, query the aggregate), the corpus path is right and your architecture is roughly correct modulo the layering fix above.
   • If you mean IND AS as a reporting framework an Indian entity reports under (one entity, its own chart of accounts, statements rendered in IND AS), that's the framework path — and it's mostly content: a frameworks/ind-as/ bundle plus a bridge, with far less new Python than the RFC anticipates. It would also be a materially smaller and more reviewable first contribution.

   Which one are you after? The answer changes Phase 2 entirely.

   To your question about existing non-US plans: yes, IFRS-adjacent frameworks are a direction we expect to go, and the mechanism we expect to use is the framework/bridge library rather than a per-jurisdiction ingestion adapter. That's why we'd like to pin down which path you mean before locking a package layout.

   Answers to your other open questions

   Taxonomy bundling (#5). Follow the SEC precedent: bundle, don't download at runtime. adapters/sec/arelle/bundles/ holds versioned tarballs (arelle-schemas-*.tar.gz, edgar-plugin-*.tar.gz) that ArelleClient unpacks into Arelle's cache, with offline mode on. Runtime taxonomy fetches are the thing that makes ingestion flaky and slow, and we don't want a second source of that.

   v1 entrypoint (#3). A just recipe over a CLI entrypoint is enough to start. Dagster only once there's a repeating pipeline worth scheduling — a one-shot ingest doesn't need it.

   Graph shape (#2). Deferred to the corpus-vs-framework answer above. If it's the corpus path, prefer a documented subset of the existing reporting nodes/edges over a parallel India-specific schema — a fork in query shape is the expensive kind of divergence.

   Suggested next step

   Rather than the xbrl skeleton PR, we'd start with:

   1. Answer the corpus-vs-framework question above.
   2. If corpus path: a no-behavior-change PR hoisting ArelleClient from adapters/sec/client/arelle.py into robosystems/arelle/, SEC calling through a wrapper, existing tests green. That stands alone, is easy to review, and is useful regardless of what happens with IND AS.
   3. Then the vertical slice against one real filing.

   On fixtures — agreed and appreciated that you raised it. Don't commit filings whose redistribution isn't clearly permitted; synthetic instances against the real taxonomy are fine for tests.

   Thanks again for taking the time to write this up properly rather than opening a large PR cold. Happy to keep iterating here.
4. MHa55anY commented on Aug 12, 2026

   

   MHa55anY

   on Aug 12, 2026

   Author

   More actions

   Thanks for the thoughtful reply — this cleared up a lot for me.

   On corpus vs framework: my long-term goal is the corpus path (SEC-style). I want Indian XBRL filings ingested and queryable so people can explore them easily, and eventually use MCP/AI to pull insights from the data — comparing companies, spotting trends, asking questions over filed facts. That's "understand what was filed," not "companies report through RoboSystems" as a first step.

   I get now that the framework path is the other side of the coin — IND AS as a reporting standard inside the product (frameworks/ind-as/, bridges, tenant reporting). That's complementary and lines up with the IFRS direction you mentioned; I'd treat it as a follow-on rather than what I'm aiming for in v1.

   On layering: fully on board with skipping adapters/xbrl/. Putting shared machinery in robosystems/arelle/ and keeping something like adapters/ind_as/ thin makes sense. Hoisting ArelleClient into robosystems/arelle/ with SEC calling through a wrapper sounds like the right first move.

   For v1, I'm still aligned with what we scoped before, with your notes folded in: S3/local upload only, bundled taxonomies (no runtime fetch), just/CLI to start, and reusing existing graph shapes where we can instead of inventing a parallel India schema. Still out of scope for v1: shared repo, MCP, crawling, full SEC parity.

   Sequencing — if this works for you:

   1. Lock in corpus as the direction (happy to tweak the RFC text if that helps)
   2. First PR: hoist ArelleClient, no behavior change, SEC wrapper, tests green
   3. Then a vertical slice on one real filing (upload → process → a few example queries)
   4. Framework/bridge and MCP once there's actually something worth querying

   On fixtures: won't commit filings unless redistribution is clearly OK; synthetic instances against the real taxonomy are fine for tests.

   If that ordering looks good, I'll start on the Arelle hoist — unless you'd rather framework content land first for sequencing reasons.
5. jfrench9

   transferred this issue fromRoboFinSystems/robosystemson Sep 14, 2026
6. jfrench9 commented on Sep 14, 2026

   

   jfrench9

   on Sep 14, 2026

   Member

   More actions

   Moving this to xbrlkit — and with an update that makes the RFC a good deal smaller.

   What changed since August

   The shared XBRL engine we discussed got extracted, but it went further than either of us planned in this thread: not into robosystems/arelle/, but out of the platform repo entirely, into xbrlkit (also on PyPI). RoboSystems now depends on it like any other library.

   Concretely, on the SEC side of the platform:

   • adapters/sec/client/arelle.py — the ~700-line ArelleClient you were going to hoist — is now 109 lines wrapping xbrlkit.parse.load_model.
   • adapters/sec/processors/xbrl_graph.py went 2,195 → 813 lines; the XBRL table schemas come from xbrlkit.schema, and the EDGAR plugin, its tarballs and the EFM validation config are gone.

   So the first PR we agreed on — hoist ArelleClient, SEC through a wrapper — no longer exists as a task. Good timing: you hadn't started it, so nothing is wasted. Sorry for not flagging it here when it landed.

   What that means for IND AS

   The neutral layer you argued for in this RFC exists, and it's jurisdiction-neutral by construction rather than by refactor. xbrlkit loads a filing from a local path, directory, zip or URL; it resolves DTS from a persistent cache; it already reaches non-US filings — ESEF and the national regimes on filings.xbrl.org, addressed by LEI rather than CIK. There is no EDGAR assumption in the load path to work around.

   That makes IND AS an xbrlkit question, not a RoboSystems one, which is why the issue is here now. It also collapses your v1:

   • No adapters/xbrl/, no adapters/ind_as/, no graph materialization. Those were all scaffolding for the separation that already happened.
   • v1 is: load a real IND AS filing and find out what breaks. Realistically that's DTS resolution and cache/bundle coverage for the Indian taxonomy hosts, packaging quirks, and the transform registry if MCA filings use their own ixt variants. Plus a fixture and a loader test.

   That's a genuinely small, self-contained first contribution with no infrastructure attached to it.

   And your actual goal gets closer than the RFC assumed

   You said the point is exploring filings and asking questions over them with MCP/AI. xbrlkit ships that already:

   uvx --from "xbrlkit[mcp]@latest" xbrlkit serve <path-or-url>
   

   is a local MCP server over the parsed filing — describe_filing, resolve_element, fact_grid, statement, calculation, search_text, read_text. No graph, no index, no hosting, no key. The moment an IND AS filing loads cleanly, that works on it, and single-filing analysis needs nothing else built.

   What that does not give you is the cross-filer corpus — many companies, aggregate queries, trends. That's the RoboSystems side, it's the expensive hosted part (manifest, storage, ops, billing), and it stays where you put it in the RFC: out of v1, a product decision rather than a contribution. It also gets much cheaper once the engine parses the filings, since the SEC pipeline already runs through xbrlkit.

   The framework path (frameworks/ind-as/ + a bridge, for entities reporting under IND AS) is unaffected by any of this and stays in RoboSystems. You said corpus, so it's a separate conversation whenever it's wanted.

   Where to pick it up

   Two things would help most:

   1. A filing to work against. Are MCA/exchange filings fetchable in any lawful way, or is this upload/local-path only? And can you point at one real instance — ideally one whose redistribution is clearly permitted, so it can be a fixture.
   2. A first PR: a loader test against an IND AS instance, plus whatever cache or taxonomy-package work it needs to resolve offline. If the answer turns out to be "it just loads", that's a fixture, a test and a line in the README — and the interesting work moves to whatever it gets wrong in the facts.

   Note the transfer dropped the RFC type and the area:adapters label (they don't exist in this repo). The design discussion above is intact and still the reference for this work.