Experiment: heterogeneous depth and breadth retrieval across GPU and CPU tiers #1008

Finding

Hybrid GPU-CPU Retrieval for Personalized Search at Ultra-Large Scale (arXiv:2609.21281, submitted 2026-09-18) reports a production architecture that separates modeling depth from inventory breadth rather than forcing one ANN tier to optimize both.

A high-depth GPU branch retrieves and interaction-pre-ranks over an independently selected inventory on the order of one billion documents. A high-breadth CPU branch searches a separately versioned inventory roughly twenty times larger with lighter personalized scoring. Branches run in parallel with independent deadlines, deduplicate before a shared ranker, and can fail or roll back independently.

Evidence class: originating-team production disclosure with large-scale A/B evidence, not independently reproduced by RuVector.

Reported production results include a nine-day hybrid treatment improving model-scored DCG@20 by 4.51% and substantive-engagement GSRR by 2.01% versus a legacy all-CPU configuration. The dedicated CPU indexing path separately reports 89.45% lower retrieval-stage compute in one offline comparison and 63.5% lower CPU use at the selected 512k-centroid operating point without detected GSRR regression in a 15-day experiment. The paper also reports a matched broad-inventory capacity plan where accelerator capacity cost is roughly four times CPU capacity cost.

These values are treatment-level results and do not isolate inventory breadth, model depth, filters, or individual kernels.

Opportunity score

4.56 / 5 using the RuV SOTA weighting model.

RuVector hypothesis

The strategic primitive is not a Meta-specific recommender. It is a heterogeneous retrieval contract:

1. independent inventory selection and versioning per tier;
2. high-depth accelerator retrieval where interaction quality justifies cost;
3. high-breadth CPU or disk-backed retrieval for rare-tail coverage;
4. parallel deadlines and failure isolation;
5. candidate source attribution and deterministic deduplication;
6. a shared downstream ranking interface;
7. a request router that can activate either or both branches under a fixed latency/cost envelope.

RuVector already has vector, graph, WASM, and heterogeneous-runtime primitives. This experiment should test whether explicit depth/breadth separation improves RuV's cost-recall-quality frontier versus one homogeneous index configuration.

Frozen arms

1. Strongest current homogeneous RuVector configuration.
2. CPU/disk breadth tier only.
3. GPU/deep tier only where compatible hardware exists.
4. Parallel heterogeneous depth plus breadth with deterministic aggregation.
5. Retrospective oracle branch selection.

Do not infer GPU results on unsupported hardware. Run CPU-only fallback experiments where GPU hardware is unavailable and record INCONCLUSIVE for accelerator claims.

Required measurements

Report exact commit, dataset, corpus size, vector dimensions, index parameters, quantization, hardware, OS, runtime versions, seeds, sample size, Recall@10/20, nDCG, rare-tail recall, candidate-source overlap, p50/p95/p99 latency, QPS, memory/HBM/DRAM footprint, CPU-seconds/query, accelerator time/query, bytes read, index-build cost, refresh cost, provider/infrastructure cost, energy where measurable, failures, timeout behavior, and reproduction commands.

Falsification

1. Protect rare decisive evidence as a separate metric; aggregate recall can hide tail loss.
2. Run an equal-cost comparison, not only equal-hardware capacity.
3. Measure branch timeout and one-branch failure.
4. Test stale or mismatched model/index versions.
5. Test duplicate-heavy inventories and source-attribution correctness.
6. Compare against a simpler single-tier increase in breadth.
7. Record whether aggregation/ranking overhead erases parallel serving gains.

Acceptance

Return ACCEPT only if heterogeneous serving delivers at least one of:

1. 10% relative Recall@20 improvement at matched p95 latency and cost;
2. 20% lower serving cost at matched recall/quality;
3. 15% better rare-tail recall with no more than 2% relative aggregate quality regression;

while retaining deterministic source attribution, independent branch rollback, and no cross-tenant candidate leakage.

Cross-stack mapping

RuVector: tiered retrieval, aggregation, attribution, versioned inventories.

RuVector WASM: lightweight local breadth branch or request-side aggregation where practical.

RuFlo: request routing under latency/cost/resource envelopes.

MetaHarness: frozen equal-cost evaluation and rare-tail protection.

RVM: resource and effect ceilings, never retrieval-derived authority.

RVF/RVForge: bind model, index, selector, and evaluation versions.

Autogenous/Dream Machine: may optimize branch selection only after baselines freeze.

MidStream: branch timing, timeout, and source-attribution events.

RuView/RuField/WorldGraph: heterogeneous local/accelerated spatial retrieval.

LatentMesh: distributed branch transport and failure isolation.

Cognitum: tenant-aware cost/latency routing and gross-margin control.

MCP: retrieval tools expose branch/source metadata without granting authority.

Governance

Reversible experiment only. No autonomous merge, deployment, resource escalation, acceptance-threshold mutation, or irreversible index migration. Preserve the homogeneous path as rollback.