Skip to content

perf(cluster): pass canonical edges directly to native Leiden - #5401

Open
Max17190 wants to merge 1 commit into
Graphify-Labs:v8from
Max17190:perf/native-leiden-edge-input
Open

Max17190 wants to merge 1 commit into
Graphify-Labs:v8from
Max17190:perf/native-leiden-edge-input

Conversation

@Max17190

@Max17190 Max17190 commented Nov 22, 2026 •

Copy link
Copy Markdown
Contributor

What does this PR do?

Why

Graphify's native Leiden path builds a second NetworkX graph and copies edge attributes into it. The next step reads that graph back as weighted endpoint triples. The stable graph matters for fallback backends. Native Leiden only needs the sorted triples and deterministic isolate ordering.
Closes #5405.

Summary

  • Send the native backend the canonical weighted edges directly for simple, undirected graphs with string node IDs.
  • Preserve endpoint orientation, edge order, weights, seed, and isolate numbering.
  • Construct the existing stable NetworkX graph when the optimized input does not apply or a fallback backend needs it.

Type of change

  • Refactor

Verification & Invariants

The optimized path passes the same ordered triples to the same Leiden call. Loaded-graph comparisons preserve exact partition IDs and complete community output. A differential sweep compares 3,000 randomized graphs and 135 edge-attribute, graph-shape, and fallback cases against v8, across three hash seeds.

  • Read the contribution guide and architecture notes.
  • Reproduced the issue or profiled the existing path and verified the affected invariant.
  • Kept the change scoped to one concern and added a regression or boundary test.
  • Documented limitations and checked the description against the implementation.

Benchmarks

Apple M5, CPython 3.13.9, NetworkX 3.7, and graspologic-native 1.3.1. Each row is five interleaved baseline/candidate pairs with a fresh output directory seeded by the same graph.json. Stdout, stderr, and SHA-256 of all five output artifacts match across both versions on each corpus.

Corpus Nodes / edges Baseline median [min, max] Candidate median [min, max] Peak RSS, median
Graphify 27,097 / 57,974 4.273 s [4.207, 4.391] 3.896 s [3.810, 3.957] 398.0 → 367.3 MiB
Django 55,368 / 132,637 9.961 s [9.520, 11.727] 9.378 s [8.872, 12.840] 757.3 → 733.5 MiB
Guava 84,232 / 252,446 18.562 s [17.637, 18.799] 17.109 s [16.191, 17.767] 1,387.9 → 1,358.7 MiB

Graphify and Guava improve in all five pairs, with median elapsed reductions of 8.8% and 7.8%. The Django CLI result is inconclusive. With the graphs already loaded, median weighted cluster() time fell by 34% to 35% on all three corpora.

How was this tested?

Test Plan

  • Ran the clustering tests and a differential comparison against v8. The focused clustering suite passed 34 tests with one optional-wrapper skip.
  • Ran the clustering, incremental, and update suites with the frozen Python 3.12 dependencies. 59 passed, including parity with the real graspologic wrapper. One Python 3.13-specific case was skipped.
  • Ran graphify cluster-only . --no-label using a fresh partition cache for each measured invocation.
  • Ran ruff check . and graphify update ..
  • Verified that the change applies cleanly with perf(cluster): count community overlaps from shared nodes #5382.
  • Full local pytest run yielded 11,816 passes, 134 skips, one expected failure, and seven failures also reproduced on v8. Six are Terraform copied-cache cases and one is the CLI source-alias exclusion case.

Graphify-specific checklist

  • Verified deterministic structural output and reviewed the changed paths for security implications.
  • Checked that the patch contains no credentials or local graph artifacts.
  • Generated skill artifacts are out of scope. The generator checks pass.
  • Disclosed AI authorship in the commit metadata.

Build the ordered weighted triples directly for simple string-keyed
graphs. Keep isolate numbering and native partition output stable, and
construct the canonical NetworkX graph on demand for fallbacks.

Builds on the partition-cache work in f51d46c and preserves Graphify-Labs#3146.

Co-Authored-By: Codex <noreply@openai.com>
@Max17190
Max17190 requested a review from safishamsi as a code owner November 22, 2026 21:31
@github-actions

Copy link
Copy Markdown

Thanks for the pull request, @Max17190. A maintainer will review it soon.

Want to talk it through while it is in review? Come join us on our Discord server. For longer-form discussion there is also GitHub Discussions.

A couple of things that speed up review: make sure the test suite passes on Python 3.11 and 3.13, and that the change keeps extraction deterministic.

@github-actions github-actions Bot added the contributor Pull request from a returning community contributor label Nov 22, 2026

@graphify-labs graphify-labs Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Graphify review View in Graphify

Nothing blocking · 2 possible issues to check · Merge confidence 5/5

2 possible issues to check · High impact

The partitioner now prepares canonical Leiden edge tuples directly for ordinary graphs with string node IDs, building the stable graph only when that fast path cannot be used.

2 possible issues we couldn't confirm
  • Partitioning can crash on non-numeric edge weights that the old sort key accepted · graphify/cluster.py:277
  • Canonical-edge test asserts inside the fake native Leiden, so a mismatch can be swallowed by the fallback path · tests/test_cluster.py:209

They don't change the score.

Code graph · 860 functions depend on this change
  • 1 function became more coupled. _partition() (4 callers).
  • 23 of 613 test files are affected by this change.
  • Gate, test list and every flagged function: in the Graphify check.

Reviewed 0e2439f · Full report in the Graphify check

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

contributor Pull request from a returning community contributor

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature]: Skip the extra graph copy before native Leiden

1 participant