Repository navigation
[python] Use Native planning and reads for REST query authorization - #10539
Merged
JingsongLi merged 1 commit intoOct 13, 2026
Merged
Conversation
leaves12138
approved these changes
Oct 13, 2026
leaves12138
left a comment
Contributor
There was a problem hiding this comment.
Reviewed the complete change at b3c503b. The Python regression tests and native plan/read authorization tests passed; no correctness issues found.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Dependency
Depends on apache/paimon-rust#1122. Keep this PR in draft until that Rust change is merged. Python Native CI continues to build paimon-rust main; the new Native authorization tests need the corresponding Rust core implementation.
What changes
Use Native planning and reads for existing REST query-authorized batch reads. Support the Java QueryAuthSplit type-5 wire format so the plan carries the captured policy alongside its data/indexed split metadata. Only original Native splits retain a live Rust authorization grant; a serialized plan or Native reader setup failure falls back through Python with the captured rules.
Align Python execution with Java ReadTransform: authorization filter on original values, column masks, the user predicate, LIMIT, then output projection. Remove masked columns from raw partition/stats/PK/bucket pruning, include hidden query fields and their mask dependencies, and include hidden query/partition operands in the REST authorization request. Scan-only partition selectors referencing masked fields are rejected as Java requires.
Fix PAD/OVERLAY integer validation and eager overflow checks despite NULL string inputs, and validate stored field families for the new string transforms. Keep row-kind metadata outside positional predicate operands.
No new public Python or Rust binding API is introduced. The new mask execution stays in Rust core.
Verification
Scope
This enables REST batch authorization through Native for the supported Rust rules. Transformed authorization row filters and streaming authorization remain separate follow-up work. Explicit authorized partition selectors remain on Python until the binding can represent their scan-only semantics.