Repository navigation

feat: project Spark 4 VARIANT columns in native Parquet scans [Splitting] - #5407

Closed

peterxcli wants to merge 29 commits into

apache:mainfrom

peterxcli:feat/native-variant-proj

Commits on Aug 21, 2026
- feat: project Spark 4 VARIANT columns in native Parquet scans
  peterxcli
  committed
  45a0ed4
Commits on Aug 22, 2026
- review
  peterxcli
  committed
  784c316
Commits on Aug 23, 2026
- review
  peterxcli
  committed
  c355fef
- add link about spark and arrow-rs issue that need to be fixed so we can get cleaner code
  peterxcli
  committed
  c556a52
- update spark issue link
  peterxcli
  committed
  9742a7b
Commits on Aug 24, 2026
- Merge branch 'main' into feat/native-variant-proj
  peterxcli
  authored
  0a079de
- widen unsigned shredded field
  peterxcli
  committed
  33e513c
Commits on Aug 25, 2026
- fix: preserve Spark Variant compatibility when unshredding
  peterxcli
  committed
  ed39c4c
Commits on Aug 26, 2026
- Merge remote-tracking branch 'upstream/main' into feat/native-variant-proj
  peterxcli
  committed
  8fb17b1
- fix: normalize shredded Variant types before unshredding
  peterxcli
  committed
  360a653
- refactor: isolate Parquet Variant normalization
  peterxcli
  committed
  a800bc8
- fix: preserve strict Variant Parquet validation
  peterxcli
  committed
  ba37a68
- review
  peterxcli
  committed
  1e832c1
- fix: normalize remaining Variant scan encodings
  peterxcli
  committed
  a784bea
Commits on Aug 27, 2026
- fix: align native Variant Parquet scan semantics
  peterxcli
  committed
  389b134
- style: remove redundant interpolation
  peterxcli
  committed
  43e7e17
- fix: preserve Spark semantics for Variant scans
  peterxcli
  committed
  23bfd83
- fix: preserve Parquet Variant physical semantics
  peterxcli
  committed
  22f588a
- Merge branch 'main' of https://github.com/apache/datafusion-comet into feat/native-variant-proj
  peterxcli
  committed
  c2778b7
- fix: narrow Variant Unicode scan fallback
  peterxcli
  committed
  a352f46
Commits on Aug 28, 2026
- test: cover entirely null Variant columns end to end
  
  peterxcli
  and
  claude
  committed
  54cfd1f
- fix: name pushVariantIntoScan in the Variant scan fallback reason
  
  peterxcli
  and
  claude
  committed
  0cc63a7
- docs: apply prettier formatting to datatypes table
  
  peterxcli
  and
  claude
  committed
  ba7677c
- fix: fail closed on Unicode-folded physical names in Variant scans
  
  peterxcli
  and
  claude
  committed
  e78da57
- fix: restrict the Unicode fold guard to referenced scan columns
  
  peterxcli
  and
  claude
  committed
  53c58be
- fix: exempt ID-resolved columns from name-duplicate validation
  
  peterxcli
  and
  claude
  committed
  e40b939
- fix: match Spark's fold-ambiguity and field-ID shadowing semantics
  
  peterxcli
  and
  claude
  committed
  4925cc2
Commits on Aug 29, 2026
- Merge branch 'main' into feat/native-variant-proj
  peterxcli
  authored
  429d598
- cargo fmt
  peterxcli
  committed
  519b281

Commits

Commits on Aug 21, 2026

Commits on Aug 22, 2026

Commits on Aug 23, 2026

Commits on Aug 24, 2026

Commits on Aug 25, 2026

Commits on Aug 26, 2026

Commits on Aug 27, 2026

Commits on Aug 28, 2026

Commits on Aug 29, 2026