docs: list the S3A settings the Iceberg native-write gate ignores

[andygrove]

Which issue does this PR close?

No issue. This follows up on #6808.

Rationale for this change

#6808 makes the native Iceberg write gate ignore five more S3A settings: fs.s3a.committer.name, fs.s3a.committer.magic.enabled, fs.s3a.committer.threads, fs.s3a.bulk.delete.page.size and fs.s3a.experimental.input.fadvise.

The user guide and the contributor guide both name the ignored settings, and both list only the vectored-read and downgrade.syncable.exceptions ones, so they would be out of date once #6808 merges. The S3A row of the user guide's eligibility table also says every other fs.s3a.* setting falls back, without pointing at the exceptions.

This describes the behavior #6808 adds, so it should merge after it.

What changes are included in this PR?

User guide (iceberg-writes.md):

• The S3A row of the eligibility table points at the ignored settings listed below it.
• The paragraph under the table lists all of the ignored settings and says Spark 4.1 sets two of them itself. It also notes that they are matched by their plain fs.s3a.<name> keys, so a per-bucket form still falls back.

Contributor guide (iceberg-writes.md):

• The gate description names IgnoredHadoopS3Keys and lists what is on it. It says when a key belongs there: only if S3A would not apply it to a data-file write request. Settings S3A does apply must keep falling back.

How are these changes tested?

Docs only, so there are no tests. npx prettier@latest --check passes on both files.

The matching rule (exact key, no per-bucket normalization) was read from unsupportedHadoopS3Settings and effectiveHadoopConf in CometIcebergNativeWrite, and the Spark 4.1 seeding from SparkContext.enableMagicCommitterIfNeeded in Spark 4.1.1. Neither was run.

[sunchao]

Reviewed the full diff at 1f4c269fd684471ad6945db1ebd6d7e1cc80fd37 against 00a4b422f0ed6560ef76dce004b94c3697613108. It contains one commit changing two documentation files. The PR is not a draft. The snapshot and current discussion contain no reviews, issue comments, inline comments, or review threads.

No introduced P1/P2 issues found within this review.

Summary

• Prior state and problem: Both Iceberg write guides list only the existing read-related S3A exceptions. They would omit the five additional exceptions proposed in #6808, and the eligibility table does not currently point readers to the exceptions.
• Design approach: Update the user-facing eligibility explanation and contributor guidance together, including exact-key matching and the criterion for adding ignored settings.
• Correctness: Checked IgnoredHadoopS3Keys, unsupportedHadoopS3Settings, effectiveHadoopConf, and surrounding detection tests. The five documented additions match #6808’s proposed list. Exact matching occurs before per-bucket classification, supporting the documented fallback for explicit target-bucket variants. No P1/P2 issue was identified in this documentation diff.
• Compatibility analysis: Spark sources for 3.4.3, 3.5.9, and 4.0.4 lack the automatic magic-committer setup. Spark 4.1.0, 4.1.1, 4.1.3, and 4.2.0 contain it. enableMagicCommitterIfNeeded checks the cloud classes and uses setIfMissing, with committer.name conditional on magic being enabled. Compared the relevant guides and gate against latest release branch branch-1.1 at e9efd9f764ee0a59b7898ff028d6985d4a7a28e1: the S3 eligibility checks are newer than that release and already present in this PR’s base. This PR adds no runtime behavior change.
• Key design decisions: Preserve the explicit allowlist and distinguish ignored settings from settings the native writer must honor. Encryption, ACL, storage-class, signing, and credential-provider settings remain subject to fallback.
• Implementation sketch: The user guide adds a table cross-reference and names the five exceptions. The contributor guide names IgnoredHadoopS3Keys and explains its admission rule and exact-key behavior.
• Performance: These Markdown edits introduce no execution, allocation, or planning overhead. Any reduction in unnecessary fallback belongs to #6808 and was not benchmarked here.
• Design: Keeping eligibility details beside the existing table and implementation guidance beside the gate description is straightforward and easy to follow.
• Abstraction & complexity: No new abstraction is introduced. Referencing the concrete allowlist connects the contributor guidance directly to the implementation without adding machinery.
• Behavioral changes worth calling out: #6808 remains open, and its five additional ignored keys are absent from this exact head’s executable code. The documentation therefore depends on the merge order explicitly stated in the PR: #6808 first. The prerequisite implementation is not part of this base-relative diff.
• Suggested improvements: No additional change meeting the P1/P2 reporting bar was identified. Preserve the stated prerequisite merge order.

Routed skills: .ai/skills/review-comet-pr/SKILL.md and .ai/skills/review-comet-iceberg-write-pr/SKILL.md. Read AGENTS.md and the relevant contributor guidance.

Exact-head CI: 23 completed checks, with 7 successful and 16 skipped. No failures or running checks. Successful checks include Required Checks, Preflight, and CodeQL. Linux/macOS builds, Spark SQL suites, Iceberg suites, and site deployment were skipped, so these results provide no runtime-suite verdict.

Validation limits: git diff --check passed and the checkout remains unchanged. Validation used source inspection, including Spark and Hadoop 3.3.4/3.4.1 sources. No JVM/native tests, documentation build, benchmarks, or live S3 writes were run for this documentation-only diff.