Native Iceberg writes: count the DataFiles a task keeps in its memory reservation #6885

What is the problem the feature request solves?

The native Iceberg writer reserves what its open files and partition feeds hold between batches (#6247), but not the DataFiles of the files it has already closed, which it keeps until the task ends to build its output manifest. Each one holds the file's per-column sizes, counts and bounds. Measured by cloning one under an allocation counter, a DataFile takes about 1 KB for a 3-column table and 25 KB for a 101-column one.

Rolled files have always had this gap. #6773 widens it, because a fanout write the pool refuses closes partitions early and ends with about (k + 1) / 2 files per partition at a pool 1/k of what it needs. In a 101-column write of 64 partitions given a quarter of the memory it needed, the 89 files closed early kept 2.2 MB of DataFiles until the task ended, about a fifth of its 10 MB pool, none of it reserved. Each close still freed far more than it kept, a median of 330 KB.

Describe the potential solution

Add an estimate for each kept DataFile to the reservation when its file closes, from its column count and the size of its bounds. Closing partitions can't give that memory back, so InnerWriter::reserve would have to leave it out of what it expects a close to free. A write could then fail on a refusal if the kept files alone outgrew the pool, which can't happen today because everything a fanout write reserves belongs to a partition it can close.

Additional context

Raised by @unikdahal in the review of #6773: #6773 (comment)