[MincongZhou]

Problem

The calibration-setup step that sizes the captured-activation buffer exists three times in this package, and only the fp8 copy got it right:

	file:line	expression
correct	angelslim/compressor/quant/modules/fp8/fp8.py:78	len(dataloader) * dataloader.batch_size
wrong	angelslim/compressor/quant/modules/int8/int8.py:77	len(dataloader)
wrong	angelslim/compressor/quant/modules/awq/awq.py:125	len(dataloader)

Catcher stores one entry per sample — its own docstring on _capture_samples says "Splits a batched input into per-sample tensors", and the loop is for i in range(inp.shape[0]): ... captured_inputs.append(sample). So len(captured_inputs) is len(dataloader) * batch_size.

int8/awq then allocate self.inps with the smaller len(dataloader) and copy with

for idx in range(min(len(captured_inputs), self.inps.shape[0])):   # int8.py:92

which silently discards every sample past the first batch, before any observer sees it. Two consequences:

1. The activation scales for int8 and AWQ are calibrated on 1/batch_size of the requested data.
2. print_info(f"nsamples:{nsamples}") reports the truncated number, so the run log states the wrong calibration-set size — the only signal the user gets is a number that looks intentional.

Reproduction

Executed against the real Catcher from this repo (96-sample dataset, batch_size=8, so len(dataloader)=12):

dataset: 96 samples, batch_size=8 -> len(dataloader)=12
Catcher captured (per-sample)          : 96
int8  inps rows = 12   fill loop keeps 12   -> DROPS 84 samples before any observer sees them
awq   inps rows = 12   fill loop keeps 12   -> DROPS 84 samples
fp8   inps rows = 96   fill loop keeps 96   -> correct

--- after this change ---
int8  inps rows = 96   fill loop keeps 96   -> DROPS 0 samples
awq   inps rows = 96   fill loop keeps 96   -> DROPS 0 samples
fp8   inps rows = 96   fill loop keeps 96   -> correct

The same input yields 12 calibrated samples on int8/AWQ and 96 on fp8 today.

Provenance

git log on the two files says this is a migration leftover, not an intentional difference:

• int8/int8.py was last touched by fix(quant): migrate int8/fp8/awq/lepto_fp8 to new Catcher API (regression from #318) #352 "migrate int8/fp8/awq/lepto_fp8 to new Catcher API" — the commit that moved these modules from a batched capture to Catcher's per-sample capture (regression from fix(gptq): fix Hessian computation, variable-length sequence support, and layer output type handling #318), and that kept the pre-migration len(dataloader).
• fp8/fp8.py received the * dataloader.batch_size form earlier, in add DeekSeek and Kimi-K2; support per-tensor and w4a8; support batch calibration in fp8 low_memory_run; support multiple nodes #29 "support batch calibration in fp8 low_memory_run".

So the fix is to make int8 and AWQ agree with fp8.

Scope

nsamples is used only as an allocation length and as a min() bound, so this cannot index out of range. At batch_size = 1 the two expressions are equal — and every low_memory: true config shipped under configs/ uses batch_size: 1 — so this is a no-op for the existing recipes and only changes runs that ask for batched calibration, which is exactly the case that is broken today. dataloader.batch_size is already dereferenced unconditionally by the fp8 copy.

Validation

• Two-line change, no control flow touched.
• python -m py_compile on both files: OK.
• The repository's own pre-commit tools, at the pinned versions from .pre-commit-config.yaml:
  • black --line-length 99 --check → 2 files would be left unchanged
  • isort --profile=black --check-only → clean
  • flake8 --ignore=E203,E704,W503,W504 --max-line-length 99 → exit 0
• The reproduction above, before and after, using the real Catcher class imported from this repository.

Not run: an end-to-end PTQ to compare the resulting quantized accuracy (needs model weights, a real calibration dataset and a GPU). The claim here is limited to what the code does: the buffer is sized for one batch instead of the whole capture set, and the surplus is dropped before the observer loop.

Competition

No issue or PR mentions this. gh api search/issues for int8 low_memory → 0 results; for nsamples → 2 (the fp8 fix #29 and an unrelated #318). The 4 currently open PRs (#351, #346, #319, #305) touch other areas. The one open issue in this neighbourhood (#48) is about FP8 low-memory and W4A8 and does not mention sample counting.

[yghstill]

@MincongZhou Please refacing code format:

pre-commit install
pre-commit run --all-files

[MincongZhou]

Applied the repo's own pre-commit config: black 24.4.0 --line-length=99, isort 5.13.2 --profile=black, flake8 7.0.0.

The diff is whitespace-only in tests/test_quant_low_memory_nsamples.py (12+/6-): three over-long lines rewrapped, no semantic change. Verified the whole tree locally — 485 files pass black / isort / flake8.

Heads-up: the two workflows on the new head (fd50303) are sitting in action_required, so they need approval from a maintainer before they start.