[chrikrah]

#217 renamed src/vllm_router/batch/ to src/vllm_router/services/batch_service/ in 95efecc, on 2025-03-04. local_processor.py moved with it and kept importing BatchInfo and BatchStatus from the old path, at line 24 and again inside retrieve_batch at line 127. The module it wants is now its own sibling.

$ gh api 'repos/vllm-project/production-stack/commits/95efecc' --jq '.files[] | select(.filename|test("batch")) | "\(.status) \(.filename) <- \(.previous_filename)"'
renamed src/vllm_router/services/batch_service/batch.py <- src/vllm_router/batch/batch.py
renamed src/vllm_router/services/batch_service/local_processor.py <- src/vllm_router/batch/local_processor.py

app.py:254 calls initialize_batch_processor whenever a user passes --enable-batch-api. That function imports LocalBatchProcessor, so the router aborts at startup:

$ python -c "from vllm_router.services.batch_service import initialize_batch_processor; initialize_batch_processor('local', '/tmp/store', None)"
  File ".../services/batch_service/local_processor.py", line 24, in <module>
    from vllm_router.batch.batch import BatchInfo, BatchStatus
ModuleNotFoundError: No module named 'vllm_router.batch'

#784 fixed the same orphaned import one layer out, in services/batch_service/__init__.py, and stopped there.

Line 22 of the same file imports aiosqlite. Neither pyproject.toml nor src/vllm_router/requirements.txt declares it, so a stock install fails one line earlier. router-e2e-test.yml:47 installs from pyproject.toml and line 80 collects src/tests/test_*.py, which now includes the new file, so this branch declares it in both places.

Two regression tests cover the path app.py takes. One builds the processor through initialize_batch_processor. The other creates a batch and reads it back. Run against the pinned aiosqlite==0.21.0:

$ python -m pytest src/tests/test_batch_service.py -q
2 passed in 0.04s

Revert the import change, keep the tests, and both fail:

2 failed in 0.04s
FAILED src/tests/test_batch_service.py::test_initialize_local_batch_processor
FAILED src/tests/test_batch_service.py::test_create_and_retrieve_batch

Correcting only the module-level import leaves the second one failing, which is what the line 127 edit is for.

No open issue covers this, so there is nothing to link. #766 reported the sibling defect and #784 closed it.

---

• Make sure the code changes pass the pre-commit checks.
• Sign-off your commit by using -s when doing git commit
• Try to classify PRs for easy understanding of the type of changes, such as [Bugfix], [Feat], and [CI].

One limit: no live vLLM backend served these endpoints, so this covers startup and the SQLite round trip only. cancel_batch also writes the literal "cancelled", which BatchStatus has no member for, so retrieve_batch raises ValueError afterwards. That is a separate defect and stays out of this change.

[gemini-code-assist]

Code Review

This pull request adds aiosqlite as a dependency, introduces unit tests for the batch service, and updates imports in the local batch processor. Feedback on the changes suggests using pytest's built-in tmp_path fixture instead of a hardcoded temporary directory to prevent potential test flakiness, and removing a redundant local import of BatchStatus in the local processor.

[gemini-code-assist]

Using a hardcoded path like /tmp/test_vllm_batch and manually cleaning it up can lead to flaky tests, permission issues, or conflicts during concurrent test execution (e.g., in CI/CD environments). It is highly recommended to use pytest's built-in tmp_path fixture, which automatically creates a unique temporary directory for each test run and handles cleanup.

pytest_plugins = ("pytest_asyncio",)


def test_initialize_local_batch_processor(tmp_path):
    """app.py calls this whenever --enable-batch-api is passed, so an import
    error inside local_processor aborts router startup."""
    storage = FileStorage(os.path.join(tmp_path, "files"))
    processor = initialize_batch_processor(
        "local", os.path.join(tmp_path, "db"), storage
    )
    assert processor.db_path.endswith("batch_queue.db")


@pytest.mark.asyncio
async def test_create_and_retrieve_batch(tmp_path):
    storage = FileStorage(os.path.join(tmp_path, "files"))
    processor = initialize_batch_processor(
        "local", os.path.join(tmp_path, "db"), storage
    )
    await processor.setup_db()

    created = await processor.create_batch(
        input_file_id="file-1",
        endpoint="/v1/chat/completions",
        completion_window="24h",
    )
    retrieved = await processor.retrieve_batch(created.id)

    assert retrieved.id == created.id
    assert retrieved.status is BatchStatus.PENDING
    assert retrieved.input_file_id == "file-1"

[gemini-code-assist]

The local import of BatchStatus is redundant because it is already imported at the module level on line 25. Removing it simplifies the code and avoids unnecessary import overhead.

Suggested change

	# Convert status string to BatchStatus enum.
	from vllm_router.batch.batch import BatchStatus
	from vllm_router.services.batch_service.batch import BatchStatus
	# Convert status string to BatchStatus enum.

[chrikrah]

Both taken, in 78962ad.

tmp_path replaces the hardcoded directory and its cleanup fixture. The hardcoded form came from the sibling test_file_storage.py, which does the same thing with /tmp/test_vllm_files, so this file no longer matches that one. tmp_path is the better pattern and it does not collide when CI runs tests concurrently.

The local BatchStatus import inside retrieve_batch existed to work around the broken module-level one. With that corrected it is dead weight, so it is gone rather than corrected in place.

Replacing the whole file with the origin/main copy still gives ModuleNotFoundError on both tests, so the module-level import is what they pin.

[Shaoting-Feng]

LGTM

[chrikrah]

This one is ready whenever it suits you. Approved on 2026-09-18, MERGEABLE and CLEAN, 16 checks green, and it has not moved since.

Nothing is owed from my side, so this is only a note that it is not waiting on me. The other three from me here, #1094, #1095 and #1097, all sit at BLOCKED on review, and they touch different files, so none of them blocks this one.