Skip to content

Python: replay an empty local_shell_call with its matching shell output - #10155

Open
DZH166 wants to merge 6 commits into
microsoft:mainfrom
DZH166:fix/empty-local-shell-call-replay
Open

DZH166 wants to merge 6 commits into
microsoft:mainfrom
DZH166:fix/empty-local-shell-call-replay

Conversation

@DZH166

@DZH166 DZH166 commented Nov 6, 2026 •

Copy link
Copy Markdown

Fixes #10373

Problem

RawOpenAIChatClient._shell_item_to_contents converts a provider local_shell_call whose action.command list is empty into a shell_tool_call carrying the raw local item, and appends no terminal result. Two things then break:

  • stateless replay goes through the managed-item route (_prepare_shell_transcript_item_for_openai expects a ResponseFunctionShellToolCall), so it raises ChatClientInvalidRequestException: Stateless replay cannot reconstruct shell_call ...;
  • service-side replay loses the empty shell_call_output because the recorded item never had a matching output.

The sibling managed shell_call branch already records shell_tool_call plus a paired empty shell_tool_result (the orphaned-call fix); the local branch was left behind.

before: contents == ['shell_tool_call']            (expected ['shell_tool_call', 'shell_tool_result'])
        stateless replay raises ChatClientInvalidRequestException
        replay emits 0 shell_call / 0 shell_call_output items
after : contents == ['shell_tool_call', 'shell_tool_result']
        stateless replay emits the empty shell_call plus its shell_call_output
        service-side replay keeps exactly one empty shell_call_output

Fix

Record the paired empty shell_tool_result for an empty local_shell_call, using the same marker as the managed branch, so history stays replayable.

A local shell call that does run commands still converts to a function_call, unchanged.

Tests

tests/openai/test_openai_chat_client.py gains tests for the recorded contents, stateless replay, and service-side replay of an empty local shell call.

pytest tests/openai/test_openai_chat_client.py -k empty_local_shell_call -q    # before: 4 failed   after: 4 passed
pytest tests/openai/test_openai_chat_client.py -k shell -q                     # 78 passed, 1 pre-existing env failure (private openai._make_tools import)

A local_shell_call with an empty command list was converted to a lone shell_tool_call with no matching result, so stateless replay raised ChatClientInvalidRequestException and service-side replay lost the empty shell_call_output. Record the paired empty shell_tool_result, as the managed shell_call branch already does.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Service-side continuation serializes local-shell results with the wrong output type and identifier, requiring preservation of the native call kind and item ID and test assertions for the correct wire format.

1 open issue
What changed in this PR

This PR updates the Python OpenAI client to record empty provider local_shell_call items as paired shell calls and results, aiming to make their history replayable.

Changes:

  • Normalize empty local shell calls to a replayable shell-call representation and append an empty result.
  • Add regression cases for stateless replay, service-side continuation, and result serialization round trips.
File Description
python/​packages/​openai/​tests/​openai/​test_openai_chat_client.py Adds parameterized empty local-shell replay tests.
python/​packages/​openai/​agent_framework_openai/​_chat_client.py Normalizes empty local shell calls and records paired empty results.

Review effort: Balanced

Comment on lines +2862 to +2865
Content.from_shell_tool_result(
call_id=local_call_id,
outputs=[],
additional_properties={_LOCAL_TOOL_RESULT_KEY: True},

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reading the serializer again, the mismatch you describe is real: the call content keeps the local-shell metadata (local_shell_call_output + the provider item id) that this branch writes before the raw_representation swap, so it still leaves through _prepare_shell_transcript_item_for_openai(..., expected_type="local_shell_call") (line 2353), while the paired result I add carries only _LOCAL_TOOL_RESULT_KEY and therefore leaves through the shell_tool_result arm as shell_call_output keyed by call_id (line 2384). So the pair can come out as local_shell_call + shell_call_output.

That is exactly the inconsistency to fix, but the fix changes what the service-side continuation stores, and the two candidates are not equivalent:

  1. give the result the same local metadata as the call (OPENAI_SHELL_OUTPUT_TYPE_KEY / local_shell_call_item_id), so the pair stays local_shell_call + local_shell_call_output keyed by the item id, and drop the raw_representation rewrite; or
  2. keep the rewrite, and give both the call and the result the shell_call metadata so the pair is shell_call + shell_call_output keyed by call_id.

Option 1 matches the native item the service already stores; option 2 matches the "an empty call that ran no commands is equivalent to shell_call" comment. My concern with 1 is that it puts back the empty local_shell_call payload that stateless replay rejects today (ChatClientInvalidRequestException), which is what this PR exists to avoid — so 1 only works if the empty action is replayed differently for a continuation than for a stateless request.

I don't want to guess at the wire contract: the repo tests can assert either shape, so whichever one the service actually accepts decides it. I've asked in the thread; if option 1 is the intended contract I'll swap the metadata and update test_empty_local_shell_call_item_replays_with_empty_output to assert local_shell_call_output keyed by the item id, and if option 2 is right the result needs the shell_call metadata instead of the current bare _LOCAL_TOOL_RESULT_KEY. Point me at the expected payload and I'll land it either way.

@DZH166
DZH166 temporarily deployed to github-app-auth November 8, 2026 20:06 — with GitHub Actions Inactive
@eavanvalkenburg

Copy link
Copy Markdown
Member

DZH166 Thanks for the contribution. This changes function-call history and replay behavior in a stable contract area, so it needs an agreed issue with explicit maintainer direction before implementation review resumes. Please link that issue and confirm the intended replay behavior against the function-calling loop contract. Approval remains blocked until that direction is established.

@DZH166
DZH166 deployed to github-app-auth November 12, 2026 03:30 — with GitHub Actions Active
@DZH166

DZH166 commented Nov 12, 2026

Copy link
Copy Markdown
Author

Status note after the latest rebase, because the related issue record changed while this PR was open:

So the question is whether the remaining gap is in scope after #9803: if it is, I will open a focused issue (state snapshot style, with a reproduction that does not need a live deployment) and link it here; if #9803 is considered to have covered this path, feel free to close this PR as redundant — I would rather not argue the point with a merged fix in the tree.

Everything else is green on the rebased head; the earlier Pre-commit Hooks (3.11) failure was an Unexpected HTTP response: 504 from the action download step, not a lint result.

# emits on replay. A call that ran no commands is equivalent to a
# ``shell_call`` with an empty action, so record that shape and pair it with
# the terminal output below instead of leaving an orphaned shell call.
raw_representation=(

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

DZH166 Persist the reconstructed call for history replay — The reconstructed shell call exists only in raw_representation, which Content.to_dict() omits. After the whole message is serialized and restored, stateless replay raises ChatClientInvalidRequestException because it cannot reconstruct the call. The new round-trip test serializes only the result, so it misses this path. Preserve sufficient call data across message serialization and test the complete message round trip.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Implemented in dcc0045, on top of the focused issue you asked for (#10373):

  • the empty local_shell_call now records the provider item id in additional_properties (which Content.to_dict() does serialize) instead of relying on raw_representation;
  • _prepare_shell_transcript_item_for_openai() falls back to _rebuild_shell_call_payload() when raw_representation is None, rebuilding {type, call_id, status, action.commands} plus the item id and environment: {"type": "local"}, i.e. exactly the payload this branch already emits in memory and the provider already validates;
  • test_empty_local_shell_call_item_replays_with_empty_output now round-trips the whole message (Message.from_json(message.to_json())) instead of only the result, so it exercises the persisted-conversation path that used to raise ChatClientInvalidRequestException.

The pair still serializes as shell_call + shell_call_output keyed by call_id (option 2 from my earlier note), which is the shape the empty-action path can rebuild from typed fields alone.

@eavanvalkenburg

Copy link
Copy Markdown
Member

DZH166 The remaining local_shell_call replay gap is in scope, but it is distinct from the completed #9697/#9803 work because current main still has a separate provider-item serialization path that can lose the paired history. Please open the focused issue you proposed with the current-main reproduction and intended wire/replay contract, then link it here. Keep this PR aligned with that issue and the function-calling loop contract.

…ialization

`Content.to_dict()` drops `raw_representation`, so a call rebuilt for an empty
`local_shell_call` vanished once the message was persisted: replaying the restored
message raised `ChatClientInvalidRequestException: Stateless replay cannot
reconstruct shell_call`.

Record the provider item id in the serialized metadata and rebuild the payload from
the typed fields when `raw_representation` is gone. The round-trip test now
serializes the whole message instead of only the result, which is the path a
checkpoint takes.

Addresses the review note on microsoft#10155; tracked by microsoft#10373.
@DZH166
DZH166 deployed to github-app-auth November 15, 2026 11:01 — with GitHub Actions Active
Keep only the fields the provider payload this branch already emits carries, so a
rebuilt call and the in-memory one serialize identically.
@DZH166
DZH166 deployed to github-app-auth November 15, 2026 11:02 — with GitHub Actions Active
@DZH166
DZH166 deployed to github-app-auth November 15, 2026 11:02 — with GitHub Actions Active

This branch was successfully deployed

1 active deployment
github-app-auth — dcc00459 Deployed Nov 15, 2026 by DZH166 via add_label #27887
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

python Usage: [Issues, PRs], Target: Python

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Python: [Bug]: a reconstructed empty local_shell_call is lost across message serialization, so stateless replay fails

3 participants