Migrate Python test infra and scripts from httpx to requests - #8419
Closed
Amaury Chamayou (achamayou) wants to merge 5 commits into
Closed
Amaury Chamayou (achamayou) wants to merge 5 commits into
Amaury Chamayou (achamayou) wants to merge 5 commits into
Conversation
Replace the httpx-based default e2e client with a requests-based RequestsClient, port HTTP signature auth to requests.auth.AuthBase, and switch the remaining scripts (tvc.py, fetch_amd_collateral.py) to requests. requests only speaks HTTP/1.1, and HTTP/2 support in CCF is experimental, so the HTTP/2 mode of the Python e2e infra (--http2 and the e2e_logging_http2 test) is removed. The h2spec compliance test in client_protocols.py now targets a dedicated HTTP/2 interface on a node whose primary interface stays HTTP/1.1. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…abilise timing-sensitive tests - Wrap TLS handshake failures as connection-establishment errors so that mid-request SSL errors are reported as CCFIOException (as with httpx) rather than being retried as connection failures by CCFClient. - Detect a caller-provided Content-Type header case-insensitively, as requests merges headers case-insensitively. - schema download: sign after every transaction so the chunk count does not depend on client speed. - long-lived forwarding: widen the node-to-node key rotation window, as requests issues requests faster than httpx did. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…face node - caps: refused (503) sessions are torn down asynchronously by the node, so wait briefly for the active session count to settle. Tolerate fatal errors at shutdown in the low-fd phase regardless of when the expected crash is observed, as ledger writes are asynchronous. - client_protocols: generate the TLS report against a single-interface node (as the golden file expects), and run h2spec against a second network with an additional dedicated HTTP/2 interface. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
urllib3 >= 2.8 no longer closes pooled connections when a PoolManager is cleared, relying on garbage collection of the pools instead. Responses keep a reference to their pool, so a retained response kept its connection, and the corresponding CCF session, open after the client was closed. This made the connections caps test see one fewer available session than expected. Also close the temporary clients used to follow redirects, which were never closed. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Member
Author
|
Superseded by #8432, which migrates to urllib3 instead of requests so that the Python HTTP/2 e2e coverage ( |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
The Python e2e infrastructure has been pinned to
httpx == 0.23.*because later releases broke the long-forwarding, extended-character-range and JWT tests.requestsis the more stable, widely used choice for a synchronous HTTP/1.1 test client, so this migrates all remaininghttpxusage torequests.Implementation summary
tests/infra/clients.py:HttpxClientbecomesRequestsClienton top ofrequests.Session.HttpSigis now arequests.auth.AuthBase. Redirects are explicitly disabled at therequestslevel (CCFClient follows them itself, as before). Exceptions are mapped to the existingTimeoutError/CCFConnectionException/CCFIOException, distinguishing connection-establishment failures from I/O errors on established connections via the wrapped urllib3 error. Response text is always decoded as UTF-8 (matching httpx, rather than requests' charset heuristics).extra_headers_count()remains 5 for HTTP/1.1.tests/connections.py,tests/tvc.py,tests/memberclient.py,scripts/fetch_amd_collateral.py: ported torequests.tvc.pygains an explicit 5s timeout, asrequestshas no default timeout (httpx did).tests/requirements.txt,scripts/requirements.txt:httpxreplaced byrequests.doc/historical_ccf_requirements.txtkeepshttpx, as older release docs still import it.requestsonly speaks HTTP/1.1 and CCF's HTTP/2 support is experimental. This removes thee2e_logging_http2test, the--http2e2e argument,reqs.no_http2()and allargs.http2branches.node.client()now raises if asked to target a non-HTTP/1.1 interface.tests/client_protocols.py: the h2spec compliance test is kept, targeting a dedicatedHTTP2RPC interface on a node whose primary interface stays HTTP/1.1 (so governance goes through the Python client normally), instead of starting a second all-HTTP/2 network.doc/use_apps/issue_commands.rst: mentionrequestsinstead of HTTPX as the example HTTP library.Safety and compatibility
Test infrastructure and developer scripts only; no runtime, API or ledger-format change. The
ccfPython SDK does not depend on httpx or requests. Server-side HTTP/2 support is unchanged and still covered by h2spec inclient_protocols; only the Python client-side HTTP/2 test mode is removed.