Skip to content

fix: cast floating inputs during policy device transfer - #2608

Merged
TATP-233 merged 1 commit into
Motphys:mainfrom
coelhoaf:acoelho/UNI-2/mps-observation-transfer
Nov 15, 2026
Merged

TATP-233 merged 1 commit into
Motphys:mainfrom
coelhoaf:acoelho/UNI-2/mps-observation-transfer

Conversation

@coelhoaf

@coelhoaf coelhoaf commented Nov 13, 2026 •

Copy link
Copy Markdown
Contributor

Summary

When a CPU environment supplies float64 observations or rewards to an Apple MPS policy, the RSL-RL adapter transfers them before converting to float32 and raises TypeError. Select float32 during the transfer so these inputs succeed, while preserving integer and boolean dtypes.

The regression uses a small CPU test environment through the public adapter on actual MPS. It covers reset, actor/critic observations, rewards, timeout flags and a large int64 value that would be rounded by an unintended float32 conversion. A stock G1 float64-input failure is not claimed.

Linked Work

Validation

  • make test-all passed on the final local head b4e60e187c9623cf886b85868f75ae04c287f526: 2,868 passed, 141 skipped, 378 deselected; Ruff, mypy and Pyright passed, with one existing Pyright warning. Import checks passed for 35 modules and 36 scripts.
  • Focused adapter and existing wrapper checks: 12 passed, including actual Apple MPS.
  • The new MPS regression fails against the original helper with the expected float64 transfer error.
  • A bounded stock G1 MuJoCo PPO comparison completed ten learning updates per version, 15,360 transitions each. All 240 rollout-step hashes, seven non-timing learning scalar series and saved actor/critic/optimizer tensors matched exactly. Both learner and environment seeds were fixed. Actual parameter updates were verified.

Commands actually run for the local gate:

uv sync --frozen --extra mujoco --extra unirl
uv run --no-sync pytest tests/rl/adapters/test_rsl_rl_vec_env.py tests/scripts/test_train_scripts.py -m 'slow or not slow' -k 'play_wrapper or float64_cpu_observations or integer_and_boolean' -q
uv run --no-sync python -m nodeenv --node=26.13.1 .venv/pyright-node
UV_NO_SYNC=1 PYRIGHT_PYTHON_ENV_DIR="$PWD/.venv/pyright-node" make test-all

The isolated environment uses Python 3.13.18, Torch 2.14.0 and MuJoCo 3.11.0 on macOS 27.0.1 arm64. A worktree-local stable Node bootstrap resolved an initial Pyright/nodeenv prerelease parsing failure; repository dependency pins are unchanged.

Both stock G1 comparison runs hit the same preexisting env.close() failure after completing learning: a deferred command-metric tensor created in inference mode is cleared outside that mode. The uninstrumented official CLI reproduces it too. The comparison establishes bounded numerical parity, not a clean training-command exit or convergence equivalence. This separate cleanup issue is outside this PR.

Remote CI route: base main; CI and Docs are awaiting upstream maintainer approval for this fork contribution. Applicable current-head checks remain required before merge.

Impact

  • Backend: adapter-level; no simulation or physics changes.
  • Platform: fixes conditional float64 input transfer on macOS MPS; existing CPU and float32 behavior is preserved in the checked cases. CUDA was not executed locally.
  • Training: permits previously rejected float64 inputs on MPS; no numerical change observed in the bounded stock float32 G1 comparison. No convergence or speed claim.

Checklist

  • Added regression tests.
  • Linked the driving issue.
  • Documented the conditional impact and separate cleanup follow-up.
  • No public API, dependency or workflow change requiring additional documentation.

@coelhoaf
coelhoaf marked this pull request as ready for review November 13, 2026 20:56
@coelhoaf
coelhoaf requested a review from TATP-233 as a code owner November 13, 2026 20:56
@TATP-233
TATP-233 merged commit d0372c2 into Motphys:main Nov 15, 2026
15 of 42 checks passed
@TATP-233 TATP-233 mentioned this pull request Nov 16, 2026
@coelhoaf
coelhoaf deleted the acoelho/UNI-2/mps-observation-transfer branch November 16, 2026 13:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bug: float64 CPU inputs fail in RSL-RL adapter on Apple MPS

2 participants