Repository navigation
Raise on implicit device transfer in __setitem__ #207
Description
Activity
Note that the current behavior (no exception on cross-device
__setitem__) of array-api-strict is aligned with PyTorch:>>> import torch >>> a = torch.ones(10, device="mps") >>> b = 2 * torch.ones(10) # CPU >>> a + b Traceback (most recent call last): Cell In[9], line 1 a + b RuntimeError: Expected all tensors to be on the same device, but found at least two devices, mps:0 and cpu! >>> b[:5] = a[:5] # implicit MPS to CPU transfer does not raise! >>> b tensor([1., 1., 1., 1., 1., 2., 2., 2., 2., 2.])
Not sure if this lack of exception could be considered a "bug" of PyTorch with MPS.
However, DPNP on some hardware architectures do raise:
ValueError: Execution queue is not compatible with allocation queuesas seen in scikit-learn/scikit-learn#32460 (comment).I asked Claude that did a source code / doc review, and
__setitem__is considered "data movement" operation (like.to(),.copy_(),.cpu(),.cuda(), ...) and therefore is not expected to raise in PyTorch (contrary to "compute" operations like+).I also checked the array API spec and there is no mention of constraints related to device transfers: https://data-apis.org/array-api/latest/API_specification/generated/array_api.array.__setitem__.html#setitem.
So I am not sure if it's ok or not for an array library to raise in this case.
Also from:
Handling devices is complex, and some frameworks have elaborate policies for handling device placement. Therefore this section only gives recommendations, rather than hard requirements:
...- Raise an exception if an operation involves arrays on different devices (i.e. avoid implicit data transfer between devices).
So I think it would make sense for array-api-strict to raise on cross-device
__setitem__if we consider assignment an "operation".would this fall under gh-168 Olivier?
We discussed this in the Array API community meeting last week, https://hackmd.io/zn5bvdZTQIeJmb3RW1B-8g?view#Meeting-minutes-30-April-2026, with a resolution to designate it as implementation-defined.
Currently the Array API spec has a general recommendation to avoid implicit transfers, however different libraries make specific choices (e.g. torch special-cases a CPU device), hence the spec itself cannot really force a specific choice on all compliant libraries.
For
array-api-strictthis means it should indeed raise an exception, because it only implements behaviors which are explicitly allowed by the spec, and rejects anything unspecified, hence implementation-defined.data-apis/array-api#1005 updates the specifications of
__setitem__and__getitem__; forarray-api-strict, we could either lump an update to #206, or have a separate PR just for__{get,set}item__(the latter's preferred, I'd think).Reacted by Olivier Grisel and Lucy Liuwe could either lump an update to #206, or have a separate PR just for
__{get,set}item__(the latter's preferred, I'd think).Raising on cross-device
__{get,set}item__is actually not directly related to the dtype restrictions of a particular device so better implement that independently to ease the review and keep a neat changelog.Reacted by Tim Head
Consider the following two arrays:
As expected, it's not possible to run operations that combine two such arrays because they live on different devices:
However, doing assignment works without raising an exception despite the fact that this causes an implicit data transfer between devices:
Shouldn't the
__setitem__call raise in this case?