[FFroehlich]

What does this pull request do, and why?

Makes numpy.array/numpy.asarray of a DoubleVector or IntVector copy the data in C++, instead of building the array element by element from the vector proxy's Python iterator. That per-element conversion is what ReturnDataView does for every field it returns (np.array(attr).reshape(field_dim) in _field_as_numpy), so it was paid after every simulation for x, w, y, sx, FIM, and so on.

• DoubleVector and IntVector get a __array__(dtype=None, copy=None) method. It calls a new _to_ndarray method, implemented in C++ in %extend, which allocates a 1D array and copies the data with memcpy (float64) or std::copy (int64, the same dtype np.array(IntVector) gave before).
• _to_ndarray creates Python objects, so its wrapper must keep the GIL. The extension is built with -threads, which releases the GIL around every wrapped call, so _to_ndarray gets %feature("nothreadallow"). I checked in the generated wrapper that its call has no SWIG_PYTHON_THREAD_BEGIN_ALLOW (unlike, e.g., DoubleVector.__len__ and stdVec2ndarray, see below).
• The result is always a new array, not a view: a view would not keep the owning ReturnData alive. Therefore np.asarray(vec, copy=False) raises ValueError, as the numpy protocol requires when a copy can't be avoided. NumPy 2 passes copy=True to __array__ from np.array and then doesn't copy again (checked), so _field_as_numpy is unchanged.

Timings on this Mac (Python 3.13, numpy 2.5.4), this branch vs. a build of #3520's head:

	before	this PR
np.array(DoubleVector), 123,000 elements (w of 82 expressions at 1500 time points, from the issue)	29 ms	0.017 ms
np.array(IntVector), 123,000 elements	29 ms	0.016 ms
_field_as_numpy of a 46×46 FIM, 18,338 times (the PEtab v2 _aggregate_s2llh case from the issue comment)	26 s	0.07 s

Not run on the Froehlich_CellSystems2018 problem or in sbmlsim.

Not addressed here:

• stdVec2ndarray still releases the GIL (stdVec2ndarray segfaults: the SWIG wrapper releases the GIL around a function which creates a numpy array #3509), and DoubleVector[i] still raises a TypeError after import amici (DoubleVector[i] and IntVector[i] raise a TypeError after import amici: libsbml is loaded before the extension and its SWIG types win #3511). This PR uses neither.
• The issue comment also mentions about 11 s spent in XArrayFactory.__init__ while creating 18,338 ReturnDataViews, possibly cyclic garbage collection (each ReturnDataView and its XArrayFactory reference each other). Not investigated here.

This is stacked on #3522, but doesn't depend on its changes.

Testing:

• New test_vector_to_ndarray in python/tests/test_swig_interface.py: replaces __iter__ of DoubleVector and IntVector with a function that raises, then checks values, dtypes, shapes, that the arrays are copies, dtype=, copy=False, empty vectors, repr, and the x, sx, y, and numsteps fields of a ReturnDataView against the element-wise conversion. Fails on Make PEtab v2 ExperimentManager lookups constant-time #3520's head ("converted via the Python iterator"), passes here.
• test_swig_interface.py, test_rdata.py, test_edata.py, test_hdf5.py, test_sbml_import.py, and petab_/test_petab_v2.py: 115 passed, 7 skipped (not checked why). The 7 JAX tests in test_petab_v2.py didn't run, since JAX/diffrax isn't installed in this environment. test_fiddy_derivative_check_leaves_solver_unchanged needs fiddy from git (--group python/sdist/pyproject.toml:fiddy); it passes with it.

Related issues

Fixes #3510

Use of AI tools

Claude Code (Claude Opus 5.5) wrote the implementation, the test, the changelog entry, and this description, and ran the tests and timings described above.

Checklist

• I have run the relevant tests locally
• I have added a changelog entry in changelog.d/ (or it is not needed)

🤖 Generated with Claude Code

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 80.24%. Comparing base (e3c8beb) to head (b317ee2).

Additional details and impacted files

@@                        Coverage Diff                         @@
##           claude/issue-3521-cache-edatas    #3524      +/-   ##
==================================================================
- Coverage                           80.30%   80.24%   -0.07%     
==================================================================
  Files                                 331      331              
  Lines                               23434    23434              
  Branches                             1631     1631              
==================================================================
- Hits                                18819    18804      -15     
- Misses                               4609     4624      +15     
  Partials                                6        6              

Flag	Coverage Δ
cpp	74.74% <ø> (-0.07%)	⬇️
cpp_python	34.64% <ø> (ø)
petab	46.92% <ø> (ø)
petab_sciml	16.60% <ø> (ø)
petab_sciml_benchmarks	15.19% <ø> (ø)
python	72.53% <ø> (-0.07%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.
see 4 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.