Skip to content

Reuse ExpDatas across PEtab v2 PetabSimulator.simulate calls - #3522

Open
FFroehlich wants to merge 1 commit into
claude/github-issue-3519-c591fcfrom
claude/issue-3521-cache-edatas
Open

FFroehlich wants to merge 1 commit into
claude/github-issue-3519-c591fcfrom
claude/issue-3521-cache-edatas

Conversation

@FFroehlich

Copy link
Copy Markdown
Member

What does this pull request do, and why?

Makes PetabSimulator.simulate (PEtab v2) create the ExpData objects only once, on its first call, and reuse them afterwards. Before, every call recreated all ExpDatas (constants, timepoints, measurements, sigmas, and the nominal parameters) and then applied the actual parameters on top, i.e. apply_parameters ran twice per experiment and call. None of that apart from apply_parameters depends on the problem parameters.

  • PetabSimulator keeps the ExpDatas from the first simulate call. Later calls only run ExperimentManager.apply_parameters on them. apply_parameters already sets all parameter-dependent values from scratch on each call (plist, free parameters, pscale, and the fixed parameters mapped to problem parameters), and the PEtab problem must not change after the ExperimentManager is created, so the reused ExpDatas stay valid.
  • ExperimentManager.create_edatas takes an optional problem_parameters argument, like create_edata already does. The first simulate call uses it, so parameters are applied once per experiment and call.
  • PetabSimulationResult.edatas holds the reused objects (in a new list). The simulate docstring already said they "may be changed by subsequent calls". It now says they are reused and must not be modified.

Behaviour changes to be aware of:

  • ExpData(model) copies the model's reinitialization settings (reinitialize_fixed_parameter_initial_states, reinitialization_state_idxs), and, for experiments without periods, its fixed parameters. These are now read on the first simulate call only, so later changes to the model's settings don't reach the reused ExpDatas. Solver settings and all other model settings still apply to every call.
  • Calling simulate on the same PetabSimulator from several Python threads at once now shares the ExpData objects between them. Copy the simulator per thread instead. ExpData deep copies and pickles keep all fields (checked separately), and test_petab_simulator_deepcopy_and_pickle pickles a simulator after it has simulated.

This is stacked on #3520 (constant-time lookups in ExperimentManager, #3519), which makes the remaining apply_parameters calls cheap.

Testing:

  • New test_simulate_reuses_edatas: counts create_edata and apply_parameters calls over two simulate calls (one creation and one application per experiment on the first call, one application per experiment on the second), checks that the same ExpData objects are returned, and that after switching parameters (including the nominal ones) the reused ExpDatas, llh, and sllh equal those of a new simulator and of freshly created ExpDatas. The problem has a non-estimated parameter mapped to a fixed model parameter and placeholders overridden by parameters in one experiment and by numbers in the other, on non-linear parameter scales. Fails on Make PEtab v2 ExperimentManager lookups constant-time #3520's head, passes here.

  • python/tests/petab_/test_petab_v2.py: all tests pass locally (25 including the JAX tests before rebasing onto Make PEtab v2 ExperimentManager lookups constant-time #3520's latest commit, which only adds a test; 18 SUNDIALS tests after the rebase).

  • A scratch script timed five simulate calls on a synthetic problem with 1000 experiments (1 state, 3 measurements each, an estimated observable placeholder override), single-threaded. The log-likelihoods are identical on both branches.

    per simulate call after the first Make PEtab v2 ExperimentManager lookups constant-time #3520 this PR
    no sensitivities 0.38 s 0.21 s
    forward sensitivities 0.58 s 0.35 s

    Not run on the Froehlich_CellSystems2018 problem from the issue.

Related issues

Fixes #3521

Use of AI tools

Claude Code (Claude Opus 5.5) wrote the implementation, the test, the changelog entry, and this description, and ran the tests and timings described above.

Checklist

  • I have run the relevant tests locally
  • I have added a changelog entry in changelog.d/ (or it is not needed)

🤖 Generated with Claude Code

`PetabSimulator.simulate` recreated the `ExpData` objects for all
experiments on every call and then applied the parameters on top of the
nominal ones, so `apply_parameters` ran twice per experiment and call.
Apart from what `apply_parameters` sets, the `ExpData`s only depend on
the PEtab problem, which must not change, and the model.

Create the `ExpData`s on the first call to `simulate`, with the given
parameters, and afterwards only apply the parameters to them.
`ExperimentManager.create_edatas` now takes the problem parameters to
apply, like `create_edata`.

Fixes #3521.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@FFroehlich
FFroehlich added this pull request to stack #3523 November 7, 2026 19:06
@FFroehlich
FFroehlich marked this pull request as ready for review November 7, 2026 19:09
@FFroehlich
FFroehlich requested a review from a team as a code owner November 7, 2026 19:09
@codecov

codecov Bot commented Nov 7, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 80.30%. Comparing base (19cea70) to head (e3c8beb).

Additional details and impacted files

Impacted file tree graph

@@                       Coverage Diff                        @@
##           claude/github-issue-3519-c591fc    #3522   +/-   ##
================================================================
  Coverage                            80.30%   80.30%           
================================================================
  Files                                  331      331           
  Lines                                23431    23434    +3     
  Branches                              1631     1631           
================================================================
+ Hits                                 18816    18819    +3     
  Misses                                4609     4609           
  Partials                                 6        6           
Flag Coverage Δ
cpp 74.80% <12.50%> (-0.01%) ⬇️
cpp_python 34.64% <12.50%> (-0.01%) ⬇️
petab 46.92% <100.00%> (+0.21%) ⬆️
petab_sciml 16.60% <12.50%> (-0.01%) ⬇️
petab_sciml_benchmarks 15.19% <12.50%> (-0.01%) ⬇️
python 72.59% <87.50%> (-0.01%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
python/sdist/amici/sim/sundials/petab/_v2.py 99.05% <100.00%> (+<0.01%) ⬆️
python/tests/petab_/test_petab_v2.py 100.00% <ø> (ø)
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

PEtab v2 PetabSimulator.simulate rebuilds all ExpDatas on every call (cache edatas)

1 participant