[jameskermode]

Summary

Infrastructure for exporting fitted ETACE potentials to standalone native libraries with
Julia 1.12's juliac --trim=safe, plus the consumers: a LAMMPS pair_style ace plugin and
a pip-installable Python/ASE package. Compiled libraries need no Julia at runtime.

Performance

The acceptance gate for this work was ≤ 1.2x pair_style pace recursive per core, in
exact :polynomial mode, on both reference models. Measured on the same core in the same
session:

Model	pair_style ace (µs/site)	vs pace recursive
Cantor (CrMnFeCoNi)	58.1	0.89x
TiAl (order 4)	94.0	0.51x

Both are faster than pace recursive outright, not merely inside the gate. The route there,
the rejected alternatives, and the measurement methodology are written up in
export/bench/FINDINGS_parity.md.

What's in it

Export pipeline

• Export ETACE models to trim-compatible Julia, compiled to a shared library (~3 MB) with a
  bundled Julia runtime (~20 MB)
• ace1_model models are converted via ETModels.convert2et() before export
• Multi-species models, including the pair term, and per-atom virials

LAMMPS

• pair_style ace plugin loading the compiled library directly
• LAMMPS native neighbour lists; MPI domain decomposition and OpenMP threading

Python/ASE (ase-ace)

Calculator	Backend	Threading	Startup	Julia required
ACELibraryCalculator	Compiled .so	Single	Instant	No
ACEJuliaCalculator	JuliaCall	Multi	10-30s	Managed by juliapkg

Julia and its five Julia packages are declared in exactly one file,
src/ase_ace/juliapkg.json, and installed on first use; no separate Julia install is needed.

Removed since this PR was opened

• :hermite_spline radial export and the splinify-for-export path it was the only
  consumer of. It could not survive --trim, and splinify() leaves no recurrence to emit.
  The mode now raises rather than silently misbehaving.
• The minimal-export path, which nothing used.
• The socket backend (ACECalculator, ASE SocketIOCalculator + IPICalculator.jl over
  i-PI). Its documented reason to exist was multi-threaded Julia, but ACEJuliaCalculator
  threads identically — both reach the same Folds.sum(..., ThreadedEx()) inside
  ACEpotentials — and the two agreed to 8e-9 relative on the same cell and model before it
  was dropped. It had also never worked on any host whose localhost resolves to IPv6,
  since ASE's SocketServer binds AF_INET only.
• ACEregistry from CI and the docs: everything resolves from General now, and the
  registry's stale entries break resolution on Julia 1.13+.

Test plan

• Exported energies, forces and virials match Julia to the gates in export/test/
• Forces checked against a finite difference; stress against -dE/dV
• LAMMPS plugin builds and runs, serial and under MPI
• Parity gate met on both reference models (above)
• ase-ace: packaging (wheel + sdist, installed into a throwaway venv), the JuliaCall
  calculator, the library calculator, and a clean-environment check that the declared
  Julia packages are sufficient
• CI green on every workflow

Requirements

• Export: Julia 1.12+ with juliac support
• LAMMPS: any version with plugin support
• Python: 3.9+, ASE; matscipy for ACELibraryCalculator, juliacall for
  ACEJuliaCalculator

[tjjarvinen]

I am not sure are you aware that you can already use ACEpotentials with ASE and LAMMPS.

It is done with IPICalculator.jl and current used by e.g. DFTK for ASE. I have also used ACEpotentials with ASE extensively, so it is well tested.

Using the i-PI socket server protocol is the approach preferred by Ask Hjorth Larsen the ASE maintainer. This was discussed with him during Julia MolSSI Workshop last year and other occasions afterwards. The implementation at IPICalculator.jl is meant to serve all AtomsCalculators compatible calculators in JuliaMolSim ecosystem by providing i-PI protocol suport and general ASE interface.

LAMMPS supports Molssi Driver Interface (MDI) in server mode https://docs.lammps.org/mdi.html, which is compatible with i-PI protocol, with the exception of not having support for Unix pipe. You can use ACEpotentials with IPICalculators in LAMMPS this way. Although I have to admit that the LAMMPS documentation is not clear at all. I have not tested ACEpotentials in LAMMPS this way, but I expect it to work

Thus, I propose that instead of using resources to implement our own ASE and LAMMPS binary interfaces - this PR - we use the existing i-PI protocol interface to connect to ASE and LAMMPS, and make documentation entries for them. This the easiest way to get ASE and LAMMPS interface going and reduces maintenance burden for ACEpotentials.

[jameskermode]

Thanks for letting me know, I was aware of the IPICalculator.jl support but not that it could be used with LAMMPS. Nonetheless, I think native code export I am pursuing here is a good path to explore because it allows MPI-parallel domain decomposition. This should be much more scalable than socket communications for very large systems. ASE is then a fringe benefit that used the same native code export - and also benefits from OpenMP parallelisation that wouldn't be easy to do with IPICalculator. Initial tests give ca. 5x speed up on 8 threads even for small systems. Moreover, so far I only spent a day or so on it and it's nearly working.

[tjjarvinen]

MPI is a valid point. Although there is the executor keyword that allows you to perform distributed calculation already in Julia

# start Julia in parallel and then use DistributedEx() instead of the default ThreadedEx()
energy_forces_virial(system, ace_calc;  executor=DistributedEx(), ntasks=nworkers() )

I am not sure what the performance is with it, but you could compare it to the MPI implementation. (DistributedEx() might not be exported in ACEpotentials so you need to get it from Folds.jl or Transducers.jl)

Also, I am really curious how did you manage get 5x performance from OpenMP? The default Julia version is multi threaded and scales well with number of threads when neighbourlist calculation time is ignored.

[jameskermode]

It would indeed be interesting to compare LAMMPS MPI performance with Julia executor, but my experience in general on HPC machines is that MPI has been heavily vendor optimisated and in comparison's Julia distributed calculations are typically not competitive. This is likely to be particularly true for LAMMPS.

Standard Julia threads can't be used with --trim=safe native code, so I added OpenMP on the loop over atoms outside the Julia call. 5x on 8 cores isn't great but this was only 128 atom system and this is including the neighbourlist overhead. I was intending it for use either from Python, or within multilevel MPI/OpenMP parallelism from LAMMPS, just to give a bit of extra speedup and make full use of available hardware threads.

[jameskermode]

Another advantage of this approach: it looks like we can bundle the Julia runtime libraries, meaning no Julia installation required for end users.

[tjjarvinen]

MPI should be faster when communication limited, if computation limited then MPI and Julia should give about the same results. The point here would be get an idea of when the situation changes to communication limited case.

The point of including Julia runtime is also good. It should make ACEpotentials easier to use when only compute is needed.

• Standard Julia for small system where multi threading inside a single node - sockets with Unix pipe is same as using MPI on local node
• Large system with MPI using several nodes (might even be valid to use MPI in Julia too at some point)
• Easy to use binary package for compute only

I should also work on to finish the new neigbourlist package. It would be faster than the current one, is multithreaded and can be made to work with MPI.

[cortner]

@jameskermode -- this is entirely outside of what I want to have any control over and would be very happy to merge this so we can proceed with the other PRs.

[jameskermode]

@Alexsp32 Thanks for your contributions here. With some AI assitance I've improved performance of LAMMPS export on CPU and now has parity with ML-PACE. Do you want to test again before I merge? I would then plan to tidy up documentation, tutorials etc and do a release, and publish the ase-ace package on PyPI.

On GPU, I've struggled with some Reactant.jl compulation bugs (one has just been fixed, so I'll try again at some point). The most promising approach at the moment for fast evaluation of ACEpotentials.jl-fitted models is to port the forward model to JAX, see ace-jax repo for an early-stage prototype if you are interested. This has parity with ML-PACE on Kokkos (although I should note MACE with cuequivariance can still be faster than large basis-set linear ACE models, work in progress on that.)

[Alexsp32]

Thanks for the update, @jameskermode. I’m happy to test this again, but won’t get around to it until next week probably. If you’re happy to wait until then, I’ll give the changes a try.

[jameskermode]

@Alexsp32 Next week is fine - given I opened this PR last Dec another week is nothing, and would be good to have a second data point before merging (although this doesn't affect the standard Julia evaluatin paths so isn't a risky merge)

[Alexsp32]

I've tried working through the new export workflow and found the following issues so far:

• export/scripts/build_deployment.jl Step 2/6 fails on Julia v.1.13 due to the Juliac.jl file no longer being in the expected path.
• export/scripts/build_deployment.jl Step 3/6 fails sometimes on Julia v1.12, this is due to ldd sometimes failing on network file systems. (I think this is worth documenting since people are likely to compile in an HPC environment they want to use)

I'll push a fix for both as soon as it's working for me. Maybe you could check it doesn't break anything on your end?

[jameskermode]

thanks @Alexsp32, will gladly check your fix. I've not tested on 1.13, can add that to CI matrix.

[Alexsp32]

That suddenly turned way more complicated than I'd liked. I've bumped the unit tests to include Julia v1.13 now, but if anything uses the ACE registry, those might fail due to ACEsuit/ACEregistry#11.

[jameskermode]

@Alexsp32 Julia 1.13 looks good, so if you're happy I'll go ahead and merge and tag a new relase?

[Alexsp32]

Sounds good, thanks for taking care of it!