Generate Battery Aging and EIS AI Datasets with PyBaMM

Generating a large, physics-labelled battery dataset with PyBaMM was never really about writing a for-loop. It is a numerical design-of-experiments problem: you have to control boundary conditions, the topology of the parameter space, thermal gradients, transient snapshots of the electrochemical state vector, the frequency grid for impedance, and the convergence tolerances of the algebraic solver. Lose any one of them and the network learns the simulator’s numerical bias rather than electrochemistry.

This article gives a pipeline that actually runs, plus the four failure modes that hand you garbage data without raising a single error. Everything produced here is a deterministic numerical solution of a PDE system — physics-based synthetic data, which has to be aligned to real cells through parameter identification before it means anything. Without that step it is a self-consistent hallucination.

PyBaMM battery modelling series (4 parts): ① Architecture and solvers → ② EIS labels → ③ Dataset pipeline → ④ Training SOH and RUL. This is part 3.

1. Stating the dataset objective formally

A well-posed battery AI dataset is this set:

$$ \mathcal{D} = \{(\mathbf{x}_i, \mathbf{y}_i, \mathbf{m}_i)\}_{i=1}^N $$

Here $\mathbf{x}_i \in \mathbb{R}^p$ is the observable feature vector (voltage transients, complex impedance $Z(\omega)$, protocol descriptors) and $\mathbf{y}_i \in \mathbb{R}^q$ holds the internal, unobservable degradation quantities (SOH, loss of lithium inventory, loss of active material).

The third term is the one people underestimate. $\mathbf{m}_i$ is the full metadata graph: model structure assumptions, thermodynamic parameters, solver configuration. Without it there is no physical causality behind $f: \mathbf{x} \rightarrow \mathbf{y}$ — and six months later, a dataset whose model class, solver tolerance and parameter set are unknown is a dataset you throw away. Metadata is not good record-keeping. It is part of the dataset.

2. Pick the model first — it caps how much data you can generate

Most tutorials start with DFN and then hit a wall on sample count. PyBaMM’s three workhorse models differ by more than an order of magnitude in cost, and that choice sets the ceiling on your dataset size.

A practical route: sweep the parameter space with SPMe, then re-run the selected subset at high fidelity with DFN. SPMe keeps electrolyte polarisation — the dominant first-order effect — at a fraction of the cost, which is what you want for broad coverage; DFN is reserved for the region you actually care about. This only works if you record which model produced each sample, which brings you back to $\mathbf{m}_i$.

Cost ratios depend heavily on electrode discretisation, cycle count and solver. Do not copy timings from any blog, including this one; the script below prints its own elapsed time, and only a measurement on your machine counts.

3. Degradation dynamics and where the labels come from

Extracting a “label” means querying the time-integrated state of a specific degradation PDE. SEI thickness $L_{SEI}$, the main source of LLI, is typically modelled as:

$$ \frac{\partial L_{SEI}}{\partial t} = \frac{M_{SEI}}{\rho_{SEI} z F} j_{SEI} \exp\left( -\frac{E_a}{R T} \right) \exp\left( -\frac{\alpha F (\phi_s – \phi_e – U_{SEI})}{R T} \right) $$

• SOH: discharge capacity of the current cycle over nominal, $Q_k / Q_0$.
• RUL-to-80: cycles remaining until the state manifold intersects the $SOH = 0.8$ boundary.
• LLI: accumulated time integral of parasitic side-reaction current, which directly suppresses coulombic efficiency.
• LAM: governed by particle fracture mechanics, triggered when local stress $\sigma_{t,max}$ exceeds yield strength. Record the two electrodes separately — their rates and mechanisms differ, and collapsing them into one scalar destroys the most discriminative signal you have.
• EIS feature vector: generated by perturbing and linearising the DFN Jacobian in the frequency domain, capturing the charge-transfer semicircle and the Warburg tail.

4. A pipeline that actually runs

The following is self-contained — copy it out and it runs, with no undefined variables. It does four things: assemble a cell with degradation submodels, cycle it, extract labels, and treat solver failure as a first-class outcome.

import time
import pybamm

# Degradation mechanisms must be declared when the model is constructed.
# Pushing SEI parameters into ParameterValues afterwards does NOT enable
# them - this is the most common "it ran but nothing degraded" mistake.
OPTIONS = {
    "SEI": "solvent-diffusion limited",
    "SEI porosity change": "true",
    "lithium plating": "partially reversible",
    "lithium plating porosity change": "true",
    "particle mechanics": ("swelling and cracking", "swelling only"),
    "SEI on cracks": "true",
    "loss of active material": "stress-driven",
    "calculate discharge energy": "true",
}

def run_one(cycles=100, model_cls=pybamm.lithium_ion.SPMe, seed_params=None):
    model = model_cls(options=OPTIONS)

    # OKane2022 is parameterised for degradation studies. With a set like
    # Chen2020, which has no degradation parameters, the SEI and plating
    # submodels fall back to defaults and the result is not physical.
    param = pybamm.ParameterValues("OKane2022")
    if seed_params:
        param.update(seed_params, check_already_exists=False)

    exp = pybamm.Experiment(
        [("Discharge at 1C until 2.5V", "Charge at 0.3C until 4.2V", "Hold at 4.2V until C/100")]
        * cycles
    )

    sim = pybamm.Simulation(
        model, parameter_values=param, experiment=exp,
        solver=pybamm.IDAKLUSolver(),   # much faster than default CasADi on DAEs
    )

    t0 = time.perf_counter()
    try:
        sol = sim.solve(calc_esoh=False)
    except pybamm.SolverError as exc:
        # Failed samples must be kept, not silently skipped. The boundary of
        # the usable parameter space is drawn precisely by these failures.
        return {"ok": False, "reason": str(exc)[:200], "seconds": time.perf_counter() - t0}

    elapsed = time.perf_counter() - t0
    sv = sol.summary_variables

    # summary_variables keys have been renamed across PyBaMM versions. Scripts
    # that hard-code them break on upgrade with a KeyError - or worse, silently
    # pick up None.
    def pick(*names):
        for n in names:
            if n in sv:
                return sv[n]
        raise KeyError(f"none of {names} present; available: {list(sv.keys())[:12]}")

    cap = pick("Capacity [A.h]")
    return {
        "ok": True,
        "seconds": elapsed,
        "cycles_completed": len(cap),
        "capacity": cap,
        "soh": [c / cap[0] for c in cap],
        "lli": pick("Loss of lithium inventory [%]"),
        "lam_neg": pick("Loss of active material in negative electrode [%]"),
        "lam_pos": pick("Loss of active material in positive electrode [%]"),
        # Metadata travels with the labels. Do not put it in a separate file.
        "meta": {
            "model": model_cls.__name__,
            "param_set": "OKane2022",
            "pybamm_version": pybamm.__version__,
            "options": OPTIONS,
        },
    }

if __name__ == "__main__":
    r = run_one(cycles=50)
    if r["ok"]:
        print(f"{r['cycles_completed']} cycles in {r['seconds']:.1f}s")
        print(f"final SOH={r['soh'][-1]:.4f}  LLI={r['lli'][-1]:.2f}%  "
              f"LAM(neg)={r['lam_neg'][-1]:.2f}%  LAM(pos)={r['lam_pos'][-1]:.2f}%")
    else:
        print(f"solver failed (this is also data): {r['reason']}")

For batch generation use the repository script, which wraps the same logic in multiprocessing and sampling:

cd pybamm-ai-data-lab
python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt

python src/run_all.py --samples 200 --workers 4 --seed 7 \
  --backend pybamm --output /tmp/pybamm-ai-dataset

Do not set --workers to your core count reflexively. Each PyBaMM process compiles its own CasADi/IDAKLU expressions and the memory footprint is significant; saturating the cores often buys you paging rather than throughput. Start at 4 and climb while watching memory.

5. Four failure modes that quietly produce garbage

This section is the most valuable part of the pipeline. None of the following raises an error. The script exits cleanly, the CSV is written, and the data is worthless.

1. Degradation was never actually enabled

Symptom: 500 cycles in, the SOH curve is nearly flat and still reads 0.999.

Cause: degradation submodels must be declared through options when the model is constructed. Building pybamm.lithium_ion.DFN() and then updating SEI parameters in ParameterValues does not switch SEI growth on — those parameters are silently ignored. Check it directly: print(model.options["SEI"]).

2. Parameter set does not match the mechanisms

Symptom: SOH collapses absurdly fast, or LLI climbs to physically impossible values.

Cause: using a set like Chen2020 that was never calibrated for degradation. Missing degradation parameters fall back to defaults, and a default is not a “reasonable value” — it is a value that lets the solver run. For degradation work use OKane2022, or a set you fitted yourself.

3. The experiment terminated early

Symptom: cycles_completed is far below what you requested, with no exception raised.

Cause: PyBaMM’s Experiment stops early when the cell has degraded past the point where a step’s termination condition can be met — for instance capacity has faded so far that discharge never reaches the 2.5 V cutoff. That is correct behaviour, but if you never check, you end up with ragged trajectories whose end-of-life distribution is systematically truncated. Always compare len(capacity) against the cycle count you asked for.

4. The EIS spectrum is a numerical artefact

Symptom: negative real parts on the Nyquist plot, or a physically impossible hook at high frequency.

Cause: linearising at a deeply degraded operating point leaves the Jacobian badly conditioned, and a sparse frequency grid produces interpolation ghosts. Test with the Kramers-Kronig relations — they must hold for any linear, causal, stable, time-invariant system, so a violation means the spectrum is not a valid impedance response at all. Flag these samples; they must not enter supervised training.

6. Parameter identification: aligning synthetic data to real cells

Before trusting synthetic data, the underlying parameter distribution has to be fitted to real measurements by minimising the residual between experimental voltage $V_{exp}$ and simulated voltage $V_{sim}(\mathbf{p})$:

$$ J(\mathbf{p}) = \sum_{k} w_k \left( V_{exp}(t_k) – V_{sim}(t_k; \mathbf{p}) \right)^2 $$

import numpy as np
import pybamm
import scipy.optimize as opt

model = pybamm.lithium_ion.SPMe()
base_param = pybamm.ParameterValues("Chen2020")

# V_exp / t_eval come from your cycler export, already aligned to one time grid
# t_eval: np.ndarray  shape (T,)
# V_exp:  np.ndarray  shape (T,)

KEYS = [
    "Negative electrode active material volume fraction",
    "Positive electrode active material volume fraction",
]
BOUNDS = [(0.4, 0.8), (0.4, 0.8)]

def objective(x):
    param = base_param.copy()          # copy, or iterations contaminate each other
    param.update(dict(zip(KEYS, x)))
    try:
        sim = pybamm.Simulation(model, parameter_values=param)
        sol = sim.solve(t_eval)
    except pybamm.SolverError:
        return 1e6                     # large penalty for non-convergent points;
                                       # raising here kills the whole optimisation
    V_sim = sol["Terminal voltage [V]"](t_eval)
    return float(np.sum((V_exp - V_sim) ** 2))

res = opt.minimize(
    objective,
    x0=[base_param[k] for k in KEYS],
    method="L-BFGS-B",
    bounds=BOUNDS,
)
print(res.x, res.fun)

Both traps are in the comments: ParameterValues must be copied before mutation or iterations pollute each other, and solver failure has to return a large penalty rather than raise, otherwise the optimiser dies at the first non-convergent point.

Skip this step and the model has learned the artificial distribution imposed by Latin hypercube sampling, not electrochemistry. For production work, PyBOP and pybamm-param already package the cost-function and optimiser combinations.

7. Orthogonal splits and leakage

If snapshots from one simulated ageing trajectory (cycles 10, 50, 100 under the same parameters) are randomly scattered across train and test, the model overfits the deterministic evolution of the ODE system. It is reciting that trajectory, not learning a generalisable degradation function.

• Split boundaries must stay physically orthogonal: split by cell_design_id, or isolate random subsets of the kinetic parameter space.
• EIS points are structurally constrained by Kramers-Kronig; adjacent frequencies within one spectrum must never straddle the train/test boundary.
• Assert it after splitting: no cell_design_id in the test set may appear in training. Putting that assertion in the pipeline is far cheaper than discovering suspiciously good metrics later.

8. Quality gates for dataset generation

At scale, sample count matters far less than whether each batch clears the physical, numerical and machine-learning gates.

9. Closing the gap to real cells

Synthetic data is irreplaceable for pre-training, for validating active-learning acquisition functions, and for architecture ablations. Crossing the sim-to-real gap additionally needs:

• Continuous calibration against differential voltage analysis (DVA) and incremental capacity analysis (ICA) from real cyclers. Both are unusually sensitive to electrode-level degradation, which makes them good probes for whether the model captured the right mechanism. To look at a curve before writing any code, the site’s dQ/dV incremental capacity analyser plots it in the browser.
• Global sensitivity analysis (Sobol indices) to quantify which unobservable parameters dominate the generated voltage signature. When the dominant terms contradict physical intuition, the parameter bounds are usually wrong.
• Strict recording of solver constraints. When SUNDIALS cannot meet a $10^{-6}$ absolute tolerance through an aggressive ageing step, those points must be flagged non-physical and discarded rather than fed to a model unexamined.

One last warning: the biggest risk in a synthetic dataset is not that it is inaccurate. It is that it is too self-consistent. Real cells have manufacturing spread, thermal non-uniformity and sensor noise; a PDE solution has none of these. Think hard about injecting noise matched to your measured distributions before training — otherwise the metrics that look excellent in simulation will collapse on real hardware.

References

• PyBaMM long experiments and summary variables
• PyBaMM coupled degradation mechanisms
• Python Battery Mathematical Modelling (PyBaMM)
• Physics-based battery model parametrisation from impedance data
• PyBOP: A Python package for battery model optimisation and parameterisation
• Synthetic dataset of LG M50 batteries with different degradation pathways

Batch generation runs into solver non-convergence more than anything else. For telling apart “tune the solver” from “fix the protocol”, see PyBaMM solver convergence failures.