Questions on NeuralCDEs

Hi!

First of all, thank you for creating this amazing library!

I'm new to neural CDEs, control theory and JAX ecosystem, so please excuse me if these are basic questions. My ultimate goal is to train a model on some time series data (physical quantities $[y_1(t), y_2(t)]$ that evolve in time and can be controlled by actuators $[C_1(t), C_2(t)]$) to replicate it and then use this model to fit PID parameters or RL policies to control the physical quantities of interest. Both control and physical quantities are normalized such that they are bound in the interval [0, 1].

Has anyone in the community attempted this before or is aware of literature where this has been done?

Initially, I decided to train RNNs, but the problem is that the time spacing of my control signal is different from my physical quantities (even the time interval of sampling between physical quantities is different). Neural differential equations seem to handle this problem better. I'm using neural CDEs for regression on some data, and I was able to overfit a neural CDE on a time series. I'm first uplifting the physical state into the hidden state of size 4, $\boldsymbol{y}$. Here's how my vector field and control interaction looks like:

$C_1$ and $C_2$ are obtained by interpolation as in the tutorial, and $mlp_i$ are small MLPs (width=4 depth=3 <10) that output a scalar value from -1 to 1. This consistently overfits the time series (checked on other trajectories as well), but it takes ~20s/train_step (in train_step, I compute loss and update model parameters).

Can Multiple Shooting #302 improve my training speed? Btw @allen-adastra, my work is mostly inspired by your paper !

I'm aware of Patrick's thesis on this topic, but any other literature recommendations on this topic are highly appreciated!

Sincerely,
Vadim

Start by reading the linked Neural CDE tutorial and the referenced Multiple Shooting issue. The post does not identify a repository file, test, or concrete change, and it does not define what completion would look like beyond answering questions about training speed and related literature.