Learning differential equations

• train feed-forward or recurrent neural networks to approximate a differential equation
  • parameterizing multi-step solver schemes link
  • learning partial differential equations (PDEs) link link
  • stochastic variational inference to solve stochastic differential equations link
  • LSTM-CNN-autoencoder based fluid simulation link
• applying Gaussian Processes (GPs)
  • learning nonlinear PDEs (from noisy observations) -> Physics Models
    • time-dependent PDEs link
    • general link
  • fit differential equations link
    • ODE-solver -> point estimates
    • apply Gaussian Process -> function estimate (with exact match at estimates)
  • GPs can naturally model continuous-time effects and interventions link link
    • causality!

Background

video (23min) link task: calculates p(y|x)

Bayesian Linear Regression

• prior knowledge + observed data to make predictions (=posterior distribution)
• Linear regression + probability distribution for the model parameters
  • uncertainty quantification
• mean linear function + covariance function -> uncertainty

  Gaussian Processes (GP)

• uses more general functions instead of linear functions (as in Bayesian Linear Regression)
• assign a distribution to every possible function that could describe a set of data points
• model complex, non-linear relationships between variables (good at sparse data, undefined relationships between variables, continual/gradual changes)
• the behavior of a Gaussian process is determined by a kernel function
  • aka covariance function
  • defines the similarity between different input points
  • the choice of kernel influences the characteristics of the functions that the Gaussian process can generate
  • common kernels (different definitions, different hyperparameters):
    • radial basis function (RBF) kernel: captures smooth, continuous functions
    • Matérn kernel: flexible and can model more abrupt changes
    • combining kernels
  • hard-to-designing