Physics-based DL models symmetry

• Physics-based DL models symmetry
  • Main resources
  • Abstract
  • Results
  • Video reviews
  • Literature
    • General
    • Models
    • Symmetry and Noether's theorem
    • Useful code links
    • Other intresting

Main

• Master Thesis
• Master Thesis presentation
• Code for experiments
• Code Usage

Abstract

В работе решается задача выбора оптимальной модели предсказания динамики физической системы. Под динамикой системы понимается изменение во времени параметров системы. Нейронные сети не имеют априорных знаний о моделируемой системе, что не позволяет получить оптимальные параметры, учитывающие физические законы. Лагранжева нейронная сеть учитывает закон сохранения энергии при моделировании динамики. В данной работе пред- лагается Нётеровская агранжева нейронная сеть, учитывающая законы сохранения импульса и момента импульса в дополнение к закону сохранения энергии. Показано, что для данной задачи Нётеровская Лагранжева нейронная сеть является оптимальной среди полносвязной модели нейронной сети, нейронной сети с долговременной кратковременной памятью и Лагранжевой нейронной сетью. Сравнение моделирования проводилось на искусственно сгенерированных данных для системы двойного маятника, которая является простейшей хаотической системой. Результаты экспериментов подтверждают гипотезу, что внесение априорного знания о физике системы повышает качество модели.

Results

Video reviews by P.Severilov

• Hamiltonian and Lagrangian Neural Networks
• Fourier Neural Operator for PDEs

Literature

General

• [book] Physics-based Deep Learning (Book Website)
• [links] Differential Equations in Deep Learning (Neural ODEs, Neural GDEs, Neural SDEs, Neural CDEs)
• [paper] A Theoretical Analysis of Deep Neural Networks and Parametric PDEs
• Overview of Physics-Based Machine Learning:
  • [paper] Physics-Guided Deep Learning for Dynamical Systems (!)
  • [paper] Integrating Machine Learning with Physics-Based Modeling
  • [paper] Integrating Scientific Knowledge with Machine Learning for Engineering and Environmental Systems
  • [links] Neural Operator approaches links
  • (optional) [paper] Scientific Machine Learning through Physics-Informed Neural Networks: Where we are and What’s next

Models

• HNN: [paper | code] (Hamiltonian Neural Networks)
• LNN: [paper | code | lecture] (Lagrangian Neural Networks)
• DeLaN: [paper1 paper2 paper3 | code | other_related_projects] (Deep Lagrangian Networks)
• FNO: [paper | code | blog_post] (Fourier Neural Operator for Parametric Partial Differential Equations)
• PINO: [paper | code] (Physics-Informed Neural Operator for Learning Partial Differential Equations)
• DeepONet: [paper | code | presentation] (DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators)
• NeuralODE: [paper | code] (Neural Ordinary Differential Equations)
• DiffCoSim: [paper | code] (Extending Lagrangian and Hamiltonian Neural Networks with Differentiable Contact Models)
• PINNs: [paper1 paper2 paper3 | code] (Physics-informed neural networks)
• (???) PGNN: [paper | code] (Physics-guided Neural Networks (PGNN): An Application in Lake Temperature Modeling)

Symmetry and Noether's theorem

• [theory] Теоремы Нётер
• [paper] Noether's Learning Dynamics: Role of Symmetry Breaking in Neural Networks
• [blog_post] Noether’s Theorem, Symmetries, and Invariant Neural Networks
• paper | lecture | slides | blog_post Neural Mechanics: Symmetry and Broken Conservation Laws in Deep Learning Dynamics
• [paper] Noether: The More Things Change, the More Stay the Same
• [paper] Interpretable conservation law estimation by deriving the symmetries of dynamics from trained deep neural networks

Useful code links

• A self-contained tutorial for LNNs.ipynb
• (???) Jupyter notebooks with 3 approaches to solve PDEs by NNs (PINNs, Feynman-Kac solver, Deep BSDE solver) (PDF)
• PhyCRNet (Physics-informed convolutional-recurrent neural networks for PDEs)
• Libraries for physics-informed learning with NNs:
  • DeepXDE (paper !!!)
  • PyDEns

Other intresting:

• [links] links to many Physics-Based Deep Learning papers
• [videos] Talks on physics and ML
• ! [paper] Unsupervised Learning of Lagrangian Dynamics from Images for Prediction and Control
• ! [paper] Noether Networks: Meta-Learning Useful Conserved Quantities | lecture
• ! [paper] Meta-Auto-Decoder for Solving Parametric Partial Differential Equations
• [paper] Identifying Physical Law of Hamiltonian Systems via Meta-Learning
• [paper] Discovering Symbolic Models from Deep Learning with Inductive Biases
• [paper] Visual Interaction Networks: Learning a Physics Simulator from Video
• [paper] Neural Network Augmented Physics Models for Systems with Partially Unknown Dynamics
• [paper] Model Reduction And Neural Networks For Parametric PDEs