Functional Data Analysis

Intelligent Data Analysis 2024

The chat-link

Requirements for the text and discussion (talk):

1. Comprehensive explanation of the method or the question we discuss
2. Only the principle, no experiments
3. Two-page text (more or less)
4. The reader is a second or third-year student
5. The picture is obligatory
6. However, a brief reference to some deep learning structure is welcome
7. Talk could be a slide or a text itself
8. The list of references with doi
9. Tell how it was generated
10. Observing a gap, put a note about it (to question later)

First series of questions

1. Canonical Correlation Analysis: forecasting model and loss function with variants
2. Canonical Correlation Analysis in tensor representation
3. CCA parameter estimation algorithm
4. Connection CCA and Cross-Attention
5. Generative CCA
6. Comparative analysis of variants of CCA like PLS and others

Motivation

Automatic generation of mediocre-quality texts increased requirements for the quality of the new messages. It makes novelty rare and makes the authorship appreciated. But it simplifies the ways of delivering. So since textbook generation has become simple, we will use generative chats to train our skills of reader persuasion. The reader is our MS-thesis defense committee.

General

1. Artificial Intelligence for Science in Quantum, Atomistic, and Continuum Systems arxiv 2023
2. Algebra, Topology, Differential Calculus, and Optimization Theory For Computer Science and Machine Learning upenn 2024
3. The Elements of Differentiable Programming arxiv 2024
4. The list from the previous year 2023.

Prerequisites

1. Understanding Deep Learning by Simon J.D. Prince mit 2023
2. Deep Learning by C.M. and H. Bishops Springer 2024 (online version)
3. A Geometric Approach to Differential Forms by David Bachman arxiv 2013
4. A Geometric Approach to Differential Forms by David Bachman arxiv 2013

1. Linear models

1. A Tutorial on Independent Component Analysis arxiv, 2014
2. On the Stability of Multilinear Dynamical Systems arxiv 2022
3. Tensor-based Regression Models and Applications by Ming Hou Thèse Uni-Laval 2017

Sp

1. Spherical Harmonics in p Dimensions arxiv 2012
2. Physics of simple pendulum a case study of nonlinear dynamics RG 2008

SSM

1. Missing Slice Recovery for Tensors Using a Low-rank Model in Embedded Space arxiv 2018

SSM+generative

1. (FLOW tex source) Masked Autoregressive Flow for Density Estimation arxiv 2017

SSM+Riemann+Gaussian process regression

• Time-series forecasting using manifold learning, radial basis function interpolation, and geometric harmonics by Ioannis G. Kevrekidis,3 and Constantinos Siettos, 2022 pdf

PINN

1. Three ways to solve partial differential equations with neural networks — A review arxiv 2021
2. NeuPDE: Neural Network Based Ordinary and Partial Differential Equations for Modeling Time-Dependent Data arxiv 2019
3. Physics-based deep learning code
4. PINN by Steve Burton yt

2. Riemmanian models

Generative

1. Riemannian Continuous Normalizing Flows arxiv 2020

3. Neural ODE

Neural Spatio-Temporal Point Processes by Ricky Chen et al. iclr 2021 (likelihood for time and space)

1. Neural Ordinary Differential Equations by Ricky Chen et al. arxiv 2018
2. Neural Controlled Differential Equations for Irregular Time Series 'Patrick Kidger et al. arxiv 2020github
3. Diffusion Normalizing Flow arxiv 2021
4. Differentiable Programming for Differential Equations: A Review arxiv 2024
5. (code tutorial) Deep Implicit Layers - Neural ODEs, Deep Equilibirum Models, and Beyond nips 2020
6. (code tutorial) 2021

CDE

Neural CDE and tensors https://ieeexplore.ieee.org/abstract/document/9979806 https://ieeexplore.ieee.org/abstract/document/9533771

4. Graph and PDEs

1. Fourier Neural Operator for Parametric Partial Differential Equations arxiv 2020

supplimentary

1. Masked Attention is All You Need for Graphs arxiv 2024

4. Neural SDE

1. Approximation of Stochastic Quasi-Periodic Responses of Limit Cycles in Non-Equilibrium Systems under Periodic Excitations and Weak Fluctuations mdpi entropy 2017 (great illustrations on the stochastic nature of a simple phase trajectory)
2. Approximation of Stochastic Quasi-Periodic Responses of Limit Cycles in Non-Equilibrium Systems under Periodic Excitations and Weak Fluctuations mdpi entropy 2017 (great illustrations on the stochastic nature of a simple phase trajectory)
3. Neural SDEs for Conditional Time Series Generation arxiv 2023 code github LSTM - CSig-WGAN
4. Neural SDEs as Infinite-Dimensional GANs 2021
5. Efficient and Accurate Gradients for Neural SDEs by Patrick Kidger arxiv 2021 code diffrax

5. PINN and Neural PDE

1. Process Model Inversion in the Data-Driven Engineering Context for Improved Parameter Sensitivities mdpi processes 2022 (nice connection pictures)
2. Physics-based Deep Learning github

6. Chains and homology

1. Operator Learning: Algorithms and Analysis arxiv 2024
2. Homotopy theory for beginners by J.M. Moeller ku.dk 2015 (is it a pertinent link?)

To research

1. Explorations in Homeomorphic Variational Auto-Encoding arxiv 2018
2. Special Finite Elements for Dipole Modelling master thesis Bauer 2011

Appendix

1. Neural Memory Networks stanford reports 2019
2. An Elementary Introduction to Information Geometry by Frank Nielsen [An Elementary Introduction to Information Geometry Frank Nielsen mdpi entropy
3. The Many Faces of Information Geometry by Frank Nielsen ams 2022 (short version)
4. Clifford Algebras and Dimensionality Reduction for Signal Separation by M. Guillemard Uni-Hamburg 2010code
5. Special Finite Elements for Dipole Modelling by Martin Bauer Master Thesis Erlangen 2012 diff p-form must read
6. Bayesian model selection for complex dynamic systems 2018
7. Visualizing 3-Dimensional Manifolds by Dugan J. Hammock 2013 umass
8. At the Interface of Algebra and Statistics by T-D. Bradley arxiv 2020
9. Time Series Handbook by Borja, 2021 github