Difference between revisions of "Mathematical forecasting"

From Research management course
Jump to: navigation, search
 
(89 intermediate revisions by 2 users not shown)
Line 1: Line 1:
−
'''[https://bit.ly/3QAOYPd Current version, October 2022, is here]'''
+
{{#seo:
 +
|title=Machine learning and forecasting
 +
|titlemode=replace
 +
|keywords=machine learning and forecasting
 +
|description=This course delivers model selection methods in machine learning and forecasting. The models are linear, tensor, deep neural networks, and neural differential equations.
 +
}}
 +
The additional part moved to [[Functional Data Analysis]]  
  
−
==Main topics==
+
==Motivation==
 +
This course delivers methods of model selection in machine learning and forecasting. The models are linear, tensor, deep neural networks, and neural differential equations. The modeling data are videos, audios, encephalograms, fMRIs, and other measurements in natural science.  The practical examples are brain-computer interfaces, weather forecasting, and various spatial-time series forecasting. The lab works are organized as paper-with-code reports.
 +
 
 +
The course joins two parts of the problem statements in Machine Learning. The first part comes from the structure of the measured data. The data come from Physics, Chemistry, and Biology and have intrinsic algebraic structures. These structures are parts of the theory that stands behind the measurement. The second part comes from errors in the measurement. The stochastic nature of errors requires statistical methods of analysis. So this course joins algebra and statistics. It is devoted to the problem of predictive model selection.
 +
 
 +
Mathematical forecasting methods play a crucial role in scientific research and industry. The distinction between forecasting and machine learning methods lies in the algebraic structures. We build forecasting models not only in vector spaces but also in vector fields. These fields include time and space and have a continuous nature. We propose a holistic approach to teaching this course: we must consider mathematical methods that combine continuous-time high-dimensional vector and tensor fields. We discuss linear, differential, and non-linear models. We introduce model ensembles to reveal the source and target space dependencies.
 +
 
 +
<!--
 +
My specialization is Mathematical Forecasting. I present a new view of this field of knowledge: the forecasting problems deal not with the vector spaces but with the vector fields. The main subject is vector and tensor fields over time and space. The modeling data are spatial-time series: audio-video streams, brain signals, and images, biomedical live measurements, wearable device sensor signals, and other signals in biology and physics. The practical applications for study and labwork are brain-computer interface, human motion, and human health monitoring. The course is organized into eight sections: autoregressive models, tensor decomposition, canonic correlation analysis, continuous-time analysis, dynamic systems, spatial-time alignment, metrics learning, and diffusion-graphical models. Each section runs labwork with various practical applications in Python.
 +
-->
 +
 
 +
== Lectures ==
 +
===Main topics===
 +
# Autoregression and singular structure analysis
 +
# Tensor decomposition and spatial-time models
 +
# Signal decoding and multi-modeling
 +
# Space alignment
 +
# Convergent cross-mapping and dynamic systems
 +
# Continuous-time forecasting and Neural ODEs
 +
<!--# Riemannian andMetrics learning
 +
# Diffusion-graph PDEs
 
# [[Autoregression]] and singular structure analysis
 
# [[Autoregression]] and singular structure analysis
 
# [[Tensor decomposition]] and spatial-time models
 
# [[Tensor decomposition]] and spatial-time models
Line 10: Line 36:
 
# Riemann, Metrics learning
 
# Riemann, Metrics learning
 
# Diffusion-graph PDEs
 
# Diffusion-graph PDEs
−
 
+
-->
 +
=== Fall semester ===
 +
# '''Introduction'''
 +
#* Semester overview, motivation, homework labs, exams
 +
#* Time and space in forecasting application problems
 +
#* Linear, neural, and memory forecasting models
 +
# '''Phase space approximate'''
 +
#* Singular spectrum analysis and forecasting
 +
#* k-linear forms, Principal component analysis
 +
#* Singular values decomposition
 +
# '''Basic models'''
 +
#* Cross-correlation
 +
#* Stochastic processes, autoregression, GARCH
 +
#* Non-parametric regression and kernels
 +
#* Error functions, residue convolution model, and analysis
 +
# '''Fourier transform'''
 +
#* Discrete transforms, wavelet transform
 +
#* Gabor transform and spectrogram
 +
#* 2d transform, Gerchberg–Saxton algorithm
 +
# '''Higher-order linear models'''
 +
#* Tensors and Penrose notation
 +
#* Tucker decomposition and alternated least squares
 +
#* Higher-order singular values decomposition
 +
# '''Neural models'''
 +
#* Convolutions for time and space
 +
#* Recursive, Hopefield, and Memory models
 +
#* Sequential models with attention
 +
# '''Canonical correlation analysis'''
 +
#* Projection to latent space
 +
#* PLS as SVD, model optimization, and selection
 +
#* Higher-order PLS
 +
# '''Time and space alignment'''
 +
#* Dynamic time warping
 +
#* Dynamic barycenter averaging
 +
#* Self-modeling regression
 +
# '''Causality detection'''
 +
#* Granger test
 +
#* Convergent cross-mapping
 +
#* Dynamic system and Taken's theorem
 +
# '''Differential models'''
 +
#* Residual neural networks
 +
#* Neuro-ODE and its solution
 +
#* Splines, Controlled neuro-ODE
 +
# '''State-space representation'''
 +
#* Linear differential models
 +
#* Partial differential models
 +
#* Memory models
 +
# '''Forecasting and control'''
 +
#* Control models
 +
#* Controllability and feedback
 +
#* Proportional integral derivative controller
 
==Lab works==
 
==Lab works==
 +
'''[https://bit.ly/3QAOYPd Current labworks, October 2022, is here]'''
 
Lab work contains a report in the pynb or TeX format and a talk with a discussion
 
Lab work contains a report in the pynb or TeX format and a talk with a discussion
−
#Title and motivated abstract
+
# Title and motivated abstract
−
#Problem statement
+
# Problem statement
−
#Model, problem solution
+
# Model, problem solution
−
#Code, analysis, and illustrative plots
+
# Code, analysis, and illustrative plots
 
#References
 
#References
−
Note: '''the model''' (and sometimes data) are '''personal contribution'''. The rest – infrastructure, error functions, and plots, are welcome to be created '''collectively''' and shared.
+
Note: '''the model''' is the '''personal''' contribution. The infrastructure: data acquisition, data uploads, error functions, and plots are welcome to be created '''collectively''' and shared.
−
 
+
===Topics of the lab works (Fall)===
−
'''Topics of the lab works are'''
+
*Autoregressive forecasting – Singular structure Analysis
−
*Autoregressive forecasting (Singular Structure Analysis)
+
*Spatial-time forecasting – Tensor decomposition
−
*Spatial-time forecasting (Tensor Decomposition)
+
*Signal decoding – Projection to latent space
−
*Signal decoding (Projection to Latent Space)
+
*Continuous-time forecasting – Neural differential equations
−
*Continuous-time forecasting (Neural Differential Equations)
+
===Example of the lab report===
−
 
+
* Put here
 +
===Format of lab works===
 +
# Create a .pynb or .py file Surname2022Lab [http://is-mipt.site in the folder]
 +
# The report also could be in the .tex file.
 +
# Find the format of your report above.
 +
# The computational experiment contains common part and individual part.
 +
# Common part:
 +
## use four short sample set [airplane], [electricity], [accelerometer hand motion], [video hand motion],
 +
## prepare the design matrix and target a scalar/vector for each time sample (in the form time, vecx, vecy),
 +
## set the forecast horizon, plot the forecast, and estimate the error.
 +
# Individual part:
 +
## select a lab work and specify your model (you can adopt any code available),
 +
## tune parameters, make your forecast according to the horizon,
 +
## write the report.
 +
# Error analysis is a part of the report: <!-- Three functions: model, quality criterion, forecast, parameter optimization (and model selection)-->
 +
# plot of the forecast,
 +
# MAPE error (and your optimization error, if available) and its standard deviation,
 +
# '''prove your model has the optimal structure''', try various structure parameters.
 +
Details:
 +
# time refers to each sample (in unix or any useful format),
 +
# the horizon is an expected fundamental period,
 +
# '''note that''' the historical time ends before the forecasting period, it means we could use either historical data or the forecasted data (the historical data are not updated after history ends),
 +
# the forecasting protocol is in [http://strijov.com/papers/Uvarov2018SuperpositionForecasting.pdf parer], [http://www.machinelearning.ru/wiki/images/5/5d/Uvarov2018Thesis.pdf text], [http://www.machinelearning.ru/wiki/images/c/cc/Uvarov2018Presentation.pdf slides] by Nikita Uvarov.
 +
Examples:
 +
# [https://sourceforge.net/p/mlalgorithms/code/HEAD/tree/TSForecasting/KernelSmoothing/Kuznetsov11Smoothing.pdf?format=raw Old format of the report]
 +
# [https://sourceforge.net/p/mlalgorithms/code/HEAD/tree/TSForecasting/KernelSmoothing/ Code and project]
 +
# Previous project [https://sourceforge.net/p/mlalgorithms/code/HEAD/tree/TSForecasting/ from Sourceforge.net]
 
==Discussion and collaboration==
 
==Discussion and collaboration==
 
* [https://t.me/is_mmp Telegram]
 
* [https://t.me/is_mmp Telegram]
 
* [https://github.com/intsystems/MathematicalForecastingMethods GitHub]
 
* [https://github.com/intsystems/MathematicalForecastingMethods GitHub]
−
 
 
==Exam and grading==
 
==Exam and grading==
 
Four lab works within deadlines and the exam on topics with problems and discussion.
 
Four lab works within deadlines and the exam on topics with problems and discussion.
 
Each lab gives 2pt, and the exam gives 2pt, so 2*4+2=10.
 
Each lab gives 2pt, and the exam gives 2pt, so 2*4+2=10.
−
 
−
==Motivation==
 
−
Mathematical forecasting methods play a crucial role in scientific research and industry. The distinction between forecasting and machine learning methods lies in the algebraic structures. We build forecasting models not only in vector spaces but in vector fields. These fields include time and space and have continuous nature. We propose a holistic approach to teaching this course: we must consider mathematical methods that combine continuous-time high-dimensional vector and tensor fields. We discuss linear, differential, and non-linear models. We introduce model ensembles to reveal both the source and the target space dependencies.
 
−
 
−
==Abstract==
 
−
This course delivers methods of model selection in machine learning and forecasting. The models are linear, tensor, deep neural networks, and neural differential equations. The modeling data are videos, audios, encephalograms, fMRIs, and other measurements in natural science.  The practical examples are brain-computer interfaces, weather forecasting and various spatial-time series forecasting. The lab works are organized as paper-with-code reports.
 
−
 
−
The course joins two parts of the problem statements in Machine Learning. The first part comes from the structure of the measured data. The data come from Physics, Chemistry, and Biology and have intrinsic algebraic structures. These structures are parts of the theory that stands behind the measurement. The second part comes from errors in the measurement. The stochastic nature of errors requires statistical methods of analysis. So this course joins algebra and statistics. It is devoted to the problem of predictive model selection.
 
−
 
 
==Terminology and notation==
 
==Terminology and notation==
−
# ''Feature selection'' in Katrutsa A.M., Strijov V.V. 2017. Comprehensive study of feature selection methods to solve multicollinearity problem according to evaluation criteria // Expert Systems with Applications [https://doi.org/10.1016/j.eswa.2017.01.048 DOI]
+
# ''Feature selection'' in Katrutsa A.M., Strijov V.V. 2017. A comprehensive study of feature selection methods to solve multicollinearity problem according to evaluation criteria // Expert Systems with Applications [https://doi.org/10.1016/j.eswa.2017.01.048 DOI]
−
# ''Tensor decomposition'' in in Motrenko A.P., Strijov V.V. 2018. Multi-way feature selection for ECoG-based brain-computer interface // Expert Systems with Applications [https://doi.org/10.1016/j.eswa.2018.06.054 DOI]
+
# ''Tensor decomposition'' in Motrenko A.P., Strijov V.V. 2018. Multi-way feature selection for ECoG-based brain-computer interface // Expert Systems with Applications [https://doi.org/10.1016/j.eswa.2018.06.054 DOI]
 
# ''Signal decoding'' in R.V.IsachenkoV.V.Strijov. 2022. Quadratic programming feature selection for multicorrelated signal decoding with partial least squares // Expert Systems with Applications [https://doi.org/10.1016/j.eswa.2022.117967 DOI]
 
# ''Signal decoding'' in R.V.IsachenkoV.V.Strijov. 2022. Quadratic programming feature selection for multicorrelated signal decoding with partial least squares // Expert Systems with Applications [https://doi.org/10.1016/j.eswa.2022.117967 DOI]
 
# ''Forecasting schedule and horizon'' in Uvarov N.D. et al. 2018. Selecting the Superpositioning of Models // Computational Mathematics and Cybernetics [https://link.springer.com/article/10.3103/S027864191804009X DOI]
 
# ''Forecasting schedule and horizon'' in Uvarov N.D. et al. 2018. Selecting the Superpositioning of Models // Computational Mathematics and Cybernetics [https://link.springer.com/article/10.3103/S027864191804009X DOI]
−
 
+
<!--
−
=Topics=
+
See PhD thesis by [http://www.frccsc.ru/sites/default/files/docs/ds/002-073-05/diss/31-isachenko/ds05_31-isachenko_main.pdf?18 Roman Isachenko]
 +
See also
 +
*Tensor terminology in the PhD thesis by [https://sourceforge.net/p/mlalgorithms/code/HEAD/tree/PhDThesis/Motrenko/doc/Motrenko2018Thesis.pdf Anastasia Motrenko]
 +
*Model selection terminology in the PhD theses by [http://www.frccsc.ru/sites/default/files/docs/ds/002-073-05/diss/26-bahteev/ds05-26-bahteev_main.pdf?28 Oleg Bakhteev] and by [https://www.frccsc.ru/sites/default/files/docs/ds/002-073-05/diss/35-grabovoy/ds05-35-grabovoy_main.pdf?90 Andery Grabovoy]
 +
-->
 +
==Topics==
 +
===Fall===
 +
* Energy forecasting example
 +
* Regression
 +
* Linear model
 +
* Model selection call
 +
* Forecasting protocol
 +
* Error functions
 +
* Singular spectrum analysis
 +
* SSA forecasting
 +
* Forecasting protocols and verification (before AR)
 +
* Autoregression
 +
* Singular values decomposition (PCA, AE, Kar-Lo)
 +
* QPFS model selection
 +
* Auto, cross-correlation, cointegration
 +
* Diagrams for ML and PLS
 +
* Projection to latent space and relation to PCA, canonical-correlation analysis
 +
* PLS-QPFS model selection
 +
* Higher-order SSA
 +
* Tensor decomposition
 +
* Tensor model selection
 +
* HOPLS
 +
* Granger causality test
 +
* Convergent cross mapping
 +
* HOCCM to invent
 +
* Taken’s theorem
 +
* ResNet, Neural ODE
 +
* Adjoint and back-propagation
 +
* Flows and forecasting
 +
===Spring===
 +
* Space state models
 +
* S4, Hippo, SaShiMi models
 +
* RNN, LSTM, attention, transformer models
 +
* Neural PDE, Lagrangian, Hamiltonian nns.
 +
* Directional regression
 +
* Harmonic functions
 +
* Phase extraction
 +
* Non-parametric regression and customer demand forecasting
 +
* Graph earth prediction
 +
* Convolutional models
 +
* Graph convolutions and spectrum
 +
* Fourier transform and phase retrieval problem
 +
* Radon transform and tomography reconstruction
 
* Forward and inverse problems, kernel regularisation
 
* Forward and inverse problems, kernel regularisation
 
* Karhunen–Loeve theorem, FPCA
 
* Karhunen–Loeve theorem, FPCA
Line 56: Line 196:
 
* Graph convolution
 
* Graph convolution
 
* Manifolds and local models
 
* Manifolds and local models
 +
* Statistics on Riemannian spaces
 +
* Statistics on stratified spaces
 +
Appendix to Spring
 +
* Probabilistic diffusion and Graphs
 +
* Graph convolution, graph representation
 +
* Neural diffusion and PDEs, GRAND
 +
* Tensors and Ricci flow, and PDE
 +
* Remmannian, Ricci tensors
 +
* Differential forms
 +
* Metrics learning and SDP
 +
* Taken's ODE
 +
Appendix-2 to Spring
 +
* ResNet, LSTM, etc
 +
* Neuro ODE, RK4
 +
* Controlled ODE, Visualization
 +
* BackProp
 +
* S4, memories
 +
* Graph convolution
 +
* Graph Laplacian
 +
* Differentiation of Graph Laplacian
 +
* Riemannian?? Py
 +
* GRAND
 +
* Neuro PDE, Galerkin
 +
* The inverse problem of brain signals
 +
* Laplacian, miltoniian NNs?
 +
Does dot product create a metric space? //
 +
Only when the dot product is considered bilinear with at least the trivial metric tensor //
 +
Singular Spectrum, Phase Space Bing HAVOK!!!! //
 +
Linear models, SVD //
 +
Convolution / ARIMA - lags //
 +
State Space (+ Kalman, etc) + Control Theory //
 +
Tensor Decomposition (Tensor ARIMA with Tucker Decomposition) //
 +
Least Squares, Alternated Least Squares //
 +
Tensor convolution Generation and Decomposition //
 +
Feature Selection (lasso-Lars-style, QPFS, Genetic, Tensor Genetic) //
 +
 +
 +
 +
Geometric Algebra
 +
Differential geometry
 +
Fields, Shiefs, etc…
  
−
L3 courses towards machine learning
 
−
* Functional analysis
 
−
* Differential geometry
 
  
−
=References=
+
==Catch-up references==
−
# Functional data analysis by James Ramsay, Bernard Silverman, 2020
+
# Kolmogorov, A.N and Fomin, S.V.: Elements of the Theory of Functions and Functional Analysis, Dover Publications, 1999.
 +
# David Bachman: A Geometric Approach to Differential Forms, Birkhauser Boston, 2006.
 +
# At the Interface of Algebra and Statistics by Tai-Danae Bradley, 2020
 +
<!-- # Functional data analysis by James Ramsay, Bernard Silverman, 2020  
 
# Riemannian geometric statistics in medical image analysis. Edited by Xavier Pennec,  Stefan Sommer, and Tom Fletcher, 2020
 
# Riemannian geometric statistics in medical image analysis. Edited by Xavier Pennec,  Stefan Sommer, and Tom Fletcher, 2020
−
# Manifolds, tensors and forms by Paul Renteln, 2014
+
# Manifolds, tensors, and forms by Paul Renteln, 2014 -->
 
# Theoretical Foundations of Functional Data Analysis, with an Introduction to Linear Operators by Tailen Hsiing, Randall Eubank, 2013
 
# Theoretical Foundations of Functional Data Analysis, with an Introduction to Linear Operators by Tailen Hsiing, Randall Eubank, 2013
−
# At the Interface of Algebra and Statistics by Tai-Danae Bradley, 2020
 

Latest revision as of 23:14, 7 October 2024

The additional part moved to Functional Data Analysis

Motivation

This course delivers methods of model selection in machine learning and forecasting. The models are linear, tensor, deep neural networks, and neural differential equations. The modeling data are videos, audios, encephalograms, fMRIs, and other measurements in natural science. The practical examples are brain-computer interfaces, weather forecasting, and various spatial-time series forecasting. The lab works are organized as paper-with-code reports.

The course joins two parts of the problem statements in Machine Learning. The first part comes from the structure of the measured data. The data come from Physics, Chemistry, and Biology and have intrinsic algebraic structures. These structures are parts of the theory that stands behind the measurement. The second part comes from errors in the measurement. The stochastic nature of errors requires statistical methods of analysis. So this course joins algebra and statistics. It is devoted to the problem of predictive model selection.

Mathematical forecasting methods play a crucial role in scientific research and industry. The distinction between forecasting and machine learning methods lies in the algebraic structures. We build forecasting models not only in vector spaces but also in vector fields. These fields include time and space and have a continuous nature. We propose a holistic approach to teaching this course: we must consider mathematical methods that combine continuous-time high-dimensional vector and tensor fields. We discuss linear, differential, and non-linear models. We introduce model ensembles to reveal the source and target space dependencies.


Lectures

Main topics

  1. Autoregression and singular structure analysis
  2. Tensor decomposition and spatial-time models
  3. Signal decoding and multi-modeling
  4. Space alignment
  5. Convergent cross-mapping and dynamic systems
  6. Continuous-time forecasting and Neural ODEs

Fall semester

  1. Introduction
    • Semester overview, motivation, homework labs, exams
    • Time and space in forecasting application problems
    • Linear, neural, and memory forecasting models
  2. Phase space approximate
    • Singular spectrum analysis and forecasting
    • k-linear forms, Principal component analysis
    • Singular values decomposition
  3. Basic models
    • Cross-correlation
    • Stochastic processes, autoregression, GARCH
    • Non-parametric regression and kernels
    • Error functions, residue convolution model, and analysis
  4. Fourier transform
    • Discrete transforms, wavelet transform
    • Gabor transform and spectrogram
    • 2d transform, Gerchberg–Saxton algorithm
  5. Higher-order linear models
    • Tensors and Penrose notation
    • Tucker decomposition and alternated least squares
    • Higher-order singular values decomposition
  6. Neural models
    • Convolutions for time and space
    • Recursive, Hopefield, and Memory models
    • Sequential models with attention
  7. Canonical correlation analysis
    • Projection to latent space
    • PLS as SVD, model optimization, and selection
    • Higher-order PLS
  8. Time and space alignment
    • Dynamic time warping
    • Dynamic barycenter averaging
    • Self-modeling regression
  9. Causality detection
    • Granger test
    • Convergent cross-mapping
    • Dynamic system and Taken's theorem
  10. Differential models
    • Residual neural networks
    • Neuro-ODE and its solution
    • Splines, Controlled neuro-ODE
  11. State-space representation
    • Linear differential models
    • Partial differential models
    • Memory models
  12. Forecasting and control
    • Control models
    • Controllability and feedback
    • Proportional integral derivative controller

Lab works

Current labworks, October 2022, is here Lab work contains a report in the pynb or TeX format and a talk with a discussion

  1. Title and motivated abstract
  2. Problem statement
  3. Model, problem solution
  4. Code, analysis, and illustrative plots
  5. References

Note: the model is the personal contribution. The infrastructure: data acquisition, data uploads, error functions, and plots are welcome to be created collectively and shared.

Topics of the lab works (Fall)

  • Autoregressive forecasting – Singular structure Analysis
  • Spatial-time forecasting – Tensor decomposition
  • Signal decoding – Projection to latent space
  • Continuous-time forecasting – Neural differential equations

Example of the lab report

  • Put here

Format of lab works

  1. Create a .pynb or .py file Surname2022Lab in the folder
  2. The report also could be in the .tex file.
  3. Find the format of your report above.
  4. The computational experiment contains common part and individual part.
  5. Common part:
    1. use four short sample set [airplane], [electricity], [accelerometer hand motion], [video hand motion],
    2. prepare the design matrix and target a scalar/vector for each time sample (in the form time, vecx, vecy),
    3. set the forecast horizon, plot the forecast, and estimate the error.
  6. Individual part:
    1. select a lab work and specify your model (you can adopt any code available),
    2. tune parameters, make your forecast according to the horizon,
    3. write the report.
  7. Error analysis is a part of the report:
  8. plot of the forecast,
  9. MAPE error (and your optimization error, if available) and its standard deviation,
  10. prove your model has the optimal structure, try various structure parameters.

Details:

  1. time refers to each sample (in unix or any useful format),
  2. the horizon is an expected fundamental period,
  3. note that the historical time ends before the forecasting period, it means we could use either historical data or the forecasted data (the historical data are not updated after history ends),
  4. the forecasting protocol is in parer, text, slides by Nikita Uvarov.

Examples:

  1. Old format of the report
  2. Code and project
  3. Previous project from Sourceforge.net

Discussion and collaboration

Exam and grading

Four lab works within deadlines and the exam on topics with problems and discussion. Each lab gives 2pt, and the exam gives 2pt, so 2*4+2=10.

Terminology and notation

  1. Feature selection in Katrutsa A.M., Strijov V.V. 2017. A comprehensive study of feature selection methods to solve multicollinearity problem according to evaluation criteria // Expert Systems with Applications DOI
  2. Tensor decomposition in Motrenko A.P., Strijov V.V. 2018. Multi-way feature selection for ECoG-based brain-computer interface // Expert Systems with Applications DOI
  3. Signal decoding in R.V.IsachenkoV.V.Strijov. 2022. Quadratic programming feature selection for multicorrelated signal decoding with partial least squares // Expert Systems with Applications DOI
  4. Forecasting schedule and horizon in Uvarov N.D. et al. 2018. Selecting the Superpositioning of Models // Computational Mathematics and Cybernetics DOI

Topics

Fall

  • Energy forecasting example
  • Regression
  • Linear model
  • Model selection call
  • Forecasting protocol
  • Error functions
  • Singular spectrum analysis
  • SSA forecasting
  • Forecasting protocols and verification (before AR)
  • Autoregression
  • Singular values decomposition (PCA, AE, Kar-Lo)
  • QPFS model selection
  • Auto, cross-correlation, cointegration
  • Diagrams for ML and PLS
  • Projection to latent space and relation to PCA, canonical-correlation analysis
  • PLS-QPFS model selection
  • Higher-order SSA
  • Tensor decomposition
  • Tensor model selection
  • HOPLS
  • Granger causality test
  • Convergent cross mapping
  • HOCCM to invent
  • Taken’s theorem
  • ResNet, Neural ODE
  • Adjoint and back-propagation
  • Flows and forecasting

Spring

  • Space state models
  • S4, Hippo, SaShiMi models
  • RNN, LSTM, attention, transformer models
  • Neural PDE, Lagrangian, Hamiltonian nns.
  • Directional regression
  • Harmonic functions
  • Phase extraction
  • Non-parametric regression and customer demand forecasting
  • Graph earth prediction
  • Convolutional models
  • Graph convolutions and spectrum
  • Fourier transform and phase retrieval problem
  • Radon transform and tomography reconstruction
  • Forward and inverse problems, kernel regularisation
  • Karhunen–Loeve theorem, FPCA
  • Parametric and non-parametric models
  • Reproductive kernel Hilbert space
  • Integral operators and Mercer theorem Convolution theorem
  • Graph convolution
  • Manifolds and local models
  • Statistics on Riemannian spaces
  • Statistics on stratified spaces

Appendix to Spring

  • Probabilistic diffusion and Graphs
  • Graph convolution, graph representation
  • Neural diffusion and PDEs, GRAND
  • Tensors and Ricci flow, and PDE
  • Remmannian, Ricci tensors
  • Differential forms
  • Metrics learning and SDP
  • Taken's ODE

Appendix-2 to Spring

  • ResNet, LSTM, etc
  • Neuro ODE, RK4
  • Controlled ODE, Visualization
  • BackProp
  • S4, memories
  • Graph convolution
  • Graph Laplacian
  • Differentiation of Graph Laplacian
  • Riemannian?? Py
  • GRAND
  • Neuro PDE, Galerkin
  • The inverse problem of brain signals
  • Laplacian, miltoniian NNs?

Does dot product create a metric space? // Only when the dot product is considered bilinear with at least the trivial metric tensor // Singular Spectrum, Phase Space Bing HAVOK!!!! // Linear models, SVD // Convolution / ARIMA - lags // State Space (+ Kalman, etc) + Control Theory // Tensor Decomposition (Tensor ARIMA with Tucker Decomposition) // Least Squares, Alternated Least Squares // Tensor convolution Generation and Decomposition // Feature Selection (lasso-Lars-style, QPFS, Genetic, Tensor Genetic) //


Geometric Algebra Differential geometry Fields, Shiefs, etc…


Catch-up references

  1. Kolmogorov, A.N and Fomin, S.V.: Elements of the Theory of Functions and Functional Analysis, Dover Publications, 1999.
  2. David Bachman: A Geometric Approach to Differential Forms, Birkhauser Boston, 2006.
  3. At the Interface of Algebra and Statistics by Tai-Danae Bradley, 2020
  4. Theoretical Foundations of Functional Data Analysis, with an Introduction to Linear Operators by Tailen Hsiing, Randall Eubank, 2013
Retrieved from ‘https://m1p.org/index.php?title=Mathematical_forecasting&oldid=1808’