Difference between revisions of "Fundamental theorems"
From m1p.org
| Line 5: | Line 5: | ||
|description=The course Fundamental Theorems of Machine Learning studies techniques and practice of theorem formulations and proofs in machine learning. | |description=The course Fundamental Theorems of Machine Learning studies techniques and practice of theorem formulations and proofs in machine learning. | ||
}} | }} | ||
| − | + | To make the results of scientific research well-founded, introduce the techniques and practices of theorem formulations and proofs in machine learning. | |
| − | + | Why does one need to convey an important message, a scientific result, as a theorem? | |
| − | |||
| − | Why one | ||
# Theorems are the most important messages in the field of research. | # Theorems are the most important messages in the field of research. | ||
# Theorems present results in the language of mathematics by generality and rigor. | # Theorems present results in the language of mathematics by generality and rigor. | ||
# Theorems are at the heart of mathematics and play a central role in its aesthetics. | # Theorems are at the heart of mathematics and play a central role in its aesthetics. | ||
| − | Theorems present the message immediately and leave reasoning | + | Theorems present the message immediately and leave reasoning for later. The direct narration puts reason first and the results later. |
* How does direct narration transform into fast narration? | * How does direct narration transform into fast narration? | ||
* How to find, state, and prove theorems in our work? | * How to find, state, and prove theorems in our work? | ||
| − | + | Both narration styles refer to progressions | |
| − | # | + | # Textbook: Definition <math>\to</math> (Axiom set) <math>\to</math> Theorem <math>\to</math> Proof <math>\to</math> Corollaries <math>\to</math> Examples <math>\to</math> Impact to applications |
| − | + | # Scientific discovery: Application problems <math>\to</math> Problem generalisations <math>\to</math> Useful algebraic platform <math>\to</math> Definitions <math>\to</math> Axiom set | |
| − | # Scientific discovery | + | In practice, we mimic the first part of the progression, then learn to discover patterns and formulate theorems. |
| − | |||
| − | |||
==Theorems== | ==Theorems== | ||
Revision as of 15:17, 8 February 2026
To make the results of scientific research well-founded, introduce the techniques and practices of theorem formulations and proofs in machine learning.
Why does one need to convey an important message, a scientific result, as a theorem?
- Theorems are the most important messages in the field of research.
- Theorems present results in the language of mathematics by generality and rigor.
- Theorems are at the heart of mathematics and play a central role in its aesthetics.
Theorems present the message immediately and leave reasoning for later. The direct narration puts reason first and the results later.
- How does direct narration transform into fast narration?
- How to find, state, and prove theorems in our work?
Both narration styles refer to progressions
- Textbook: Definition \(\to\) (Axiom set) \(\to\) Theorem \(\to\) Proof \(\to\) Corollaries \(\to\) Examples \(\to\) Impact to applications
- Scientific discovery: Application problems \(\to\) Problem generalisations \(\to\) Useful algebraic platform \(\to\) Definitions \(\to\) Axiom set
In practice, we mimic the first part of the progression, then learn to discover patterns and formulate theorems.
Contents
Theorems
- Fundamental theorem of linear algebra S
- Singular values decomposition and spectral theorem W
- Gauss–Markov-(Aitken) theorem W
- Principal component analysis W
- Karhunen–Loève theorem W
- Kolmogorov–Arnold representation theorem W
- Universal approximation theorem by Cybenko W
- Deep neural network theorem Mark
- Inverse function theorem and Jacobian W
- No free lunch theorem by Wolpert W
- RKHS by Aronszajn and Mercer's theorem W
- Representer theorem by Schölkopf, Herbrich, and Smola W
- Convolution theorem (FT, convolution, correlation with CNN examples) W
- Fourier inversion theorem W
- Wiener–Khinchin theorem about autocorrelation and spectral decomposition W
- Parseval's theorem (and uniform, non-uniform convergence) W
- Probably approximately correct learning with the theorem about compression means learnability
- Bernstein–von Mises theorem W
- Holland's schema theorem W
- Variational approximation
- Convergence of random variables and Kloek's theorem W
- Exponential family of distributions and Nelder's theorem
- Multi-armed bandit theorem
- Copulas and Sklar's theorem W
- Boosting theorem Freud, Shapire, 1996, 1995
- Bootstrap theorem (statistical estimations): Ergodic theorem
Each class contains
- A lecturer's talk on one of fundamental theorems (\(40' = 30' + 10'\) discussion)
- Two students' talks (each \(20' = 15' + 5'\) discussion)
Each student delivers two talks
- On a theorem, which is formulated in a paper from the list of student thesis work's references
- On a theorem, which is formulated and proved by the student
It is welcome to
- Make variants of our formulations and proofs
- Re-formulate significant messages of researchers and formulate these messages as theorems
Plan of the talk
- Introduction: the main message briefly
- If necessary (it could be introduced during the talk)
- Axiom sets
- Definitions
- Algebraic structures
- Notations
- Theorem formulation and exact proof
- The author's variant of the proof could be ameliorated
- Corollaries
- Theorem significance and applications
Typography
- As one (or two) text page example, template to download
- Please
- set the font size \(\geqslant 14\)pt
- include plots, diagrams, freehand drawings
The organization
- GitHub project to upload your text Intelligent-Systems-Phystech/FundamentalTheoremsML to the group folder upload the pdf, tex, fig files named as Surname2021Literature, Surname2021Research
- See the Youtube channel Machine Learning
- Spring semester, Wednesdays 14:30 at Zoom m1p.org/go_zoom
Scoring
- Talks and text 0-4 points, according to comparison
- Out-of-schedule drops a half
- The exam 2 points: schemes of proof of various theorems
- time-limit test (as Physics state exam) and discussion
- theorem formulation and poof scheme are hand-written
- two random theorems from the list below, 10 min to write the text
Theorem types
- Uniqueness, existence
- Universality
- Convergence
- Complexity
- Properties of estimations
- Bounds
Schedule
Spring semester 2021
Student talks
| Speaker | References |
|---|---|
| Bishuk Anton | 17.2 link |
| Weiser Kirill | 17.2 link, link |
| Grebenkova Olga | 24.2 link |
| Gunaev Ruslan | 24.2 link |
| Zholobov Vladimir | 3.3 link |
| Islamov Rustem | 3.3 link |
| Pankratov Victor | 10.3 link |
| Savelyev Nikolay | 10.3 link |
| Filatov Andrey | 10.3 link |
| Filippova Anastasia | 17.3 link |
| Khar Alexandra | 17.3 link |
| Khristolyubov Maxim | 24.3 link |
| Shokorov Vyacheslav | 24.3 link |
Invited talks
| Speaker | Link |
|---|---|
| Strijov, Potanin | 10.2 link |
| Mark Potanin | 17.2 link |
| Mark Potanin | 24.2 link |
| Andriy Grabovyi | 10.3 link |
| Andriy Grabovyi | 17.3 link |
| Andriy Grabovyi | 24.3 link |
| Mark Potanin | 31.3 link |
Out of schedule
- Three works by Greg Yang arXiv:1910.12478, arXiv:2006.14548, arXiv:2009.10685 Youtube Rus
- Theorems on flows by Johann Brehmera and Kyle Cranmera arXiv:2003.13913v2
References
- Mathematical statistics by A.A. Borovkov, 1998
- Learning Theory from First Principles by Francis Bach, 2021
- Theoretical foundations of potential function method in pattern recognition by M. A. Aizerman, E. M. Braverman, and L. I. Rozonoer // Avtomatica i Telemekhanica, 1964. Vol. 25, pp. 917-936.
Proof techniques
- Proofs and Mathematical Reasoning by Agata Stefanowicz, 2014
- The nuts and bolts of proofs by Antonella Cupillari, 2013
- Theorems, Corollaries, Lemmas, and Methods of Proof by Richard J. Rossi, 1956
- Problem Books in Mathematics by P.R. Halmos (editor), 1990
- Les contre-exemples en mathématique par Bertrand Hauchecorne, 2007
- Kolmogorov and Mathematical Logic by Vladimir A. Uspensky // The Journal of Symbolic Logic, Vol. 57, No. 2 (Jun., 1992), 385-412.
- Что такое аксиоматический метод? В.А. Успенский, 2001
- Аксиоматический метод. Е.Е. Золин, 2015
Methodology
- Introduction to Metamathematics by Stephen Cole Kleene, 1950
- Science and Method by Henry Poincare, 1908
- A Summary of Scientific Method by Peter Kosso, 2011
- Being a Researcher: An Informatics Perspective by Carlo Ghezzi, 2020
- The definitive glossary of higher mathematical jargon by Math Vault, 2015
- The definitive guide to learning higher mathematics: 10 principles to mathematical transcendence by Math Vault, 2020
- List of mathematical jargon on Wikipedia
- Пикабу. Типичные методы доказательства, 2018 (если вы чувствуете, что несет не туда)
