Course syllabus: Category theory for Machine Learning
Contents
Category Theory for Machine Learning (a practical course)
This course introduces the language of category theory as a tool for model selection in machine learning. In this course, we create templates of problem statements and model descriptions. We use these templates in the introductory parts of our research projects, both academic and industrial. It brings us clarity and precision in our main goal: to convey the message to the reader.
The course comprises a series of lab-works and instructions
In a lab-work a student has to 1) formulate a machine learning problem in the CT language, 2) Write an essay, a short document with text, diagram, and plot, 3) presents it, and 4) collects critics to improve the document.
In an instruction group 1) selects a problem to discuss, 2) distributes materials, and 3) schedules personal lab-works. Each week 6 students deliver essays and 6 students write critiques.
Lab-work themes
1. IDEF0 (UML) and CT, examples of engineering systems
2. Structures of linear models, NNs, multilinear models
3. Elements of CT as a programming tool
4. Backprop in CT, differential programming
5. Metric learning, metric tensors, graphs, and RKHS,
6. Multi-domain problems: CCA, attention
7. Plate notation and Bayesian model selection
8. Dimensionality reduction and tensor decomposition
9. Discriminative and generative models, stochastic decomposition
10. Riemannian generative models
11. Continuous time, autoregressive, and memory models
12. Continuous spatial models and Clifford algebra
The challenge is to (slowly) select lab-work themes
from the cartesian product of 1) notions of category theory, 2) elements of tensor analysis, differential geometry, homology theory, 3) machine learning problems, 4) pieces of code and papers and put it in a sequence of essays.
Here is an example to develop (present an nn as a combination of linear operators).
In a nutshell
The goal is to make a collection of simple, precise, and short explanations of ML principles in a diagram-plot language. For example, the diagram in the paper “Attention is all you need” delivers absolutely no principles of attention.
The notions of quadratic form and metric tensor deliver that.