Course syllabus: Category theory for Machine Learning

Category Theory for Machine Learning (a practical course)

This course introduces the language of category theory as a tool for model selection in machine learning. In this course, we create templates of problem statements and model descriptions. We use these templates in the introductory parts of our research projects, both academic and industrial. It brings us clarity and precision in our main goal: to convey the message to the reader.

The course comprises a series of lab-works and instructions

In a lab-work a student has to 1) formulate a machine learning problem in the CT language, 2) Write an essay, a short document with text, diagram, and plot, 3) presents it, and 4) collects critics to improve the document.

In an instruction group 1) selects a problem to discuss, 2) distributes materials, and 3) schedules personal lab-works. Each week 6 students deliver essays and 6 students write critiques.

Lab-work themes

1. IDEF0 (UML) and CT, examples of engineering systems

2. Structures of linear models, NNs, multilinear models

3. Elements of CT as a programming tool

4. Backprop in CT, differential programming

5. Metric learning, metric tensors, graphs, and RKHS,

6. Multi-domain problems: CCA, attention

7. Plate notation and Bayesian model selection

8. Dimensionality reduction and tensor decomposition

9. Discriminative and generative models, stochastic decomposition

10. Riemannian generative models

11. Continuous time, autoregressive, and memory models

12. Continuous spatial models and Clifford algebra

The challenge is to (slowly) select lab-work themes

from the cartesian product of 1) notions of category theory, 2) elements of tensor analysis, differential geometry, homology theory, 3) machine learning problems, 4) pieces of code and papers and put it in a sequence of essays.

Here is an example to develop (present an nn as a combination of linear operators).

In a nutshell

The goal is to make a collection of simple, precise, and short explanations of ML principles in a diagram-plot language. For example, the diagram in the paper “Attention is all you need” delivers absolutely no principles of attention.

The notions of quadratic form and metric tensor deliver that.