Course syllabus: Neural architecture search

Twelve lectures with practical exercises. The first part of the class is devoted to the theoretical search for architecture. It ends with a technical application. As part of the practical work, the architecture of a neural network of a given type is analyzed.

1. Overview of neural network types and architecture descriptions
2. Genetic Algorithms from GMDH to WANN
3. Structure selection quality criteria
4. A priori hypothesis for individual models, types of distributed structural parameters
5. Structural parameter analysis
6. Online learning and multi-armed bandits to generate structure
7. Reinforcement learning to generate structure
8. Transfer of knowledge between neural networks and optimization of structural parameters
9. Random processes for generating models
10. Generative Adversarial Networks and Search Structure
11. Creation and rejection of structure
12. Bilevel Bayesian Selection and Metropolis-Hastings Sampling

Laboratory works

The laboratory work is based on the application of the architecture search method. The first job is to evaluate the finished method, the second job is to propose and program your own method. Work report - a page of text with a formal description of the method with sufficient detail to recover the code, and error analysis (basic diagnostic criteria, cases, cases). The interface to the class is constant and common to all, just like the selections. There are general tables with results, and a private analysis of the errors of each method.

Computational experiment with the report Each student makes a short report in 3 minutes on 7 and 14 weeks on the first laboratory work and the second, respectively.

Grading Total 10 points, two points for answering questions during classes, and four points for two laboratory works. It is not the accuracy of the approximation that is evaluated but the quality of the code and error analysis.