The Art of Scientific Research

This is a preparatory course for the main part of m1p.

Goals of the seminar

1. Gather tools and train skills to run a scientific research
2. Elaborate competencies of the scientific problem statement and reporting
3. Fit your research society, find a high-quality scientific advisor, and select an important problem to engage

Organizers' goals

1. Boost the performance of the MS thesis works, namely change the magical presentation of the machine learning models for the theoretical-based one
2. Persuade the scientific advisers to set complex and well-elaborated problems with high-quality planning

Outline of a seminar

1. Test (five open or closed questions) with a brief analysis
2. Theoretical part (15 minutes) and references to study
3. Practice and homework handout
4. Talks and discussion (20 minutes)

Coursework and talks

• Module 1: A formal description of a method, a two-page text plus a three-slide talk
• Module 2: An error analysis, a plan of a computational experiment with model selection plus a talk

Homeworks

Since it is a preparatory course, the change of research subject for different HWs is welcome.

Module 1 

1. Select and read the reference paper 
2. Reconstruct its abstract 
3. Formulate highlights 
4. Collect a SOTA review
5. Extract the principles of the paper
6. Prepare one-slide talk

Module 2

1. State the problem 
2. State statistical hypotheses
3. Construct algebraic structures  
4. Gather the theory 
5. Select a model
6. Plan the experiment 

The student's response-based syllabus

1. Step 0: We start
2. Prepare your tools
3. Check the foundations
4. How to measure impact
5. Describe your system
6. Write the abstract
7. Write the intro
8. Review the paper
9. Deliver a message
10. Your one-slide talk
11. Blind management game
12. List your ideas
13. List the foundations
14. Suggest an impactful theorem
15. Review for your topic
16. Good, bad, ugly: tell the difference
17. Tell about a scientific society
18. Reproducible computational experiment
19. Computer-supported brainstorming
20. Conferences and journals, reviews, and schedules
21. Writing a grant proposal

addendum

• Annotate and highlight (rules of annotation and highlighting applied)
• Write a review
• Boost a review by gathering your colleagues' efforts
• Make long and short lists of your ideas and solutions
• Select a topic from the list
• Find the data if you need something special, it takes time and efforts
• Structure of a thesis work and bureaucracy of thesis defense

The theory to discuss

1. Machine learning at one go
2. Linear models (and processes) with time (regression, SVD, PCA, NN)
3. Tensor indexing and decomposition, Tucker, HOSVD, TT (getting rid of time by constructing a state space)
4. Types of optimization (what is the gradient and Jacoby matrix)
5. Convolution and Fourier transform is a linear operator
6. Kernel methods and RKHS
7. Graph convolution, metric spaces (if possible)
8. Canonical correlation analysis and autoencoders
9. Bayesian inference and regularization, optimization
10. Model selection
11. Multimodeling (privilege, distilling, domain transfer)
12. Introduction to sampling and generative models

• Goals for the next year are CaТ, NODE, SDE, Diffusion, Riemannian, Tensors as tensors, Advanced calculus, Clifford algebra, Homology

Scoring

1. Tests at the beginning of a seminar
2. Talks at the end of a seminar
3. Downloads of the homework
4. The coursework

Weekly homework. All points are added up and scaled to [0,10]. Deadlines are strict. Normally there is no exam.

Student's risks

Despite m1p (it flourishes over years), this is a new course, so:

1. It might end abruptly, after one week, one month, or one module.
2. There will be no resources to check and review your texts.
3. Most likely there will be no possibilities to listen to all of your talks.
4. So feedback is limited.

Student prerequisites

Briefly: it is for 3rd year BS students.

1. Discrete Analysis and Set Theory
2. Calculus and Mathematical Analysis
3. Probability and Statistics
4. Algebra, Group theory is welcome
5. Functional Analysis is welcome
6. General Physics is highly welcome
7. Machile learning by C.P. Bishop is a must!

Main references

1. (long reading 2196 pages) Algebra, Topology, Differential Calculus, and Optimization Theory for Computer Science and Machine Learning by Jean Gallier and Jocelyn Quaintance, 2024. pdf, github
2. Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges by M.M. Bronstein, J. Bruna, T. Cohen, P. Veličković, 2021. arxiv
3. Deep Learning: Foundations and Concepts by C.M. Bishop, H. Bishop, 2024 version'06
4. Mathematics for Physicists: Introductory Concepts and Methods by A. Altland. J. von Delf, 2017 pdf
5. Mathematics for Machine Learning by M.P. Deisenroth, A.A. Faisal, C.S. Ong pdf
6. Python for Probability, Statistics, and Machine Learning by J. Unpingco, 2016 github

• Cath-up references are in the Week 0 of the main course

Dates

Sat 14:40 – 16:00 zoom | Sept 14 21 28 | Oct 5 12 19 || skip 26, 2 || Now 9 16 23 30 | Dec 7 14 |