Difference between revisions of "The Art of Scientific Research"

From Research management course
Jump to: navigation, search
 
(40 intermediate revisions by the same user not shown)
Line 1: Line 1:
−
The Art of Scientific Research
+
This is a preparatory course for the [[Course schedule|main part of m1p]].
  
−
This is a preparatory course for the main part of m1p.
+
=== Goals of the seminar ===
 +
# Gather tools and train skills to run a scientific research
 +
# Elaborate competencies of the scientific problem statement and reporting
 +
# Fit your research society, find a high-quality scientific advisor, and select an important problem to engage
 +
 
 +
====Organizers' goals====
 +
# Boost the performance of the MS thesis works, namely change ''the magical presentation'' of the machine learning models for ''the theoretical-based one''
 +
# Persuade the scientific advisers to set complex and well-elaborated problems with high-quality planning
 +
 
 +
=== Outline of a seminar ===
 +
# Test (five open or closed questions) with a brief analysis
 +
# Theoretical part (15 minutes) and references to study
 +
# Practice and homework handout
 +
# Talks and discussion (20 minutes)
 +
 
 +
=== Coursework and talks ===
 +
* Module 1: A formal description of a problem (project), a two-page text plus a two-slide talk
 +
* Module 2: An error analysis, a plan of a computational experiment with model selection plus a talk
 +
 
 +
=== Homeworks ===
 +
Since it is a preparatory course, the change of research subject for different HWs is welcome.
 +
 
 +
==== Module 1 ====
 +
# Select and read the reference paper 
 +
# Reconstruct its abstract 
 +
# Formulate highlights 
 +
# Collect a SOTA review
 +
# Extract the principles of the paper
 +
# Prepare the two-slide talk
 +
 
 +
==== Module 2 ====
 +
# State the problem 
 +
# State statistical hypotheses
 +
# Construct algebraic structures  
 +
# Gather the theory 
 +
# Select a model
 +
# Plan the experiment 
  
 
== The student's response-based syllabus ==
 
== The student's response-based syllabus ==
−
# We start
+
# [[Step 0]]: We start
 
# Prepare your tools  
 
# Prepare your tools  
 
# Check the foundations   
 
# Check the foundations   
−
# How to measure impact?
+
# How to measure impact
 
# Describe your system
 
# Describe your system
 
# Write the abstract
 
# Write the abstract
Line 22: Line 58:
 
# Tell about a scientific society
 
# Tell about a scientific society
 
# Reproducible computational experiment
 
# Reproducible computational experiment
−
# Computer supported brainstorming
+
# Computer-supported brainstorming
−
# Conferences and journals, review and schedules
+
# Conferences and journals, reviews, and schedules
 
# Writing a grant proposal
 
# Writing a grant proposal
 +
 +
=== addendum ===
 +
* Annotate and highlight (rules of annotation and highlighting applied)
 +
* Write a review
 +
* Boost a review by gathering your colleagues' efforts
 +
* Make long and short lists of your ideas and solutions
 +
* Select a topic from the list
 +
* Find the data if you need something special, it takes time and efforts
 +
* Structure of a thesis work and bureaucracy of thesis defense
 +
 +
==The theory to discuss==
 +
# Machine learning at one go
 +
# Linear models (and processes) with time (regression, SVD, PCA, NN)
 +
# Tensor indexing and decomposition, Tucker, HOSVD, TT (getting rid of time by constructing a state space)
 +
# Types of optimization (what is the gradient and Jacoby matrix)
 +
# Convolution and Fourier transform is a linear operator
 +
# Kernel methods and RKHS
 +
# Graph convolution, metric spaces (if possible)
 +
# Canonical correlation analysis and autoencoders
 +
# Bayesian inference and regularization, optimization
 +
# Model selection
 +
# Multimodeling (privilege, distilling, domain transfer)
 +
# Introduction to sampling and generative models
 +
* Goals for the next year are CaТ, NODE, SDE, Diffusion, Riemannian, Tensors as tensors, Advanced calculus, Clifford algebra, Homology
  
 
==Scoring==  
 
==Scoring==  
Line 32: Line 92:
 
# The coursework
 
# The coursework
  
−
== Similar courses ==  
+
Weekly homework. All points are added up and scaled to [0,10]. Deadlines are strict. Normally there is no exam.
−
# Around
+
 
 +
==Student's risks==
 +
Despite [[Course schedule|m1p]] (it flourishes over years), this is a new course, so:
 +
# It might end abruptly, after one week, one month, or one module.
 +
# There will be no resources to check and review your texts.
 +
# Most likely there will be no possibilities to listen to all of your talks.
 +
# So feedback is limited.
 +
 
 +
<!-- == Similar courses ==  
 +
# Around -->
 +
 
 +
==Student prerequisites==
 +
Briefly: it is for 3rd year BS students.
 +
# Discrete Analysis and Set Theory
 +
# Calculus and Mathematical Analysis
 +
# Probability and Statistics
 +
# Algebra, Group theory is welcome
 +
# Functional Analysis is welcome
 +
# General Physics is highly welcome
 +
# Machile learning by C.P. Bishop is a must!
  
 
== Main references ==
 
== Main references ==
−
# Algebra, Topology, Differential Calculus, and Optimization Theory for Computer Science and Machine Learning by Jean Gallier and Jocelyn Quaintance, 2024. [https://www.cis.upenn.edu/~jean/math-deep.pdf pdf], [https://github.com/akhauriyash/MathForCS_ML github] (2196 pages)
+
# (long reading 2196 pages) Algebra, Topology, Differential Calculus, and Optimization Theory for Computer Science and Machine Learning by Jean Gallier and Jocelyn Quaintance, 2024. [https://www.cis.upenn.edu/~jean/math-deep.pdf pdf], [https://github.com/akhauriyash/MathForCS_ML github]
 +
# Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges by M.M. Bronstein, J. Bruna, T. Cohen, P. Veličković, 2021. [https://arxiv.org/abs/2104.13478 arxiv]
 +
# Deep Learning: Foundations and Concepts by C.M. Bishop, H. Bishop, 2024 [https://www.microsoft.com/en-us/research/uploads/prod/2006/01/Bishop-Pattern-Recognition-and-Machine-Learning-2006.pdf version'06]
 +
# Mathematics for Physicists: Introductory Concepts and Methods by  A. Altland. J. von Delf, 2017 [https://klassfeldtheorie.wordpress.com/wp-content/uploads/2018/10/mathematische-methoden-310117.pdf pdf]
 +
# Mathematics for Machine Learning by M.P. Deisenroth, A.A. Faisal, C.S. Ong [https://mml-book.github.io/book/mml-book.pdf pdf]
 +
# Python for Probability, Statistics, and Machine Learning by J. Unpingco, 2016 [https://github.com/YikaiZhangskye/ML/blob/master/Unpingco%20J.%20-%20Python%20for%20Probability,%20Statistics,%20and%20Machine%20Learning%20-%202016.pdf github]
 +
 
 +
* Cath-up references are in the [[Week 0]] of the main course
  
−
==Cath-up==
+
==Dates==
−
Check and develop your [https://www.typingclub.com/sportal/program-3.game typing skills]
+
2024 on Sat 11:10 – 12:50 [http://m1p.org/go_zoom zoom] |
 +
Sept 14 21 28 |
 +
Oct 5 12 19 || skip 26, 2 ||
 +
Now 9 16 23 30 |
 +
Dec 7 14 |

Latest revision as of 00:21, 9 November 2024

This is a preparatory course for the main part of m1p.

Goals of the seminar

  1. Gather tools and train skills to run a scientific research
  2. Elaborate competencies of the scientific problem statement and reporting
  3. Fit your research society, find a high-quality scientific advisor, and select an important problem to engage

Organizers' goals

  1. Boost the performance of the MS thesis works, namely change the magical presentation of the machine learning models for the theoretical-based one
  2. Persuade the scientific advisers to set complex and well-elaborated problems with high-quality planning

Outline of a seminar

  1. Test (five open or closed questions) with a brief analysis
  2. Theoretical part (15 minutes) and references to study
  3. Practice and homework handout
  4. Talks and discussion (20 minutes)

Coursework and talks

  • Module 1: A formal description of a problem (project), a two-page text plus a two-slide talk
  • Module 2: An error analysis, a plan of a computational experiment with model selection plus a talk

Homeworks

Since it is a preparatory course, the change of research subject for different HWs is welcome.

Module 1 

  1. Select and read the reference paper 
  2. Reconstruct its abstract 
  3. Formulate highlights 
  4. Collect a SOTA review
  5. Extract the principles of the paper
  6. Prepare the two-slide talk

Module 2

  1. State the problem 
  2. State statistical hypotheses
  3. Construct algebraic structures  
  4. Gather the theory 
  5. Select a model
  6. Plan the experiment 

The student's response-based syllabus

  1. Step 0: We start
  2. Prepare your tools
  3. Check the foundations
  4. How to measure impact
  5. Describe your system
  6. Write the abstract
  7. Write the intro
  8. Review the paper
  9. Deliver a message
  10. Your one-slide talk
  11. Blind management game
  12. List your ideas
  13. List the foundations
  14. Suggest an impactful theorem
  15. Review for your topic
  16. Good, bad, ugly: tell the difference
  17. Tell about a scientific society
  18. Reproducible computational experiment
  19. Computer-supported brainstorming
  20. Conferences and journals, reviews, and schedules
  21. Writing a grant proposal

addendum

  • Annotate and highlight (rules of annotation and highlighting applied)
  • Write a review
  • Boost a review by gathering your colleagues' efforts
  • Make long and short lists of your ideas and solutions
  • Select a topic from the list
  • Find the data if you need something special, it takes time and efforts
  • Structure of a thesis work and bureaucracy of thesis defense

The theory to discuss

  1. Machine learning at one go
  2. Linear models (and processes) with time (regression, SVD, PCA, NN)
  3. Tensor indexing and decomposition, Tucker, HOSVD, TT (getting rid of time by constructing a state space)
  4. Types of optimization (what is the gradient and Jacoby matrix)
  5. Convolution and Fourier transform is a linear operator
  6. Kernel methods and RKHS
  7. Graph convolution, metric spaces (if possible)
  8. Canonical correlation analysis and autoencoders
  9. Bayesian inference and regularization, optimization
  10. Model selection
  11. Multimodeling (privilege, distilling, domain transfer)
  12. Introduction to sampling and generative models
  • Goals for the next year are CaТ, NODE, SDE, Diffusion, Riemannian, Tensors as tensors, Advanced calculus, Clifford algebra, Homology

Scoring

  1. Tests at the beginning of a seminar
  2. Talks at the end of a seminar
  3. Downloads of the homework
  4. The coursework

Weekly homework. All points are added up and scaled to [0,10]. Deadlines are strict. Normally there is no exam.

Student's risks

Despite m1p (it flourishes over years), this is a new course, so:

  1. It might end abruptly, after one week, one month, or one module.
  2. There will be no resources to check and review your texts.
  3. Most likely there will be no possibilities to listen to all of your talks.
  4. So feedback is limited.


Student prerequisites

Briefly: it is for 3rd year BS students.

  1. Discrete Analysis and Set Theory
  2. Calculus and Mathematical Analysis
  3. Probability and Statistics
  4. Algebra, Group theory is welcome
  5. Functional Analysis is welcome
  6. General Physics is highly welcome
  7. Machile learning by C.P. Bishop is a must!

Main references

  1. (long reading 2196 pages) Algebra, Topology, Differential Calculus, and Optimization Theory for Computer Science and Machine Learning by Jean Gallier and Jocelyn Quaintance, 2024. pdf, github
  2. Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges by M.M. Bronstein, J. Bruna, T. Cohen, P. Veličković, 2021. arxiv
  3. Deep Learning: Foundations and Concepts by C.M. Bishop, H. Bishop, 2024 version'06
  4. Mathematics for Physicists: Introductory Concepts and Methods by A. Altland. J. von Delf, 2017 pdf
  5. Mathematics for Machine Learning by M.P. Deisenroth, A.A. Faisal, C.S. Ong pdf
  6. Python for Probability, Statistics, and Machine Learning by J. Unpingco, 2016 github
  • Cath-up references are in the Week 0 of the main course

Dates

2024 on Sat 11:10 – 12:50 zoom | Sept 14 21 28 | Oct 5 12 19 || skip 26, 2 || Now 9 16 23 30 | Dec 7 14 |

Retrieved from ‘https://m1p.org/index.php?title=The_Art_of_Scientific_Research&oldid=1842’