Todo list

The todo lists here corresponds to the Course schedule. Each list must be completed before the day of review. It is Wednesday 06:00 am for the 2020 Spring semester.

Todo P: Problem statement

In the paradigm Idea\(\to\)Formula\(\to\)Code state the problem to find an optimal solution.

1. Discuss the problem statement with your adviser.
2. See the examples below and in the past projects.
3. Discuss terminology and notation see [pdf] and [tex] with notations and a useful style file.
4. In the beginning of Problem statement write a general problem description.
5. Describe the elements of your problem statement:
   1. the sample set,
   2. its origin, or its algebraic structure,
   3. statistical hypotheses of data generation,
   4. [conditions of measurements] ,
   5. [restrictions of the sample set and its values],
   6. your model in the class of models,
   7. restrictions on the class of models,
   8. the error function (and its inference) or a loss function, or a quality criterion,
   9. cross-validation procedure,
   10. restrictions to the solutions,
   11. external (industrial) quality criteria,
   12. the optimization statement as \(\arg\min\).
6. Define the main termini: what is called the model, the solution, the algorithm.

Note that:

• The model is a parametric family of functions to map design space to target space.
• The criterion (error function) is a function to optimize in order to obtain an optimal solution (model parameters, a function).
• The algorithm transforms solution space, usually iteratively.
• The method combines a model, a criterion, and an algorithm to produce a solution.

Check it:

• the regression model,
• the sum of squared errors,
• the Newton-Raphson algorithm,
• the method of least squares.

Resources

• Slides with a plan of Problem statement
• Examples of problem statements
  1. Katrutsa A.M., Strijov V.V. Stresstest procedure for feature selection algorithms // Chemometrics and Intelligent Laboratory Systems, 2015, 142 : 172-183 article
  2. Katrutsa A.M., Strijov V.V. Comprehensive study of feature selection methods to solve multicollinearity problem according to evaluation criteria // Expert Systems with Applications, 2017 article
  3. Motrenko A., Strijov V., Weber G.-W. Bayesian sample size estimation for logistic regression // Journal of Computational and Applied Mathematics, 2014, 255 : 743-752. article
  4. Kulunchakov A.S., Strijov V.V. Generation of simple structured Information Retrieval functions by genetic algorithm without stagnation // Expert Systems with Applications, 2017, 85 : 221-230. article
  5. Ivkin N.P. Feature generation for classification and forecasting problems, MIPT, 2013 draft
• Notations for wiki Ru
• Basic notations, pdf
• Recommended notations, 2019: pdf and .tex with .sty)]
• Simple and useful notations
• Notations for Bayesian model selection, pdf

Todo A: Abstract

1. Write a draft of your abstract.

• The abstract shall not exceed 600 characters. It may contain:
  • wide-range field of the investigated problem,
  • narrow problem to focus on,
  • features and conditions of the problem,
  • [the novelty],
  • application to illustrate with.
• For joint projects it is important that each team-member writes its own text.

Resources

• How to Read a Paper, 2016, S. Keshav
• Examples of rewiev-and-planning drafts LinkReview раз, два.

Todo B: Beginner's-talk

Short 45-second introductory talk. Plan of the talk:

1. The project goal. What is the motivation, the goal to reach?
2. The main idea. What is the message?
3. The expected result. What is your delivery, your impact, novelty?

There is no time to show a slide or draw a plot on the blackboard. It is recommended to rehearse the report.

Todo I: Introduction

The introductory part includes research goals and motivations. It reasons the research with fundamental and state-of-the-arts references. It delivers the main message of the work to the reader. This message shows novelty of this work in comparison to recent results.

1. Create a file ProjectN.bib for the group project, or Surname2018Title.bib for your personal project.
2. Move from the file LinkReview useful bibliographic records in the BibTeX format.
   • Check the correctness of the BibTeX database (styles of authors names, volumes of journals, page numbers).
   • Use bibliographic databases to facilitate your work.
   • Use the default style \bibliographystyle{plain} before the bibliography section \bibliography{ProjectN}.
   • Important! Wikipedia is not the source of information, but it contains many useful sources.
   • Important! ArXiv is not a peer-review source of information. Look for the copies of papers that are published in peer-review scientific journals. If after one or two years after its ArXiv version, the pare did not appear in a peer-review journal, be careful to use it: this paper might be non-verified since it was rejected by the other journals.
3. Write Introduction. The expected size is one page. The expected plan is:
   1. the research goal (and its motivations),
   2. the object of research (introduce main termini),
   3. the problem (what is the challenge),
   4. methodology: literature review and state-of-the-art
   5. the project tasks,
   6. the proposed solution, its novelty and advantages,
   7. the profs and cons of recent works,
   8. goal of the experiment, set up, data sets, workflow.

The goal of this week is comprehend the goal at its whole and write about it.

Resources

• Bibliographic databases
  • The Collection of Computer Science Bibliographies
  • List of academic databases and search engines in Wikipedia
  • Refer to BibTeX in Wikipedia
  • An introduction updated after a peer-review.

Todo L: Literature

We use the LinkReview draft format to share our evanescent ephemeral ideas and impressions we have during the literature reading.

1. Collect the list of references including:
   1. state-of-the-art reviews, tutorials,
   2. fundamental solutions to the problem,
   3. the basic algorithm to solve your problem,
   4. alternative algorithms,
   5. [changes in the research directions],
   6. data sets and experiments,
   7. the papers that use these data sets
   8. applications of the results,
   9. names of researchers, who solve this problem,
   10. their students and teams,
   11. those, who refer to their works.
2. Balance the list of the new and well-known works.
3. Keep up-to date the list of keywords to search with.
4. Continuously fill your LinkReview.
5. Plan Introduction (see the next todo list), namely collect:
   • keywords as the basic termini; those who brigs good search results are useful,
   • what the paper devoted to,
   • the investigated problem,
   • the central idea,
   • literature review,
   • the authors' contribution.

Todo 1: Select your project

To select your project:

1. Look through the list of projects.
2. Find information about the experts and consultants.
3. Select your projects in the questionnaire before Wednesday 22:00pm.
4. Wait for confirmation.
5. Put confirmed topics to the Group table on Machine learning

Todo 0: Prepare necessary tools

1. Editing. Install LaTeX: MikTeX for Windown, TeX Live for Linux, and for Mac OS. Sign up V2 OverLeaf ShareLaTeX.
2. Install the editor TeXnic Center or its alternative WinEdt for Windows, TeXworks for Linux, and TeXmakerfor Mac OS.
   • Read LaTeX on MachineLearning (Ru).
   • Useful: Wikibooks LaTeX, К.В.Воронцов. LaTeX2e в примерах.
   • Read Львовский С. М. Набор и верстка в системе LaTeX.
3. Download the paper template, ZIP and compile it.
4. Read BibTeX.
   • An example of a bibliographic database.
   • An example of a bibliographic record.
   • An example of draft review LinkReview.
   • List of databases and search engines.
   • List of data-sets for Machine Learning projects.
5. Install bibliographic collection software JabRef (can be postponed).
6. Communications. Sign up GitHub.
   • Important: address and login like Name.Surname or Name-Surname (it depends on system conventions) is welcome.
   • Introductory sliders on Version Control System.
   • Introduction to GitHub.
   • The first steps in GitHub.
7. Download a shell: Desktop.GitHub, or use a command line to synchronise your project.
8. Sign up MachineLearning.ru. Send a logon to your coordinator of to mlalgorithms [at] gmail [dot] com.
9. To state a problem (write essay) using notebook see example in MathJax.
   • Create your page example.
10. Install Hangouts, Skype - read instructions.
11. Programming. Install Python Anaconda, PyCharm (alternative Visual Studio), Notebook online Google.Colab.
    • Development for ML: PyTorch
    • Style formatting: Codestyle pep8
12. Add. As alternative install and try Matlab (MIPT provides free version), (alternative Octave), R-project, Wofram Mathematica.
    • Read Introduction to Matlab].
    • Read Matlab code style, reporting and documenting].
    • Read Matlab Programming Style Guidelines.
13. Add. Read with pleasure Кутателадзе С. С. Советы эпизодическому переводчику and Сосинский А. Б. Как написать математическую статью по-английски.

Resources

• Announcements: Telegram m1p_news
• Ask to email mlalgorithms [at] gmail [dot] com
• Slides.
• Short course description.

References to catch up

• A Brief Introduction to Machine Learning for Engineers by Osvaldo Simeone, 2017-2018
• Understanding Machine Learning: From Theory to Algorithms by Shai Shalev-Shwartz, Shai Ben-David, 2014
• Mathematics for Machine learning by Marc Peter Deisenroth, A. Aldo Faisal, and Cheng Soon Ong
• Mathematics for Physicists: Introductory Concepts and Methods by Alexander Altland & Jan von Delf
• Python notes for professionals by GoalKicker.com Free Programming Books.
• Лагутин М.Б. Наглядная математическая статистика, М.: Бином, 2009. См. также вырезку.
• Bishop C.P. Pattern recognition and machine learning, Berlin: Springer, 2008.
• MackKay D. Information Theory, Pattern Recognition and Neural Networks, Inference.org.uk, 2009.

Todo -1: Subscribe to the course

Todo before 06:00 Wednesday, February 12 th:

1. pick up a problem from the page Try-on programming problems (get the oldest problems, they are simpler),
2. plot one figure to illustrate the problem (plot data or analysis),
3. write explanatory comments to the figure (what the reader sees on the figure, what conclusions follow up),
4. an example of the figure formatting is here
5. upload your notebook to your github repository,
6. send the link to this notebook to mlalgorithms [at] gmail [dot] com, with the subject "Application m1p"

• Example of a nice simple problem: bread regression.
• Examples of plots: one many solutions from this project.
• Examples of old problems Problem 7, Problem 1, Problem15.