Week 4

The goal is to get the simplest possible solution to your problem: it is models and its parameters. So make the model fit data with the minimum of your efforts.

X: Experiment planning

Plan your computational experiment.

1. Discuss the experiment goal with your adviser and team.
   • Put this goal in the section Computational experiment
2. Describe your basic data set, a synthetic, or a simple real one:
   • put in the text the title, source, and set up of measurements (it is the technical description, the theoretical one is in the problem statement section),
   • write down the number of objects, and features, describe general statistics,
   • for a synthetic data set describe the generation model, and its parameters (for example, uniform random independent sampling at some given interval).
3. Describe the configuration of the algorithm run.
4. Plan the whole experimental part.
5. List expected tables and figures:
   • make short and long list, for each
   • describe axes,
   • make a draft with a pencil.

R: Preliminary report

1. Make sure that the obtained results are in not logical (sic!) contradiction with the goals of the computational experiment.
2. Illustrate the obtained results with the preliminary plot. Optimally this plot is hand-made. Just draw it with a pencil on a piece of paper. See an example. For the final version use this format.
3. Write a mini-report on the results with
   1. a short description of the figure: what the reader could see, what are the consequences,
   2. the results in numbers and comments on it,
   3. put the report to the section computational experiment.

B: Run basic code

Select the basic algorithm and run it using a simple data set.

1. Run your basic algorithm:
   • select the simplest algorithm (with your adviser) to (partially) solve the problem you set.
2. Collect a synthetic data set or download a simple real-world data set of small size.
3. Upload your data to the repository (in case the data size exceeds 5MB or the data set consists of numerous files, please discuss with your adviser and team).
4. Run the basic algorithm on the synthetic data set, and estimate the error.
5. Describe the basic algorithm, analyze its features, and list competitive models. Here the examples of the description style.
   1. Description refers to the name of some black box model. It is advisable to indicate the source, where the contents of the black box model are described in detail. The description specifies the structural parameters of the black box.
   2. Description defines a model as a map from the design space of features to the space of target variables. Since the model has its parameters the description may refer to the algorithm for optimizing the model parameters in the form of a black box.
   3. Description of the model and algorithm for optimizing its parameters in the form of pseudocode.

Resources

• Slides for week 4. Slides 2022.
• Video for week 4.
• See examples of the reports.
  1. Бахтеев О.Ю. Системы и средства глубокого обучения, статья
  2. Мотренко А.П. Повышение качества классификации, статья
  3. Исаченко Р.В. Снижение размерности в задаче декодирования, статья
  4. The goals of computational experiments А. Грабовой, В. Алексеев, А. Рогозина, И. Игашов, Н. Уваров
  5. Example of the measurement description, Bishop C.P. Pattern recognition and machine learning, 2006. Pp. 677-683.]
• Top 8 Sources For Machine Learning Datasets

Homework

1. Watch the video.
2. Find the code that works.
3. Write the goal of your computational experiment. A couple of sentences help you focus your efforts.
4. Write a draft of your desired report and draw a plot for the error analysis next step.
5. Run the code on the simplest dataset.

References

1. Research Methodology: Methods and Techniques by C.R. Kothari, 2004 pdf or pdf