Difference between revisions of "Python: snippets"

From Research management course
Jump to: navigation, search
(Created page with "Papers on Machine learning refer to the code of computational experiments to reproduce. Selecting a shell and setting up an environment to execute an alien project takes time....")
 
 
(6 intermediate revisions by the same user not shown)
Line 1: Line 1:
 +
Main page: [[Reviews#Programming]]
 +
 
Papers on Machine learning refer to the code of computational experiments to reproduce. Selecting a shell and setting up an environment to execute an alien project takes time.  Below we collect some useful tips on how to start and organize work on alien projects.
 
Papers on Machine learning refer to the code of computational experiments to reproduce. Selecting a shell and setting up an environment to execute an alien project takes time.  Below we collect some useful tips on how to start and organize work on alien projects.
 +
 +
 +
For reference
 +
List of the Colab magic functions
 +
%lsmagic
 +
 +
List of the bash commands in Colab
 +
!compgen -c
 +
 +
Import modules from custom py-files, mount the Google-drive
 +
from google.colab import drive
 +
drive.mount('/content/drive')
 +
import sys
 +
sys.path.insert(0,'/content/drive/MyDrive/MyProject')
 +
% download the files to the folder, namely utils.py
 +
% and import desired functions from it
 +
from utils import *
 +
# also
 +
!python myproj.py
 +
 +
Clone a github project to the Colab folder
 +
!git clone https://github.com/AuthorName/ProjectName
 +
!cd contents/ProjectName
 +
!pwd
 +
!ls
 +
 +
Pip tricks
 +
!pip3 show numpy
 +
 +
Three ways to install a Conda environment
 +
One
 +
! pip install -q condacolab
 +
import condacolab
 +
condacolab.install()
 +
!which conda # should return /usr/local/bin/conda
 +
 +
Two
 +
! wget https://repo.anaconda.com/miniconda/Miniconda3-py37_4.8.2-Linux-x86_64.sh
 +
! chmod +x Miniconda3-py37_4.8.2-Linux-x86_64.sh
 +
! bash ./Miniconda3-py37_4.8.2-Linux-x86_64.sh -b -f -p /usr/local
 +
import sys
 +
sys.path.append('/usr/local/lib/python3.7/site-packages/')
 +
!which conda
 +
!conda -h
 +
 +
Three
 +
!which python
 +
!python --version
 +
%env PYTHONPATH=
 +
%%bash
 +
MINICONDA_INSTALLER_SCRIPT=Miniconda3-py39_23.1.0-1-Linux-x86_64.sh
 +
MINICONDA_PREFIX=/usr/local/bin/
 +
wget https://repo.continuum.io/miniconda/$MINICONDA_INSTALLER_SCRIPT
 +
%
 +
!chmod +x $MINICONDA_INSTALLER_SCRIPT/$MINICONDA_PREFIX
 +
!chmod --help
 +
%
 +
 +
How Conda works: [https://www.youtube.com/watch?v=1VVCd0eSkYc video], [https://www.youtube.com/watch?v=23aQdrS58e0 more general video]
 +
! conda update -n base -c conda-forge conda
 +
!conda init
 +
!conda create -h
 +
!conda create -p /content/MyProj/env.yml
 +
!conda env create --file /content/MyProj/environment.yml
 +
!less environment.yml
 +
%cd /content/MyProj
 +
!conda env create --file environment.yml
 +
!conda env list
 +
!conda init --all
 +
!conda activate tf
 +
!conda env export --file test.yml
 +
!less test.yml
 +
!conda env list
 +
!conda list
 +
 +
Debuging
 +
* [https://stackoverflow.com/questions/52656692/debugging-in-google-colab Debugging in Google Colab]

Latest revision as of 20:48, 22 November 2024

Main page: Reviews#Programming

Papers on Machine learning refer to the code of computational experiments to reproduce. Selecting a shell and setting up an environment to execute an alien project takes time. Below we collect some useful tips on how to start and organize work on alien projects.


For reference List of the Colab magic functions 

%lsmagic

List of the bash commands in Colab 

!compgen -c

Import modules from custom py-files, mount the Google-drive 

from google.colab import drive
drive.mount('/content/drive')
import sys
sys.path.insert(0,'/content/drive/MyDrive/MyProject')
% download the files to the folder, namely utils.py
% and import desired functions from it
from utils import *
# also
!python myproj.py

Clone a github project to the Colab folder 

!git clone https://github.com/AuthorName/ProjectName
!cd contents/ProjectName
!pwd
!ls

Pip tricks 

!pip3 show numpy

Three ways to install a Conda environment One 

! pip install -q condacolab
import condacolab
condacolab.install()
!which conda # should return /usr/local/bin/conda

Two 

! wget https://repo.anaconda.com/miniconda/Miniconda3-py37_4.8.2-Linux-x86_64.sh
! chmod +x Miniconda3-py37_4.8.2-Linux-x86_64.sh
! bash ./Miniconda3-py37_4.8.2-Linux-x86_64.sh -b -f -p /usr/local
import sys
sys.path.append('/usr/local/lib/python3.7/site-packages/')
!which conda
!conda -h

Three 

!which python
!python --version
%env PYTHONPATH=
%%bash
MINICONDA_INSTALLER_SCRIPT=Miniconda3-py39_23.1.0-1-Linux-x86_64.sh
MINICONDA_PREFIX=/usr/local/bin/
wget https://repo.continuum.io/miniconda/$MINICONDA_INSTALLER_SCRIPT
%
!chmod +x $MINICONDA_INSTALLER_SCRIPT/$MINICONDA_PREFIX
!chmod --help
%

How Conda works: video, more general video 

! conda update -n base -c conda-forge conda
!conda init
!conda create -h
!conda create -p /content/MyProj/env.yml
!conda env create --file /content/MyProj/environment.yml
!less environment.yml
%cd /content/MyProj
!conda env create --file environment.yml
!conda env list
!conda init --all
!conda activate tf
!conda env export --file test.yml
!less test.yml
!conda env list
!conda list

Debuging

Retrieved from ‘https://m1p.org/index.php?title=Python:_snippets&oldid=1899’