Python: snippets

Papers on Machine learning refer to the code of computational experiments to reproduce. Selecting a shell and setting up an environment to execute an alien project takes time. Below we collect some useful tips on how to start and organize work on alien projects.

For reference List of the Colab magic functions

%lsmagic

List of the bash commands in Colab

!compgen -c

Import modules from custom py-files, mount the Google-drive

from google.colab import drive
drive.mount('/content/drive')
import sys
sys.path.insert(0,'/content/drive/MyDrive/MyProject')
% download the files to the folder, namely utils.py
% and import desired functions from it
from utils import *
# also
!python myproj.py

Clone a github project to the Colab folder

!git clone https://github.com/AuthorName/ProjectName
!cd contents/ProjectName
!pwd
!ls

Pip tricks

!pip3 show numpy

Three ways to install a Conda environment One

! pip install -q condacolab
import condacolab
condacolab.install()
!which conda # should return /usr/local/bin/conda

Two

! wget https://repo.anaconda.com/miniconda/Miniconda3-py37_4.8.2-Linux-x86_64.sh
! chmod +x Miniconda3-py37_4.8.2-Linux-x86_64.sh
! bash ./Miniconda3-py37_4.8.2-Linux-x86_64.sh -b -f -p /usr/local
import sys
sys.path.append('/usr/local/lib/python3.7/site-packages/')
!which conda
!conda -h

Three

!which python
!python --version
%env PYTHONPATH=
%%bash
MINICONDA_INSTALLER_SCRIPT=Miniconda3-py39_23.1.0-1-Linux-x86_64.sh
MINICONDA_PREFIX=/usr/local/bin/
wget https://repo.continuum.io/miniconda/$MINICONDA_INSTALLER_SCRIPT
%
!chmod +x $MINICONDA_INSTALLER_SCRIPT/$MINICONDA_PREFIX
!chmod --help
%

How Conda works: video, more general video

! conda update -n base -c conda-forge conda
!conda init
!conda create -h
!conda create -p /content/MyProj/env.yml
!conda env create --file /content/MyProj/environment.yml
!less environment.yml
%cd /content/MyProj
!conda env create --file environment.yml
!conda env list
!conda init --all
!conda activate tf
!conda env export --file test.yml
!less test.yml
!conda env list
!conda list

Debuging

Debugging in Google Colab