Python: snippets
Papers on Machine learning refer to the code of computational experiments to reproduce. Selecting a shell and setting up an environment to execute an alien project takes time. Below we collect some useful tips on how to start and organize work on alien projects.
For reference List of the Colab magic functions
%lsmagic
List of the bash commands in Colab
!compgen -c
Import modules from custom py-files, mount the Google-drive
from google.colab import drive
drive.mount('/content/drive')
import sys
sys.path.insert(0,'/content/drive/MyDrive/MyProject')
% download the files to the folder, namely utils.py
% and import desired functions from it
from utils import *
# also
!python myproj.py
Clone a github project to the Colab folder
!git clone https://github.com/AuthorName/ProjectName !cd contents/ProjectName !pwd !ls
Pip tricks
!pip3 show numpy
Three ways to install a Conda environment One
! pip install -q condacolab import condacolab condacolab.install() !which conda # should return /usr/local/bin/conda
Two
! wget https://repo.anaconda.com/miniconda/Miniconda3-py37_4.8.2-Linux-x86_64.sh ! chmod +x Miniconda3-py37_4.8.2-Linux-x86_64.sh ! bash ./Miniconda3-py37_4.8.2-Linux-x86_64.sh -b -f -p /usr/local import sys sys.path.append('/usr/local/lib/python3.7/site-packages/') !which conda !conda -h
Three !which python !python --version %env PYTHONPATH= %%bash MINICONDA_INSTALLER_SCRIPT=Miniconda3-py39_23.1.0-1-Linux-x86_64.sh MINICONDA_PREFIX=/usr/local/bin/ wget https://repo.continuum.io/miniconda/$MINICONDA_INSTALLER_SCRIPT % !chmod +x $MINICONDA_INSTALLER_SCRIPT/$MINICONDA_PREFIX !chmod --help %
How Conda works: video, more general video ! conda update -n base -c conda-forge conda !conda init !conda create -h !conda create -p /content/MyProj/env.yml !conda env create --file /content/MyProj/environment.yml !less environment.yml %cd /content/MyProj !conda env create --file environment.yml !conda env list !conda init --all !conda activate tf !conda env export --file test.yml !less test.yml !conda env list !conda list
Debuging