7  General Tips

7.1 Running Jupyterlab from a server

Before running the server it is a good idea to generate the user configuration file:

jupyter-lab --generate-config

By default it will be located at ~/.jupyter/jupyter_lab_config.py. Now you can add your own access token that will simplify the following steps (and allow for reproducible connections in the future).

c.IdentityProvider.token = '<YOUR_TOKEN>'

The idea is illustrated in this thread; first on the server side you need to start a headless service as provided below. Once Jupyter starts running, copy the token it will generate if you skipped the user configuration step above.

jupyter-lab --no-browser --port=8080

On the host side (the computer from where you wish to edit the notebooks) establish a ssh tunel exposing and mapping the port chose to serve Jupyter. Instructions are provided in Chapter 2. Now you can browse to http://localhost:8080/ and add the token you copied earlier or your user-token you added to the configuration file. Notice that you need to keep the terminal you used to launch the port forwarding open while you work.

7.2 Downloading from YouTube

#programming/python/tips

Retrieving a video or playlist from YouTube can be automated with help of yt-dlp.

To get the tool working under Ubuntu you can do the following:

# Install Python venv to create a local virtual environment:
sudo apt install python3-venv

# Create an homonymous environment:
python3 -m venv venv

# Activate the local environment:
source venv/bin/activate

# Use pip to install the tool:
pip install -U --pre "yt-dlp[default]"

NOTE: alternative applications as youtube-dl and pytube are now considered to be legacy as discussed in this post.

7.3 Installing Python packages behind proxy

#programming/python/tips

To install a package behind a proxy requiring SSL one can enforce trusted hosts to avoid certificate hand-shake and allow installation. This is done with the following options:

pip install --trusted-host pypi.org --trusted-host files.pythonhosted.org <pkg>

7.4 Regular expressions

Regular expressions (or simply regex) processing is a must-have skill for anyone doing scientific computing. Most programs produce results or logs in plain text and do not support specific data extraction from those. There regex becomes your best friend. Unfortunately during the years many flavors of regex appeared, each claiming to offer advantages or to be more formal than its predecessors. Due to this, learning regex is often language-specific (most of the time you create and process regex from your favorite language) and sometimes even package-specific. Needless to say, regex may be more difficult to master than assembly programming.

Useful tools:

• regex101
• regexr

Matching between two strings: match all characters between two strings with lookbehind and look ahead patterns. Notice that this will require the enclosing strings to be fixed (at least under PCRE). For processing WallyTutor.jl documentation I have used a more generic approach but less general than what is proposed here.

Match any character across multiple lines: as described here with (.|\n)*.

Regex in Julia: currently joining regexes in Julia might be tricky (because of escaping characters); a solution is proposed here and seems to work just fine with minimal extra coding.

7.5 Using nteract with a virtual environment

Warning

nteract project underwent a major restructuration and the following instructions might not work anymore.

# Activate the virtual environment:
. .venv\Scripts\Activate.ps1

# Give the kernel an unique name:
$kName = "your-kernel-name"

# Install the kernel:
python -m ipykernel install --user `
    --name $kName --display-name "Python ($kName)"

# Run nteract:
nteract.exe