Running Jupyter notebook on a remote server

1 minute read

Published:

Running a python notebook in need of large computing resources, expecially the GPUs, are always a concern for programmers. A good solution is to run the notebook on a remote cluster.

Secure a notebook server

This document describes how you can secure a notebook server and how to run it on a public interface.

Notes:

  1. When generating ‘mycert.pem’ the FQDN is required. A fully qualified domain name (FQDN) is the complete domain name for a specific computer, or host, on the internet. The FQDN consists of two parts: the hostname and the domain name. For example, an FQDN for a hypothetical mail server might be mymail.somecollege.edu. The hostname is mymail, and the host is located within the domain somecollege.edu.

  2. To set

    c.NotebookApp.ip = '*'
    

    makes the connection failure in some version. A possible change is to set

    c.NotebookApp.ip = '0.0.0.0'
    

SSH login without password

Given local host A and server B, you want to automatic login from A to B without entering a key every time. You can run the following commands to avoid it.

a@A:~> ssh-keygen -t rsa
a@A:~> ssh b@B mkdir -p .ssh
a@A:~> cat .ssh/id_rsa.pub | ssh b@B 'cat >> .ssh/authorized_keys'
a@A:~> ssh b@B

The details can be found here

Remote Access to notebooks via SSH

To get remote access to notebooks via SSH, first you need to run the notebook in no-browser mode with a given port on the server. Then create a tunnel through B doing local port forwarding and listening to the given port on the server. The command to do local port forwarding is ‘ssh -L’. Otherwise, if doing remote port forwarding, it’s ‘ssh -R’

b@B:~> jupyter notebook --no-browser --port=8889
a@A:~> ssh -N -L localhost:8888:(localhost/node***):8889 b@B

To check and close the tunnel, look for the process and kill it manually.

ps aux | grep localhost:8889
kill (process)

The explanation can be find here.

Remote Tensorboard Access

b@B:~> tensorboard --logdir logs --port 6006
a@A:~> ssh -N -f -L localhost:16006:localhost:6006 b@B

Good luck!