Compute Canada provides the Canadian research community access to advanced computing resources. Jupyter Notebooks are a great way to create and share reproducible research. In this tutorial, we will show you how to set up a Jupyter Notebook server on the Compute Canada cloud system that you can access remotely.
You’ll need to apply for a Compute Canada Database (CCDB) account and get your PI to approve it.
Using your CCDB username, you’ll need to apply for a separate Compute Canada Cloud account.
I spent some time trying to login to the East and West cloud with my CCDB account without applying for a Cloud account, which obviously didn’t work. The need to register for a separate Cloud account wasn’t very clear to me because the Compute Canada website had the registration link and instructions hidden in the middle of their Cloud service page.
Step 1 — Setting up your Cloud instance
Instances are virtual machines and volumes are disk space that can be attached to instances. Data on instances are destroyed when you terminate an instance, but you can store persistent data on volumes.
Step 1.1 — Security
We first need to open up some ports to access the Jupyter Notebook server remotely. In the Access & Security tab, click on Manage Rules for the default security group. Click Add Rule and add SSH, HTTP, HTTPS and a Custom TCP Rule for Port 9999 with direction ingress.
Step 1.2 — Volume
In the Volumes tab, click on the Create Volume button and fill up the name and description. Use Image as the Volume Source and Ubuntu-16.04 as the image source. For the size, anything above 5 GiB should be sufficient. Use nova for the availability zone.
Step 1.3 — Instance
In the Instances tab, click on Launch Instance. Give it a name and use nova for the availability zone. In Instance Boot Source select Boot from volume, then choose the volume that you’ve just created.
You can select the size of your new virtual machine as pre-defined flavours. You have 2 main choices, a p (persistent) flavour and a c (compute) flavour. Briefly, c flavours are intended for running shorter, intensive computations and for development and testing purposes while p flavours are meant for running longer computations. You can find more information in the Compute Canada wiki.
DO NOT CHECK the Delete Volume on Instance Delete box or you will lose your data when you terminate the instance.
Step 1.4 — Creating an SSH key pair
Under Access & Security, select the default security group.
Click on the + button in the Key Pair form to create a new key pair. We will follow the instructions in the new popup window to import a new key pair. Give your key pair a name first, i.e.
If you’re on Linux or Mac, open up your terminal (
Ctrl+Alt+T for Ubuntu) the run the following commands:
Use ssh-keygen to create a key pair named
canadacloud (you can name it anything you like). It will ask you for a passphrase, just leave it blank if you want.
ssh-keygen -t rsa -f canadacloud
Grab and copy the public key using this command
Then paste the contents back in the popup window in the Public Key box then click the Import button.
Once your done, click launch and wait for your instance to boot up.
Step 1.5 — Associating a Floating IP
To connect to our new instance, we need to assign a public IP to it. In the Access & Security tab, select Floating IPs and click on Allocate IP to Project. Select the default pool and click on the Allocate IP button.
Click on the Associate button for the floating IP address and then select your instance in Port to be associated.
Step 2 — Installing Anaconda and Jupyter
In your terminal, login to the instance with the following command, replacing
xxx.xxx.xxx.xxx with your floating IP address and
computecanada with your key name.
ssh -i computecanada email@example.com
If you’re having trouble logging in, try restarting your instance using a soft (or hard) reboot from the dropdown menu in the Instances page.
First, update your system with the following commands:
sudo apt-get update sudo apt-get upgrade
Once that’s done, download and install Anaconda:
wget https://repo.continuum.io/archive/Anaconda3-4.4.0-Linux-x86_64.sh bash Anaconda3-4.4.0-Linux-x86_64.sh
Step 3 — Configuring Jupyter Notebook
Once Anaconda is installed, you should be able to launch Jupyter with this command:
Ctrl+C to close the server.
We will follow the instructions here to set up Jupyter as a public server and secure it with HTTPS and a password.
Step 3.1 — Generate a Jupyter configuration file
First, create a configuration file
jupyter notebook --generate-config
and generate a hashed password
jupyter notebook password
Step 3.2 — Create an SSL certificate
openssl req -x509 -nodes -days 365 -newkey rsa:1024 -keyout mykey.key -out mycert.pem
Step 3.3 — Edit the configuration file
First copy your hashed password (the string starting with sha1) with the following command and save it somewhere safe:
Open the Jupyter config file in
Then add the following lines to the top of the file. Replace YourPassword with the password that you copied.
c.NotebookApp.certfile = u'/home/ubuntu/mycert.pem' c.NotebookApp.keyfile = u'/home/ubuntu/mykey.key' c.NotebookApp.ip = '*' c.NotebookApp.password = u'YourPassword' c.NotebookApp.open_browser = False c.NotebookApp.port = 9999
To close and save the file, press
Ctrl+X and select Yes.
Step 4 — Testing your new Notebook
Start the Jupyter notebook:
Point your browser to the following address, replace xxx.xxx.xxx.xxx with your floating IP:
If everything works, you should see a login screen and you will be able to enter your password and login to your Notebook!
Step 5 — Persistent Notebooks
Closing your SSH connection will close the notebook server.
To create a persistent notebook, we will use the screen terminal multiplexer. First, close the server with
Ctrl+C then run:
Then in the new window, run the Jupyter notebook:
Now if you close the terminal, the Notebook server will still be running.
Step 6 — Conclusion
Congratulations, you should now have a Jupyter Notebook running on the Compute Canada Cloud that you can access remotely. For a quick tour on using Jupyter notebooks, visit this link.
If you get an SSL security error in your browser, it’s because you’re using a self-signed certificate. To get rid of the error, you’ll need to use a custom domain and use a Let’s Encrypt certificate instead of a self-signed certificate:
sudo add-apt-repository ppa:certbot/certbot sudo apt-get update sudo apt-get install certbot sudo certbot certonly --standalone -d your.domain.name
Then replace the certificate and key in your Jupyter configuration file with the Let’s Encrypt certificates
c.NotebookApp.certfile = u'/etc/letsencrypt/live/your.domain.name/fullchain.pem' c.NotebookApp.keyfile = u'/etc/letsencrypt/live/your.domain.name/privkey.pem'