Working with ipython notebooks

There are many python distributions, many python versions (2.7.xx, 3.nn) and… many ways to use python:

• interactively or by running a script (or running a script with python -i script.py and then typing interactive commands)
• using python in a Linux terminal, the Spyder GUI (Graphical User Interface) a Windows cmd terminal or powershell, a Mac terminal, or whatever GUI the distribution you have installed on Windows or Mac provides
• using the default python interpreter or the ipython interpreter
  

   > python
  Python 3.6.7 | packaged by conda-forge | (default, Feb 28 2019, 09:07:38)
  [GCC 7.3.0] on linux
  Type "help", "copyright", "credits" or "license" for more information.
  >>> "python interpreter in a terminal".upper()
  'PYTHON INTERPRETER IN A TERMINAL'
  >>>
  
   > ipython
  Python 3.6.7 | packaged by conda-forge | (default, Feb 28 2019, 09:07:38)
  Type 'copyright', 'credits' or 'license' for more information
  IPython 7.3.0 -- An enhanced Interactive Python. Type '?' for help.
  
  In [1]: "ipython interpreter in a terminal".upper()
  Out[1]: 'IPYTHON INTERPRETER IN A TERMINAL'
  
  In [2]:
  Do you really want to exit ([y]/n)? y

• using python interactively in a ipython notebooks, which means that
  • you will use an ipython interpreter
  • in a web browser (rather than a terminal)
  • a notebook is the content of the web page where you have interacted with python. You can save it (.ppynb file) and share it, and it will display both the commands you have used and the output/results of your commands
  • the notebooks have cells with python input, python output (including graphics), and cells that can be used to format the document (titles, formatted text, equations, images, etc…)
  • the notebooks support several other languages (e.g. R)

Using an ipython notebooks implies the following steps:

1. determine if you will use python on your local computer or a remote computer and go to that computer
   • the selected computer mostly depends on the data you need to work with. The idea is to move the computation (i.e. python) near the data, and not move/replicate the data
2. choose/initialize the distribution and version of python you need to use on the selected computer
   • e.g. you can use the CDAT distribution on the LSCE servers
3. start the ipython notebook server with the appropriate parameters (on the selected computer, with the selected python)
   • this will start a temporary web server that you can connect to, in order to use the notebooks
     • the server will display the URL that you have to use. The URL has an authentication token and only you (or the person who has a copy of the full URL) can connect to your server
   • the basic way to start a server is to just type jupyter notebook, but only do this if you want to work locally on your desktop/laptop! See the next section for working on a remote computer
     • the jupyter notebook command will start the notebook server, and it will automatically start a web browser on the same computer. That's OK on your local computer, but on a remote server, the browser will probably seem to be slow (and probably out of date) and will use unnecessary resources of the server. This will be frustrating for you and the other users of the server
   • if you have to work on a remote computer, the idea is to start the ipython server on the remote computer and use a web browser on your local computer (thanks to an ssh tunnel)
4. in a web browser, open the temporary URL displayed by the ipython server and start using the ipython notebooks
5. when you are finished, do not forget to kill the ipython server by typing ^C^C (2x CTRL-C) in the terminal where you started the server. Otherwise the server and the python processes started by the server will keep on using resources on the computer
   • Warning! Closing the browser, or using the Logout button in the browser will not stop the ipython server. You can stop the server by clicking on the XXXX button

• Make sure you read and understand the How does it work? section above
• If you have a python installed on your own desktop/laptop, you can just start a local notebook server and browser with
  jupyter notebook
• On a remote and shared server, only start the notebook server and use a local browser to connect to the notebook server through an ssh tunnel.

You need to meet the following prerequisites in order to create an ssh tunnel to the remote server where your notebook server will run

• We assume that:
  • you have a local web browser installed!
  • you have an account on the remote server: replace <LOGIN> below with your account name
  • you have an assigned a port number on the remote server: replace <PPPP> with your specific port number
    • List of assigned port numbers at LSCE: check the Shared TCP ports table
      • Ask your advisor for a port number at LSCE if you do not have one
    • List of assigned port numbers on ciclad: check the /ciclad-home/jservon/Evaluation/CliMAF/climaf_installs/climaf_1.2.10/bin/cmip6.notebook.txt file
      • Ask Jérôme Servonnat for a port number on ciclad if you do not have one
• System dependent requirements:
  • Linux and Mac: you should already be able to open a local terminal, and ssh is installed by default
  • Windows: you either need to have Putty installed, or you need to know how to open a Windows PowerShell (CTRL+Esc to open the start menu, then type PowerShell and start a PowerShell)
    • Windows 7: you have to install Putty in order to use ssh:
      • In the instructions below, replace ssh with plink if you are using Windows 7, because plink is the name of the Putty ssh command-line executable
    • Windows 10: ssh is already installed, but you may still want to install Putty in order to use predefined sessions
      • if you have installed Windows Subsystem for Linux and Ubuntu, you can also use a Ubuntu terminal

The ssh tunnels will allow you to connect a web browser running locally on your computer to a notebook server running on a remote server

ssh -L 7225:localhost:7225 jypeter@obelix

In that case, we have to go through the ssh1 gateway before connecting to obelix

plink -L 7225:localhost:7225 jypeter@obelix


plink -L 7225:localhost:7225 jypeter@obelix

ssh -X -t -L7100:localhost:7100 jypeter@ssh1.lsce.ipsl.fr ssh -L7100:localhost:7100 obelix

plink -X -t -L7100:localhost:7100 jypeter@ssh1.lsce.ipsl.fr ssh -L7100:localhost:7100 obelix

plink -ssh -X -t -L 7100:localhost:7100 jypeter@ssh1.lsce.ipsl.fr ssh -L7100:localhost:7100 obelix

plink -X -t -L 7100:localhost:7100 jypeter@ssh1.lsce.ipsl.fr ssh -L7100:localhost:7100 obelix

ssh -X -t -L 7100:localhost:7100 jypeter@ssh1.lsce.ipsl.fr ssh -L 7100:localhost:7100 obelix

(cdatm_py3) jypeter@obelix3 - ...jypeter - 44 >jupyter notebook --no-browser --port=7100
[I 23:37:26.017 NotebookApp] [nb_conda_kernels] enabled, 14 kernels found
[I 23:37:26.977 NotebookApp] [nb_conda] enabled
[I 23:37:26.978 NotebookApp] Serving notebooks from local directory: /home/users/jypeter
[I 23:37:26.978 NotebookApp] The Jupyter Notebook is running at:
[I 23:37:26.978 NotebookApp] http://localhost:7100/?token=256f8fa26c8478f136720e0c0c24c4995b6682d9febf6387
[I 23:37:26.978 NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
[C 23:37:26.992 NotebookApp]

    To access the notebook, open this file in a browser:
        file:///home/users/jypeter/.local/share/jupyter/runtime/nbserver-1442-open.html
    Or copy and paste one of these URLs:
        http://localhost:7100/?token=256f8fa26c8478f136720e0c0c24c4995b6682d9febf6387

• Jérôme's script for starting notebooks on the ciclad cluster: /ciclad-home/jservon/Evaluation/CliMAF/climaf_installs/climaf_1.2.10/bin/climaf-notebook
• Patrick's page 1
• Patricks' page 2
• JYP's Testing CLIMAF on ciclad page

[ PMIP3 Wiki Home ] - [ Help! ] - [ Wiki syntax ]