other:uvcdat:cdat_conda:ipnb
Table of Contents
Working with ipython notebooks
What does this mean?
There are many python distributions, many python versions (2.7.xx
, 3.nn
) and… many ways to use python:
- interactively or by running a script (or running a script with
python -i script.py
and then typing interactive commands) - using python in a Linux terminal, the Spyder GUI (Graphical User Interface) a Windows cmd terminal or powershell, a Mac terminal, or whatever GUI the distribution you have installed on Windows or Mac provides
- using the default
python
interpreter or theipython
interpreter
> python Python 3.6.7 | packaged by conda-forge | (default, Feb 28 2019, 09:07:38) [GCC 7.3.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> "python interpreter in a terminal".upper() 'PYTHON INTERPRETER IN A TERMINAL' >>> > ipython Python 3.6.7 | packaged by conda-forge | (default, Feb 28 2019, 09:07:38) Type 'copyright', 'credits' or 'license' for more information IPython 7.3.0 -- An enhanced Interactive Python. Type '?' for help. In [1]: "ipython interpreter in a terminal".upper() Out[1]: 'IPYTHON INTERPRETER IN A TERMINAL' In [2]: Do you really want to exit ([y]/n)? y
- using python interactively in a ipython notebooks, which means that
- you will use an ipython interpreter
- in a web browser (rather than a terminal)
- a notebook is the content of the web page where you have interacted with python. You can save it (
.ppynb
file) and share it, and it will display both the commands you have used and the output/results of your commands - the notebooks have cells with python input, python output (including graphics), and cells that can be used to format the document (titles, formatted text, equations, images, etc…)
- the notebooks support several other languages (e.g. R)
How does it work?
Using an ipython notebooks implies the following steps:
- determine if you will use python on your local computer or a remote computer and go to that computer
- the selected computer mostly depends on the data you need to work with. The idea is to move the computation (i.e. python) near the data, and not move/replicate the data
- choose/initialize the distribution and version of python you need to use on the selected computer
- e.g. you can use the CDAT distribution on the LSCE servers
- start the ipython notebook server with the appropriate parameters (on the selected computer, with the selected python)
- this will start a temporary web server that you can connect to, in order to use the notebooks
- the server will display the URL that you have to use. The URL has an authentication token and only you (or the person who has a copy of the full URL) can connect to your server
- the basic way to start a server is to just type
jupyter notebook
, but only do this if you want to work locally on your desktop/laptop! See the next section for working on a remote computer- the
jupyter notebook
command will start the notebook server, and it will automatically start a web browser on the same computer. That's OK on your local computer, but on a remote server, the browser will probably seem to be slow (and probably out of date) and will use unnecessary resources of the server. This will be frustrating for you and the other users of the server
- if you have to work on a remote computer, the idea is to start the ipython server on the remote computer and use a web browser on your local computer (thanks to an ssh tunnel)
- in a web browser, open the temporary URL displayed by the ipython server and start using the ipython notebooks
- when you are finished, do not forget to kill the ipython server by typing
^C^C
(2xCTRL-C
) in the terminal where you started the server. Otherwise the server and the python processes started by the server will keep on using resources on the computer- Warning! Closing the browser, or using the
Logout
button in the browser will not stop the ipython server. You can stop the server by clicking on theXXXX
button
Working correctly with the notebooks
Reminder
- Make sure you read and understand the How does it work? section above
- If you have a python installed on your own desktop/laptop, you can just start a local notebook server and browser with
jupyter notebook
- On a remote and shared server, only start the notebook server and use a local browser to connect to the notebook server through an ssh tunnel.
Prerequisites
You need to meet the following prerequisites in order to create an ssh tunnel to the remote server where your notebook server will run
- We assume that:
- you have a local web browser installed!
- you have an account on the remote server: replace
<LOGIN>
below with your account name - you have an assigned a port number on the remote server: replace
<PPPP>
with your specific port number- List of assigned port numbers at LSCE: check the Shared TCP ports table
- Ask your advisor for a port number at LSCE if you do not have one
- List of assigned port numbers on ciclad: check the
/ciclad-home/jservon/Evaluation/CliMAF/climaf_installs/climaf_1.2.10/bin/cmip6.notebook.txt
file- Ask Jérôme Servonnat for a port number on ciclad if you do not have one
- System dependent requirements:
- Linux and Mac: you should already be able to open a local terminal, and
ssh
is installed by default - Windows: you either need to have Putty installed, or you need to know how to open a Windows PowerShell (
CTRL
+Esc
to open the start menu, then typePowerShell
and start a PowerShell)- Windows 7: you have to install Putty in order to use
ssh
:- In the instructions below, replace
ssh
withplink
if you are using Windows 7, because plink is the name of the Putty ssh command-line executable
- Windows 10: ssh is already installed, but you may still want to install Putty in order to use predefined sessions
- if you have installed Windows Subsystem for Linux and Ubuntu, you can also use a Ubuntu terminal
Digging ssh tunnels
The ssh tunnels will allow you to connect a web browser running locally on your computer to a notebook server running on a remote server
From a local LSCE computer to obelix
ssh -L 7225:localhost:7225 jypeter@obelix
From a computer outside LSCE to obelix
In that case, we have to go through the ssh1 gateway before connecting to obelix
plink -L 7225:localhost:7225 jypeter@obelix plink -L 7225:localhost:7225 jypeter@obelix ssh -X -t -L7100:localhost:7100 jypeter@ssh1.lsce.ipsl.fr ssh -L7100:localhost:7100 obelix plink -X -t -L7100:localhost:7100 jypeter@ssh1.lsce.ipsl.fr ssh -L7100:localhost:7100 obelix plink -ssh -X -t -L 7100:localhost:7100 jypeter@ssh1.lsce.ipsl.fr ssh -L7100:localhost:7100 obelix plink -X -t -L 7100:localhost:7100 jypeter@ssh1.lsce.ipsl.fr ssh -L7100:localhost:7100 obelix ssh -X -t -L 7100:localhost:7100 jypeter@ssh1.lsce.ipsl.fr ssh -L 7100:localhost:7100 obelix (cdatm_py3) jypeter@obelix3 - ...jypeter - 44 >jupyter notebook --no-browser --port=7100 [I 23:37:26.017 NotebookApp] [nb_conda_kernels] enabled, 14 kernels found [I 23:37:26.977 NotebookApp] [nb_conda] enabled [I 23:37:26.978 NotebookApp] Serving notebooks from local directory: /home/users/jypeter [I 23:37:26.978 NotebookApp] The Jupyter Notebook is running at: [I 23:37:26.978 NotebookApp] http://localhost:7100/?token=256f8fa26c8478f136720e0c0c24c4995b6682d9febf6387 [I 23:37:26.978 NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation). [C 23:37:26.992 NotebookApp] To access the notebook, open this file in a browser: file:///home/users/jypeter/.local/share/jupyter/runtime/nbserver-1442-open.html Or copy and paste one of these URLs: http://localhost:7100/?token=256f8fa26c8478f136720e0c0c24c4995b6682d9febf6387
Extra resources
- Jérôme's script for starting notebooks on the ciclad cluster:
/ciclad-home/jservon/Evaluation/CliMAF/climaf_installs/climaf_1.2.10/bin/climaf-notebook
- JYP's Testing CLIMAF on ciclad page
[ PMIP3 Wiki Home ] - [ Help! ] - [ Wiki syntax ]
other/uvcdat/cdat_conda/ipnb.txt · Last modified: 2019/05/20 11:10 by jypeter