other:uvcdat:cdat_conda:cdat_8_0_py2
Differences
This shows you the differences between two versions of the page.
| Next revision | Previous revision Next revision Both sides next revision | ||
|
other:uvcdat:cdat_conda:cdat_8_0_py2 [2018/11/07 17:08] jypeter created |
other:uvcdat:cdat_conda:cdat_8_0_py2 [2018/11/08 16:10] jypeter |
||
|---|---|---|---|
| Line 16: | Line 16: | ||
| ===== Installation with Miniconda3 ===== | ===== Installation with Miniconda3 ===== | ||
| - | We have installed a new version of Miniconda3, based on the old [[other:uvcdat:conda_notes#installing_miniconda|installing miniconda]] instructions | + | ==== Installing Miniconda3 ==== |
| - | * Follow the [[https://conda.io/docs/install/quick.html|Quick install]] steps to install Miniconda | + | We have installed a new version of Miniconda3, by following the existing [[other:uvcdat:conda_notes#installing_miniconda|installing miniconda]] instructions |
| + | |||
| + | * Follow the [[https://conda.io/docs/install/quick.html|Quick install]] steps to install Miniconda | ||
| * Get the [[https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh|Python 3.7 Linux 64-bit (bash installer)]] (unless there is a [[https://conda.io/miniconda.html|more recent version?]]) | * Get the [[https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh|Python 3.7 Linux 64-bit (bash installer)]] (unless there is a [[https://conda.io/miniconda.html|more recent version?]]) | ||
| * ''cd /home/scratch01/jypeter'' (some temporary storage space for the installer) | * ''cd /home/scratch01/jypeter'' (some temporary storage space for the installer) | ||
| Line 44: | Line 46: | ||
| 12K x86_64-conda_cos6-linux-gnu</code> | 12K x86_64-conda_cos6-linux-gnu</code> | ||
| * Accept the automatic update of the //PATH// variable in ''.bashrc'' (note: the installer automatically creates a backup of ''.bashrc'') | * Accept the automatic update of the //PATH// variable in ''.bashrc'' (note: the installer automatically creates a backup of ''.bashrc'') | ||
| - | * ''# added by Miniconda2 4.3.21 installer''\\ ''export PATH="/home/share/unix_files/cdat/miniconda2/bin:$PATH"'' | + | * ''# added by Miniconda3 installer''\\ ''export PATH="/home/share/unix_files/cdat/miniconda3/bin:$PATH"'' |
| - | * Open a new terminal (if you use bash) or type ''bash'' and check if the python from the conda distribution has been correctly installed | + | * Open a new terminal (if you use bash) or open a new terminal and type ''bash'' |
| - | * Update and clean the new installation | + | * check if the python from the conda distribution has been correctly installed\\ <code> > bash |
| - | * ''conda update %%--%%all'' | + | $ which python |
| - | * ''conda clean %%--%%all'' | + | /home/share/unix_files/cdat/miniconda3/bin/python |
| - | * <code>bash-4.1$ which python | + | $ which conda |
| - | /home/share/unix_files/cdat/miniconda2/bin/python | + | /home/share/unix_files/cdat/miniconda3/bin/conda</code> |
| - | bash-4.1$ du -sh /home/share/unix_files/cdat/miniconda2 | + | * Update and clean the new installation\\ <code>$ conda update --all |
| - | 114M /home/share/unix_files/cdat/miniconda2 | + | $ du -sh /home/share/unix_files/cdat/miniconda3 |
| - | bash-4.1$ conda list | + | 515M /home/share/unix_files/cdat/miniconda3 |
| - | # packages in environment at /home/share/unix_files/cdat/miniconda2: | + | $ conda clean --all |
| + | $ du -sh /home/share/unix_files/cdat/miniconda3 | ||
| + | 270M /home/share/unix_files/cdat/miniconda3 | ||
| + | $ conda list | ||
| + | # packages in environment at /home/share/unix_files/cdat/miniconda3: | ||
| # | # | ||
| - | cffi 1.9.1 py27_0 | + | # Name Version Build Channel |
| - | conda 4.3.13 py27_0 | + | |
| - | conda-env 2.6.0 0 | + | |
| - | cryptography 1.7.1 py27_0 | + | |
| [...] | [...] | ||
| + | python 3.7.1 h0371630_3 | ||
| + | readline 7.0 h7b6447c_5 | ||
| + | requests 2.20.0 py37_0 | ||
| + | [...]</code> | ||
| + | * Remove the miniconda3 initialization lines that were automatically added at the end of ''.bashrc'' and put them (and other useful commands) in a special initialization file (see ''~jypeter/.conda3_jyp.sh'' below), that can be sourced only when we actually want to use conda and CDAT\\ <code>bash-4.2$ which python | ||
| + | /usr/bin/python | ||
| + | $ cat ~jypeter/.conda3_jyp.sh | ||
| + | # Conda initialization by JYP | ||
| + | # | ||
| + | # Use this for working with conda and CDAT centrally managed by JYP | ||
| + | # | ||
| + | # More details in: | ||
| + | # https://wiki.lsce.ipsl.fr/pmip3/doku.php/other:python:starting#conda-based_v ersions_of_uv-cdat | ||
| + | # https://wiki.lsce.ipsl.fr/pmip3/doku.php/other:uvcdat:conda_notes | ||
| + | # | ||
| + | # Jean-Yves Peterschmitt - LSCE - 11/2018 | ||
| - | <note important>\\ We used the [[other:uvcdat:cdat_conda:cdat_2_8_0#downgrading_conda|recommended downgraded version of conda]] that was available after installing UV-CDAT 2.8</note> | + | export PATH="/home/share/unix_files/cdat/miniconda3/bin:$PATH" |
| - | DIsk space used before 2.10 installation (after running ''conda clean --all'') | + | # Use the alias below to easily determine where your python |
| + | # interpreter is located | ||
| + | alias wp="which python" | ||
| - | <code> > du -sh /home/share/unix_files/cdat/miniconda2 | + | # Where are ALL the python interpreters in the search path |
| - | 6.5G /home/share/unix_files/cdat/miniconda2 | + | alias wpa="which -a python" |
| - | > du -sh /home/share/unix_files/cdat/miniconda2/* | + | # The end |
| - | 4.4M /home/share/unix_files/cdat/miniconda2/bin | + | $ source ~jypeter/.conda3_jyp.sh |
| - | 4.0K /home/share/unix_files/cdat/miniconda2/conda-bld | + | $ wp |
| - | 272K /home/share/unix_files/cdat/miniconda2/conda-meta | + | /home/share/unix_files/cdat/miniconda3/bin/python |
| - | 6.4G /home/share/unix_files/cdat/miniconda2/envs | + | </code> |
| - | 20K /home/share/unix_files/cdat/miniconda2/etc | + | |
| - | 3.8M /home/share/unix_files/cdat/miniconda2/include | + | |
| - | 99M /home/share/unix_files/cdat/miniconda2/lib | + | |
| - | 4.0K /home/share/unix_files/cdat/miniconda2/LICENSE.txt | + | |
| - | 76K /home/share/unix_files/cdat/miniconda2/pkgs | + | |
| - | 376K /home/share/unix_files/cdat/miniconda2/share | + | |
| - | 364K /home/share/unix_files/cdat/miniconda2/ssl | + | |
| - | + | ||
| - | >du -sh /home/share/unix_files/cdat/miniconda2/envs/* | + | |
| - | 2.8G /home/share/unix_files/cdat/miniconda2/envs/cdatm14 | + | |
| - | 1.7G /home/share/unix_files/cdat/miniconda2/envs/keras | + | |
| - | 2.0G /home/share/unix_files/cdat/miniconda2/envs/uvcdat-2.8.0</code> | + | |
| - | + | ||
| - | Actual installation | + | |
| - | <code>bash-4.1$ conda create -n uvcdat-2.10 -c conda-forge -c uvcdat uvcdat=2.10 | + | ==== Installing CDAT 8.0 ==== |
| + | <code>conda create -n cdat-8.0_py2 -c cdat/label/v80 -c conda-forge -c cdat python=2.7 cdat | ||
| # Generate the list of installed packages | # Generate the list of installed packages | ||
| - | conda list -n uvcdat-2.10 > /home/scratch01/jypeter/uvcdat-2.10_list_170721.txt</code> | + | conda list -n cdat-8.0_py2 > /home/scratch01/jypeter/cdat-8.0_py2_list_181108.txt</code> |
| - | List of installed packages: {{:other:uvcdat:cdat_conda:uvcdat-2.10_list_170721.txt}} | + | List of installed packages: {{:other:uvcdat:cdat_conda:cdat-8.0_py2_list_181108.txt}} |
| - | Disk space after 2.10 installation | + | Disk space after 8.0 installation |
| - | <code>bash-4.1$ conda env list | + | <code>$ du -sh /home/share/unix_files/cdat/miniconda3 |
| - | cdatm14 /home/share/unix_files/cdat/miniconda2/envs/cdatm14 | + | 4.3G /home/share/unix_files/cdat/miniconda3 |
| - | keras /home/share/unix_files/cdat/miniconda2/envs/keras | + | $ du -sh /home/share/unix_files/cdat/miniconda3/* |
| - | uvcdat-2.10 /home/share/unix_files/cdat/miniconda2/envs/uvcdat-2.10 | + | 15M /home/share/unix_files/cdat/miniconda3/bin |
| - | uvcdat-2.8.0 /home/share/unix_files/cdat/miniconda2/envs/uvcdat-2.8.0 | + | 2.4M /home/share/unix_files/cdat/miniconda3/compiler_compat |
| - | root * /home/share/unix_files/cdat/miniconda2 | + | 4.0K /home/share/unix_files/cdat/miniconda3/conda-bld |
| + | 3.7M /home/share/unix_files/cdat/miniconda3/conda-meta | ||
| + | 2.8G /home/share/unix_files/cdat/miniconda3/envs | ||
| + | 28K /home/share/unix_files/cdat/miniconda3/etc | ||
| + | 4.9M /home/share/unix_files/cdat/miniconda3/include | ||
| + | 156M /home/share/unix_files/cdat/miniconda3/lib | ||
| + | 8.0K /home/share/unix_files/cdat/miniconda3/LICENSE.txt | ||
| + | 1.4G /home/share/unix_files/cdat/miniconda3/pkgs | ||
| + | 1.3M /home/share/unix_files/cdat/miniconda3/share | ||
| + | 24K /home/share/unix_files/cdat/miniconda3/ssl | ||
| + | 12K /home/share/unix_files/cdat/miniconda3/x86_64-conda_cos6-linux-gnu | ||
| + | $ du -sh /home/share/unix_files/cdat/miniconda3/envs/* | ||
| + | 2.8G /home/share/unix_files/cdat/miniconda3/envs/cdat-8.0_py2</code> | ||
| - | > du -sh /home/share/unix_files/cdat/miniconda2 | ||
| - | 11G /home/share/unix_files/cdat/miniconda2 | ||
| - | > du -sh /home/share/unix_files/cdat/miniconda2/* | + | ===== Cloning cdat to add specific packages for LSCE ===== |
| - | 4.4M /home/share/unix_files/cdat/miniconda2/bin | + | |
| - | 4.0K /home/share/unix_files/cdat/miniconda2/conda-bld | + | |
| - | 272K /home/share/unix_files/cdat/miniconda2/conda-meta | + | |
| - | 9.2G /home/share/unix_files/cdat/miniconda2/envs | + | |
| - | 20K /home/share/unix_files/cdat/miniconda2/etc | + | |
| - | 3.8M /home/share/unix_files/cdat/miniconda2/include | + | |
| - | 99M /home/share/unix_files/cdat/miniconda2/lib | + | |
| - | 4.0K /home/share/unix_files/cdat/miniconda2/LICENSE.txt | + | |
| - | 1.4G /home/share/unix_files/cdat/miniconda2/pkgs | + | |
| - | 376K /home/share/unix_files/cdat/miniconda2/share | + | |
| - | 364K /home/share/unix_files/cdat/miniconda2/ssl | + | |
| - | + | ||
| - | > du -sh /home/share/unix_files/cdat/miniconda2/envs/* | + | |
| - | 2.8G /home/share/unix_files/cdat/miniconda2/envs/cdatm14 | + | |
| - | 1.7G /home/share/unix_files/cdat/miniconda2/envs/keras | + | |
| - | 2.8G /home/share/unix_files/cdat/miniconda2/envs/uvcdat-2.10 | + | |
| - | 2.0G /home/share/unix_files/cdat/miniconda2/envs/uvcdat-2.8.0 | + | |
| - | </code> | + | |
| - | + | ||
| - | Disk space after removing the keras environment that we don't need any more | + | |
| - | + | ||
| - | <code>jypeter@asterix2 - ...cdat - 67 >du -sh /home/share/unix_files/cdat/miniconda2 | + | |
| - | 9.0G /home/share/unix_files/cdat/miniconda2 | + | |
| - | jypeter@asterix2 - ...cdat - 68 >du -sh /home/share/unix_files/cdat/miniconda2/* | + | |
| - | 4.4M /home/share/unix_files/cdat/miniconda2/bin | + | |
| - | 4.0K /home/share/unix_files/cdat/miniconda2/conda-bld | + | |
| - | 272K /home/share/unix_files/cdat/miniconda2/conda-meta | + | |
| - | 7.5G /home/share/unix_files/cdat/miniconda2/envs | + | |
| - | 20K /home/share/unix_files/cdat/miniconda2/etc | + | |
| - | 3.8M /home/share/unix_files/cdat/miniconda2/include | + | |
| - | 99M /home/share/unix_files/cdat/miniconda2/lib | + | |
| - | 4.0K /home/share/unix_files/cdat/miniconda2/LICENSE.txt | + | |
| - | 1.4G /home/share/unix_files/cdat/miniconda2/pkgs | + | |
| - | 376K /home/share/unix_files/cdat/miniconda2/share | + | |
| - | 364K /home/share/unix_files/cdat/miniconda2/ssl | + | |
| - | jypeter@asterix2 - ...cdat - 69 >du -sh /home/share/unix_files/cdat/miniconda2/envs/* | + | |
| - | 2.8G /home/share/unix_files/cdat/miniconda2/envs/cdatm14 | + | |
| - | 2.8G /home/share/unix_files/cdat/miniconda2/envs/uvcdat-2.10 | + | |
| - | 2.0G /home/share/unix_files/cdat/miniconda2/envs/uvcdat-2.8.0</code> | + | |
| - | + | ||
| - | Disk space used in <wrap em>a brand new directory hierarchy</wrap> (''cdat/miniconda2b'', on the obelix servers) | + | |
| - | + | ||
| - | <code>bash-4.2$ conda env list | + | |
| - | uvcdat-2.10 /home/share/unix_files/cdat/miniconda2b/envs/uvcdat-2.10 | + | |
| - | root * /home/share/unix_files/cdat/miniconda2b | + | |
| - | + | ||
| - | bash-4.2$ du -sh /home/share/unix_files/cdat/miniconda2b | + | |
| - | 4.3G /home/share/unix_files/cdat/miniconda2b | + | |
| - | + | ||
| - | bash-4.2$ du -sh /home/share/unix_files/cdat/miniconda2b/* | + | |
| - | 4.4M /home/share/unix_files/cdat/miniconda2b/bin | + | |
| - | 4.0K /home/share/unix_files/cdat/miniconda2b/conda-bld | + | |
| - | 280K /home/share/unix_files/cdat/miniconda2b/conda-meta | + | |
| - | 2.8G /home/share/unix_files/cdat/miniconda2b/envs | + | |
| - | 20K /home/share/unix_files/cdat/miniconda2b/etc | + | |
| - | 3.8M /home/share/unix_files/cdat/miniconda2b/include | + | |
| - | 100M /home/share/unix_files/cdat/miniconda2b/lib | + | |
| - | 8.0K /home/share/unix_files/cdat/miniconda2b/LICENSE.txt | + | |
| - | 1.4G /home/share/unix_files/cdat/miniconda2b/pkgs | + | |
| - | 16K /home/share/unix_files/cdat/miniconda2b/share | + | |
| - | 376K /home/share/unix_files/cdat/miniconda2b/ssl | + | |
| - | </code> | + | |
| - | + | ||
| - | + | ||
| - | ===== Cloning uvcdat to add specific packages for LSCE ===== | + | |
| <WRAP center round tip 60%> | <WRAP center round tip 60%> | ||
| - | This section is about the creation of the **cdatm15** environment | + | This section is about the creation of the **cdatm17** environment |
| - | Notes about [[https://wiki.lsce.ipsl.fr/pmip3/doku.php/other:python:starting#conda-based_versions_of_uv-cdat|actually using the cdatm15 conda-based python]] | + | Notes about [[https://wiki.lsce.ipsl.fr/pmip3/doku.php/other:python:starting#conda-based_versions_of_uv-cdat|actually using the cdatm17 conda-based python]] |
| </WRAP> | </WRAP> | ||
| - | Note: using hard links, cloning a full environment //only// adds an extra 500M of disk space | + | Note: using hard links, cloning a full environment //only// adds an extra 582M of disk space |
| - | <code>bash-4.1$ du -sh /home/share/unix_files/cdat/miniconda2 | + | <code>$ du -sh /home/share/unix_files/cdat/miniconda3/envs/* |
| - | 9.0G /home/share/unix_files/cdat/miniconda2 | + | 2.8G /home/share/unix_files/cdat/miniconda3/envs/cdat-8.0_py2 |
| + | $ conda create -n cdatm17_py2 --clone cdat-8.0_py2 | ||
| + | Source: /home/share/unix_files/cdat/miniconda3/envs/cdat-8.0_py2 | ||
| + | Destination: /home/share/unix_files/cdat/miniconda3/envs/cdatm17_py2 | ||
| + | Packages: 226 | ||
| + | Files: 3 | ||
| - | bash-4.1$ du -sh /home/share/unix_files/cdat/miniconda2/envs/* | ||
| - | 2.8G /home/share/unix_files/cdat/miniconda2/envs/cdatm14 | ||
| - | 2.8G /home/share/unix_files/cdat/miniconda2/envs/uvcdat-2.10 | ||
| - | 2.0G /home/share/unix_files/cdat/miniconda2/envs/uvcdat-2.8.0 | ||
| - | |||
| - | bash-4.1$ conda create -n cdatm15 --clone uvcdat-2.10 | ||
| - | Source: /home/share/unix_files/cdat/miniconda2/envs/uvcdat-2.10 | ||
| - | Destination: /home/share/unix_files/cdat/miniconda2/envs/cdatm15 | ||
| [...] | [...] | ||
| - | bash-4.1$ du -sh /home/share/unix_files/cdat/miniconda2 | + | $ du -sh /home/share/unix_files/cdat/miniconda3/envs/* |
| - | 9.5G /home/share/unix_files/cdat/miniconda2 | + | 2.8G /home/share/unix_files/cdat/miniconda3/envs/cdat-8.0_py2 |
| + | 582M /home/share/unix_files/cdat/miniconda3/envs/cdatm17_py2 | ||
| - | bash-4.1$ du -sh /home/share/unix_files/cdat/miniconda2/envs/* | ||
| - | 2.8G /home/share/unix_files/cdat/miniconda2/envs/cdatm14 | ||
| - | 2.8G /home/share/unix_files/cdat/miniconda2/envs/cdatm15 | ||
| - | 573M /home/share/unix_files/cdat/miniconda2/envs/uvcdat-2.10 | ||
| - | 2.0G /home/share/unix_files/cdat/miniconda2/envs/uvcdat-2.8.0 | ||
| - | </code> | ||
| - | |||
| - | Disk space used after cloning, in <wrap em>in the new directory hierarchy</wrap> (''cdat/miniconda2b'', on the obelix servers) | ||
| - | |||
| - | <code>bash-4.2$ du -sh /home/share/unix_files/cdat/miniconda2b | ||
| - | 4.8G /home/share/unix_files/cdat/miniconda2b | ||
| - | |||
| - | bash-4.2$ du -sh /home/share/unix_files/cdat/miniconda2b/* | ||
| - | 4.4M /home/share/unix_files/cdat/miniconda2b/bin | ||
| - | 4.0K /home/share/unix_files/cdat/miniconda2b/conda-bld | ||
| - | 280K /home/share/unix_files/cdat/miniconda2b/conda-meta | ||
| - | 3.4G /home/share/unix_files/cdat/miniconda2b/envs | ||
| - | 20K /home/share/unix_files/cdat/miniconda2b/etc | ||
| - | 3.8M /home/share/unix_files/cdat/miniconda2b/include | ||
| - | 100M /home/share/unix_files/cdat/miniconda2b/lib | ||
| - | 8.0K /home/share/unix_files/cdat/miniconda2b/LICENSE.txt | ||
| - | 1.4G /home/share/unix_files/cdat/miniconda2b/pkgs | ||
| - | 16K /home/share/unix_files/cdat/miniconda2b/share | ||
| - | 376K /home/share/unix_files/cdat/miniconda2b/ssl | ||
| - | |||
| - | bash-4.2$ du -sh /home/share/unix_files/cdat/miniconda2b/envs/* | ||
| - | 2.8G /home/share/unix_files/cdat/miniconda2b/envs/cdatm15 | ||
| - | 574M /home/share/unix_files/cdat/miniconda2b/envs/uvcdat-2.10 | ||
| </code> | </code> | ||
| Line 234: | Line 157: | ||
| We create a **//cdatm// symbolic link** in the ''envs'' directory, that has a //stable name// but can be moved to point to the latest default CDAT. In that case, most users can just activate this //cdatm// version and always get the latest stable version | We create a **//cdatm// symbolic link** in the ''envs'' directory, that has a //stable name// but can be moved to point to the latest default CDAT. In that case, most users can just activate this //cdatm// version and always get the latest stable version | ||
| - | <code>bash-4.2$ cd /home/share/unix_files/cdat/miniconda2b/envs | + | <code>$ cd /home/share/unix_files/cdat/miniconda3/envs |
| - | + | $ conda env list | |
| - | bash-4.2$ conda env list | + | base * /home/share/unix_files/cdat/miniconda3 |
| - | cdatm15 /home/share/unix_files/cdat/miniconda2b/envs/cdatm15 | + | cdat-8.0_py2 /home/share/unix_files/cdat/miniconda3/envs/cdat-8.0_py2 |
| - | uvcdat-2.10 /home/share/unix_files/cdat/miniconda2b/envs/uvcdat-2.10 | + | cdatm17_py2 /home/share/unix_files/cdat/miniconda3/envs/cdatm17_py2 |
| - | root * /home/share/unix_files/cdat/miniconda2b | + | $ ln -s cdatm17_py2 cdatm |
| - | + | $ conda env list | |
| - | bash-4.2$ ln -s cdatm15 cdatm | + | base * /home/share/unix_files/cdat/miniconda3 |
| - | + | cdat-8.0_py2 /home/share/unix_files/cdat/miniconda3/envs/cdat-8.0_py2 | |
| - | bash-4.2$ conda env list | + | cdatm /home/share/unix_files/cdat/miniconda3/envs/cdatm |
| - | cdatm /home/share/unix_files/cdat/miniconda2b/envs/cdatm | + | cdatm17_py2 /home/share/unix_files/cdat/miniconda3/envs/cdatm17_py2 |
| - | cdatm15 /home/share/unix_files/cdat/miniconda2b/envs/cdatm15 | + | $ source activate cdatm |
| - | uvcdat-2.10 /home/share/unix_files/cdat/miniconda2b/envs/uvcdat-2.10 | + | |
| - | root * /home/share/unix_files/cdat/miniconda2b | + | |
| - | + | ||
| - | bash-4.2$ source activate cdatm | + | |
| (cdatm) bash-4.2$ conda env list | (cdatm) bash-4.2$ conda env list | ||
| - | cdatm * /home/share/unix_files/cdat/miniconda2b/envs/cdatm | + | base /home/share/unix_files/cdat/miniconda3 |
| - | cdatm15 /home/share/unix_files/cdat/miniconda2b/envs/cdatm15 | + | cdat-8.0_py2 /home/share/unix_files/cdat/miniconda3/envs/cdat-8.0_py2 |
| - | uvcdat-2.10 /home/share/unix_files/cdat/miniconda2b/envs/uvcdat-2.10 | + | cdatm * /home/share/unix_files/cdat/miniconda3/envs/cdatm |
| - | root /home/share/unix_files/cdat/miniconda2b | + | cdatm17_py2 /home/share/unix_files/cdat/miniconda3/envs/cdatm17_py2</code> |
| - | </code> | + | |
| ===== Customizing UV-CDAT for LSCE ===== | ===== Customizing UV-CDAT for LSCE ===== | ||
other/uvcdat/cdat_conda/cdat_8_0_py2.txt · Last modified: 2021/02/27 13:55 by jypeter
