This shows you the differences between two versions of the page.
Next revision | Previous revision Next revision Both sides next revision | ||
other:uvcdat:cdat_conda:cdat_8_0_py2 [2018/11/07 17:08] jypeter created |
other:uvcdat:cdat_conda:cdat_8_0_py2 [2020/01/22 16:49] jypeter [Installing Miniconda3] |
||
---|---|---|---|
Line 16: | Line 16: | ||
===== Installation with Miniconda3 ===== | ===== Installation with Miniconda3 ===== | ||
- | We have installed a new version of Miniconda3, based on the old [[other:uvcdat:conda_notes#installing_miniconda|installing miniconda]] instructions | + | ==== Installing Miniconda3 ==== |
- | * Follow the [[https://conda.io/docs/install/quick.html|Quick install]] steps to install Miniconda | + | * You don't need to be //root// when you install Miniconda3 (actually **you should not be //root//**) |
- | * Get the [[https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh|Python 3.7 Linux 64-bit (bash installer)]] (unless there is a [[https://conda.io/miniconda.html|more recent version?]]) | + | * If you are installing Miniconda3 in a Linux environment on a Windows 10 computer using Windows Subsystem for Linux (WSL), pay special attention to the instructions on the WSL lines |
- | * ''cd /home/scratch01/jypeter'' (some temporary storage space for the installer) | + | |
- | * ''wget https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh'' (60 Mo, 11 Nov 2018) | + | We have installed a new version of Miniconda3 (steps below adapted from the [[other:uvcdat:conda_notes#installing_miniconda|installing miniconda]] instructions) |
- | * Execute the installer | + | |
- | * ''bash Miniconda3-latest-Linux-x86_64.sh'' | + | * Get the [[https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh|Python 3.7 Linux 64-bit (bash installer)]] (unless there is a [[https://conda.io/miniconda.html|more recent version?]]) |
- | * Accept the license | + | * Find some temporary space on Linux or Windows |
- | * Specify an explicit installation path **outside of the home directory**, with enough disk space (more than 3G if you are going to install UV-CDAT and some extra packages), preferably on a disk that is not backed up: ''/home/share/unix_files/cdat/miniconda3'' | + | * Linux:''cd /home/scratch01/jypeter'' |
- | * Resulting ''miniconda3'' directory size is 302M\\ <code> >du -sh miniconda3 | + | * WSL: ''cd /mnt/h/Scratch/jypeter/'' (assuming that there is a ''H:\Scratch\jypeter'' directory) |
+ | * ''wget https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh'' (67 Mo, 15 Feb 2019) | ||
+ | * Execute the installer | ||
+ | * ''bash Miniconda3-latest-Linux-x86_64.sh'' | ||
+ | * Accept the license | ||
+ | * Specify an explicit installation path **outside of the home directory**, with enough disk space (more than 3G if you are going to install UV-CDAT and some extra packages), preferably on a disk that is not backed up: | ||
+ | * Linux LSCE: ''/home/share/unix_files/cdat/miniconda3'' | ||
+ | * Linux ciclad: ''/data/jypmce/cdat/miniconda3'' | ||
+ | * WSL: installing to a directory that is not in ''/home/'' <wrap hi>does not work</wrap> (e.g. ''/mnt/h/CDAT/miniconda3'',assuming there is a ''H:\CDAT\'' directory, does not work)\\ Need to accept the installation in the default location: ''/home/jypeter/miniconda3'' | ||
+ | * Resulting ''miniconda3'' directory size is 302M\\ <code> >du -sh miniconda3 | ||
302M miniconda3 | 302M miniconda3 | ||
>cd miniconda3 | >cd miniconda3 | ||
Line 43: | Line 52: | ||
92K ssl | 92K ssl | ||
12K x86_64-conda_cos6-linux-gnu</code> | 12K x86_64-conda_cos6-linux-gnu</code> | ||
- | * Accept the automatic update of the //PATH// variable in ''.bashrc'' (note: the installer automatically creates a backup of ''.bashrc'') | + | * Accept the automatic update of the //PATH// variable in ''.bashrc'' (note: the installer automatically creates a backup of ''.bashrc''). This will some extra lines at the end of ''.bashrc'' used by the //old-style conda initialization// (we will later switch to the //new-style//). The added lines should look like: |
- | * ''# added by Miniconda2 4.3.21 installer''\\ ''export PATH="/home/share/unix_files/cdat/miniconda2/bin:$PATH"'' | + | * ''# added by Miniconda3 installer''\\ ''export PATH="/home/share/unix_files/cdat/miniconda3/bin:$PATH"'' |
- | * Open a new terminal (if you use bash) or type ''bash'' and check if the python from the conda distribution has been correctly installed | + | * ''# added by Miniconda4.xxx installer''\\ ''[some really fancy bash shell commands]'' |
- | * Update and clean the new installation | + | * Remove the installer: ''rm Miniconda3-latest-Linux-x86_64.sh'' |
- | * ''conda update %%--%%all'' | + | * Open a new terminal (if you use bash) or open a new terminal and type ''bash'' |
- | * ''conda clean %%--%%all'' | + | * check if the python from the conda distribution has been correctly installed\\ <code> > bash |
- | * <code>bash-4.1$ which python | + | $ which python |
- | /home/share/unix_files/cdat/miniconda2/bin/python | + | /home/share/unix_files/cdat/miniconda3/bin/python |
- | bash-4.1$ du -sh /home/share/unix_files/cdat/miniconda2 | + | $ which conda |
- | 114M /home/share/unix_files/cdat/miniconda2 | + | /home/share/unix_files/cdat/miniconda3/bin/conda</code> |
- | bash-4.1$ conda list | + | * Update and clean the new installation\\ <code>$ conda update --all |
- | # packages in environment at /home/share/unix_files/cdat/miniconda2: | + | $ du -sh /home/share/unix_files/cdat/miniconda3 |
+ | 515M /home/share/unix_files/cdat/miniconda3 | ||
+ | $ conda clean --all | ||
+ | $ du -sh /home/share/unix_files/cdat/miniconda3 | ||
+ | 270M /home/share/unix_files/cdat/miniconda3 | ||
+ | $ conda list | ||
+ | # packages in environment at /home/share/unix_files/cdat/miniconda3: | ||
# | # | ||
- | cffi 1.9.1 py27_0 | + | # Name Version Build Channel |
- | conda 4.3.13 py27_0 | + | |
- | conda-env 2.6.0 0 | + | |
- | cryptography 1.7.1 py27_0 | + | |
[...] | [...] | ||
+ | python 3.7.1 h0371630_3 | ||
+ | readline 7.0 h7b6447c_5 | ||
+ | requests 2.20.0 py37_0 | ||
+ | [...]</code> | ||
- | <note important>\\ We used the [[other:uvcdat:cdat_conda:cdat_2_8_0#downgrading_conda|recommended downgraded version of conda]] that was available after installing UV-CDAT 2.8</note> | + | ==== Post-Miniconda3 installation ==== |
- | DIsk space used before 2.10 installation (after running ''conda clean --all'') | + | The idea is to remove the miniconda3 initialization lines that were automatically added at the end of ''.bashrc'' and put them (and other useful commands) in a special initialization file, that can be sourced only when we actually want to use conda and CDAT (in order to avoid potentiel side effects) |
- | <code> > du -sh /home/share/unix_files/cdat/miniconda2 | + | === New style initialization === |
- | 6.5G /home/share/unix_files/cdat/miniconda2 | + | |
- | > du -sh /home/share/unix_files/cdat/miniconda2/* | + | Note: this is the **new** ''conda activate some_version'' style |
- | 4.4M /home/share/unix_files/cdat/miniconda2/bin | + | |
- | 4.0K /home/share/unix_files/cdat/miniconda2/conda-bld | + | |
- | 272K /home/share/unix_files/cdat/miniconda2/conda-meta | + | |
- | 6.4G /home/share/unix_files/cdat/miniconda2/envs | + | |
- | 20K /home/share/unix_files/cdat/miniconda2/etc | + | |
- | 3.8M /home/share/unix_files/cdat/miniconda2/include | + | |
- | 99M /home/share/unix_files/cdat/miniconda2/lib | + | |
- | 4.0K /home/share/unix_files/cdat/miniconda2/LICENSE.txt | + | |
- | 76K /home/share/unix_files/cdat/miniconda2/pkgs | + | |
- | 376K /home/share/unix_files/cdat/miniconda2/share | + | |
- | 364K /home/share/unix_files/cdat/miniconda2/ssl | + | |
- | >du -sh /home/share/unix_files/cdat/miniconda2/envs/* | + | See the ''~jypeter/.conda3_jyp.sh'' file below, and how to use it, **in a bash shell**. In a **tcsh shell**, see the ''~jypeter/.conda3_jyp.csh'' further down. In both shell cases, if you are installing your own version of python, you need to use your own location of the initialization files in the ''source'' lines, and you can use another file name than ''conda3_jyp'' |
- | 2.8G /home/share/unix_files/cdat/miniconda2/envs/cdatm14 | + | |
- | 1.7G /home/share/unix_files/cdat/miniconda2/envs/keras | + | |
- | 2.0G /home/share/unix_files/cdat/miniconda2/envs/uvcdat-2.8.0</code> | + | |
- | Actual installation | + | <code>$ which python |
+ | /usr/bin/python | ||
- | <code>bash-4.1$ conda create -n uvcdat-2.10 -c conda-forge -c uvcdat uvcdat=2.10 | + | $ cat ~jypeter/.conda3_jyp.sh |
+ | # Conda initialization by JYP, NEW style | ||
+ | # | ||
+ | # Use this for working with conda and CDAT centrally managed by JYP | ||
+ | # | ||
+ | # Execute this file in a BASH shell with | ||
+ | # source path/this_file | ||
+ | # Then get the list of available python distributions with | ||
+ | # conda env list | ||
+ | # Then activate a specific distribution with | ||
+ | # conda activate version_name | ||
+ | # | ||
+ | # More details in: | ||
+ | # https://wiki.lsce.ipsl.fr/pmip3/doku.php/other:python:starting#conda-based_versions_of_uv-cdat | ||
+ | # https://wiki.lsce.ipsl.fr/pmip3/doku.php/other:uvcdat:conda_notes | ||
+ | # | ||
+ | # Jean-Yves Peterschmitt - LSCE - 11/2018 | ||
- | # Generate the list of installed packages | + | source /home/share/unix_files/cdat/miniconda3/etc/profile.d/conda.sh |
- | conda list -n uvcdat-2.10 > /home/scratch01/jypeter/uvcdat-2.10_list_170721.txt</code> | + | |
- | List of installed packages: {{:other:uvcdat:cdat_conda:uvcdat-2.10_list_170721.txt}} | + | # Use the alias below to easily determine where your python |
+ | # interpreter is located | ||
+ | alias wp="which python" | ||
- | Disk space after 2.10 installation | + | # Where are ALL the python interpreters in the search path |
+ | alias wpa="which -a python" | ||
- | <code>bash-4.1$ conda env list | + | # The end |
- | cdatm14 /home/share/unix_files/cdat/miniconda2/envs/cdatm14 | + | |
- | keras /home/share/unix_files/cdat/miniconda2/envs/keras | + | |
- | uvcdat-2.10 /home/share/unix_files/cdat/miniconda2/envs/uvcdat-2.10 | + | |
- | uvcdat-2.8.0 /home/share/unix_files/cdat/miniconda2/envs/uvcdat-2.8.0 | + | |
- | root * /home/share/unix_files/cdat/miniconda2 | + | |
- | > du -sh /home/share/unix_files/cdat/miniconda2 | + | $ source ~jypeter/.conda3_jyp.sh |
- | 11G /home/share/unix_files/cdat/miniconda2 | + | |
- | > du -sh /home/share/unix_files/cdat/miniconda2/* | + | $ conda activate |
- | 4.4M /home/share/unix_files/cdat/miniconda2/bin | + | (base) $ which python |
- | 4.0K /home/share/unix_files/cdat/miniconda2/conda-bld | + | /home/share/unix_files/cdat/miniconda3/bin/python |
- | 272K /home/share/unix_files/cdat/miniconda2/conda-meta | + | |
- | 9.2G /home/share/unix_files/cdat/miniconda2/envs | + | |
- | 20K /home/share/unix_files/cdat/miniconda2/etc | + | |
- | 3.8M /home/share/unix_files/cdat/miniconda2/include | + | |
- | 99M /home/share/unix_files/cdat/miniconda2/lib | + | |
- | 4.0K /home/share/unix_files/cdat/miniconda2/LICENSE.txt | + | |
- | 1.4G /home/share/unix_files/cdat/miniconda2/pkgs | + | |
- | 376K /home/share/unix_files/cdat/miniconda2/share | + | |
- | 364K /home/share/unix_files/cdat/miniconda2/ssl | + | |
- | > du -sh /home/share/unix_files/cdat/miniconda2/envs/* | ||
- | 2.8G /home/share/unix_files/cdat/miniconda2/envs/cdatm14 | ||
- | 1.7G /home/share/unix_files/cdat/miniconda2/envs/keras | ||
- | 2.8G /home/share/unix_files/cdat/miniconda2/envs/uvcdat-2.10 | ||
- | 2.0G /home/share/unix_files/cdat/miniconda2/envs/uvcdat-2.8.0 | ||
</code> | </code> | ||
- | Disk space after removing the keras environment that we don't need any more | + | **tcsh shell** usage example |
- | <code>jypeter@asterix2 - ...cdat - 67 >du -sh /home/share/unix_files/cdat/miniconda2 | + | <code> >which python |
- | 9.0G /home/share/unix_files/cdat/miniconda2 | + | /usr/bin/python |
- | jypeter@asterix2 - ...cdat - 68 >du -sh /home/share/unix_files/cdat/miniconda2/* | + | |
- | 4.4M /home/share/unix_files/cdat/miniconda2/bin | + | |
- | 4.0K /home/share/unix_files/cdat/miniconda2/conda-bld | + | |
- | 272K /home/share/unix_files/cdat/miniconda2/conda-meta | + | |
- | 7.5G /home/share/unix_files/cdat/miniconda2/envs | + | |
- | 20K /home/share/unix_files/cdat/miniconda2/etc | + | |
- | 3.8M /home/share/unix_files/cdat/miniconda2/include | + | |
- | 99M /home/share/unix_files/cdat/miniconda2/lib | + | |
- | 4.0K /home/share/unix_files/cdat/miniconda2/LICENSE.txt | + | |
- | 1.4G /home/share/unix_files/cdat/miniconda2/pkgs | + | |
- | 376K /home/share/unix_files/cdat/miniconda2/share | + | |
- | 364K /home/share/unix_files/cdat/miniconda2/ssl | + | |
- | jypeter@asterix2 - ...cdat - 69 >du -sh /home/share/unix_files/cdat/miniconda2/envs/* | + | |
- | 2.8G /home/share/unix_files/cdat/miniconda2/envs/cdatm14 | + | |
- | 2.8G /home/share/unix_files/cdat/miniconda2/envs/uvcdat-2.10 | + | |
- | 2.0G /home/share/unix_files/cdat/miniconda2/envs/uvcdat-2.8.0</code> | + | |
- | Disk space used in <wrap em>a brand new directory hierarchy</wrap> (''cdat/miniconda2b'', on the obelix servers) | + | >cat ~jypeter/.conda3_jyp.csh |
+ | # Conda initialization by JYP, NEW style | ||
+ | # | ||
+ | # Use this for working with conda and CDAT centrally managed by JYP | ||
+ | # | ||
+ | # Execute this file in a TCSH shell with | ||
+ | # source path/this_file | ||
+ | # Then get the list of available python distributions with | ||
+ | # conda env list | ||
+ | # Then activate a specific distribution with | ||
+ | # conda activate version_name | ||
+ | # | ||
+ | # More details in: | ||
+ | # https://wiki.lsce.ipsl.fr/pmip3/doku.php/other:python:starting#conda-based_versions_of_uv-cdat | ||
+ | # https://wiki.lsce.ipsl.fr/pmip3/doku.php/other:uvcdat:conda_notes | ||
+ | # | ||
+ | # Jean-Yves Peterschmitt - LSCE - 11/2018 | ||
- | <code>bash-4.2$ conda env list | + | source /home/share/unix_files/cdat/miniconda3/etc/profile.d/conda.csh |
- | uvcdat-2.10 /home/share/unix_files/cdat/miniconda2b/envs/uvcdat-2.10 | + | |
- | root * /home/share/unix_files/cdat/miniconda2b | + | |
- | bash-4.2$ du -sh /home/share/unix_files/cdat/miniconda2b | + | # Use the alias below to easily determine where your python |
- | 4.3G /home/share/unix_files/cdat/miniconda2b | + | # interpreter is located |
+ | alias wp "which python" | ||
- | bash-4.2$ du -sh /home/share/unix_files/cdat/miniconda2b/* | + | # The end |
- | 4.4M /home/share/unix_files/cdat/miniconda2b/bin | + | |
- | 4.0K /home/share/unix_files/cdat/miniconda2b/conda-bld | + | |
- | 280K /home/share/unix_files/cdat/miniconda2b/conda-meta | + | |
- | 2.8G /home/share/unix_files/cdat/miniconda2b/envs | + | |
- | 20K /home/share/unix_files/cdat/miniconda2b/etc | + | |
- | 3.8M /home/share/unix_files/cdat/miniconda2b/include | + | |
- | 100M /home/share/unix_files/cdat/miniconda2b/lib | + | |
- | 8.0K /home/share/unix_files/cdat/miniconda2b/LICENSE.txt | + | |
- | 1.4G /home/share/unix_files/cdat/miniconda2b/pkgs | + | |
- | 16K /home/share/unix_files/cdat/miniconda2b/share | + | |
- | 376K /home/share/unix_files/cdat/miniconda2b/ssl | + | |
- | </code> | + | |
+ | >source ~jypeter/.conda3_jyp.csh | ||
- | ===== Cloning uvcdat to add specific packages for LSCE ===== | + | >conda activate |
- | <WRAP center round tip 60%> | + | (base) >which python |
- | This section is about the creation of the **cdatm15** environment | + | /home/share/unix_files/cdat/miniconda3/bin/python</code> |
- | Notes about [[https://wiki.lsce.ipsl.fr/pmip3/doku.php/other:python:starting#conda-based_versions_of_uv-cdat|actually using the cdatm15 conda-based python]] | + | You probably don't want to type the //source// line each time you need to use your conda based python, so you can add a ''source ~jypeter/.conda3_jyp.sh'' line in your ''~/.bashrc'' file, and ''source ~jypeter/.conda3_jyp.csh'' line in your ''~/.cshrc'' file. Then, when you need a specific python environment, just type ''conda activate name_of_the_specific_environment'' |
+ | |||
+ | === Old style initialization === | ||
+ | |||
+ | <WRAP center round tip 60%> | ||
+ | The //old style// notes below are just for reference sake and you can skip them | ||
</WRAP> | </WRAP> | ||
- | Note: using hard links, cloning a full environment //only// adds an extra 500M of disk space | + | Note: this is the **old** ''source activate some_version'' style |
- | <code>bash-4.1$ du -sh /home/share/unix_files/cdat/miniconda2 | + | See the ''~jypeter/.conda3old_jyp.sh'' file below, and how to use it, **in a bash shell** |
- | 9.0G /home/share/unix_files/cdat/miniconda2 | + | |
- | bash-4.1$ du -sh /home/share/unix_files/cdat/miniconda2/envs/* | + | <code>bash-4.2$ which python |
- | 2.8G /home/share/unix_files/cdat/miniconda2/envs/cdatm14 | + | /usr/bin/python |
- | 2.8G /home/share/unix_files/cdat/miniconda2/envs/uvcdat-2.10 | + | |
- | 2.0G /home/share/unix_files/cdat/miniconda2/envs/uvcdat-2.8.0 | + | |
- | bash-4.1$ conda create -n cdatm15 --clone uvcdat-2.10 | + | $ cat .conda3old_jyp.sh |
- | Source: /home/share/unix_files/cdat/miniconda2/envs/uvcdat-2.10 | + | # Conda initialization by JYP, OLD style |
- | Destination: /home/share/unix_files/cdat/miniconda2/envs/cdatm15 | + | # |
- | [...] | + | # Use this for working with conda and CDAT centrally managed by JYP |
+ | # | ||
+ | # Execute this file in a BASH shell with | ||
+ | # source path/this_file | ||
+ | # Then get the list of available python distributions with | ||
+ | # conda env list | ||
+ | # Then activate a specific distribution with | ||
+ | # source activate version_name | ||
+ | # | ||
+ | # More details in: | ||
+ | # https://wiki.lsce.ipsl.fr/pmip3/doku.php/other:python:starting#conda-based_versions_of_uv-cdat | ||
+ | # https://wiki.lsce.ipsl.fr/pmip3/doku.php/other:uvcdat:conda_notes | ||
+ | # | ||
+ | # Jean-Yves Peterschmitt - LSCE - 11/2018 | ||
- | bash-4.1$ du -sh /home/share/unix_files/cdat/miniconda2 | + | export PATH="/home/share/unix_files/cdat/miniconda3/bin:$PATH" |
- | 9.5G /home/share/unix_files/cdat/miniconda2 | + | |
- | bash-4.1$ du -sh /home/share/unix_files/cdat/miniconda2/envs/* | + | # Use the alias below to easily determine where your python |
- | 2.8G /home/share/unix_files/cdat/miniconda2/envs/cdatm14 | + | # interpreter is located |
- | 2.8G /home/share/unix_files/cdat/miniconda2/envs/cdatm15 | + | alias wp="which python" |
- | 573M /home/share/unix_files/cdat/miniconda2/envs/uvcdat-2.10 | + | |
- | 2.0G /home/share/unix_files/cdat/miniconda2/envs/uvcdat-2.8.0 | + | |
- | </code> | + | |
- | Disk space used after cloning, in <wrap em>in the new directory hierarchy</wrap> (''cdat/miniconda2b'', on the obelix servers) | + | # Where are ALL the python interpreters in the search path |
+ | alias wpa="which -a python" | ||
- | <code>bash-4.2$ du -sh /home/share/unix_files/cdat/miniconda2b | + | # The end |
- | 4.8G /home/share/unix_files/cdat/miniconda2b | + | |
- | bash-4.2$ du -sh /home/share/unix_files/cdat/miniconda2b/* | + | $ source ~jypeter/.conda3old_jyp.sh |
- | 4.4M /home/share/unix_files/cdat/miniconda2b/bin | + | |
- | 4.0K /home/share/unix_files/cdat/miniconda2b/conda-bld | + | |
- | 280K /home/share/unix_files/cdat/miniconda2b/conda-meta | + | |
- | 3.4G /home/share/unix_files/cdat/miniconda2b/envs | + | |
- | 20K /home/share/unix_files/cdat/miniconda2b/etc | + | |
- | 3.8M /home/share/unix_files/cdat/miniconda2b/include | + | |
- | 100M /home/share/unix_files/cdat/miniconda2b/lib | + | |
- | 8.0K /home/share/unix_files/cdat/miniconda2b/LICENSE.txt | + | |
- | 1.4G /home/share/unix_files/cdat/miniconda2b/pkgs | + | |
- | 16K /home/share/unix_files/cdat/miniconda2b/share | + | |
- | 376K /home/share/unix_files/cdat/miniconda2b/ssl | + | |
- | bash-4.2$ du -sh /home/share/unix_files/cdat/miniconda2b/envs/* | + | $ wp |
- | 2.8G /home/share/unix_files/cdat/miniconda2b/envs/cdatm15 | + | /home/share/unix_files/cdat/miniconda3/bin/python |
- | 574M /home/share/unix_files/cdat/miniconda2b/envs/uvcdat-2.10 | + | |
</code> | </code> | ||
- | ===== Getting ready for a moving default CDAT ===== | + | ==== Installing CDAT 8.0 ==== |
- | We create a **//cdatm// symbolic link** in the ''envs'' directory, that has a //stable name// but can be moved to point to the latest default CDAT. In that case, most users can just activate this //cdatm// version and always get the latest stable version | + | === Python 2.7 version === |
- | <code>bash-4.2$ cd /home/share/unix_files/cdat/miniconda2b/envs | + | <code>$ conda create -n cdat-8.0_py2 -c cdat/label/v80 -c conda-forge -c cdat python=2.7 cdat |
+ | # Generate the list of installed packages | ||
+ | $ conda list -n cdat-8.0_py2 > /home/scratch01/jypeter/cdat-8.0_py2_list_181108.txt</code> | ||
- | bash-4.2$ conda env list | + | List of installed packages: {{:other:uvcdat:cdat_conda:cdat-8.0_py2_list_181108.txt}} |
- | cdatm15 /home/share/unix_files/cdat/miniconda2b/envs/cdatm15 | + | |
- | uvcdat-2.10 /home/share/unix_files/cdat/miniconda2b/envs/uvcdat-2.10 | + | |
- | root * /home/share/unix_files/cdat/miniconda2b | + | |
- | bash-4.2$ ln -s cdatm15 cdatm | + | Disk space after 8.0 installation |
- | bash-4.2$ conda env list | + | <code>$ du -sh /home/share/unix_files/cdat/miniconda3 |
- | cdatm /home/share/unix_files/cdat/miniconda2b/envs/cdatm | + | 4.3G /home/share/unix_files/cdat/miniconda3 |
- | cdatm15 /home/share/unix_files/cdat/miniconda2b/envs/cdatm15 | + | $ du -sh /home/share/unix_files/cdat/miniconda3/* |
- | uvcdat-2.10 /home/share/unix_files/cdat/miniconda2b/envs/uvcdat-2.10 | + | 15M /home/share/unix_files/cdat/miniconda3/bin |
- | root * /home/share/unix_files/cdat/miniconda2b | + | 2.4M /home/share/unix_files/cdat/miniconda3/compiler_compat |
+ | 4.0K /home/share/unix_files/cdat/miniconda3/conda-bld | ||
+ | 3.7M /home/share/unix_files/cdat/miniconda3/conda-meta | ||
+ | 2.8G /home/share/unix_files/cdat/miniconda3/envs | ||
+ | 28K /home/share/unix_files/cdat/miniconda3/etc | ||
+ | 4.9M /home/share/unix_files/cdat/miniconda3/include | ||
+ | 156M /home/share/unix_files/cdat/miniconda3/lib | ||
+ | 8.0K /home/share/unix_files/cdat/miniconda3/LICENSE.txt | ||
+ | 1.4G /home/share/unix_files/cdat/miniconda3/pkgs | ||
+ | 1.3M /home/share/unix_files/cdat/miniconda3/share | ||
+ | 24K /home/share/unix_files/cdat/miniconda3/ssl | ||
+ | 12K /home/share/unix_files/cdat/miniconda3/x86_64-conda_cos6-linux-gnu | ||
+ | $ du -sh /home/share/unix_files/cdat/miniconda3/envs/* | ||
+ | 2.8G /home/share/unix_files/cdat/miniconda3/envs/cdat-8.0_py2</code> | ||
- | bash-4.2$ source activate cdatm | + | === Python 3.6 version === |
+ | |||
+ | <code>$ conda create -n cdat-8.0_py3 -c cdat/label/v80 -c conda-forge -c cdat python=3.6 cdat</code> | ||
+ | |||
+ | ==== Installing CDAT nightly ==== | ||
+ | |||
+ | Notes: | ||
+ | |||
+ | * This page is about CDAT 8.0, but we have added a short nightly section here as a convenient shortcut. This should probably be moved to a stand-alone nightly page later | ||
+ | * [[https://github.com/CDAT/cdat/wiki/Obtain-CDAT-nightly-packages|Official nightly page]] | ||
+ | |||
+ | === Python 2.7 version === | ||
+ | |||
+ | <code>$ conda create -n cdat-nightly_py2 -c cdat/label/nightly -c conda-forge -c cdat python=2.7 cdat</code> | ||
+ | |||
+ | <wrap hi>Not tested!</wrap> Can this be updated with ''conda update -n cdat-nightly_py2 -c cdat/label/nightly -c conda-forge -c cdat --all'' ??? | ||
+ | |||
+ | === Python 3.6 version === | ||
+ | |||
+ | <code>$ conda create -n cdat-nightly_py3 -c cdat/label/nightly -c conda-forge -c cdat python=3.6 cdat</code> | ||
+ | |||
+ | ===== Cloning cdat to add specific packages for LSCE ===== | ||
+ | |||
+ | <WRAP center round tip 60%> | ||
+ | This section is about the creation of the **cdatm17** environment | ||
+ | |||
+ | Notes about [[https://wiki.lsce.ipsl.fr/pmip3/doku.php/other:python:starting#conda-based_versions_of_uv-cdat|actually using the cdatm17 conda-based python]] | ||
+ | </WRAP> | ||
+ | |||
+ | |||
+ | Note: using hard links, cloning a full environment //only// adds an extra 582M of disk space | ||
+ | |||
+ | <code>$ conda create -n cdatm17_py2 --clone cdat-8.0_py2 | ||
+ | Source: /home/share/unix_files/cdat/miniconda3/envs/cdat-8.0_py2 | ||
+ | Destination: /home/share/unix_files/cdat/miniconda3/envs/cdatm17_py2 | ||
+ | Packages: 226 | ||
+ | Files: 3 | ||
+ | |||
+ | [...] | ||
+ | |||
+ | $ du -sh /home/share/unix_files/cdat/miniconda3/envs/* | ||
+ | 2.5G /home/share/unix_files/cdat/miniconda3/envs/cdat-8.0_py2 | ||
+ | 392M /home/share/unix_files/cdat/miniconda3/envs/cdatm17_py2 | ||
+ | 839M /home/share/unix_files/cdat/miniconda3/envs/cdat-nightly_py2 | ||
- | (cdatm) bash-4.2$ conda env list | ||
- | cdatm * /home/share/unix_files/cdat/miniconda2b/envs/cdatm | ||
- | cdatm15 /home/share/unix_files/cdat/miniconda2b/envs/cdatm15 | ||
- | uvcdat-2.10 /home/share/unix_files/cdat/miniconda2b/envs/uvcdat-2.10 | ||
- | root /home/share/unix_files/cdat/miniconda2b | ||
</code> | </code> | ||
+ | |||
+ | ==== cdat nightly case ==== | ||
+ | |||
+ | ''conda create -n cdatm-nightly_py2 %%--%%clone cdat-nightly_py2'' | ||
+ | |||
+ | ===== Getting ready for a moving default CDAT ===== | ||
+ | |||
+ | <note warning>This step should probably be listed at the **end**, especially in a multi-user environment! If there is already a ''cdatm'' link, make sure that the new version is stable and working correctly before updating the ''cdatm'' link</note> | ||
+ | |||
+ | We create a **//cdatm// symbolic link** in the ''envs'' directory, that has a //stable name// but can be moved to point to the latest default CDAT. In that case, most users can just activate this //cdatm// version and always get the latest stable version | ||
+ | |||
+ | <code>$ cd /home/share/unix_files/cdat/miniconda3/envs | ||
+ | $ conda env list | ||
+ | base * /home/share/unix_files/cdat/miniconda3 | ||
+ | cdat-8.0_py2 /home/share/unix_files/cdat/miniconda3/envs/cdat-8.0_py2 | ||
+ | cdatm17_py2 /home/share/unix_files/cdat/miniconda3/envs/cdatm17_py2 | ||
+ | $ ln -s cdatm17_py2 cdatm | ||
+ | $ conda env list | ||
+ | base * /home/share/unix_files/cdat/miniconda3 | ||
+ | cdat-8.0_py2 /home/share/unix_files/cdat/miniconda3/envs/cdat-8.0_py2 | ||
+ | cdatm /home/share/unix_files/cdat/miniconda3/envs/cdatm | ||
+ | cdatm17_py2 /home/share/unix_files/cdat/miniconda3/envs/cdatm17_py2 | ||
+ | $ source activate cdatm | ||
+ | (cdatm) bash-4.2$ conda env list | ||
+ | base /home/share/unix_files/cdat/miniconda3 | ||
+ | cdat-8.0_py2 /home/share/unix_files/cdat/miniconda3/envs/cdat-8.0_py2 | ||
+ | cdatm * /home/share/unix_files/cdat/miniconda3/envs/cdatm | ||
+ | cdatm17_py2 /home/share/unix_files/cdat/miniconda3/envs/cdatm17_py2</code> | ||
===== Customizing UV-CDAT for LSCE ===== | ===== Customizing UV-CDAT for LSCE ===== | ||
Line 263: | Line 328: | ||
==== Downloading cdms2/vcs test data ==== | ==== Downloading cdms2/vcs test data ==== | ||
- | You should download the test data and use it in the example scripts that you want to distribute, and scripts you write for reporting the errors you find (if any...) | + | You should download the test data (174M of data...) and use it in the example scripts that you want to distribute, and scripts you write for reporting the errors you find (if any...) |
- | <code>bash-4.1$ source activate cdatm15 | + | <code>$ source activate cdatm17_py2 |
- | (cdatm15) bash-4.1$ python -c 'import vcs; vcs.download_sample_data_files(); print "\nFinished downloading sample data to", vcs.sample_data' | + | (cdatm17_py2) $ python -c 'import vcs; vcs.download_sample_data_files(); print "\nFinished downloading sample data to", vcs.sample_data' |
[...] | [...] | ||
- | Finished downloading sample data to /home/share/unix_files/cdat/miniconda2/envs/cdatm15/share/uvcdat/sample_data | + | Finished downloading sample data to /home/share/unix_files/cdat/miniconda3/envs/cdatm17_py2/share/uvcdat/sample_data |
- | (cdatm15) bash-4.1$ du -sh /home/share/unix_files/cdat/miniconda2/envs/cdatm15/share/uvcdat/sample_data | + | (cdatm17_py2) $ du -sh /home/share/unix_files/cdat/miniconda3/envs/cdatm17_py2/share/uvcdat/sample_data |
- | 174M /home/share/unix_files/cdat/miniconda2/envs/cdatm15/share/uvcdat/sample_data</code> | + | 174M /home/share/unix_files/cdat/miniconda3/envs/cdatm17_py2/share/uvcdat/sample_data |
+ | </code> | ||
Line 278: | Line 344: | ||
After cloning, we are ready to install some extra packages that may be requested by LSCE users | After cloning, we are ready to install some extra packages that may be requested by LSCE users | ||
+ | |||
+ | * We first try to install together as many packages as possible that don't require other channels than //conda-forge//, and that don't request a downgrade of what is already installed | ||
+ | * We then install individual extra packages with ''conda install'' or ''pip install'' | ||
<code># Keep a trace of what will be installed | <code># Keep a trace of what will be installed | ||
- | bash-4.1$ conda install --dry-run -n cdatm15 -c conda-forge -c uvcdat -c pcmdi pillow pandas statsmodels seaborn scikit-image seawater gsw netcdf4 pyferret basemap-data-hires pcmdi_metrics xlsxwriter cmor shapely cartopy rpy2 > /home/scratch01/jypeter/lsce-extra_01_install_170808.txt | + | $ conda install --dry-run -n cdatm17_py2 -c conda-forge pillow pandas statsmodels seaborn scikit-image seawater gsw netcdf4 pyferret basemap-data-hires xlsxwriter cmocean rpy2 gdal windspharm > /home/scratch01/jypeter/lsce-extra_01_install_190304.txt |
# Install... | # Install... | ||
- | bash-4.1$ conda install -n cdatm15 -c conda-forge -c uvcdat -c pcmdi pillow pandas statsmodels seaborn scikit-image seawater gsw netcdf4 pyferret basemap-data-hires pcmdi_metrics xlsxwriter cmor shapely cartopy rpy2 | + | $ conda install -n cdatm17_py2 -c conda-forge pillow pandas statsmodels seaborn scikit-image seawater gsw netcdf4 pyferret basemap-data-hires xlsxwriter cmocean rpy2 gdal windspharm |
[...] | [...] | ||
+ | |||
+ | # Install extra packages in the nightly version (if you need 'nightly') | ||
+ | $ conda install -n cdatm-nightly_py2 -c conda-forge pillow pandas statsmodels seaborn scikit-image seawater gsw netcdf4 pyferret basemap-data-hires xlsxwriter cmocean rpy2 gdal windspharm | ||
# Check the disk space after installation | # Check the disk space after installation | ||
- | bash-4.1$ du -sh /home/share/unix_files/cdat/miniconda2 | + | $ du -sh /home/share/unix_files/cdat/miniconda3 |
- | 11G /home/share/unix_files/cdat/miniconda2 | + | 7.5G /home/share/unix_files/cdat/miniconda3 |
- | bash-4.1$ du -sh /home/share/unix_files/cdat/miniconda2/envs/* | + | $ du -sh /home/share/unix_files/cdat/miniconda3/envs/* |
- | 0 /home/share/unix_files/cdat/miniconda2/envs/cdatm | + | 2.5G /home/share/unix_files/cdat/miniconda3/envs/cdat-8.0_py2 |
- | 2.8G /home/share/unix_files/cdat/miniconda2/envs/cdatm14 | + | 1.2G /home/share/unix_files/cdat/miniconda3/envs/cdatm17_py2 |
- | 3.7G /home/share/unix_files/cdat/miniconda2/envs/cdatm15 | + | 938M /home/share/unix_files/cdat/miniconda3/envs/cdatm-nightly_py2 |
- | 574M /home/share/unix_files/cdat/miniconda2/envs/uvcdat-2.10 | + | 450M /home/share/unix_files/cdat/miniconda3/envs/cdat-nightly_py2</code> |
- | 2.0G /home/share/unix_files/cdat/miniconda2/envs/uvcdat-2.8.0 | + | |
- | # Check the disk space again after cleaning | + | List of installed packages: {{ :other:uvcdat:cdat_conda:lsce-extra_01_install_190304.txt |}} |
- | bash-4.1$ conda clean --all | + | |
- | + | ||
- | bash-4.1$ du -sh /home/share/unix_files/cdat/miniconda2 | + | |
- | 9.9G /home/share/unix_files/cdat/miniconda2</code> | + | |
- | + | ||
- | List of installed packages: {{ :other:uvcdat:cdat_conda:lsce-extra_01_install_170808.txt |}} | + | |
- | + | ||
- | Disk space used after adding the extra packages, in <wrap em>in the new directory hierarchy</wrap> (''cdat/miniconda2b'', on the obelix servers) | + | |
- | + | ||
- | <code>bash-4.2$ du -sh /home/share/unix_files/cdat/miniconda2b | + | |
- | 6.4G /home/share/unix_files/cdat/miniconda2b | + | |
- | + | ||
- | bash-4.2$ du -sh /home/share/unix_files/cdat/miniconda2b/envs/* | + | |
- | 0 /home/share/unix_files/cdat/miniconda2b/envs/cdatm | + | |
- | 3.8G /home/share/unix_files/cdat/miniconda2b/envs/cdatm15 | + | |
- | 630M /home/share/unix_files/cdat/miniconda2b/envs/uvcdat-2.10 | + | |
- | + | ||
- | # Check the disk space again after cleaning | + | |
- | bash-4.2$ conda clean --all | + | |
- | + | ||
- | bash-4.2$ du -sh /home/share/unix_files/cdat/miniconda2b | + | |
- | 5.4G /home/share/unix_files/cdat/miniconda2b</code> | + | |
Packages installed with //pip// | Packages installed with //pip// | ||
Line 326: | Line 375: | ||
* Update with: ''pip install %%--%%upgrade dreqPy'' | * Update with: ''pip install %%--%%upgrade dreqPy'' | ||
* Get version number with:\\ <code>$ drq -v | * Get version number with:\\ <code>$ drq -v | ||
- | dreqPy version 01.00.06 [Version 01.00.06]</code> | + | dreqPy version 01.00.29 [Version 01.00.29]</code> |
- | * [[https://github.com/pierre-rouanet/dtw|dtw]]: DTW (Dynamic Time Warping) python module | + | |
- | * ''pip install dtw'' | + | |
- | * Requested by Sentia Goursaud | + | |
The following packages have no dependency problems and were installed (or updated) //later// | The following packages have no dependency problems and were installed (or updated) //later// | ||
- | * [[http://www.earthsystemmodeling.org/esmf_releases/non_public/ESMF_7_0_0/esmpy_doc/html/index.html|ESMPy]]: ESMF Python Regridding Interface | ||
- | * We used the following to get a more up-to-date version of esmpy than the one bundled with 2.10 | ||
- | * <code>bash-4.1$ conda list | grep -i esm | ||
- | esmf 7.0.0 6 conda-forge | ||
- | esmpy 7.0.0 py27_1 conda-forge | ||
- | bash-4.1$ conda install -n cdatm15 -c nesii/channel/dev-esmf -c conda-forge esmpy=7.1.0.dev32 | + | * [[http://www.ifremer.fr/vacumm/|vacumm]]: Validation, Analysis, Comparison - Utilities written in Python to validate and analyze Multi-Model outputs, and compare them to observations |
- | [...] | + | * ''conda install -n cdatm17_py2 -c conda-forge -c vacumm vacumm'' |
- | bash-4.1$ conda list | grep -i esm | ||
- | esmf 7.1.0.dev32 1 nesii/channel/dev-esmf | ||
- | esmpy 7.1.0.dev32 py27_1 nesii/channel/dev-esmf</code> | ||
- | * [[http://matplotlib.org/cmocean/|cmocean]]: beautiful colormaps for oceanography | ||
- | * ''conda install -n cdatm15 -c conda-forge cmocean'' | ||
- | * [[http://scitools.org.uk/iris/index.html|iris]]: A Python library for Meteorology and Climatology | ||
- | * ''conda install -n cdatm15 -c conda-forge -c scitools iris'' | ||
- | * [[http://www.gdal.org/|OSGeo/GDAL]]: Geospatial Data Abstraction Library. GDAL is a translator library for raster and vector geospatial data formats | ||
- | * ''conda install -c conda-forge gdal'' | ||
- | * [[https://pcjericks.github.io/py-gdalogr-cookbook/|Python GDAL/OGR Cookbook]] | ||
* [[https://github.com/stefraynaud/spanlib|spanlib]]: Spectral Analysis Library | * [[https://github.com/stefraynaud/spanlib|spanlib]]: Spectral Analysis Library | ||
- | * ''conda install -n cdatm15 -c stefraynaud -c conda-forge spanlib'' | + | * ''conda install -n cdatm17_py2 -c stefraynaud -c conda-forge spanlib'' |
* Test: ''python -c 'from spanlib.analyzer import Analyzer' '' | * Test: ''python -c 'from spanlib.analyzer import Analyzer' '' | ||
- | * [[http://wrf-python.readthedocs.io/en/latest/|wrf-python]]: A collection of diagnostic and interpolation routines for use with output from the Weather Research and Forecasting (WRF-ARW) Model | + | |
- | * ''conda install -n cdatm15 -c conda-forge wrf-python'' | + | * [[https://climaf.readthedocs.io/en/latest/|CliMAF]]: a Climate Model Assessment Framework |
- | * [[https://glances.readthedocs.io/en/stable/index.html|glances]]: a cross-platform monitoring tool (similar to ''top'') | + | * [[other:uvcdat:cdat_conda:climaf_install|Installation notes]] |
- | * ''conda install -n cdatm15 -c conda-forge glances'' | + | |
- | * [[https://github.com/nicolargo/glances|glances@github]] | + | |
- | * [[https://ajdawson.github.io/windspharm/latest/|windspharm]]: spherical harmonic wind analysis in Python | + | |
- | * ''conda install -n cdatm15 -c conda-forge windspharm'' | + | |
- | * [[https://github.com/ajdawson/windspharm|windspharm@github]] | + | |
==== TODO ==== | ==== TODO ==== | ||
- | //Add here packages that would be useful but have some problems that prevent their installation// | + | //Add here packages that would be useful and have not been installed yet, or have some problems that prevent their installation// |
- | * [[http://www.ifremer.fr/vacumm/|vacumm]]: Validation, Analysis, Comparison - Utilities written in Python to validate and analyze Multi-Model outputs, and compare them to observations | + | * [[http://scitools.org.uk/iris/index.html|iris]]: A Python library for Meteorology and Climatology |
- | * As of August 08 2017, installing vacumm in 2.10 requires downgrading to 2.10 (see [[https://github.com/VACUMM/vacumm/issues/9|github issue]])! | + | * ''conda install -n cdatxxx -c conda-forge iris'' |
- | * <wrap hi>Problem solved</wrap> with **vacumm 3.4.1-1** on August 31st! Installation was OK with\\ ''conda install -n cdatm15 -c conda-forge -c vacumm vacumm'' | + | |
- | * Install PySPLIT for Sentia\\ [[https://github.com/mscross/pysplit|PySPLIT]] A package for generating HYSPLIT air parcel trajectories trajectories, performing moisture uptake analyses, expediting HYSPLIT cluster analysis, and for visualizing trajectories, clusters, and along-trajectory meteorological data\\ Works with [[https://ready.arl.noaa.gov/HYSPLIT.php|HYSPLiT]] | + | * [[http://wrf-python.readthedocs.io/en/latest/|wrf-python]]: A collection of diagnostic and interpolation routines for use with output from the Weather Research and Forecasting (WRF-ARW) Model |
- | - ''conda install -n cdatm15 -c conda-forge fiona geopandas'' | + | * ''conda install -n cdatxxx -c conda-forge wrf-python'' |
- | - ''pip install pysplit'' | + | |
+ | * [[https://glances.readthedocs.io/en/stable/index.html|glances]]: a cross-platform monitoring tool (similar to ''top'') | ||
+ | * ''conda install -n cdatxxx -c conda-forge glances'' | ||
+ | * [[https://github.com/nicolargo/glances|glances@github]] | ||
==== Other packages ==== | ==== Other packages ==== | ||
Line 387: | Line 414: | ||
* [[https://cmor.llnl.gov/|CMOR]] | * [[https://cmor.llnl.gov/|CMOR]] | ||
- | * Update with: ''conda update -n cdatm15 -c conda-forge -c pcmdi -c uvcdat cmor'' | + | * Update with: ''conda update -n cdatxxx -c conda-forge -c pcmdi -c uvcdat cmor'' |
* Get version number with: ''python -c 'from cmor import *; print CMOR_VERSION_MAJOR, CMOR_VERSION_MINOR, CMOR_VERSION_PATCH' '' | * Get version number with: ''python -c 'from cmor import *; print CMOR_VERSION_MAJOR, CMOR_VERSION_MINOR, CMOR_VERSION_PATCH' '' | ||
Line 393: | Line 420: | ||
* Update with: ''pip install %%--%%upgrade dreqPy'' | * Update with: ''pip install %%--%%upgrade dreqPy'' | ||
* Get version number with: ''drq -v'' | * Get version number with: ''drq -v'' | ||
+ | |||
+ | ==== Installing CLIMAF ==== | ||
+ | |||
+ | The [[https://climaf.readthedocs.io/|CLIMAF]] installation is a special case, because we may want to use it with the Python installed for CDAT, but it is not installed using conda | ||
+ | |||
+ | We follow the steps for the [[https://climaf.readthedocs.io/en/latest/installing.html#installing-climaf-if-you-are-not-at-cnrm-on-ciclad-or-climserv|installation of a non-standard server]] | ||
+ | |||
+ | * Go to the appropriate installation directory (we choose the directory where we have also installed miniconda3): | ||
+ | * LSCE: ''cd /home/share/unix_files/cdat'' | ||
+ | * ciclad: ''/data/jypmce/cdat/'' | ||
+ | * ''git clone https://github.com/senesis/climaf climaf'' | ||
+ | * Add CLIMAF to the PYTHONPATH environment variable | ||
+ | * LSCE & tcsh: ''setenv PYTHONPATH ${PYTHONPATH}:/home/share/unix_files/cdat/climaf'' | ||
+ | * LSCE & bash: ''export PYTHONPATH=${PYTHONPATH}:/home/share/unix_files/cdat/climaf'' | ||
+ | * ciclad: same as above, using ''/data/jypmce/cdat/climaf'' | ||
+ | * Test the installation | ||
+ | * make sure the [[https://climaf.readthedocs.io/en/latest/requirements.html#requirements|Requirements]] are available: | ||
+ | * LSCE: ''module load netcdf/4 nco cdo ncl ncview'' | ||
+ | * ciclad: ''module load netcdf4 nco cdo ncl/6.3.0'' | ||
+ | * optional: start an X server | ||
+ | * <code>cd climaf/testing | ||
+ | ./test_install.sh</code> | ||
+ | * Once the tests are OK, add the required //module load// and environment variables definitions to the ''.basrc'' and ''.cshrc'' files. <wrap hi>Note: maybe these lines should go to the ''.profile'' and ''.login'' files...(check [[https://www.gnu.org/software/bash/manual/html_node/Bash-Startup-Files.html|Bash Startup Files]])</wrap> | ||
+ | * LSCE: add to your .cshrc | ||
+ | * <code># Prerequisites for using CLIMAF | ||
+ | module load netcdf/4 nco cdo ncl ncview | ||
+ | |||
+ | # CLIMAF environment variables | ||
+ | setenv CLIMAF /home/share/unix_files/cdat/climaf | ||
+ | setenv PYTHONPATH "${PYTHONPATH}:${CLIMAF}" | ||
+ | #setenv PYTHONPATH $CLIMAF | ||
+ | setenv PATH "${PATH}:${CLIMAF}/bin" | ||
+ | setenv CLIMAF_CACHE "/home/scratch01/${USER}/climaf_cache" | ||
+ | setenv TMPDIR $CLIMAF_CACHE | ||
+ | setenv CLIMAF_LOG_DIR $CLIMAF_CACHE | ||
+ | |||
+ | #conda activate cdatm-nightly_py2</code> | ||
+ | * ciclad: add to your .bashrc | ||
+ | * <code># Prerequisites for using CLIMAF when not using the | ||
+ | # default 'module load climaf' | ||
+ | module load netcdf4 nco cdo ncl/6.3.0 | ||
+ | |||
+ | # CLIMAF environment variables | ||
+ | export CLIMAF=/data/jypmce/cdat/climaf | ||
+ | #export PYTHONPATH=$PYTHONPATH:$CLIMAF | ||
+ | export PYTHONPATH=$CLIMAF | ||
+ | export PATH=$PATH:$CLIMAF/bin | ||
+ | export CLIMAF_CACHE=/data/jypmce/climaf_cache | ||
+ | export TMPDIR=$CLIMAF_CACHE | ||
+ | export CLIMAF_LOG_DIR=$CLIMAF_CACHE | ||
+ | |||
+ | #conda activate cdatm-nightly_py2 | ||
+ | </code> | ||
+ | * Open a new shell and <wrap em>run the tests again</wrap>. If there is an error that was not there, check what you have added to the shell init files | ||
+ | * [[other:uvcdat:cdat_conda:climaf_test|Test CLIMAF on ciclad]] | ||
==== Cleaning up things ==== | ==== Cleaning up things ==== | ||
Line 399: | Line 481: | ||
We check if some of the installed files are missing read access for the //group// or //other//, and we manually change the permissions | We check if some of the installed files are missing read access for the //group// or //other//, and we manually change the permissions | ||
- | <code>(cdatm15) bash-4.2$ find cdatm15 \! -perm /g+r,o+r -ls | + | <code> >find /home/share/unix_files/cdat/miniconda3/envs \! -perm /g+r,o+r -ls |
- | 50333969 4 -rw------- 2 jypeter lsce 234 Apr 4 20:35 cdatm15/lib/python2.7/site-packages/cdp-1.0.3-py2.7.egg-info/PKG-INFO | + | # Everything OK!</code> |
- | 50333971 4 -rw------- 2 jypeter lsce 291 Apr 4 20:35 cdatm15/lib/python2.7/site-packages/cdp-1.0.3-py2.7.egg-info/SOURCES.txt | + | |
- | 50333962 4 -rw------- 2 jypeter lsce 4 Apr 4 20:35 cdatm15/lib/python2.7/site-packages/cdp-1.0.3-py2.7.egg-info/top_level.txt | + | |
- | 50333961 4 -rw------- 2 jypeter lsce 1 Apr 4 20:35 cdatm15/lib/python2.7/site-packages/cdp-1.0.3-py2.7.egg-info/dependency_links.txt | + | |
- | 50333960 4 -rw------- 2 jypeter lsce 1 Apr 4 20:35 cdatm15/lib/python2.7/site-packages/cdp-1.0.3-py2.7.egg-info/not-zip-safe | + | |
- | 50333987 8 -rw------- 2 jypeter lsce 5149 Apr 4 20:35 cdatm15/lib/python2.7/site-packages/cdp/cdp_viewer.pyc | + | |
- | 50333986 8 -rw------- 2 jypeter lsce 4208 Apr 4 20:35 cdatm15/lib/python2.7/site-packages/cdp/cdp_metric.py | + | |
- | 50333966 4 -rw------- 2 jypeter lsce 150 Apr 4 20:35 cdatm15/lib/python2.7/site-packages/cdp/__init__.pyc | + | |
- | 50333968 4 -rw------- 2 jypeter lsce 201 Apr 4 20:35 cdatm15/lib/python2.7/site-packages/cdp/cdp_provenance.py | + | |
- | 50333988 8 -rw------- 2 jypeter lsce 5365 Apr 4 20:35 cdatm15/lib/python2.7/site-packages/cdp/cdp_metric.pyc | + | |
- | 50333982 4 -rw------- 2 jypeter lsce 1821 Apr 4 20:35 cdatm15/lib/python2.7/site-packages/cdp/cdp_parser.py | + | |
- | 50333981 4 -rw------- 2 jypeter lsce 1773 Apr 4 20:35 cdatm15/lib/python2.7/site-packages/cdp/cdp_parameter.py | + | |
- | 50333973 4 -rw------- 2 jypeter lsce 670 Apr 4 20:35 cdatm15/lib/python2.7/site-packages/cdp/cdp_output.py | + | |
- | 50333980 4 -rw------- 2 jypeter lsce 1642 Apr 4 20:35 cdatm15/lib/python2.7/site-packages/cdp/cdp_output.pyc | + | |
- | 50333979 4 -rw------- 2 jypeter lsce 1474 Apr 4 20:35 cdatm15/lib/python2.7/site-packages/cdp/_cache.pyc | + | |
- | 50333967 4 -rw------- 2 jypeter lsce 187 Apr 4 20:35 cdatm15/lib/python2.7/site-packages/cdp/cdp_tool.py | + | |
- | 50333989 0 -rw------- 2 jypeter lsce 0 Apr 4 20:35 cdatm15/lib/python2.7/site-packages/cdp/__init__.py | + | |
- | 50333985 4 -rw------- 2 jypeter lsce 3843 Apr 4 20:35 cdatm15/lib/python2.7/site-packages/cdp/cdp_viewer.py | + | |
- | 50333984 4 -rw------- 2 jypeter lsce 2436 Apr 4 20:35 cdatm15/lib/python2.7/site-packages/cdp/cdp_parameter.pyc | + | |
- | 50333970 4 -rw------- 2 jypeter lsce 288 Apr 4 20:35 cdatm15/lib/python2.7/site-packages/cdp/cdp_io.py | + | |
- | 50333974 4 -rw------- 2 jypeter lsce 692 Apr 4 20:35 cdatm15/lib/python2.7/site-packages/cdp/cdp_tool.pyc | + | |
- | 50333978 4 -rw------- 2 jypeter lsce 878 Apr 4 20:35 cdatm15/lib/python2.7/site-packages/cdp/cdp_io.pyc | + | |
- | 50333976 4 -rw------- 2 jypeter lsce 734 Apr 4 20:35 cdatm15/lib/python2.7/site-packages/cdp/_cache.py | + | |
- | 50333983 4 -rw------- 2 jypeter lsce 2269 Apr 4 20:35 cdatm15/lib/python2.7/site-packages/cdp/cdp_parser.pyc | + | |
- | 50333975 4 -rw------- 2 jypeter lsce 724 Apr 4 20:35 cdatm15/lib/python2.7/site-packages/cdp/cdp_provenance.pyc | + | |
- | (cdatm15) bash-4.2$ chmod -R a+r cdatm15/lib/python2.7/site-packages/cdp* | + | |
- | (cdatm15) bash-4.2$ find cdatm15 \! -perm /g+r,o+r -ls</code> | + | |
Line 452: | Line 508: | ||
* [[https://rpy2.readthedocs.io/en/version_2.8.x/|rpy2]]: providing simple and robust access to R from within Python | * [[https://rpy2.readthedocs.io/en/version_2.8.x/|rpy2]]: providing simple and robust access to R from within Python | ||
* [[http://matplotlib.org/cmocean/|cmocean]]: beautiful colormaps for oceanography | * [[http://matplotlib.org/cmocean/|cmocean]]: beautiful colormaps for oceanography | ||
- | * [[http://scitools.org.uk/iris/index.html|iris]]: A Python library for Meteorology and Climatology | ||
* [[http://www.gdal.org/|OSGeo/GDAL]]: Geospatial Data Abstraction Library. GDAL is a translator library for raster and vector geospatial data formats | * [[http://www.gdal.org/|OSGeo/GDAL]]: Geospatial Data Abstraction Library. GDAL is a translator library for raster and vector geospatial data formats | ||
+ | * [[https://pcjericks.github.io/py-gdalogr-cookbook/|Python GDAL/OGR Cookbook]] | ||
* [[https://github.com/stefraynaud/spanlib|spanlib]]: Spectral Analysis Library | * [[https://github.com/stefraynaud/spanlib|spanlib]]: Spectral Analysis Library | ||
- | * [[http://wrf-python.readthedocs.io/en/latest/|wrf-python]]: A collection of diagnostic and interpolation routines for use with output from the Weather Research and Forecasting (WRF-ARW) Model | ||
- | * [[https://glances.readthedocs.io/en/stable/index.html|glances]]: a cross-platform monitoring tool (similar to ''top'') | ||
* [[https://ajdawson.github.io/windspharm/latest/|windspharm]]: spherical harmonic wind analysis in Python | * [[https://ajdawson.github.io/windspharm/latest/|windspharm]]: spherical harmonic wind analysis in Python | ||
Line 468: | Line 522: | ||
^ Environment\\ name ^ Server ^ ''conda list'' ^ | ^ Environment\\ name ^ Server ^ ''conda list'' ^ | ||
- | | uvcdat-2.10 | LSCE | {{ |package list}} | | + | | cdat-8.0_py2 | LSCE | | |
- | | cdatm15 | LSCE | {{ |package list}} | | + | | cdatm17_py2 | LSCE | | |
+ | | cdat-nightly_py2 | LSCE\\ ciclad | | | ||
+ | | cdatm-nightly_py2 | LSCE\\ ciclad | | | ||
/* standard page footer */ | /* standard page footer */ |