====== Windows Subsystem for Linux (WSL), and Ubuntu ====== [[other:index#windows_10_notes|Other Windows 10 related notes]] What is described on this page was tested with **WSL v1**, and not WSL 2 [[https://docs.microsoft.com/en-us/windows/wsl/wsl2-index|WSL 2]] is available in ''Windows 10 v2004'' (April 2020) and more recent, and should also be available in [[https://devblogs.microsoft.com/commandline/wsl-2-support-is-coming-to-windows-10-versions-1903-and-1909/|1903 and 1909 versions]] The user can choose to install Linux using WSL1 or WSL2. Check the [[https://docs.microsoft.com/en-us/windows/wsl/compare-versions|differences between WSL1 and WSL2]] before choosing ===== What is WSL ? ===== Useful reading: * Have a look at the beginning of the [[https://docs.microsoft.com/en-us/windows/wsl/about|Windows Subsystem for Linux Documentation]] for a quick **WSL** description on the Microsoft site * Read [[https://wiki.ubuntu.com/WSL|What is WSL]] on the Ubuntu site * There is more useful information on the //WSL@Ubuntu page//, you can read the rest later In a few words, WSL is not Linux, it's more a way to [[https://blogs.msdn.microsoft.com/wsl/2016/06/15/wsl-file-system-support/|share files between Windows and Linux]] (it's OK to skip this rather technical link!). [[https://docs.microsoft.com/en-us/windows/wsl/faq#are-files-in-the-linux-drive-different-from-the-mounted-windows-drive|Files under the Linux root (i.e. /) are controlled by WSL which mimics Linux specific behavior [...follow the link for details...]]]. That's why you need //to activate WSL **and then install a Linux distribution**//. ===== Prerequisites ===== * A computer running at least //Windows 10 Anniversary Update//, that is with a version number of ''1607'' (July 2016) or higher * [[other:win10maintain#checking_the_installed_version|Check the installed version on your computer]] * [[other:win10misc#windows_10_versions|Check the available versions]] * [[other:win10maintain#updating_windows_10|See if you can update your Windows]] if your installed version is not the most receent available version * Note: there is sometimes a delay between the official release and the time when Windows Update will actually tell you that a major update is available for **your** computer (e.g. the //September YY09// update may become available for you only in December) * //Enough// space on ''C:\''. Microsoft instructions say //The Windows Subsystem for Linux only runs on your system drive (usually this is your C: drive)// * Exporting a //simple// Ubuntu configuration (after doing everything from //Base installation// to //Advanced configuration// below) with the [[other:win10wsl#creating_a_backup_of_the_linux_running_in_wsl|wsl --export]] command generated a 2.7 Gb tar file (uncompressed), so it's probably good to **start with at least 3 Gb of free space on ''C:\''** (and you always need to keep enough free space on ''C:\'' for regular Windows operation) * The Linux ''/'' and ''/home'' partitions will always be on ''C:'', but the data files can be on any Windows disk/partition (or external drive) of the computer * the Windows partitions will be available and accessible in ''/mnt/'', so you can store data on other partitions (e.g. the content of the Windows ''D:\'' partition will be available in ''/mnt/d'' in Linux) * The files will be installed in **hidden partitions of the current user's directory on ''C:\''**: e.g. ''C:\Users\''. Other users of the same Windows 10 computer would have to install Ubuntu again, in their own user directory * Checking the properties of ''C:\Users\'' will **not** report an accurate size and number of files, because lots of files and directories are hidden! On the test laptop, the reported size of the user backed up directory (with [[other:win10apps#syncbackfree|SyncBackFree]]) was ~20 Gb (user data, WSL+Ubuntu, custom install of the CDAT 8.1 python distribution) * Be sure to monitor the remaining free space on ''C:''! * Note: it seems that [[other:win10wsl#restoring_a_wsl_backup|wsl --import]] can restore a backed-up Linux to a user specified location. Maybe this can be used for moving Ubuntu... * Check [[https://github.com/MicrosoftDocs/WSL/issues/412|how to move the vhdx of wsl2 to other disk]] and [[https://github.com/microsoft/WSL/issues/4320|[WSL 2] move VHD File]] for relocating a WSL (1 or 2) installation if you don't have enough space on ''C:'' ===== Base installation ===== * [[https://support.microsoft.com/en-us/help/4027538/windows-create-a-system-restore-point|Create a system restore point]] ([[https://support.microsoft.com/fr-fr/help/4027538/windows-create-a-system-restore-point|Créer un point de restauration système]]), and give it a meaningful name (e.g. //Before installing WSL//), in the unlikely case you have to restore it later * Follow **closely** the steps listed in [[https://docs.microsoft.com/en-us/windows/wsl/install-win10|WSL Windows 10 Installation Guide]]. Open this [[https://docs.microsoft.com/en-us/windows/wsl/install-win10|WSL Windows 10 Installation Guide]] in another window, so that you can **read the useful hints below at the same time!** * When you have to //Open PowerShell as Administrator//, go to ''Start''=>''W''=>''Windows PowerShell'' then right-clik on ''Windows PowerShell'' and select //Run as Administrator// * FIXME At this stage, we could try to restore/clone a previously installed/customized/backed-up version of Ubuntu if we have one (see [[other:win10wsl#restoring_a_wsl_backup|Restoring a WSL backup]]) * In the //Install your Linux Distribution of Choice//, we will use the very easy //install from the Microsoft Store// option * Open the **Microsoft Store** with: ''Start''=>''Microsoft Store'' * In the **Microsoft Store**, just search //Linux// and select the //generic// **Ubuntu** (select the //Ubuntu// that has **no version number after the name**) * Select ''Get'' (''Télécharger'') to download the installer package. You can ignore (''No thanks'') the popups asking you to use a Microsoft account, if you are not using a Microsoft account yet on this machine * Select ''Launch'' (''Lancer'') to start the installation, and give a user name (e.g. your LSCE login) and password (something that you can remember, but maybe not the same password as your LSCE account) * Type some basic Linux commands to check that things are working, e.g. check the available disks\\ $ df -h Filesystem Size Used Avail Use% Mounted on rootfs 238G 58G 181G 25% / none 238G 58G 181G 25% /dev none 238G 58G 181G 25% /run none 238G 58G 181G 25% /run/lock none 238G 58G 181G 25% /run/shm none 238G 58G 181G 25% /run/user cgroup 238G 58G 181G 25% /sys/fs/cgroup C:\ 238G 58G 181G 25% /mnt/c D:\ 932G 8.1G 924G 1% /mnt/d * Update your Ubuntu Linux with: * ''sudo apt update && sudo apt upgrade'' * Use the Left arrow to select ''''+''Enter'' when some services have to be restarted * [[other:win10wsl#updating_wsl_and_ubuntu|More details about updating WSL and/or Ubuntu]] (you can safely skip this for now) ===== After the base installation ===== Once you have completed the base installation steps, you'll get a Ubuntu machine, with a //root// and user account (pre-configured to use ''sudo'' for commands that need root access) and lots of pre-installed applications (but no graphics). ==== Opening a terminal ==== * Check the next sections to find out the different ways to open terminals * Opening the first terminal will start WSL+Ubuntu * Closing all terminals will not stop WSL+Ubuntu. Read the [[#rebooting_wsl|rebooting WSL section]] if you need a clean start of WSL without rebooting Windows Having a terminal you can efficiently interact with is the basis of your future work with WSL+Ubuntu! You can **optionally** read the [[https://nickjanetakis.com/blog/conemu-vs-hyper-vs-terminus-vs-mobaxterm-terminator-vs-ubuntu-wsl|ConEmu vs Hyper vs Terminus vs MobaXTerm Terminator vs Ubuntu WSL]] blog if you really want more details === The default terminal === The **Default WSL+Ubuntu //terminal//** is basically a //bash// shell running inside a regular Windows ''cmd'' text console. You do not need to have an //X server// running to use it! There are several ways to open a default terminal: * Select //Ubuntu// from the ''Start'' menu * Click on the //Ubuntu// icon on the task bar * if you don't have a //Ubuntu// icon on the task bar yet: right-click on the //Ubuntu// icon (in the ''Start'' menu) -> More -> Pin to the task bar * to open more terminals: right-click on the //Ubuntu// icon on the task bar -> Ubuntu * Type ''ubuntu''+ or ''wsl''+'''' in the (Cortana) Search field on the task bar == Copy/paste in the default terminal == Follow the steps in [[https://blogs.msdn.microsoft.com/commandline/2018/04/13/copy-and-paste-arrives-for-linuxwsl-consoles/|Copy and Paste arrives for Linux/WSL Consoles]] and you will then be able to **copy/paste by using ''Shift+Ctrl+C/V''** * The //QuickEdit mode// should already be selected by default and you can also: * **select** text with the left mouse button * **paste** the selected text with the right mouse button (click **twice** after a new selection) * Use ''Alt''+'''' to toggle the terminal to full screen (maybe more useful options [[https://docs.microsoft.com/en-us/previous-versions/orphan-topics/ws.11/mt427362(v=ws.11)|here?]]) === The WSLtty terminal === * This is the recommended (by JYP) option for now * If you can't open a terminal after Windows upgrades from WSL to WSL v2, get and install the latest version of WSLtty ! [[https://github.com/mintty/wsltty|WSLtty]] allows you to use the efficient //Mintty// as a terminal for WSL. **You do not need to have an //X server// running to use it!** * Download and execute the latest [[https://github.com/mintty/wsltty/releases|64bit installer]] (the most recent ''wsltty-*-install-x86_64.exe'' file) * After installing, you will get several ways to open a terminal: * Double-click on the ''WSL terminal'' icon on the desktop * ''U''=>''Ubuntu terminal'' and ''W''=>''WSL Terminal'' in the ''Start'' menu * There is also a ''W''=>''WSLtty'' folder, than you don't really need * Open a terminal and right-click in it to check the available options * You can access even more options with CTRL+right-click * You can cycle between the open WSLtty terminals with CTRL+Tab and Shift+CTRL+Tab * [[https://github.com/mintty/mintty/wiki|Technical stuff]] (on the Mintty wiki) === The new Windows Terminal === Microsoft is working on a powerful replacement of the old ''cmd'' terminal! Check [[https://github.com/microsoft/terminal|Windows Terminal, Console and Command-Line @ github]] Note: the //Windows Terminal// looks really promising, but will be nice when there is a GUI for editing the options (rather the editing a //json// file with a text editor...) and a way to select //xterm-style// copy-pasting. See [[https://github.com/microsoft/terminal/issues/7646|Feature request: add xterm-style select/copy/paste options]] * [[https://aka.ms/terminal|Getting the Windows Terminal from the Microsoft Store]] * [[https://docs.microsoft.com/en-us/windows/terminal/|Documentation]] === Terminals started from Linux === You can install additional //graphical// terminals inside the Linux running in WSL, but **you will need to have a running //X server// if you want to open them**. In these terminals, the //usual// Linux copy/paste rules will apply! == xterm == * Install with: ''apt-get install xterm'' * Start from another terminal with: ''xterm &'' == xfce-terminal == * Install with: ''apt-get install xfce4-terminal'' * Start from another terminal with: ''xfce4-terminal &'' ==== Location of the files ==== You can optionally read the detailed [[https://blogs.msdn.microsoft.com/wsl/2016/06/15/wsl-file-system-support/|WSL File System Support]] page if you want to (try to) understand how things work and how both systems can share files. === Linux files seen from Windows === * You can access the Linux files in ''/'' from Windows by opening the //virtual// ''\\wsl$'' path in the Windows explorer (since [[https://docs.microsoft.com/en-us/windows/wsl/release-notes#build-18342|Windows build 18342]]) * Your Linux //home// directory is located in ''\\wsl$\Ubuntu\home\'' * The content of the Ubuntu ''/'' directory is actually located in the following //hidden// sub-directory of the user's Windows //home// directory: ''C:\Users\\AppData\Local\Packages\CanonicalGroupLimited.UbuntuonWindows_79rhkp1fndgsc\LocalState\rootfs'' [[https://blogs.msdn.microsoft.com/commandline/2016/11/17/do-not-change-linux-files-using-windows-apps-and-tools/|Do not change Linux files in '/' from Windows!]] (it's safe **only** if you use the ''\\wsl$'' path) If you need to easily update files from both Linux and Windows, put them in a regular Windows directory (not in the hidden directory mentioned above, where the Linux system is installed) === Windows files seen from Linux === Note: When you are in an explorer window, you can open a Windows terminal with Linux by holding Shift and right-clicking on a directory and choosing Linux in the context menu. The Windows partitions are mounted under ''/mnt'' in Linux: $ df -h Filesystem Size Used Avail Use% Mounted on rootfs 111G 30G 82G 27% / none 111G 30G 82G 27% /dev none 111G 30G 82G 27% /run none 111G 30G 82G 27% /run/lock none 111G 30G 82G 27% /run/shm none 111G 30G 82G 27% /run/user C: 111G 30G 82G 27% /mnt/c H: 366G 138M 366G 1% /mnt/h Read the [[#mounting_disks_and_network_drives|Mounting disks and network drives]] section if you want to access remote server disks on your local network, or just want to access USB disks/sticks on your local computer. === Misc === * The ''wslpath'' command can be used to make Windows<->Linux path conversion ==== Linux version installed ==== $ date Thu Feb 20 11:18:57 CET 2020 $ lsb_release -a No LSB modules are available. Distributor ID: Ubuntu Description: Ubuntu 18.04.4 LTS Release: 18.04 Codename: bionic $ cat /proc/version Linux version 4.4.0-18362-Microsoft (Microsoft@Microsoft.com) (gcc version 5.4.0 (GCC) ) #476-Microsoft Fri Nov 01 16:53:00 PST 2019 $ uname -a Linux some_machine_name 4.4.0-18362-Microsoft #476-Microsoft Fri Nov 01 16:53:00 PST 2019 x86_64 x86_64 x86_64 GNU/Linux ==== Ubuntu packages installed ==== $ dpkg --list Desired=Unknown/Install/Remove/Purge/Hold | Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend |/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad) ||/ Name Version Architecture Description +++-=====================-===============-===============-================================================ ii accountsservice 0.6.45-1ubuntu1 amd64 query and manipulate user account information ii acl 2.2.52-3build1 amd64 Access control list utilities ii acpid 1:2.0.28-1ubunt amd64 Advanced Configuration and Power Interface event ii adduser 3.116ubuntu1 all add and remove users and groups [...] ii xterm 330-1ubuntu2 amd64 X terminal emulator ii xxd 2:8.0.1453-1ubu amd64 tool to make (or reverse) a hex dump ii xz-utils 5.2.2-1.3 amd64 XZ-format compression utilities ii zerofree 1.0.4-1 amd64 zero free blocks from ext2, ext3 and ext4 file-s ii zlib1g:amd64 1:1.2.11.dfsg-0 amd64 compression library - runtime $ dpkg --list > /mnt/c/Scratch/jyp/pkg_initial_201601.txt $ wc -l /mnt/c/Scratch/jyp/pkg_initial_201601.txt 628 /mnt/c/Scratch/jypeter/pkg_initial_201601.txt Full (example) list of installed packages: {{:other:pkg_initial_201601.txt|pkg_initial_201601.txt}} ==== Update ubuntu ==== Use the following to update your ubuntu installation: $ sudo -s $ apt-get update $ apt-get upgrade ==== Managing WSL ==== This is just place to store some technical information, that will probably not be needed by most people * [[https://docs.microsoft.com/en-us/windows/wsl/wsl-config#managing-multiple-linux-distributions|command-line options]] * [[https://docs.microsoft.com/en-us/windows/wsl/wsl-config#set-wsl-launch-settings|Changing the start-up settings]] * [[https://docs.microsoft.com/fr-fr/windows/wsl/interop#invoking-wsl-from-the-windows-command-line|Invoking WSL from the Windows Command Line]] * [[https://docs.microsoft.com/fr-fr/windows/wsl/interop#invoking-windows-binaries-from-wsl|Invoking Windows binaries from WSL]] * News in the [[https://devblogs.microsoft.com/commandline/|Windows Command Line Tools For Developers]] WSL blog ===== Getting ready for some real work ===== We often assume below that you are using the **root** account (''sudo -s''), or ''sudo'' when executing the ''apt*'' installation commands, and othe admin commands ==== Specifying correctly the display language ==== WSL will automatically try to use in the terminals the same language as the one used by the Windows 10 computer ([[https://docs.microsoft.com/en-us/windows/wsl/troubleshooting#changing-the-display-language|ref]]), but this could lead later to some difficult to debug **side-effects** when you connect to remote Linux servers (e.g. the ''sort'' command may give different results !). It is **safer** to force the system to use a //standard// English version $ sudo update-locale LANG=en_US.UTF8 $ echo $LANG en_US.UTF8 ==== Package management ==== [[https://www.howtogeek.com/63997/how-to-install-programs-in-ubuntu-in-the-command-line/|apt-get tutorial]] if you are more used to Fedora/RedHat ''yum''/''dnf'' and //rpm// packages * Install the **aptitude** package manager: ''apt-get install aptitude'' * [[https://doc.ubuntu-fr.org/aptitude|details]] (fr) * Install the **synaptic** package manager: ''aptitude install synaptic'' * [[https://doc.ubuntu-fr.org/synaptic|details]] (fr) === Useful commands === * ''dpkg -l'': list installed packages * ''dpkg -L package_name'': list files installed from ''package_name'' * ''dpkg -S path/command'': name of the package used to install ''path/command'' === Example === $ dpkg -l | grep -i wsl ii ubuntu-wsl 1.417.3 amd64 Ubuntu on Windows tools - Windows Subsystem for Linux integration ii wslu 2.3.2-0ubuntu2~18.04.3 all collection of utilities for the Windows 10 Linux Subsystem $ dpkg -L wslu /. /etc /etc/profile.d /usr /usr/bin /usr/bin/wslfetch /usr/bin/wslsys /usr/bin/wslupath /usr/bin/wslusc /usr/bin/wslvar /usr/bin/wslview [...] ==== Using an X server ==== You will need to have an **X server** running locally on your Windows 10 machine if you want to display graphics generated by the Linux running locally, or on a remote server. Applications running in text mode only (e.g. ''vi'') do not require a running X server. You will get a //Can't open display// error if no server is running (even if the ''DISPLAY'' variable is correctly defined) your_login@your_machine:~$ echo $DISPLAY localhost:0.0 your_login@your_machine:~$ xterm & xterm: Xt error: Can't open display: localhost:0.0 [Everything is fine after starting a local X server] jyour_login@your_machine:~$ xterm & [1] 55 ==== Installing an X server ==== Read the [[other:x_conf|Installing and configuring an X server]] page ==== Configuration (in your local Linux account) ==== * Set the ''DISPLAY'' variable in the shell by hand, or automatically in the shell configuration file(s): * //bash// shell: ''export DISPLAY=localhost:0.0''\\ Can be defined in the ''~/.bashrc'' file * //tcsh// shell: ''setenv DISPLAY localhost:0.0''\\ Can be defined in the ''~/.login'', or the ''~/.cshrc'' file * Warning! ''export DISPLAY=:0.0'' will also work for using graphics locally, but **will fail when trying to use graphics on a remote computer**, with a ''connect /tmp/.X11-unix/X0: No such file or directory'' error message ([[https://unix.stackexchange.com/questions/57138/why-does-my-x11-forwarding-attempt-fail-with-connect-tmp-x11-unix-x0-no-such/|source]]) * FIXME **//X server// and WSL2** : check ''export DISPLAY=$(cat /etc/resolv.conf | grep nameserver | awk '{print $2}'):0.0'' in [[https://solvercube.com/how-to-install-ubuntu-20-04-on-windows-using-wsl/|How to install Ubuntu 20.04 on Windows using WSL]] ==== Adding useful packages ==== Reminder: you need to use the ''root'' account (with ''sudo -s'') in order to use the following commands $ apt-get install gcc libglu1-mesa mesa-utils libsm6 x11-apps # Check the "Terminals started from Linux" section # if you need to install extra terminals # Install emacs $ apt-get install emacs # Install pdf (evince) and image (eog) viewers $ apt-get install evince eog # Install wslu to improve the Windows<->Linux communication $ apt-get install wslu # The packages below are useful for dealing with climate data # in netCDF files (and are also requirements of CLIMAF) $ aptitude install imagemagick nco cdo netcdf-bin ncview exiv2 ===== Updating WSL and ubuntu ===== You don't have to worry about the //Upgrade// section if you have just installed WSL+Ubuntu, because you are already using the latest Ubuntu stable release! ==== WSL ==== **Nothing to do!** WSL is automatically updated, provided you have **enabled** the [[other:win10maintain#updating_microsoft_office|Get updates for other Microsoft products]] option [[https://docs.microsoft.com/en-us/windows/wsl/wsl-config#managing-multiple-linux-distributions| WSL command line options]] ==== Updating ubuntu ==== The ''Ubuntu **16**.04.3 LTS'' that was initially installed from the //Windows Store// when the test computer was running ''Win 10 1709'' was automatically updated to ''Ubuntu **18**.04.2 LTS'' when the computer was updated to ''Win 10 1809''. $ lsb_release -a No LSB modules are available. Distributor ID: Ubuntu Description: Ubuntu 18.04.2 LTS Release: 18.04 Codename: bionic We also ran the ''do-release-upgrade'' command to complete the upgrade (from 16.04 to 18.04) [ [[https://devblogs.microsoft.com/commandline/upgrading-ubuntu/|More info from WSL team]] - [[https://doc.ubuntu-fr.org/migration|more info from Ubuntu]] ] And we then ran the usual upgrade commands. These commands can be run any time for updating the installed packages $ sudo -s $ apt-get update $ apt-get upgrade Full list of the resulting {{ :other:pkg_190221.txt |installed packages}}, generated with $ dpkg --list >$ dpkg --list > /mnt/h/Scratch/jyp/pkg_190221.txt $ wc -l /mnt/h/Scratch/jyp/pkg_190221.txt 578 /mnt/h/Scratch/jyp/pkg_190221.txt ==== Upgrading ubuntu ==== You can skip this section if you have just installed ubuntu, since you are already running the latest stable release available The **updates** are for maintaining the same version of ubuntu (e.g. you go from version ''18.04.2'' to ''18.04.5''), while **upgrades** are for going from one major version to the next (e.g ''16.xx'' => ''18.xx'' => ''20.xx''). We try to only use the stable major releases, that have an even version number. We use the **LTS versions** (//Long Time Support//) to ensure even more stability, which means that we will get updates of a given major release for quite some time after the next major release is available, so it is not mandatory to updgrade ubuntu if you are still receiving updates for the release you use There should not be any risk, but it may be a good idea to [[other:win10wsl#creating_a_backup_of_the_linux_running_in_wsl|create a backup]] of your current ubuntu before upgrading it! Use ''sudo'' for each command below, or just become root by using ''sudo su -'' * before the Linux upgrade: * [[other:win10wsl#creating_a_backup_of_the_linux_running_in_wsl|create a backup]] * Check how much space is left on ''C:'' * make sure you have a network connection * read the official WSL [[https://devblogs.microsoft.com/commandline/upgrading-ubuntu/#upgrading-your-distribution|Upgrading your distribution]] instructions * Launch the upgrade with the ''do-release-upgrade'' command and follow the instructions * if you get a //There is no development version of an LTS available// message, use ''do-release-upgrade -d'' * if you get a //Please install all available updates for your release before upgrading// message, use ''apt-get dist-upgrade'' and then try ''do-release-upgrade -d'' again (see [[https://www.liquidweb.com/kb/troubleshooting-please-install-available-updates-release-upgrading/|How to Solve the Upgrade Ubuntu Install Updates Error]]) * if you get ''sleep: cannot read realtime clock: Invalid argument'' error, temporarily replace the ''sleep'' binary with an executable empty file. Do not forget to restore the original sleep command after the upgrade!! * # sleep 1 sleep: cannot read realtime clock: Invalid argument # mv /bin/sleep /bin/sleep.back # ls -l /bin/sleep.back -rwxr-xr-x 1 root root 35000 Jan 18 2018 /bin/sleep.back # touch /bin/sleep # chmod +x /bin/sleep # sleep 1 # * Note: you may have to use the //empty sleep command trick// again, if the sleep command itself is updated and breaks the upgrading process again. Do not forget to restore the original sleep command after the upgrade!!\\ [...] Restarting services possibly affected by the upgrade: cron: stopping...sleep: cannot read realtime clock: Invalid argument dpkg: error processing package libpam0g:amd64 (--configure): installed libpam0g:amd64 package post-installation script subprocess returned error exit status 1 E: Sub-process /usr/bin/dpkg returned an error code (1) # ls -l /bin/sleep -rwxr-xr-x 1 root root 39256 Sep 5 2019 /bin/sleep # mv /bin/sleep /bin/sleep.back2 # touch /bin/sleep # chmod +x /bin/sleep # * see also [[https://www.how2shout.com/how-to/how-to-upgrade-ubuntu-18-04-lts-to-20-04-lts-on-wsl-windows-10.html|How to Upgrade Ubuntu 18.04 LTS to 20.04 LTS on WSL (Windows 10)]] * Do not forget to restore the correct ''sleep'' executable, if you had to move it * # ls -l /bin/sleep* -rwxr-xr-x 1 root root 0 Aug 23 23:57 /bin/sleep -rwxr-xr-x 1 root root 35000 Jan 18 2018 /bin/sleep.back -rwxr-xr-x 1 root root 39256 Sep 5 2019 /bin/sleep.back2 # cp -p /bin/sleep.back2 /bin/sleep # ls -l /bin/sleep* -rwxr-xr-x 1 root root 39256 Sep 5 2019 /bin/sleep -rwxr-xr-x 1 root root 35000 Jan 18 2018 /bin/sleep.back -rwxr-xr-x 1 root root 39256 Sep 5 2019 /bin/sleep.back2 ==== Cleaning up things ==== * [[https://linoxide.com/how-to-clear-apt-cache/|How to Clear Apt Cache]] ===== Advanced configuration ===== ==== Get Linux configuration files ==== * You can either adapt your existing Linux configuration files (''.profile'', ''.bashrc'', ...) from another Linux computer, or download some example files in your WSL home directory and use (and improve) them as suggested below (or as you wish) * $ cd $ scp -pr @ssh1.lsce.ipsl.fr:/home/users/jypeter/WSL_config . * You get a default ''.bashrc'' file in your home directory when your WSL account is created. Add the following line at the end of ''~/.bashrc'' to use what is defined in the downloaded ''WSL_config'' directory:\\ ''source ~/WSL_config/bashrc_extra.sh'' * If you use the //emacs// text editor, you can use your own ''.emacs'' configuration file, or get an example configuration file the following way: * $ cd $ ln -s WSL_config/emacs .emacs * Font configuration (type, size, ...): FIXME * You can either copy the full ''.ssh'' directory of your Linux account, or just get the //private key// files: * $ cd $ scp -pr @ssh1.lsce.ipsl.fr:.ssh . * Make sure you **also** read the [[other:win10wsl#ssh_agent_and_keys_related_questions|ssh, agent and keys related questions]] section below! ==== ssh, agent and keys related questions ==== This assumes that you have already created a private/public ssh key pair and that the private key is available somewhere in the Windows 10 computer === ssh from WSL === == Authorizing ssh to use old dsa keys == //dsa// ssh keys are disabled by default (becoming obsolete). ''ssh'' will ask for your password even if you have a ''~/.ssh/id_dsa'' file. ''ssh -v'' will print ''Skipping ssh-dss key /home//.ssh/id_dsa - not in PubkeyAcceptedKeyTypes''. The solution is to generate a new couple of public/private keys not based on //dsa// encryption, or to use the user ssh ''config'' file to authorize //dsa// ([[https://unix.stackexchange.com/a/247614|source]]) * Add ''PubkeyAcceptedKeyTypes +ssh-dss'' to ''~/.ssh/config'' (**before** the ''HOST'' sections) * Make sure the config file has the correct access rights: ''chmod 600 config'' == Using Putty Pageant as an ssh agent == This assumes that you have installed the full [[other:win10apps#putty_pageant|Putty]] package (with the //msi// installer), converted your private ssh key with ''PuTTYgen'', and unlocked the converted ssh private key in Pageant. The converted ssh private key is located in a Windows 10 folder * Download the required binaries from the [[https://github.com/vuori/weasel-pageant|weasel-pageant github page]] and extract them to a Windows directory, e.g. ''C:\Utils\weasel-pageant-x.x'' * Add the following line to your ''~/.bashrc'' file:\\ ''eval $(/mnt/c/Utils/weasel-pageant-x.x/weasel-pageant -rb -a $HOME/.weasel-pageant.sock)'' * This is done by default in the ''~/WSL_config/bashrc_extra.sh'' if you have downloaded and adapted it * Open a new terminal: * Type ''ssh-add -l'': it will display either ''The agent has no identities'', or as many lines as you have entered keys in Pageant * Type ''ssh user@host'' to connect to a computer where you have your public key Note: the other similar solutions are [[https://github.com/rupor-github/ssh-agent-wsl|ssh-agent-wsl]] (the key is stored in the Windows ssh-agent instead of Pageant) or [[https://github.com/benpye/wsl-ssh-pageant|wsl-ssh-pageant]] == Using Ubuntu as an ssh-agent == The ssh key (and other ssh settings) are in the ''~/.ssh'' directory of your WSL+Ubuntu account This is the most simple setting, **but** there will be one running //ssh-agent// per terminal, and the key will be stored independently for each terminal (which means that you will have to type ''ssh-add'' and your pass-phrase for each terminal) * It's not possible by default to add a key to ssh-agent ('ssh-add' will print ''Could not open a connection to your authentication agent'') * If you want to use ''ssh-add'' in a terminal, you first need to type ''eval `ssh-agent -s`'' in the (or ''eval $(ssh-agent)'') * In that case, it's just easier to add ''eval $(ssh-agent)'' to the ''.bashrc'' file! * This is done by default in the ''~/WSL_config/bashrc_extra.sh'' if you have downloaded and adapted it == Notes & TODO == * See if a solution appears for [[https://unix.stackexchange.com/questions/504083/wsl-keeping-ssh-private-key-accessible-as-long-as-i-dont-restart-windows-10-l/|WSL: Keeping ssh private key accessible as long as I don't restart Windows 10 (like the behaviour of pageant)]] * See if we can use something from [[https://devblogs.microsoft.com/commandline/sharing-ssh-keys-between-windows-and-wsl-2/|Sharing SSH keys between Windows and WSL 2]] === ssh to WSL === FIXME Maybe check [[https://docs.microsoft.com/en-us/windows/wsl/troubleshooting#openssh-server-connection-issues|OpenSSH-Server connection issues]] Maybe it could be useful to do a ''ssh'' from the Win10 part of the machine to the WSL part, and from outside the machine to the WSL part It would probably already be enough to enable the ssh server on Win10 (the [[https://devblogs.microsoft.com/powershell/using-the-openssh-beta-in-windows-10-fall-creators-update-and-windows-server-1709/|ssh 1709 release beta features]] are now officially in Win10) [[https://www.google.com/search?client=firefox-b-d&q=wsl+ssh|Google search: wsl ssh]] * [[https://gist.github.com/dentechy/de2be62b55cfd234681921d5a8b6be11|How to automatically start ssh server on boot on Windows Subsystem for Linux]] * [[https://www.illuminiastudios.com/dev-diaries/ssh-on-windows-subsystem-for-linux/|SSH on Windows Subsystem for Linux]] * [[https://superuser.com/questions/1123552/how-to-ssh-into-wsl|How to SSH into WSL]] ==== Creating shortcuts to Linux programs on the Windows desktop ==== * Install [[https://github.com/wslutilities/wslu|wslu]] (A collection of utilities for WSL) if it is not already installed * ''apt-get install wslu'' * Have a quick look at the wslu [[https://github.com/wslutilities/wslu|github]] page and [[https://github.com/wslutilities/wslu/wiki|wiki]] to find about the available tools (that you may need later) * You can create a shortcut by using ''wslusc'' in any Linux terminal * e.g. you can create a shortcut on the Windows desktop to start emacs (in WSL!) with:\\ ''wslusc -g emacs'' * This will make it easier to start emacs, but since this is a graphical Linux application, **you need to have an X server running before you click on the emacs shortcut**! * You can optionally get some information about the [[https://github.com/Microsoft/WSL/issues/3404|technical details]] ==== Mounting disks and network drives ==== By default, the local disks and the external USB disks available (and decrypted!) when WSL is started (i.e. when the first WSL terminal is opened) are automatically available (//mounted//) in ''/mnt'' : the content of ''C:\'' is available in ''/mnt/c'', etc... The list of available disks is refreshed when the Windows session is restarted (the user logs out and in, or Windows is rebooted) or when just [[other:win10wsl#rebooting_wsl|WSL is restarted]]. Extra local disks or //network drives// (what you can access by opening ''\\dfshost\dfs\'' in the file explorer when you are on the LSCE network) can be either mounted on the fly with the ''mount'' command, or automatically by configuring the ''/etc/fstab'' file. You will access the network drives with the access rights you have in the current Windows session (and you may have to access the drives in Windows before you can mount them in WSL) Warning! In all cases, the owner, groups and permissions displayed may not be accurate (but the actual access rights will apply). * the user/group of the files may appear as ''your_login''/''your_login'' or ''root''/''root''. Example of a local file belonging to the current Windows //jypeter// user, and a system file * $ ls -l /mnt/c/Users/jypeter/Desktop/File_on_Desktop.txt -rwxrwxrwx 1 jypeter jypeter 0 Jan 21 11:50 /mnt/c/Users/jypeter/Desktop/File_on_Desktop.txt $ ls -l /mnt/c/Windows/notepad.exe -r-xr-xr-x 3 jypeter jypeter 181248 Mar 19 2019 /mnt/c/Windows/notepad.exe * the access rights may appear as ''rwxrwxrwx'' even if the actual rights are more restrictive. Example after mounting the LSCE ''/home/scratch01'' disk * # File access rights seen from WSL+Ubuntu $ ls -l /mnt/scratch01/jypeter/Chrome_backup.zip -rwxrwxrwx 1 root root 2755154 Jan 13 15:10 /mnt/scratch01/jypeter/Chrome_backup.zip # Actual access rights (on the Linux server) >ls -l /home/scratch01/jypeter/Chrome_backup.zip -rw-r--r-- 1 jypeter lsce 2755154 Jan 13 15:10 /home/scratch01/jypeter/Chrome_backup.zip # You can't change the access rights from WSL+Ubuntu, but you can remove a file, or create a new one $ chmod 600 /mnt/scratch01/jypeter/Chrome_backup.zip chmod: changing permissions of '/mnt/scratch01/jypeter/Chrome_backup.zip': Operation not permitted $ rm /mnt/scratch01/jypeter/Chrome_backup.zip $ ls -l /mnt/scratch01/jypeter/Chrome_backup.zip ls: cannot access '/mnt/scratch01/jypeter/Chrome_backup.zip': No such file or directory $ touch /mnt/scratch01/jypeter/new_file_from_wsl.txt jypeter@lsce5203:/mnt/scratch01/jypeter$ ls -l /mnt/scratch01/jypeter/new_file_from_wsl.txt -rwxrwxrwx 1 root root 0 Jan 21 11:37 /mnt/scratch01/jypeter/new_file_from_wsl.txt # The new file has the correct (default) access rights on the Linux server >ls -l /home/scratch01/jypeter/new_file_from_wsl.txt -rw-r--r-- 1 jypeter lsce 0 Jan 21 11:37 /home/scratch01/jypeter/new_file_from_wsl.txt === Enabling metadata === Some access rights handling can be slightly improved by automatically enabling the ''metadata'' mount option. This will allow you to use ''chmod''/''chown'' on **local NTFS disks** (disks directly connected to your computer, e.g. ''/mnt/c/Users/your_login''). - become //root// in a terminal with: ''sudo -s'' - create a ''/etc/wsl.conf'' with the following content:[automount] options = "metadata" You need to [[other:win10wsl#rebooting_wsl|restart WSL+ubuntu]] to activate the new content of ''/etc/wsl.conf''. See below an example of the ''mount -l'' output: * Before: ''C: on /mnt/c type drvfs (rw,noatime,uid=1000,gid=1000,case=off)'' * After enabling //metadata//: ''C: on /mnt/c type drvfs (rw,noatime,uid=1000,gid=1000,metadata,case=off)'' Optional: more details about ''wsl.conf'' in [[https://docs.microsoft.com/en-us/windows/wsl/wsl-config#set-wsl-launch-settings|WSL launch settings]], [[https://devblogs.microsoft.com/commandline/automatically-configuring-wsl/|How to use wsl.conf]] and [[https://devblogs.microsoft.com/commandline/chmod-chown-wsl-improvements/|metadata and access rights management]] === On-the-fly mount === You have to create a //mount point// in the existing ''/mnt'' directory and then use the ''mount'' command # Must be root for most 'mount' related commands $ sudo -s # Create the mount point (once), e.g. for accessing /home/scratch01 on the local LSCE servers # This works when the Windows 10 computer is on the LSCE network $ mkdir /mnt/scratch01 # Mount the remote disk $ mount -t drvfs '\\dfshost\dfs\scratch01' /mnt/scratch01 # List the mounted disks $ mount -l [...] \\dfshost\dfs\scratch01 on /mnt/scratch01 type drvfs (rw,relatime,case=off) # Unmount the disk, or just shutdown Windows or WSL when you are finished $ umount /mnt/scratch01 If you connect a USB disk/stick and windows sees it as a new ''J:\'' drive, you can also manually mount it $ sudo -s # Create the mount point (once) $ mkdir /mnt/j # Mount the USB disk $ mount -t drvfs J: /mnt/j # Unmount the disk, or just shutdown Windows or WSL when you are finished $ umount /mnt/j === Automatic mount === The disks defined in ''/etc/fstab'' will be automatically mounted when you start WSL and if they are actually accessible from your Windows session (e.g. ''\\dfshost\dfs'' can only be accessed from the LSCE network) You need to add 1 line per mount point to ''/etc/fstab'', and create (once) the directories that will be used for mounting the disks $ sudo -s # Create the mount points (once) # The example below is to access the LSCE /home/users and /home/scrath01 disks $ mkdir /mnt/users /mnt/scratch01 # Make a backup copy of /etc/fstab before modifying it (for safety reasons!) $ cp -p /etc/fstab /etc/fstab.bak # Add lines to /etc/fstab associating mount points and directories $ cat /etc/fstab LABEL=cloudimg-rootfs / ext4 defaults 0 0 \\dfshost\dfs\users /mnt/users drvfs defaults 0 0 \\dfshost\dfs\scratch01 /mnt/scratch01 drvfs defaults 0 0 # Try to mount all the disks in /etc/fstab $ mount -a # Check the result $ mount -l [...] \\dfshost\dfs\users on /mnt/users type drvfs (rw,relatime,case=off) \\dfshost\dfs\scratch01 on /mnt/scratch01 type drvfs (rw,relatime,case=off) === Accessing remote file systems with sshfs === FIXME It seems this can be done with a combination of [[https://github.com/billziss-gh/winfsp|WinFsp]] and [[https://github.com/billziss-gh/sshfs-win|SSHFS-Win]] (see [[https://superuser.com/questions/1423371/sshfs-remote-directory-mounting-syntax|SSHFS remote directory mounting syntax]] for details === Extra resources === Useful ''mount'' options: * ''mount -a'' (must be //root//): mount everything specified in ''/etc/fstab'' * ''umont /mnt/scratch01'' (must be //root//): unmount the disk accessible through ''/mnt/scratch01'' * ''mount -l'': list all the mounted disks and the mount options (useful when you are using the default mount options and want to know which options were actually applied) Useful links (used for writing this section): * [[https://devblogs.microsoft.com/commandline/automatically-configuring-wsl/|Automatically Configuring WSL]] * [[https://blogs.msdn.microsoft.com/wsl/2017/04/18/file-system-improvements-to-the-windows-subsystem-for-linux/|File System Improvements to the Windows Subsystem for Linux]] * [[https://devblogs.microsoft.com/commandline/chmod-chown-wsl-improvements/|Chmod/Chown WSL Improvements]] (the new //metadata// option) * [[https://github.com/Microsoft/WSL/issues/2636|Automount additonal filesystems from fstab]] * [[https://gist.github.com/sgtoj/f82990bcd9e89db49b84e2d2e70b281d|Ubuntu for Windows: Mounting C: Drive to WSL's Root]] ==== Rebooting WSL ==== It is not possible to **stop or reboot WSL** like a regular Linux computer * ''shutdown -r now'' will generate an error message * WSL keeps on running even when all the Linux terminals are closed Shutting down or rebooting the Windows computer will of course stop WSL but it's useful to be able to just stop/restart WSL in order to: * force WSL to **recognize new disk drives** and make them available as ''/mnt/Your Linux //home// directory (''/home/''), or some specific sub-directories of the //Ubuntu// running in WSL (sub-directories of ''/''): all these files are located in a [[other:win10wsl#location_of_the_files|hidden sub-directory of you Windows directory]], and the best way to save them is probably to **create a tar file from Linux**, and put this tar file in a specific Windows //WSL backup folder// (we will use below the directory ''/mnt/c/Scratch/'' or ''c:\Scratch\'') * $ cd /home $ du -sh $USER 232K # We assume below that there is an existing /mnt/c/Scratch// directory # It's a WINDOWS directory C:\Scratch\ $ tar cfz /mnt/c/Scratch/$USER/${USER}_home_`date +%y%m%d_%H%M`.tgz $USER # Check the size of the new (and existing old) backup(s) $ ls -ltr /mnt/c/Scratch/$USER/${USER}_*.tgz 81K Jan 21 2020 /mnt/c/Scratch//_home_200121.tgz 93K Aug 22 2020 /mnt/c/Scratch//_home_200822_1151.tgz 95K Feb 24 12:09 /mnt/c/Scratch//_home_210224_1209.tgz * Regular Windows folders where you store data used by WSL: you can use ''tar'' in a Linux terminal, as above, or use any usual way of backing up Windows files (drag and drop, some specific [[other:win10apps#backup_software|backup software]]) * The **full** Linux/Ubuntu installation running in WSL (based on [[https://docs.microsoft.com/en-us/windows/wsl/faq#how-can-i-back-up-my-wsl-distros-or-move-them-from-one-drive-to-another|How can I back up my WSL distros?]]). The following steps will save the full installation in a tar file, that can be used later for restoring the full installation (//restoration// not tested yet) * Open a **Windows terminal** (preferably a //PowerShell// if you need easy copy/paste)\\ The //Windows// terminal is where you will type the ''wsl'' commands shown below * Determine the name of the current Linux installation used in WSL (and see if it is running, and using //WSL 1// or //WSL 2//) * > wsl --list --all Distributions du sous-système Windows pour Linux : Ubuntu (par défaut) > wsl --list --verbose NAME STATE VERSION * Ubuntu Running 1 * [[other:win10wsl#rebooting_wsl|Stop WSL+Linux]] * > wsl --list --verbose NAME STATE VERSION * Ubuntu Running 1 > wsl --shutdown > wsl --list --verbose NAME STATE VERSION * Ubuntu Stopped 1 * Export the current Linux to a //tar// file: * > wsl --export Ubuntu C:\Scratch\\_ubuntu--full_.tar * You can optionally open a new terminal (this will start again WSL+Ubuntu) to get some information about the created file (size and number of saved files/directories), and compress it in order to save some disk space * $ cd /mnt/c/Scratch/ $ ls -ltrh *ubuntu*tar* 976M Jan 21 2020 _200121.tar.gz 2.8G Aug 22 2020 -16.04-full_200822.tar 3.4G Feb 24 14:06 -20-04-1-full_210224.tar # Determine the number of files/directories in the backup $ tar tvf -20-04-1-full_210224.tar | wc -l 112711 $ gzip *ubuntu*.tar $ ls -ltrh *ubuntu*tar* 976M Jan 21 2020 _200121.tar.gz 1.2G Aug 22 2020 -16.04-full_200822.tar.gz 1.8G Feb 24 14:06 -20-04-1-full_210224.tar.gz ==== Restoring a WSL backup ==== FIXME Read [[https://www.howtogeek.com/426562/how-to-export-and-import-your-linux-systems-on-windows-10/|How to Export and Import Your Linux Systems on Windows 10]] and [[https://winaero.com/blog/export-import-wsl-linux-distro-windows-10/|Export and Import WSL Linux Distro in Windows 10]] and similar threads FIXME Check if [[https://wslhub.com/|WSL Manager]] is maintained and can be useful This assumes that you have followed the steps above to create a full backup of your Linux distribution in a ''ubuntu__.tar'' file. This file can theoretically be restored with: * ''wsl %%--%%import ubuntu__.tar'' * Note: found a page mentioning that you should be in a terminal with admin rights when using ''%%--%%import'' * Can we use the same '''' (e.g. ''Ubuntu'') as the one we are trying to restore without first removing it with ''unregister'' ([[https://docs.microsoft.com/en-us/windows/wsl/wsl-config#unregister-and-reinstall-a-distribution|Unregister and reinstall a distribution]])? * If we use a new distribution name, it will probably not appear anymore in the Microsoft Store and will not be updated. Is this a problem? * It seems we can use '''' to install anywhere! What should we do if we want to restore to the default location (in the hidden directory)? * Can we install out of the current user's Windows folder? * Can we install out of ''C:\''? * Can we restore (//clone//) a backed up Linux distribution on another Windows 10 computer for another user? * if the default user (and its password) has to be changed, you probably have to follow the steps to [[https://docs.microsoft.com/en-us/windows/wsl/user-support#for-fall-creators-update-and-later|reset the Linux password]] and find a way to create a new account when the ''root'' password is activated ==== Uninstalling WSL ==== Not too sure about this part... * Make a backup of whatever you had in your Ubuntu home directory, and all the files and settings that are not in the ''/mnt'' directories * Find how to unregister Ubuntu on the [[https://docs.microsoft.com/en-us/windows/wsl/wsl-config|Manage and configure Windows Subsystem for Linux]] page * and then disable WSL??? ===== Miscellaneous ===== * Determining from **inside** the running Linux if you are using WSL (or another type of virtual machine)\\ $ systemd-detect-virt wsl ===== Other WSL resources and links ===== * [[https://docs.microsoft.com/en-us/windows/wsl/release-notes|Release Notes for Windows Subsystem for Linux]] * [[https://docs.microsoft.com/en-us/windows/wsl/faq|Frequently Asked Questions]] * [[https://docs.microsoft.com/en-us/windows/wsl/troubleshooting|Troubleshooting]] * [[https://docs.microsoft.com/en-us/windows/wsl/user-support|User Accounts and Permissions]] * [[https://devblogs.microsoft.com/commandline/|Windows Command Line Tools For Developers]] * Very useful Microsoft blogs about //Windows Console, Command-Line, Windows Subsystem for Linux, WSL, Linux// * [[https://github.com/Microsoft/WSL|WSL @ github]] * [[https://stackoverflow.com/questions/tagged/windows-subsystem-for-linux|WSL @ stackoverflow]] * [[https://wpdev.uservoice.com/forums/266908-command-prompt/filters/top|WSL @ UserVoice portal]] * [[https://github.com/sirredbeard/Awesome-WSL|Awesome-WSL]]: An Awesome collection of Windows Subsystem for Linux information, distributions, and tools * [[https://virtualizationreview.com/articles/2018/01/10/hands-on-with-wsl-installation-and-new-features.aspx|Hands-On with WSL: Installation & New Features]] (five nice and useful articles) * [[https://github.com/ethanhs/WSL-Programs|A list of which programs work...]] * [[https://doc.ubuntu-fr.org/wsl]] * [[https://github.com/UV-CDAT/uvcdat/wiki/CDAT-under-windows|CDAT under windows]] /* standard page footer */ \\ \\ \\ ---- [ [[pmip3:|PMIP3 Wiki Home]] ] - [ [[pmip3:wiki_help|Help!]] ] - [ [[wiki:syntax|Wiki syntax]] ]