Table of Contents

Windows Subsystem for Linux (WSL), and Ubuntu

Other Windows 10 related notes

What is described on this page was tested with WSL v1, and not WSL 2

WSL 2 is available in Windows 10 v2004 (April 2020) and more recent, and should also be available in 1903 and 1909 versions

The user can choose to install Linux using WSL1 or WSL2. Check the differences between WSL1 and WSL2 before choosing

What is WSL ?

Useful reading:

In a few words, WSL is not Linux, it's more a way to share files between Windows and Linux (it's OK to skip this rather technical link!). Files under the Linux root (i.e. /) are controlled by WSL which mimics Linux specific behavior [...follow the link for details...]. That's why you need to activate WSL and then install a Linux distribution.

Prerequisites

Base installation

After the base installation

Once you have completed the base installation steps, you'll get a Ubuntu machine, with a root and user account (pre-configured to use sudo for commands that need root access) and lots of pre-installed applications (but no graphics).

Opening a terminal

Having a terminal you can efficiently interact with is the basis of your future work with WSL+Ubuntu! You can optionally read the ConEmu vs Hyper vs Terminus vs MobaXTerm Terminator vs Ubuntu WSL blog if you really want more details

The default terminal

The Default WSL+Ubuntu terminal is basically a bash shell running inside a regular Windows cmd text console. You do not need to have an X server running to use it!

There are several ways to open a default terminal:

Copy/paste in the default terminal

Follow the steps in Copy and Paste arrives for Linux/WSL Consoles and you will then be able to copy/paste by using Shift+Ctrl+C/V

The WSLtty terminal

  • This is the recommended (by JYP) option for now
  • If you can't open a terminal after Windows upgrades from WSL to WSL v2, get and install the latest version of WSLtty !

WSLtty allows you to use the efficient Mintty as a terminal for WSL. You do not need to have an X server running to use it!

The new Windows Terminal

Microsoft is working on a powerful replacement of the old cmd terminal! Check Windows Terminal, Console and Command-Line @ github

Note: the Windows Terminal looks really promising, but will be nice when there is a GUI for editing the options (rather the editing a json file with a text editor…) and a way to select xterm-style copy-pasting. See Feature request: add xterm-style select/copy/paste options

Terminals started from Linux

You can install additional graphical terminals inside the Linux running in WSL, but you will need to have a running X server if you want to open them. In these terminals, the usual Linux copy/paste rules will apply!

xterm
xfce-terminal

Location of the files

You can optionally read the detailed WSL File System Support page if you want to (try to) understand how things work and how both systems can share files.

Linux files seen from Windows

Do not change Linux files in '/' from Windows! (it's safe only if you use the \\wsl$ path)

If you need to easily update files from both Linux and Windows, put them in a regular Windows directory (not in the hidden directory mentioned above, where the Linux system is installed)

Windows files seen from Linux

Note: When you are in an explorer window, you can open a Windows terminal with Linux by holding Shift and right-clicking on a directory and choosing Linux in the context menu.

The Windows partitions are mounted under /mnt in Linux: 

$ df -h
Filesystem      Size  Used Avail Use% Mounted on
rootfs          111G   30G   82G  27% /
none            111G   30G   82G  27% /dev
none            111G   30G   82G  27% /run
none            111G   30G   82G  27% /run/lock
none            111G   30G   82G  27% /run/shm
none            111G   30G   82G  27% /run/user
C:              111G   30G   82G  27% /mnt/c
H:              366G  138M  366G   1% /mnt/h

Read the Mounting disks and network drives section if you want to access remote server disks on your local network, or just want to access USB disks/sticks on your local computer.

Misc

Linux version installed

$ date
Thu Feb 20 11:18:57 CET 2020

$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 18.04.4 LTS
Release:        18.04
Codename:       bionic

$ cat /proc/version
Linux version 4.4.0-18362-Microsoft (Microsoft@Microsoft.com) (gcc version 5.4.0 (GCC) ) #476-Microsoft Fri Nov 01 16:53:00 PST 2019

$ uname -a
Linux some_machine_name 4.4.0-18362-Microsoft #476-Microsoft Fri Nov 01 16:53:00 PST 2019 x86_64 x86_64 x86_64 GNU/Linux

Ubuntu packages installed

$ dpkg --list
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name                  Version         Architecture    Description
+++-=====================-===============-===============-================================================
ii  accountsservice       0.6.45-1ubuntu1 amd64           query and manipulate user account information
ii  acl                   2.2.52-3build1  amd64           Access control list utilities
ii  acpid                 1:2.0.28-1ubunt amd64           Advanced Configuration and Power Interface event
ii  adduser               3.116ubuntu1    all             add and remove users and groups

[...]
ii  xterm                 330-1ubuntu2    amd64           X terminal emulator
ii  xxd                   2:8.0.1453-1ubu amd64           tool to make (or reverse) a hex dump
ii  xz-utils              5.2.2-1.3       amd64           XZ-format compression utilities
ii  zerofree              1.0.4-1         amd64           zero free blocks from ext2, ext3 and ext4 file-s
ii  zlib1g:amd64          1:1.2.11.dfsg-0 amd64           compression library - runtime


$ dpkg --list > /mnt/c/Scratch/jyp/pkg_initial_201601.txt
$ wc -l /mnt/c/Scratch/jyp/pkg_initial_201601.txt
628 /mnt/c/Scratch/jypeter/pkg_initial_201601.txt

Full (example) list of installed packages: pkg_initial_201601.txt

Update ubuntu

Use the following to update your ubuntu installation: 

$ sudo -s
$ apt-get update
$ apt-get upgrade

Managing WSL

This is just place to store some technical information, that will probably not be needed by most people

Getting ready for some real work

We often assume below that you are using the root account (sudo -s), or sudo when executing the apt* installation commands, and othe admin commands

Specifying correctly the display language

WSL will automatically try to use in the terminals the same language as the one used by the Windows 10 computer (ref), but this could lead later to some difficult to debug side-effects when you connect to remote Linux servers (e.g. the sort command may give different results !).

It is safer to force the system to use a standard English version 

$ sudo update-locale LANG=en_US.UTF8
$ echo $LANG
en_US.UTF8

Package management

apt-get tutorial if you are more used to Fedora/RedHat yum/dnf and rpm packages

Useful commands

Example

$ dpkg -l | grep -i wsl
ii  ubuntu-wsl                      1.417.3                            amd64        Ubuntu on Windows tools - Windows Subsystem for Linux integration
ii  wslu                            2.3.2-0ubuntu2~18.04.3             all          collection of utilities for the Windows 10 Linux Subsystem

$ dpkg -L wslu
/.
/etc
/etc/profile.d
/usr
/usr/bin
/usr/bin/wslfetch
/usr/bin/wslsys
/usr/bin/wslupath
/usr/bin/wslusc
/usr/bin/wslvar
/usr/bin/wslview
[...]

Using an X server

You will need to have an X server running locally on your Windows 10 machine if you want to display graphics generated by the Linux running locally, or on a remote server. Applications running in text mode only (e.g. vi) do not require a running X server.

You will get a Can't open display error if no server is running (even if the DISPLAY variable is correctly defined) 

your_login@your_machine:~$ echo $DISPLAY
localhost:0.0
your_login@your_machine:~$ xterm &
xterm: Xt error: Can't open display: localhost:0.0
[Everything is fine after starting a local X server]
jyour_login@your_machine:~$ xterm &
[1] 55

Installing an X server

Read the Installing and configuring an X server page

Configuration (in your local Linux account)

Adding useful packages

Reminder: you need to use the root account (with sudo -s) in order to use the following commands 

$ apt-get install gcc libglu1-mesa mesa-utils libsm6 x11-apps

# Check the "Terminals started from Linux" section
# if you need to install extra terminals

# Install emacs
$ apt-get install emacs

# Install pdf (evince) and image (eog) viewers
$ apt-get install evince eog

# Install wslu to improve the Windows<->Linux communication
$ apt-get install wslu

# The packages below are useful for dealing with climate data
# in netCDF files (and are also requirements of CLIMAF)
$ aptitude install imagemagick nco cdo netcdf-bin ncview exiv2

Updating WSL and ubuntu

You don't have to worry about the Upgrade section if you have just installed WSL+Ubuntu, because you are already using the latest Ubuntu stable release!

WSL

Nothing to do! WSL is automatically updated, provided you have enabled the Get updates for other Microsoft products option

WSL command line options

Updating ubuntu

The Ubuntu 16.04.3 LTS that was initially installed from the Windows Store when the test computer was running Win 10 1709 was automatically updated to Ubuntu 18.04.2 LTS when the computer was updated to Win 10 1809. 

$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 18.04.2 LTS
Release:        18.04
Codename:       bionic

We also ran the do-release-upgrade command to complete the upgrade (from 16.04 to 18.04) [ More info from WSL team - more info from Ubuntu ]

And we then ran the usual upgrade commands. These commands can be run any time for updating the installed packages 

$ sudo -s
$ apt-get update
$ apt-get upgrade

Full list of the resulting installed packages, generated with 

$ dpkg --list >$ dpkg --list > /mnt/h/Scratch/jyp/pkg_190221.txt
$ wc -l /mnt/h/Scratch/jyp/pkg_190221.txt
578 /mnt/h/Scratch/jyp/pkg_190221.txt

Upgrading ubuntu

You can skip this section if you have just installed ubuntu, since you are already running the latest stable release available

The updates are for maintaining the same version of ubuntu (e.g. you go from version 18.04.2 to 18.04.5), while upgrades are for going from one major version to the next (e.g 16.xx ⇒ 18.xx ⇒ 20.xx). We try to only use the stable major releases, that have an even version number.

We use the LTS versions (Long Time Support) to ensure even more stability, which means that we will get updates of a given major release for quite some time after the next major release is available, so it is not mandatory to updgrade ubuntu if you are still receiving updates for the release you use

There should not be any risk, but it may be a good idea to create a backup of your current ubuntu before upgrading it!

Use sudo for each command below, or just become root by using sudo su -

Cleaning up things

* How to Clear Apt Cache

Advanced configuration

Get Linux configuration files

This assumes that you have already created a private/public ssh key pair and that the private key is available somewhere in the Windows 10 computer

ssh from WSL

Authorizing ssh to use old dsa keys

dsa ssh keys are disabled by default (becoming obsolete). ssh will ask for your password even if you have a ~/.ssh/id_dsa file. ssh -v will print Skipping ssh-dss key /home/<your_login>/.ssh/id_dsa - not in PubkeyAcceptedKeyTypes. The solution is to generate a new couple of public/private keys not based on dsa encryption, or to use the user ssh config file to authorize dsa (source)

Using Putty Pageant as an ssh agent

This assumes that you have installed the full Putty package (with the msi installer), converted your private ssh key with PuTTYgen, and unlocked the converted ssh private key in Pageant.

The converted ssh private key is located in a Windows 10 folder

Note: the other similar solutions are ssh-agent-wsl (the key is stored in the Windows ssh-agent instead of Pageant) or wsl-ssh-pageant

Using Ubuntu as an ssh-agent

The ssh key (and other ssh settings) are in the ~/.ssh directory of your WSL+Ubuntu account

This is the most simple setting, but there will be one running ssh-agent per terminal, and the key will be stored independently for each terminal (which means that you will have to type ssh-add and your pass-phrase for each terminal)

Notes & TODO

ssh to WSL

FIXME Maybe check OpenSSH-Server connection issues

Maybe it could be useful to do a ssh from the Win10 part of the machine to the WSL part, and from outside the machine to the WSL part

It would probably already be enough to enable the ssh server on Win10 (the ssh 1709 release beta features are now officially in Win10)

Google search: wsl ssh

Creating shortcuts to Linux programs on the Windows desktop

Mounting disks and network drives

By default, the local disks and the external USB disks available (and decrypted!) when WSL is started (i.e. when the first WSL terminal is opened) are automatically available (mounted) in /mnt : the content of C:\ is available in /mnt/c, etc… The list of available disks is refreshed when the Windows session is restarted (the user logs out and in, or Windows is rebooted) or when just WSL is restarted.

Extra local disks or network drives (what you can access by opening \\dfshost\dfs\ in the file explorer when you are on the LSCE network) can be either mounted on the fly with the mount command, or automatically by configuring the /etc/fstab file.

You will access the network drives with the access rights you have in the current Windows session (and you may have to access the drives in Windows before you can mount them in WSL)

Warning! In all cases, the owner, groups and permissions displayed may not be accurate (but the actual access rights will apply).

Enabling metadata

Some access rights handling can be slightly improved by automatically enabling the metadata mount option. This will allow you to use chmod/chown on local NTFS disks (disks directly connected to your computer, e.g. /mnt/c/Users/your_login).

  1. become root in a terminal with: sudo -s
  2. create a /etc/wsl.conf with the following content:
    [automount]
options = "metadata"

You need to restart WSL+ubuntu to activate the new content of /etc/wsl.conf. See below an example of the mount -l output:

Optional: more details about wsl.conf in WSL launch settings, How to use wsl.conf and metadata and access rights management

On-the-fly mount

You have to create a mount point in the existing /mnt directory and then use the mount command 

# Must be root for most 'mount' related commands
$ sudo -s

# Create the mount point (once), e.g. for accessing /home/scratch01 on the local LSCE servers
# This works when the Windows 10 computer is on the LSCE network
$ mkdir /mnt/scratch01

# Mount the remote disk
$ mount -t drvfs '\\dfshost\dfs\scratch01' /mnt/scratch01

# List the mounted disks
$ mount -l
[...]
\\dfshost\dfs\scratch01 on /mnt/scratch01 type drvfs (rw,relatime,case=off)

# Unmount the disk, or just shutdown Windows or WSL when you are finished
$ umount /mnt/scratch01

If you connect a USB disk/stick and windows sees it as a new J:\ drive, you can also manually mount it 

$ sudo -s

# Create the mount point (once)
$ mkdir /mnt/j

# Mount the USB disk
$ mount -t drvfs J: /mnt/j

# Unmount the disk, or just shutdown Windows or WSL when you are finished
$ umount /mnt/j

Automatic mount

The disks defined in /etc/fstab will be automatically mounted when you start WSL and if they are actually accessible from your Windows session (e.g. \\dfshost\dfs can only be accessed from the LSCE network)

You need to add 1 line per mount point to /etc/fstab, and create (once) the directories that will be used for mounting the disks 

$ sudo -s

# Create the mount points (once)
# The example below is to access the LSCE /home/users and /home/scrath01 disks
$ mkdir /mnt/users /mnt/scratch01

# Make a backup copy of /etc/fstab before modifying it (for safety reasons!)
$ cp -p /etc/fstab /etc/fstab.bak

# Add lines to /etc/fstab associating mount points and directories
$ cat /etc/fstab
LABEL=cloudimg-rootfs   /        ext4   defaults        0 0
\\dfshost\dfs\users /mnt/users drvfs defaults 0 0
\\dfshost\dfs\scratch01 /mnt/scratch01 drvfs defaults 0 0

# Try to mount all the disks in /etc/fstab
$ mount -a

# Check the result
$ mount -l
[...]
\\dfshost\dfs\users on /mnt/users type drvfs (rw,relatime,case=off)
\\dfshost\dfs\scratch01 on /mnt/scratch01 type drvfs (rw,relatime,case=off)

Accessing remote file systems with sshfs

FIXME

It seems this can be done with a combination of WinFsp and SSHFS-Win (see SSHFS remote directory mounting syntax for details

Extra resources

Useful mount options:

Useful links (used for writing this section):

Rebooting WSL

It is not possible to stop or reboot WSL like a regular Linux computer

Shutting down or rebooting the Windows computer will of course stop WSL but it's useful to be able to just stop/restart WSL in order to:

WSL can be stopped the following way. Warning: ​all the open terminals and running processes ​will be terminated!

Creating a backup of the Linux running in WSL

There are several types of WSL related directories that you may want to back up:

Restoring a WSL backup

FIXME Read How to Export and Import Your Linux Systems on Windows 10 and Export and Import WSL Linux Distro in Windows 10 and similar threads

FIXME Check if WSL Manager is maintained and can be useful

This assumes that you have followed the steps above to create a full backup of your Linux distribution in a ubuntu_<your_login>_<some_date>.tar file. This file can theoretically be restored with:

Uninstalling WSL

Not too sure about this part…

Miscellaneous




[ PMIP3 Wiki Home ] - [ Help! ] - [ Wiki syntax ]