User Tools

Site Tools


other:win10wsl

Windows Subsystem for Linux, and Ubuntu

What is described on this page was tested with WSL v1, and not WSL2 (that will be introduced mid-2019). It will be possible to install both WSL v1 and v2 on the same computer and WSL v2 will theoretically have faster I/O, but we recommend you use WSL v1 for now

Read just the top of the Windows Subsystem for Linux Documentation for a quick WSL description on the Microsoft site

In a few words, WSL is not Linux, it's more a way to share files between Windows and Linux (it's OK to skip this rather technical link!). That's why you need WSL and a Linux distribution.

Prerequisites

  • up-to-date Windows 10 64-bit: this should automatically be the case. This page is based on a laptop running the Fall (September) 2018 edition, and Microsoft is starting to push the March 2019 (aka 1903) edition
  • Enough space on C:. Microsoft instructions say The Windows Subsystem for Linux only runs on your system drive (usually this is your C: drive)
    • The initial installation of Ubuntu does not seem to use much disk space (< 1 Gb), but you will get a basic Linux with no graphics, and you may need space for installing more applications
    • / and /home will be on C:, but the data files can be on any disk/partition (or external drive) of the computer, and all the partitions will be available and accessible in /mnt/, so you can store data on other partitions
    • The files will be installed in hidden partitions of the current user's directory on C:: e.g. C:\Users\johndoe. Other users of the same Windows 10 computer would have to install Ubuntu again, in their own user directory
      • Checking the properties of C:\Users\johndoe will not report an accurate size and number of files, because lots of files and directories are hidden! On the test laptop, the reported size of the user backed up directory (with SyncBackFree) was ~20 Gb (user data, WSL+Ubuntu, custom install of the CDAT 8.1 python distribution)
      • Be sure to monitor the remaining free space on C:!

Installation

After the installation

Once you have completed the installation steps, you'll get a Ubuntu machine, with a root and user account (preconfigured to use sudo) and lots of pre-installed applications (but no graphics).

Opening a terminal

  • Opening the first terminal will start WSL+Ubuntu
  • Closing all terminals will not stop WSL+Ubuntu. Read the rebooting WSL section if you need a clean start without rebooting Windows

Having a terminal you can efficiently interact with is the basis of your future work with WSL+Ubuntu! You can optionally read the ConEmu vs Hyper vs Terminus vs MobaXTerm Terminator vs Ubuntu WSL blog if you want more details

The default terminal

The Default WSL+Ubuntu terminal is basically a bash shell running inside a regular Windows cmd window. You do not need to have an X server running to use it

There are several ways to open a default terminal:

  • select Ubuntu from the Start menu
    You should also add the Ubuntu icon to the task bar
    • right-click on the Ubuntu icon (in the Start menu) → More → Pin to the task bar
  • click on the Ubuntu icon on the task bar
    • to open more windows: right-click on the Ubuntu icon on the task bar → Ubuntu
  • type wsl+<Return> in the (Cortana) Search field on the task bar
Copy/paste in the default terminal

Follow the steps in Copy and Paste arrives for Linux/WSL Consoles and you will then be able to copy/paste by using Shift+Ctrl+C/V

  • The QuickEdit mode should already be selected by default and you can also:
    • select text with the left mouse button
    • paste the selected text with the right mouse button (click twice after a new selection)
  • Use Alt+<Return> to toggle the terminal to full screen (maybe more useful options here?)

The WSLtty terminal

WSLtty allows you to use Mintty as a terminal for WSL. You do not need to have an X server running to use it

  • Download and execute the latest installer (the most recent wsltty-*-install.exe file)
  • You will then have different ways to start a terminal:
    • WSL terminal shorcut on the desktop
    • Ubuntu terminal and WSL Terminal in the Start menu
  • Open a terminal and right-click in it to discover/use the options
    • More options with CTRL+right-click
    • You can cycle between the open WSLtty terminals with CTRL+Tab and Shift+CTRL+Tab
  • Technical stuff (on the Mintty wiki)

Terminals started from Linux

You can install additional graphical terminals inside the Linux running in WSL, but you will need to have a running X server if you want to open them. In these terminals, the usual Linux copy/paste rules will apply!

xterm
  • Install with: apt-get install xterm
  • Start from another terminal with: xterm &
xfce-terminal
  • Install with: apt-get install xfce4-terminal
  • Start from another terminal with: xfce4-terminal &

Location of the files

You can optionally read the detailed WSL File System Support page if you want to (try to) understand how things work and how both systems can share files.

Linux files seen from Windows

The content of the Linux / directory is actually located in the following hidden sub-directory of the user's Windows home directory: C:\Users\johndoe\AppData\Local\Packages\CanonicalGroupLimited.UbuntuonWindows_79rhkp1fndgsc\LocalState\rootfs

Warning! Do not change Linux files using Windows apps and Tools! If you need to update files from both Linux and Windows, put them in a regular Windows directory (not in the hidden directory mentioned above, where the Linux system is installed)

Windows files seen from Linux

The Windows partitions are mounted under /mnt in Linux:

$ df -h
Filesystem      Size  Used Avail Use% Mounted on
rootfs          111G   30G   82G  27% /
none            111G   30G   82G  27% /dev
none            111G   30G   82G  27% /run
none            111G   30G   82G  27% /run/lock
none            111G   30G   82G  27% /run/shm
none            111G   30G   82G  27% /run/user
C:              111G   30G   82G  27% /mnt/c
H:              366G  138M  366G   1% /mnt/h

Read the Mounting disks and network drives section if you want to access remote server disks on your local network, or just wan to access USB disks/sticks on your local computer.

Misc

  • The wslpath command can be used to make Windows↔Linux path conversion

Linux version installed

$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 16.04.3 LTS
Release:        16.04
Codename:       xenial

$ cat /proc/version
Linux version 4.4.0-43-Microsoft (Microsoft@Microsoft.com) (gcc version 5.4.0 (GCC) ) #1-Microsoft Wed Dec 31 14:42:53 PST 2014

$ uname -a
Linux DESKTOP-QNS0KAA 4.4.0-43-Microsoft #1-Microsoft Wed Dec 31 14:42:53 PST 2014 x86_64 x86_64 x86_64 GNU/Linux

Ubuntu packages installed

$ dpkg --list
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name                                                  Version                         Architecture                    Description
+++-=====================================================-===============================-===============================-=======================================================
ii  accountsservice                                       0.6.40-2ubuntu11.3              amd64                           query and manipulate user account information
ii  acl                                                   2.2.52-3                        amd64                           Access control list utilities
ii  acpid                                                 1:2.0.26-1ubuntu2               amd64                           Advanced Configuration and Power Interface event daemon
ii  adduser                                               3.113+nmu3ubuntu4               all                             add and remove users and groups
[...]
ii  xml-core                                              0.13+nmu2                       all                             XML infrastructure and XML catalog file support
ii  xz-utils                                              5.1.1alpha+20120614-2ubuntu2    amd64                           XZ-format compression utilities
ii  zerofree                                              1.0.3-1                         amd64                           zero free blocks from ext2, ext3 and ext4 file-systems
ii  zlib1g:amd64                                          1:1.2.8.dfsg-2ubuntu4.1         amd64                           compression library - runtime

$ dpkg --list > /mnt/h/Scratch/jyp/pkg_initial.txt

$ wc -l /mnt/h/Scratch/jyp/pkg_initial.txt
437 /mnt/h/Scratch/jyp/pkg_initial.txt

Full list of installed packages: pkg_initial.txt

Update ubuntu

Use the following to update your ubuntu installation:

$ sudo -s
$ apt-get update
$ apt-get upgrade

Managing WSL

This is just place to store some technical information, that will probably not be needed by most people

Getting ready for some real work

We often assume below that you are using the root account (sudo -s), or sudo when executing the apt* installation commands, and othe admin commands

Package management

apt-get tutorial if you are more used to Fedora/RedHat yum/dnf and rpm packages

  • Install the aptitude package manager: apt-get install aptitude
  • Install the synaptic package manager: aptitude install synaptic

Useful commands

  • dpkg -l: list installed packages
  • dpkg -L package_name: list files installed from package_name
  • dpkg -S path/command: name of the package used to install path/command

Using an X server

You will need to have an X server running locally on your Windows 10 machine if you want to display graphics generated by the Linux running locally, or on a remote server. Applications running in text mode only (e.g. vi) do not require a running X server.

You will get a Can't open display error if no server is running (even if the DISPLAY variable is correctly defined)

your_login@your_machine:~$ echo $DISPLAY
localhost:0.0
your_login@your_machine:~$ xterm &
xterm: Xt error: Can't open display: localhost:0.0
[Everything is fine after starting a local X server]
jyour_login@your_machine:~$ xterm &
[1] 55

Installing an X server

After the X server is installed and running, some graphical applications and images (or the text displayed by those applications) may appear blurry. This is due to some servers or some Linux applications not handling high resolution (HiDPI) screens correctly

X410 has several options for this (see Running X410 on HiDPI Screens) but it may also work correctly with non commercial applications

Installing x410

Type: commercial

x410 is a low cost commercial X server that can be directly installed from the Microsoft store. Once installed, it will be automatically updated by the Microsoft Store application.

Starting, configuring and exiting X410: you can start X410 by clicking on it in the Windows Start Menu, or choose to Automatically Start X410 on Login

  • Note: nothing actually happens when you start X410, as it is just waiting to display graphics! You will just get a new X icon in the hidden icons of the taskbar.
  • You can configure (and exit) X410 by right clicking on its X icon
Installing VcXrsv

Type: free

Download the installer from VcXrsv

Installing MobaXterm

Type: commercial

Available at MobaXterm, not tested. This seems to be more than just an X server, and may be an overkill if you just need an X server

Configuration (in your local Linux account)

  • Set the DISPLAY variable in the shell (by hand, or in the shell init file)
    • export DISPLAY=localhost:0.0
    • Warning! export DISPLAY=:0.0 will also work for using graphical windows locally, but will fail when trying to open a window on a remote computer, with a connect /tmp/.X11-unix/X0: No such file or directory error message (source)
  • Need to determine if/when export LIBGL_ALWAYS_INDIRECT=1 is useful
    • it breaks glxgears with x410

Resources

Adding useful packages

Reminder: you need to use the root account (with sudo -s) in order to use the following commands

$ apt-get install gcc libglu1-mesa mesa-utils libsm6 x11-apps

# Check the "Terminals started from Linux" section
# if you need to install extra terminals

# Install emacs
$ apt-get install emacs25

# Install pdf (evince) and image (eog) viewers
$ apt-get install evince eog

# Install wslu to improve the Windows<->Linux communication
$ apt-get install wslu

# The packages below are useful for dealing with climate data
# in netCDF files (and are also requirements of CLIMAF)
$ aptitude install imagemagick nco cdo netcdf-bin ncview exiv2

Updating WSL and ubuntu

WSL

WSL is automatically updated when Windows 10 is updated

WSL command line options

Updating ubuntu

The Ubuntu 16.04.3 LTS that was initially installed from the Windows Store when the test computer was running Win 10 1709 was automatically updated to Ubuntu 18.04.2 LTS when the computer was updated to Win 10 1809.

$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 18.04.2 LTS
Release:        18.04
Codename:       bionic

We also ran the do-release-upgrade command to complete the upgrade (from 16.04 to 18.04) [ More info from WSL team - more info from Ubuntu ]

And we then ran the usual upgrade commands. These commands can be run any time for updating the installed packages

$ sudo -s
$ apt-get update
$ apt-get upgrade

Full list of the resulting installed packages, generated with

$ dpkg --list >$ dpkg --list > /mnt/h/Scratch/jyp/pkg_190221.txt
$ wc -l /mnt/h/Scratch/jyp/pkg_190221.txt
578 /mnt/h/Scratch/jyp/pkg_190221.txt

Advanced configuration

Get configuration files

  • You can either use your existing Linux configuration files, or download some example files in your WSL home directory and use (and improve) them as directed (or as you wish)
    • $ cd
      $ scp -pr jypeter@ssh1.lsce.ipsl.fr:WSL_config .
    • You get a default .bashrc file in your home directory when your WSL account is created. Add the following line at the end of ~/.bashrc to use what is defined in the downloaded WSL_config directory:
      source ~/WSL_config/bashrc_extra.sh
  • You can either copy the full .ssh directory of your Linux account, or just get the private key files:

This assumes that you have already created a private/public ssh key pair and that the private key is available somewhere in the Windows 10 computer

ssh from WSL

Authorizing ssh to use old dsa keys

dsa ssh keys are disabled by default (becoming obsolete). ssh will ask for your password even if you have a ~/.ssh/id_dsa file. ssh -v will print Skipping ssh-dss key /home/jyp/.ssh/id_dsa - not in PubkeyAcceptedKeyTypes. The solution is to use the user config file to authorize dsa (source)

  • Add PubkeyAcceptedKeyTypes +ssh-dss to ~/.ssh/config
  • Make sure the config file has the correct access rights: chmod 600 config
Using Putty Pageant as an ssh agent

This assumes that you have converted your private ssh key with PuTTYgen and unlocked it in Pageant

  • Download the required binaries from the weasel-pageant github page and extract them to a Windows directory, e.g. C:\Utils\weasel-pageant-x.x
  • Add the following line to your ~/.bashrc file:
    eval $(C:\Utils\weasel-pageant-x.x/weasel-pageant -rb -a $HOME/.weasel-pageant.sock)
  • Open a new terminal:
    • Type ssh-add -l: it will display either The agent has no identities, or as many lines as you have entered keys in Pageant
    • Type ssh user@host to connect to a computer where you have your public key

Note: the other similar solutions are ssh-agent-wsl (the key is stored in the Windows ssh-agent instead of Pageant) or wsl-ssh-pageant

Using Ubuntu as an ssh-agent

We do not use this now, because there will be one running ssh-agent per terminal, and the key is stored only for the current terminal

  • It's not possible to add a key to ssh-agent. 'ssh-add' will print Could not open a connection to your authentication agent
    • type eval `ssh-agent -s` (or eval $(ssh-agent)) before using ssh-add
Notes & TODO

ssh to WSL

FIXME

Need to find a way to ssh from the Win10 part of the machine to the WSL part, and from outside the machine to the WSL part

It would probably already be enough to enable the ssh server on Win10 (the ssh 1709 release beta features are now officially in Win10)

Google search: wsl ssh

Creating shortcuts to Linux programs on the Windows desktop

  • Install wslu (A collection of utilities for WSL) if it is not already installed
    • aptget install wslu
  • Have a quick look at the wslu github page and wiki to find about the available tools (that you may need later)
  • Create a shortcut (e.g. to emacs) with: wslusc -g emacs
  • You can optionally get some information about the technical details

emacs configuration

  • Use the provided emacs configuration file (or your own):
    • $ cd
      $ ln -s WSL_config/emacs .emacs
  • Use wslusc to create a shortcut to emacs on your Windows desktop:
    wslusc -g emacs
    • This will make it easier to start emacs, but since this is a graphical Linux application, you need to have an X server running before you click on the shortcut!

Mounting disks and network drives

By default, the local disks and the external USB disks available (and decrypted!) when the first WSL terminal is opened are automatically available (mounted) in /mnt : the content of C:\ is available in /mnt/c, etc… The list is refreshed when Windows (or just WSL) is rebooted .

Extra local disks or network drives (what you can access by opening \\dfshost\dfs\ in the file explorer when you are on the LSCE network) can be either mounted on the fly with the mount command, or automatically by configuring the /etc/fstab file.

You will access the network drives with the access rights you have in the current Windows session (and you may have to access the drives in Windows before you can mount them in WSL)

Warning! In all cases, the owner, groups and permissions displayed may not be accurate (but the actual access rights will apply).

  • the user/group of the files may appear as your_login/your_login or root/root
  • the access rights may appear as rwxrwxrwx even if the actual rights are more restrictive
  • more details in the metadata section

Enabling metadata

Some access rights handling can be partially corrected by automatically enabling the metadata mount option. This will allow you to use chmod/chown on local NTFS disks (e.g. /mnt/c/Users/your_login). sudo -s and then create a /etc/wsl.conf with the following content:

[automount]
options = "metadata"

You need to restart WSL+ubuntu to activate the new content of /etc/wsl.conf. See below an example of the mount -l output:

  • Before: C: on /mnt/c type drvfs (rw,noatime,uid=1000,gid=1000,case=off)
  • After: C: on /mnt/c type drvfs (rw,noatime,uid=1000,gid=1000,metadata,case=off)

Optional: more details about wsl.conf and metadata and access rights management

On-the-fly mount

You have to create a mount point in the existing /mnt directory and then use the mount command

$ sudo -s

# Create the mount point (once)
$ mkdir /mnt/scratch01

# Mount the remote disk
$ mount -t drvfs '\\dfshost\dfs\scratch01' /mnt/scratch01

# Unmount the disk, or just shutdown Windows or WSL when you are finished
$ umount /mnt/scratch01

If you connect a USB disk/stick and windows sees it as a new J:\ drive, you can also manually mount it

$ sudo -s

# Create the mount point (once)
$ mkdir /mnt/j

# Mount the USB disk
$ mount -t drvfs J: /mnt/j

# Unmount the disk, or just shutdown Windows or WSL when you are finished
$ umount /mnt/j

Automatic mount

The disks defined in /etc/fstab will be automatically mounted when you start WSL and if they are actually accessible from your Windows session (e.g. \\dfshost\dfs can only be accessed from the LSCE network)

You need to add 1 line per mount point to /etc/fstab, and create (once) the directories that will be used for mounting the disks

$ sudo -s

# Create the mount points (once)
$ mkdir /mnt/home_jyp /mnt/scratch_jyp

# Make a backup copy of /etc/fstab
$ cp -p /etc/fstab /etc/fstab.bak

# Add lines to /etc/fstab associating mount points and directories
$ cat /etc/fstab
LABEL=cloudimg-rootfs   /        ext4   defaults        0 0
\\dfshost\dfs\users\jyp /mnt/home_jyp drvfs defaults 0 0
\\dfshost\dfs\scratch01\jyp /mnt/scratch_jyp drvfs defaults 0 0

# Try to mount all the disks in /etc/fstab
$ mount -a

Resources

Useful mount options:

  • mount -a (must be root): mount everything specified in /etc/fstab
  • umont /mnt/scratch01 (must be root): unmount the disk accessible through /mnt/scratch01
  • mount -l: list all the mounted disks and the mount options (useful when you are using the default mount options and want to know which options were actually applied)

Useful links (used for writing this section):

Rebooting WSL

It is not possible to reboot WSL like a regular Linux computer

  • shutdown -r now will generate an error message
  • WSL keeps on running even when all the terminals are closed

Rebooting the Windows computer will of course reboot WSL but it's useful to be able to just reboot WSL:

  • to force WSL to recognize new disk drives and make them available as /mnt/<new_drive_letter
  • to force WSL to recognize drives that were still encrypted when the first terminal was started

WSL can be rebooted (all the terminals and processed will be closed/killed) by restarting the WSL service (LxssManager)

  • from the Services tab of the Task manager or the Services program (launch services.msc)
    • Right-click on the LxssManager service and select Restart
  • from a Powershell in admin mode (right-click on the Windows Start button)
    • Type Get-Service LxssManager | Restart-Service

Source: google wsl lxssmanager (most usable answer)

Creating a backup

Best thing is probably to create tar file of the user's home directory, and get a list of installed applications and updated settings (e.g /etc/fstab)

Check How to backup settings and list of installed packages

Uninstalling WSL

Not too sure about this part…

  • Make a backup of whatever you had in your Ubuntu home directory, and all the files and settings that are not in the /mnt directories
  • Find how to unregister Ubuntu on the Manage and configure Windows Subsystem for Linux page
  • and then disable WSL???





[ PMIP3 Wiki Home ] - [ Help! ] - [ Wiki syntax ]

other/win10wsl.txt · Last modified: 2019/10/24 10:56 by jypeter