Return to site

Docker Slow Ext4 Partition

broken image


Unregister docker-desktop-data from wsl, note that after this, your ext4.vhdx file would automatically be removed (so back it up first if you have important existing image/container): wsl -unregister docker-desktop-data Import the docker-desktop-data back to wsl, but now the ext4.vhdx would reside in different drive/directory. Right-click on the EXT4 partition and select 'Properties'. Open and expand the folders on the left pane to check the content of EXT4 partition. If the Linux EXT4 partition contains some valuable files, move to the next phase, and you'll learn how to access and regain data from a Linux partition on Windows.

This is going to be a multipart-series of how to setup the ULTIMATE WSL2 and Docker environment, to get the most out of it, during your everyday developer’s life.

Now, let’s first tackle the meaning of ULTIMATE, since everyone has it’s own interpretation on what it means. S

Docker slow ext4 partition recovery
Docker slow ext4 partition software
Docker slow ext4 partition recovery
  • Reliable – The setup has to be bulletproof, in case of system failures or data loss, and it has to be organized, so that everything is on it’s proper place.
  • Fast – Working on projects, especially web projects, where you may have to reload a site multiple times to see changes, has to perform like a bare-metal setup
  • Convenient – There’s nothing more frustrating than having to turn knobs across multiple places, to get the most basic things to work.

Now that we’re on the same page about the meaning of ULTIMATE, let’s get started!

The problem with the initial WSL2 setup

Let’s start this series off with the proper setup of WSL2.

I assume that you already have a successfully running WSL2 distro on your Windows machine. If you installed the distro from the Microsoft Store, it’s probably located somewhere in your AppData folder, God knows where.

Also, you may be working on your fancy NodeJS server, which is probably located under /mnt/c/Users/<username>/Projects/NodeJS/my-most-paying-client or even /mnt/d if you’re on the secure side and have your sensitive data NOT stored on C: .

So, there are two problems here.

First, remember when I said it has to be reliable. Well, that’s absolutely not the case here. Since Windows can crash, and your AppData folder cannot be moved outside of C:Usersusername , you’re most likely to lose all your hard-efforts you put into setting up your distro.

The second issue is speed. As you probably may know (or may not know): to get the best performance out of WSL2 (which is near bare-metal) you MUST store your project-files inside WSL2, like /home/<username> .

The ULTIMATE solution

Fortunately we can tackle both of these problems with some setup.

Recently, WSL has introduced a nice feature --mount, which allows you to mount ext4 disk partitions into WSL and access them natively like you would do with any other ext4 partition. This also comes with a huge performance benefit, eliminating the need of the slow P9 filesystem.

  • Create an empty vhd-file and format it as ext4
  • Mount the vhd-file into WSL as a partition
  • Move all projects into that partition
  • Symlink the partition as our /home/<username> folder

1. Creating the virtual hard disk (VHD)

First you need to create a virtual disk. You can easily do that with Windows Disk Management. Make sure to create a dynamically expanding disk, which will slowly grow in size as you copy files to it.

If you’re a power-user, you probably have a separate partition for your projects, images, etc. Make sure to save the newly created virtual disk somewhere save, in case of a system crash. Also make sure to include it into your backup routine.

2. Formatting VHD to ext4

There are many ways on how to format a disk to ext4 under Windows. The easiest I’ve found so far, is AOMEI Partition Assistant which comes with a free trial, and is enough for this purpose (not sponsored).

Simply open up AOMEI Partition Assistant and create an ext4 partition on your newly created and mounted disk.

3. Mounting VHD into WSL2

Now you should be able to access the disk under /mnt/wsl/dataDisk and see a lost+found folder in there.

You may also want to save this script somewhere on your computer and run it with Task Scheduler every time you login into your PC.

Simply add this line into the action of Start a program :

-ExecutionPolicy Bypass -File D:path_tovhd.ps1

4. Migrating user data to VHD

Let’s create our user folder there, move all our /home/<username>/* files into the new folder and symlink it.

The ultimate bonus

Now that you have migrated your user data to the VHD, let’s tackle the convenient part of the ULTIMATE setup. This allows you to switch your Linux distros as you wish, while keeping all your data in one central place.

You want to run Kali alongside Ubuntu while still being able to access your user data? This comes in handy, if sometimes things don’t work as expected and you want to make sure if the problem is in Ubuntu or elsewhere. Just make sure to have the same username as you have on Ubuntu and execute these commands.

I hope you learned something, and stay tuned for the next part, where I’ll cover the proper setup of Docker inside WSL2.

Part 2

Continue to the second part:

Slow

Docker Slow Ext4 Partition Download

-->

If you want to access a Linux disk format that isn't supported by Windows, you can use WSL 2 to mount your disk and access its content. This tutorial will cover the steps to identify the disk and partition to attach to WSL2, how to mount them, and how to access them.

Note Access 2010 for mac free download.

Administrator access is required to attach a disk to WSL 2.The WSL 2 mount command does not support mounting a disk (or partitions that belong to the disk) that is currently in use. wsl --mount always attaches the entire disk even if only a partition is requested. You can't mount the Windows installation disk.

Prerequisites

You will need to be on Windows 10 Build 20211 or higher to access this feature. You can join the Windows Insiders Program to get the latest preview builds.

Mounting an unpartitioned disk

In this simplest case, if you have a disk that doesn't have any partitions, you can mount it directly using the wsl --mount command. First you need to identify the disk.

Identify the disk

To list the available disks in Windows, run:

The disks paths are available under the 'DeviceID' columns. Usually under the .PHYSICALDRIVE* format.

Mount the disk

Then in Powershell you can mount the disk using the Disk path discovered above.

Mounting a partitioned disk

If you have a disk that you aren't sure what file format it is in, or what partitions it has, you can follow the steps below to mount it.

Identify the disk

To list the available disks in Windows, run:

The disks paths are listed after 'DeviceID', usually in the .PHYSICALDRIVE* format

List and select the partitions to mount in WSL 2

Once the disk is identified, run:

This will make the disk available in WSL 2. (In the case of our example, the <DiskPath> is .PHYSICALDRIVE*.

Once attached, the partition can be listed by running the following command inside WSL 2:

This will display the available block devices and their partitions.

Inside Linux, a block device is identified as /dev/<Device><Partition>. For example, /dev/sdb3, is the partition number 3 of disk sdb.

Example output:

Identifying the filesystem type

If you don't know the type of filesystem of a disk or partition, you can use this command:

This will output the detected filesystem type (under the TYPE='<Filesystem>' format).

Mount the selected partitions

Once you have identified the partitions you want to mount, run this command on each partition:

Note

If you wish to mount the entire disk as a single volume (i.e. if the disk isn't partitioned), --partition can be omitted.

If omitted, the default filesystem type is 'ext4'.

Access the disk content

Once mounted, the disk can be accessed under the path pointed to by the config value: automount.root. The default value is /mnt/wsl.

From Windows, the disk can be accessed from File Explorer by navigating to: wsl$<Distro><Mountpoint> (pick any Linux distribution).

Unmount the disk

If you want to unmount and detach the disk from WSL 2, run:

Docker Slow Ext4 Partition Windows 10

Mount a VHD in WSL

You can also mount virtual hard disk files (VHD) into WSL using wsl --mount. To do this, you first need to mount the VHD into Windows using the Mount-VHD command in Windows. Be sure to run this command with administrator privileges. Below is an example where we use this command, and also output the disk path. Be sure to replace <pathToVHD> with your actual VHD path.

You can use the output above to obtain the disk path for this VHD and mount that into WSL following the instructions in the previous section.

You can also use this technique to mount and interact with the virtual hard disks of other WSL distros, as each WSL 2 distro is stored via a virtual hard disk file called: ext4.vhdx. By default the VHDs for WSL 2 distros are stored in this path: C:Users[user]AppDataLocalPackages[distro]LocalState[distroPackageName], please exercise caution accessing these system files, this is a power user workflow. Make sure to run wsl --shutdown before interacting with this disk to ensure the disk is not in use.

Command line reference

Mounting a specific filesystem

By default, WSL 2 will attempt to mount the device as ext4. To specify another filesystem, run:

For example, to mount a disk as fat, run:

Note

To list the available filesystems in WSL2, run: cat /proc/filesystems
When a disk has been mounted via WSL2 (Linux file system), it is no longer available to mount via an ext4 driver on the Windows file system.

Mounting a specific partition

By default, WSL 2 attempts to mount the entire disk. To mount a specific partition, run:

This only works if the disk is either MBR (Master Boot Record) or GPT (GUID Partition Table). Read about partition styles - MBR and GPT.

Docker slow ext4 partition tool
  • Reliable – The setup has to be bulletproof, in case of system failures or data loss, and it has to be organized, so that everything is on it’s proper place.
  • Fast – Working on projects, especially web projects, where you may have to reload a site multiple times to see changes, has to perform like a bare-metal setup
  • Convenient – There’s nothing more frustrating than having to turn knobs across multiple places, to get the most basic things to work.

Now that we’re on the same page about the meaning of ULTIMATE, let’s get started!

The problem with the initial WSL2 setup

Let’s start this series off with the proper setup of WSL2.

I assume that you already have a successfully running WSL2 distro on your Windows machine. If you installed the distro from the Microsoft Store, it’s probably located somewhere in your AppData folder, God knows where.

Also, you may be working on your fancy NodeJS server, which is probably located under /mnt/c/Users/<username>/Projects/NodeJS/my-most-paying-client or even /mnt/d if you’re on the secure side and have your sensitive data NOT stored on C: .

So, there are two problems here.

First, remember when I said it has to be reliable. Well, that’s absolutely not the case here. Since Windows can crash, and your AppData folder cannot be moved outside of C:Usersusername , you’re most likely to lose all your hard-efforts you put into setting up your distro.

The second issue is speed. As you probably may know (or may not know): to get the best performance out of WSL2 (which is near bare-metal) you MUST store your project-files inside WSL2, like /home/<username> .

The ULTIMATE solution

Fortunately we can tackle both of these problems with some setup.

Recently, WSL has introduced a nice feature --mount, which allows you to mount ext4 disk partitions into WSL and access them natively like you would do with any other ext4 partition. This also comes with a huge performance benefit, eliminating the need of the slow P9 filesystem.

  • Create an empty vhd-file and format it as ext4
  • Mount the vhd-file into WSL as a partition
  • Move all projects into that partition
  • Symlink the partition as our /home/<username> folder

1. Creating the virtual hard disk (VHD)

First you need to create a virtual disk. You can easily do that with Windows Disk Management. Make sure to create a dynamically expanding disk, which will slowly grow in size as you copy files to it.

If you’re a power-user, you probably have a separate partition for your projects, images, etc. Make sure to save the newly created virtual disk somewhere save, in case of a system crash. Also make sure to include it into your backup routine.

2. Formatting VHD to ext4

There are many ways on how to format a disk to ext4 under Windows. The easiest I’ve found so far, is AOMEI Partition Assistant which comes with a free trial, and is enough for this purpose (not sponsored).

Simply open up AOMEI Partition Assistant and create an ext4 partition on your newly created and mounted disk.

3. Mounting VHD into WSL2

Now you should be able to access the disk under /mnt/wsl/dataDisk and see a lost+found folder in there.

You may also want to save this script somewhere on your computer and run it with Task Scheduler every time you login into your PC.

Simply add this line into the action of Start a program :

-ExecutionPolicy Bypass -File D:path_tovhd.ps1

4. Migrating user data to VHD

Let’s create our user folder there, move all our /home/<username>/* files into the new folder and symlink it.

The ultimate bonus

Now that you have migrated your user data to the VHD, let’s tackle the convenient part of the ULTIMATE setup. This allows you to switch your Linux distros as you wish, while keeping all your data in one central place.

You want to run Kali alongside Ubuntu while still being able to access your user data? This comes in handy, if sometimes things don’t work as expected and you want to make sure if the problem is in Ubuntu or elsewhere. Just make sure to have the same username as you have on Ubuntu and execute these commands.

I hope you learned something, and stay tuned for the next part, where I’ll cover the proper setup of Docker inside WSL2.

Part 2

Continue to the second part:

Docker Slow Ext4 Partition Download

-->

If you want to access a Linux disk format that isn't supported by Windows, you can use WSL 2 to mount your disk and access its content. This tutorial will cover the steps to identify the disk and partition to attach to WSL2, how to mount them, and how to access them.

Note Access 2010 for mac free download.

Administrator access is required to attach a disk to WSL 2.The WSL 2 mount command does not support mounting a disk (or partitions that belong to the disk) that is currently in use. wsl --mount always attaches the entire disk even if only a partition is requested. You can't mount the Windows installation disk.

Prerequisites

You will need to be on Windows 10 Build 20211 or higher to access this feature. You can join the Windows Insiders Program to get the latest preview builds.

Mounting an unpartitioned disk

In this simplest case, if you have a disk that doesn't have any partitions, you can mount it directly using the wsl --mount command. First you need to identify the disk.

Identify the disk

To list the available disks in Windows, run:

The disks paths are available under the 'DeviceID' columns. Usually under the .PHYSICALDRIVE* format.

Mount the disk

Then in Powershell you can mount the disk using the Disk path discovered above.

Mounting a partitioned disk

If you have a disk that you aren't sure what file format it is in, or what partitions it has, you can follow the steps below to mount it.

Identify the disk

To list the available disks in Windows, run:

The disks paths are listed after 'DeviceID', usually in the .PHYSICALDRIVE* format

List and select the partitions to mount in WSL 2

Once the disk is identified, run:

This will make the disk available in WSL 2. (In the case of our example, the <DiskPath> is .PHYSICALDRIVE*.

Once attached, the partition can be listed by running the following command inside WSL 2:

This will display the available block devices and their partitions.

Inside Linux, a block device is identified as /dev/<Device><Partition>. For example, /dev/sdb3, is the partition number 3 of disk sdb.

Example output:

Identifying the filesystem type

If you don't know the type of filesystem of a disk or partition, you can use this command:

This will output the detected filesystem type (under the TYPE='<Filesystem>' format).

Mount the selected partitions

Once you have identified the partitions you want to mount, run this command on each partition:

Note

If you wish to mount the entire disk as a single volume (i.e. if the disk isn't partitioned), --partition can be omitted.

If omitted, the default filesystem type is 'ext4'.

Access the disk content

Once mounted, the disk can be accessed under the path pointed to by the config value: automount.root. The default value is /mnt/wsl.

From Windows, the disk can be accessed from File Explorer by navigating to: wsl$<Distro><Mountpoint> (pick any Linux distribution).

Unmount the disk

If you want to unmount and detach the disk from WSL 2, run:

Docker Slow Ext4 Partition Windows 10

Mount a VHD in WSL

You can also mount virtual hard disk files (VHD) into WSL using wsl --mount. To do this, you first need to mount the VHD into Windows using the Mount-VHD command in Windows. Be sure to run this command with administrator privileges. Below is an example where we use this command, and also output the disk path. Be sure to replace <pathToVHD> with your actual VHD path.

You can use the output above to obtain the disk path for this VHD and mount that into WSL following the instructions in the previous section.

You can also use this technique to mount and interact with the virtual hard disks of other WSL distros, as each WSL 2 distro is stored via a virtual hard disk file called: ext4.vhdx. By default the VHDs for WSL 2 distros are stored in this path: C:Users[user]AppDataLocalPackages[distro]LocalState[distroPackageName], please exercise caution accessing these system files, this is a power user workflow. Make sure to run wsl --shutdown before interacting with this disk to ensure the disk is not in use.

Command line reference

Mounting a specific filesystem

By default, WSL 2 will attempt to mount the device as ext4. To specify another filesystem, run:

For example, to mount a disk as fat, run:

Note

To list the available filesystems in WSL2, run: cat /proc/filesystems
When a disk has been mounted via WSL2 (Linux file system), it is no longer available to mount via an ext4 driver on the Windows file system.

Mounting a specific partition

By default, WSL 2 attempts to mount the entire disk. To mount a specific partition, run:

This only works if the disk is either MBR (Master Boot Record) or GPT (GUID Partition Table). Read about partition styles - MBR and GPT.

Specifying mount options

To specify mount options, run:

Example:

Note

Only filesystem specific options are supported at this time. Generic options such as ro, rw, noatime, .. are not supported.

Attaching the disk without mounting it

If the disk scheme isn't supported by any of the above options, you can attach the disk to WSL 2 without mounting it by running:

Docker Slow Ext4 Partition Tool

This will make the block device available inside WSL 2 so it can be mounted manually from there. Use lsblk to list the available block devices inside WSL 2.

Detaching a disk

To detach a disk from WSL 2, run:

If Diskpath is omitted, all attached disks are unmounted and detached.

Note

If one disk fails to unmount, WSL 2 can be forced to exit by running wsl --shutdown, which will detach the disk.

Limitations

Docker Slow Ext4 Partition Tool

  • At this time, only entire disks can be attached to WSL 2, meaning that it's not possible to attach only a partition. Concretely, this means that it's not possible to use wsl --mount to read a partition on the boot device, because that device can't be detached from Windows.

  • USB flash drives and SD cards are not supported at this time and will fail to attach to WSL 2. USB disks are supported though.

  • Only filesystems that are natively supported in the kernel can be mounted by wsl --mount. This means that it's not possible to use installed filesystem drivers (such as ntfs-3g for example) by calling wsl --mount.





broken image