bootandy/archived-dust
1
0
Fork 0
mirror of https://github.com/bootandy/dust.git synced 2026-06-08 11:29:05 +03:00
Code Issues 21 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #295] Quirky output of Windows NTFS mount inside WSL #128

New Issue
Closed
opened 2026-06-08 11:25:48 +03:00 by zhus · 5 comments
zhus commented 2026-06-08 11:25:48 +03:00
Owner
Copy Link
Copy Source

Originally created by @danie-dejager on GitHub (Jan 15, 2023).
Original GitHub issue: https://github.com/bootandy/dust/issues/295

Hi,

I see that most Linux apps struggle to get the actual file size of a NTFS drive mounted in WSL.
Using Dust 0.8.3

From my Linux instance I go to /mnt/d/ and see the following results
Normal output

# dust
13294035T       ┌── QtGraphicalEffects
13196115T       │   ┌── eo
13458252T       │   │ ┌── LC_MESSAGES
13458252T       │   ├─┴ bs
13574211T       │   ├── fi
13589332T       │   │ ┌── LC_MESSAGES
13589332T       │   ├─┴ nb
14721323T       │   │ ┌── LC_MESSAGES
14721323T       │   ├─┴ mk
14721323T       │   │ ┌── LC_MESSAGES
14721323T       │   ├─┴ uz@cyrillic
14721323T       │   │ ┌── LC_MESSAGES
14721323T       │   ├─┴ cy
14871114T       │   │ ┌── LC_MESSAGES
14871114T       │   ├─┴ lt
15056111T       │ ┌─┴ locale
14673252T       ├─┴ data
10797842T     ┌─┴ bin
10797842T   ┌─┴ KDE
 4117825T ┌─┴ .

with apparent-size things are looking better

# dust -s -b
 17G   ┌── Data
 45G   ├── Hyper-V
 13G   │ ┌── ubuntu 22.04
 14G   │ │ ┌── FreeBSD13.vdi
 14G   │ ├─┴ FreeBSD13
 16G   │ │ ┌── FreeBSD.vdi
 16G   │ ├─┴ FreeBSD 12
 16G   │ ├── Amazon Linux 2
 17G   │ │ ┌── Oracle Linux 7.6.vdi
 17G   │ ├─┴ Oracle Linux 7
 18G   │ │   ┌── {a1599947-7d7c-492d-b377-7b4187c508a7}.vdi
 19G   │ │ ┌─┴ Snapshots
 38G   │ │ ├── Sophos.vdi
 58G   │ ├─┴ Windows7
 28G   │ │   ┌── {17213b5e-2726-444d-9ecc-41254bcc1fc1}.vdi
 28G   │ │ ┌─┴ Snapshots
 32G   │ │ ├── Centos 6.vdi
 60G   │ ├─┴ Centos 6
260G   ├─┴ VM
332G ┌─┴ .

Is there something wrong with how WSL is presenting the drive as other tools like Dust is also not printing sizes correctly. gdu also needs to use apparent size to display correctly.
https://github.com/dundee/gdu/issues/162

Originally created by @danie-dejager on GitHub (Jan 15, 2023). Original GitHub issue: https://github.com/bootandy/dust/issues/295 Hi, I see that most Linux apps struggle to get the actual file size of a NTFS drive mounted in WSL. Using `Dust 0.8.3` From my Linux instance I go to `/mnt/d/` and see the following results Normal output ``` # dust 13294035T ┌── QtGraphicalEffects 13196115T │ ┌── eo 13458252T │ │ ┌── LC_MESSAGES 13458252T │ ├─┴ bs 13574211T │ ├── fi 13589332T │ │ ┌── LC_MESSAGES 13589332T │ ├─┴ nb 14721323T │ │ ┌── LC_MESSAGES 14721323T │ ├─┴ mk 14721323T │ │ ┌── LC_MESSAGES 14721323T │ ├─┴ uz@cyrillic 14721323T │ │ ┌── LC_MESSAGES 14721323T │ ├─┴ cy 14871114T │ │ ┌── LC_MESSAGES 14871114T │ ├─┴ lt 15056111T │ ┌─┴ locale 14673252T ├─┴ data 10797842T ┌─┴ bin 10797842T ┌─┴ KDE 4117825T ┌─┴ . ``` with apparent-size things are looking better ``` # dust -s -b 17G ┌── Data 45G ├── Hyper-V 13G │ ┌── ubuntu 22.04 14G │ │ ┌── FreeBSD13.vdi 14G │ ├─┴ FreeBSD13 16G │ │ ┌── FreeBSD.vdi 16G │ ├─┴ FreeBSD 12 16G │ ├── Amazon Linux 2 17G │ │ ┌── Oracle Linux 7.6.vdi 17G │ ├─┴ Oracle Linux 7 18G │ │ ┌── {a1599947-7d7c-492d-b377-7b4187c508a7}.vdi 19G │ │ ┌─┴ Snapshots 38G │ │ ├── Sophos.vdi 58G │ ├─┴ Windows7 28G │ │ ┌── {17213b5e-2726-444d-9ecc-41254bcc1fc1}.vdi 28G │ │ ┌─┴ Snapshots 32G │ │ ├── Centos 6.vdi 60G │ ├─┴ Centos 6 260G ├─┴ VM 332G ┌─┴ . ``` Is there something wrong with how WSL is presenting the drive as other tools like Dust is also not printing sizes correctly. gdu also needs to use `apparent size` to display correctly. https://github.com/dundee/gdu/issues/162
zhus closed this issue 2026-06-08 11:25:48 +03:00
zhus commented 2026-06-08 11:25:48 +03:00
Author
Owner
Copy Link
Copy Source

@bootandy commented on GitHub (Jan 18, 2023):

I've no idea how WSL works, but if it is fixed in gdu then I should be able to replicate that here

apparent size uses file length instead of counting the blocks that the file uses. It is possible that the block size in WSL is strange:
https://doc.rust-lang.org/std/os/linux/fs/trait.MetadataExt.html#tymethod.st_blocks

<!-- gh-comment-id:1386256875 --> @bootandy commented on GitHub (Jan 18, 2023): I've no idea how WSL works, but if it is fixed in gdu then I should be able to replicate that here apparent size uses file length instead of counting the blocks that the file uses. It is possible that the block size in WSL is strange: https://doc.rust-lang.org/std/os/linux/fs/trait.MetadataExt.html#tymethod.st_blocks
zhus commented 2026-06-08 11:25:48 +03:00
Author
Owner
Copy Link
Copy Source

@bootandy commented on GitHub (Apr 2, 2025):

can we close this @danie-dejager ? It's been 2 years and I don't have a windows box to replicate this

<!-- gh-comment-id:2770835899 --> @bootandy commented on GitHub (Apr 2, 2025): can we close this @danie-dejager ? It's been 2 years and I don't have a windows box to replicate this
zhus commented 2026-06-08 11:25:48 +03:00
Author
Owner
Copy Link
Copy Source

@frendsick commented on GitHub (Apr 19, 2025):

I have the same issue. I'm taking some time to debug the issue and will let you know if I find a solution. The --apparent-size flag works for me, too.

<!-- gh-comment-id:2816647705 --> @frendsick commented on GitHub (Apr 19, 2025): I have the same issue. I'm taking some time to debug the issue and will let you know if I find a solution. The `--apparent-size` flag works for me, too.
zhus commented 2026-06-08 11:25:48 +03:00
Author
Owner
Copy Link
Copy Source

@frendsick commented on GitHub (Apr 20, 2025):

I successfully reproduced the problem using the stat function with _FILE_OFFSET_BITS set to 64; see the C program below. It should be the same as stat64, what Rust's std::fs::metadata uses internally.

#define _FILE_OFFSET_BITS 64
#include <fcntl.h>
#include <stdio.h>
#include <sys/stat.h>
#include <sys/types.h>

int main() {
  struct stat buffer;
  stat("/mnt/c/Users/User/AppData/Local/Microsoft/OneDrive/logs/Personal/FeedbackHub/SubmissionPayload.json",
       &buffer);
  printf("%lu", buffer.st_blocks); // prints: 8180440366363314
  return 0;
}

This script shows that the problem cannot be fixed from Rust's side as long as we are using std::fs::metadata. The bug is in the GNU C library's stat function, specifically in how it handles NTFS files, at least when using WSL on Windows. Luckily, this happens very rarely, and the output is usually correct.

PR #487 does not completely fix the issue, but it makes it less prevalent for dust when it does occur. The expected allocated_size should be close, if not the actual size.

<!-- gh-comment-id:2816882524 --> @frendsick commented on GitHub (Apr 20, 2025): I successfully reproduced the problem using the `stat` function with `_FILE_OFFSET_BITS` set to 64; see the C program below. It should be the same as `stat64`, what Rust's [std::fs::metadata](https://doc.rust-lang.org/std/fs/fn.metadata.html) uses internally. ```c #define _FILE_OFFSET_BITS 64 #include <fcntl.h> #include <stdio.h> #include <sys/stat.h> #include <sys/types.h> int main() { struct stat buffer; stat("/mnt/c/Users/User/AppData/Local/Microsoft/OneDrive/logs/Personal/FeedbackHub/SubmissionPayload.json", &buffer); printf("%lu", buffer.st_blocks); // prints: 8180440366363314 return 0; } ``` This script shows that the problem cannot be fixed from Rust's side as long as we are using `std::fs::metadata`. The bug is in the GNU C library's `stat` function, specifically in how it handles NTFS files, at least when using WSL on Windows. Luckily, this happens very rarely, and the output is usually correct. PR #487 does not completely fix the issue, but it makes it less prevalent for `dust` when it does occur. The expected [allocated_size](https://github.com/bootandy/dust/blob/1b928534fc6c8d795f3bbd27abd90411d8a9e86b/src/platform.rs#L48) should be close, if not the actual size.
zhus commented 2026-06-08 11:25:48 +03:00
Author
Owner
Copy Link
Copy Source

@frendsick commented on GitHub (Apr 20, 2025):

The rabbit hole deepens. The problem seems not to be in GLIBC, but rather probably in the WSL's DrvFs driver. Unfortunately, DrvFs is not open source, so I cannot dig deeper.

Side note: I copied the SubmissionPayload.json file from the previous comment's test script to the same folder, renaming it to SubmissionPayload_new.json. After copying, both files show correct block counts on WSL's side. Somehow, touching the original file fixed it for the stat command in WSL.

<!-- gh-comment-id:2816914482 --> @frendsick commented on GitHub (Apr 20, 2025): The rabbit hole deepens. The problem seems not to be in GLIBC, but rather probably in the WSL's DrvFs driver. Unfortunately, DrvFs is not open source, so I cannot dig deeper. Side note: I copied the _SubmissionPayload.json_ file from the previous comment's test script to the same folder, renaming it to _SubmissionPayload_new.json_. After copying, both files show correct block counts on WSL's side. Somehow, touching the original file fixed it for the `stat` command in WSL.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
good first issue
help wanted
pull-request
Mirrored from GitHub Pull Request
 windows help wanted
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
zhus (Sergey Zhukov)
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: bootandy/archived-dust#128
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?