mirror of
https://github.com/bootandy/dust.git
synced 2026-06-08 11:29:05 +03:00
[GH-ISSUE #443] Error occurred while processing a large number of files #195
Assignees
zhus (Sergey Zhukov)
Clear assignees
zhus (Sergey Zhukov)
No Assignees
Notifications
Due Date
No due date set.
Dependencies
No dependencies set.
Reference: bootandy/archived-dust#195
Reference in New Issue
Block a user
Delete Branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @iwancat on GitHub (Oct 10, 2024).
Original GitHub issue: https://github.com/bootandy/dust/issues/443
used version :dust-v1.1.1-i686-unknown-linux-musl
./dust -d 1 /mnt/countdir Indexing: /mnt/countdir 18349815 files, 36T ... |memory allocation of 560000 bytes failed Aborted (core dumped)Then I added parameters -S ,Error message: Printing becomes longer
./dust -d 1 -S88388608 /mnt/countdir Indexing: /mnt/countdir 18018277 files, 36T ... /memory allocation of memory allocation of 150memory allocation of bytes failed memory allocation of 150150 bytes failed memory allocation of bytes failed memory allocation of memory allocation of 150150150 bytes failed bytes failed 150 bytes failed bytes failed Aborted (core dumped)gdb dust -c core file :
[New LWP 46342] Core was generated by./dust -d 1 -S88388608 /mnt/countdir'.Program terminated with signal 6, Aborted.
#0 0xf77f5430 in __kernel_vsyscall ()
(gdb) n`
From the top command , %Cpu(s): 1.0 us, 7.6 sy, and the dust process's RES was at 3.8 GB when the program experienced a core dump.
@tatref commented on GitHub (Oct 14, 2024):
Can you try the 64bits version?
@bootandy commented on GitHub (Oct 18, 2024):
What if you keep increasing the '-S' parameter ?
@iwancat commented on GitHub (Oct 29, 2024):
Continue with this error message
”Indexing: “ number close to before and then abort
@iwancat commented on GitHub (Oct 29, 2024):
I have tried that version [dust-v1.1.1-x86_64-unknown-linux-musl.tar.gz]，but the error is still the same.
May I ask the maximum number of files that this dust supports for retrieval.
@bootandy commented on GitHub (Nov 2, 2024):
It's clearly run out of memory. I don't think I can do anything about that. You can keep increasing the -S parameter, you could also try a non musl build but I doubt that would help.