sigoden/dufs
1
0
Fork 0
mirror of https://github.com/sigoden/dufs.git synced 2026-04-08 16:49:02 +03:00
Code Issues 8 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #336] Corrupted Zip files when downloading large folders #180

New Issue
Closed
opened 2026-04-08 16:50:55 +03:00 by zhus · 0 comments
zhus commented 2026-04-08 16:50:55 +03:00
Owner
Copy Link

Originally created by @wasle on GitHub (Jan 7, 2024).
Original GitHub issue: https://github.com/sigoden/dufs/issues/336

Problem

When downloading large folders with several files in the GB range the resulting .zip files often is corrupted.

Log

  1. I created a set of files with a script python script main.zip.
    But probably just an arbitrary set of files with around 10GB will do.
    Put all files in a folder.
  2. start dufs
  3. download the folder
  4. try to extract the files (optionally with the validateFile() function in the python script).

% ./dufs ~/TMP/dufs-data --port 1235 --allow-all --compress none
Listening on:
...
http://127.0.0.1:1235/
...

2024-01-06T22:54:33+01:00 INFO - 127.0.0.1 "GET /" 200
2024-01-06T22:54:38+01:00 INFO - 127.0.0.1 "GET /test/?zip" 200

Environment:

  • Tested dufs versions:
    • build of current master: 17063454d3
    • docker image with 0.38.0
  • Browser:
    • Firefox 115.6.0esr(64Bit)
    • Chrome 120.0.6099.199 (Official Build) (64-bit)
  • OS Info:
    • debian/bookworm

Notes

  • The compression setting does not seam to have an influence
  • The first impression is it is not directly related to a 4GB file size. It mostly started some ware around 6-8GB.
Originally created by @wasle on GitHub (Jan 7, 2024). Original GitHub issue: https://github.com/sigoden/dufs/issues/336 **Problem** When downloading large folders with several files in the GB range the resulting .zip files often is corrupted. **Log** 1. I created a set of files with a script python script [main.zip](https://github.com/sigoden/dufs/files/13851477/main.zip). But probably just an arbitrary set of files with around 10GB will do. Put all files in a folder. 2. start dufs 3. download the folder 4. try to extract the files (optionally with the validateFile() function in the python script). % ./dufs ~/TMP/dufs-data --port 1235 --allow-all --compress none Listening on: ... http://127.0.0.1:1235/ ... 2024-01-06T22:54:33+01:00 INFO - 127.0.0.1 "GET /" 200 2024-01-06T22:54:38+01:00 INFO - 127.0.0.1 "GET /test/?zip" 200 **Environment:** - Tested dufs versions: - build of current master: 17063454d35ba3ca7d1e8008197d02c756927029 - docker image with 0.38.0 - Browser: - Firefox 115.6.0esr(64Bit) - Chrome 120.0.6099.199 (Official Build) (64-bit) - OS Info: - debian/bookworm **Notes** - The compression setting does not seam to have an influence - The first impression is it is not directly related to a 4GB file size. It mostly started some ware around 6-8GB.
zhus closed this issue 2026-04-08 16:50:55 +03:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
bug
enhancement
help wanted
pull-request
Mirrored from GitHub Pull Request
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
zhus
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: sigoden/dufs#180
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?