[GH-ISSUE #308] Option to download folders as tar #163

Closed
opened 2026-04-08 16:50:47 +03:00 by zhus · 6 comments
Owner

Originally created by @abe1242 on GitHub (Dec 2, 2023).
Original GitHub issue: https://github.com/sigoden/dufs/issues/308

Specific Demand

zip is really slow when sending large folders with multiple files, since it has to do compression.

Having an option to download as tar (without any compression like gz, bz xz, etc.) would be the better option.

Originally created by @abe1242 on GitHub (Dec 2, 2023). Original GitHub issue: https://github.com/sigoden/dufs/issues/308 ## Specific Demand zip is really slow when sending large folders with multiple files, since it has to do compression. Having an option to download as tar (without any compression like gz, bz xz, etc.) would be the better option.
zhus closed this issue 2026-04-08 16:50:47 +03:00
Author
Owner

@sigoden commented on GitHub (Dec 2, 2023):

If only one compression format is supported, zip is the only choice. Because zip is the most versatile, the evidence is that windows can directly open zip, but other formats cannot.

Dufs is not prepared to support two compression formats, just like github has always only supported zip.

Dufs supports downloading compressed streams. The compression algorithm won't have a huge impact on download times. The biggest impact has always been bandwidth.

<!-- gh-comment-id:1837039905 --> @sigoden commented on GitHub (Dec 2, 2023): If only one compression format is supported, zip is the only choice. Because zip is the most versatile, the evidence is that windows can directly open zip, but other formats cannot. Dufs is not prepared to support two compression formats, just like github has always only supported zip. Dufs supports downloading compressed streams. The compression algorithm won't have a huge impact on download times. The biggest impact has always been bandwidth.
Author
Owner

@abe1242 commented on GitHub (Dec 2, 2023):

The compression algorithm won't have a huge impact on download times. The biggest impact has always been bandwidth.

In my case bandwidth is not the problem. I can send files using dufs with an average speed of 40 MB/s. Whereas, for a folder it around 6 MB/s. Tried to download the same folder with the .tar option in miniserve and I got close to 40 MB/s.

If only one compression format is supported, zip is the only choice. Because zip is the most versatile, the evidence is that windows can directly open zip, but other formats cannot.

I do understand that the universal support of zip makes it a better default. You could implement tar by providing a flag like --archive-format which can be set to tar if needed, and by default it is set to zip.

Correction: speed of zip is around 3.5 MB/s when I tested it right now.

<!-- gh-comment-id:1837052677 --> @abe1242 commented on GitHub (Dec 2, 2023): > The compression algorithm won't have a huge impact on download times. The biggest impact has always been bandwidth. In my case bandwidth is not the problem. I can send files using dufs with an average speed of 40 MB/s. Whereas, for a folder it around 6 MB/s. Tried to download the same folder with the .tar option in `miniserve` and I got close to 40 MB/s. > If only one compression format is supported, zip is the only choice. Because zip is the most versatile, the evidence is that windows can directly open zip, but other formats cannot. I do understand that the universal support of zip makes it a better default. You could implement tar by providing a flag like `--archive-format` which can be set to tar if needed, and by default it is set to zip. Correction: speed of zip is around 3.5 MB/s when I tested it right now.
Author
Owner

@sigoden commented on GitHub (Dec 2, 2023):

image
The download speed of zip stream can reach 20M.
If there are many small files, the speed may be slower.

This speed is acceptable.

<!-- gh-comment-id:1837072649 --> @sigoden commented on GitHub (Dec 2, 2023): ![image](https://github.com/sigoden/dufs/assets/4012553/0e133d8c-b998-490b-8cd0-7d2556456b3f) The download speed of zip stream can reach 20M. If there are many small files, the speed may be slower. This speed is acceptable.
Author
Owner

@abe1242 commented on GitHub (Dec 2, 2023):

My phone has the computing power of a potato so that explains the slow speed of zip. I guess I'm gonna have to stick to miniserve for sending folders and dufs for anything else.

Thanks for the response anyway

<!-- gh-comment-id:1837081979 --> @abe1242 commented on GitHub (Dec 2, 2023): My phone has the computing power of a potato so that explains the slow speed of zip. I guess I'm gonna have to stick to miniserve for sending folders and dufs for anything else. Thanks for the response anyway
Author
Owner

@sigoden commented on GitHub (Dec 14, 2023):

#319 supports --compress: none which is similar to tar and is extremely fast. you may try it.

<!-- gh-comment-id:1855622361 --> @sigoden commented on GitHub (Dec 14, 2023): #319 supports `--compress: none` which is similar to tar and is extremely fast. you may try it.
Author
Owner

@abe1242 commented on GitHub (Dec 17, 2023):

Tried it on my PC and it works really well. Getting 170M+ speeds whereas with compression it is 20M. Haven't tried it on my phone, but I'm hoping it will be much faster than before.
Thank you so much for working on this. This was much needed

<!-- gh-comment-id:1859057310 --> @abe1242 commented on GitHub (Dec 17, 2023): Tried it on my PC and it works really well. Getting 170M+ speeds whereas with compression it is 20M. Haven't tried it on my phone, but I'm hoping it will be much faster than before. Thank you so much for working on this. This was much needed
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: sigoden/dufs#163