[GH-ISSUE #654] Some search results missing if broken symlinks are present #397

New Issue

zhus · 2026-04-08T16:52:36+03:00

zhus commented

2026-04-08 16:52:36 +03:00

Originally created by @jamesscottbrown on GitHub (Jan 5, 2026).
Original GitHub issue: https://github.com/sigoden/dufs/issues/654

Problem

The collect_dir_entries function doesn't handle errors when listing files in the serve path.

I was using dufs to serve a directory that contained some broken symbolic links, and this was causing other files to be missing from the list of search results. This problem occurs whether or not the user specifies --allow-symlink, as the WalkDir iterator has follow_links enabled (let mut it = WalkDir::new(&dir).follow_links(true).into_iter()).

It took me a while to realise what the problem was, as no errors are logged or returned as a response to the search query.

Originally created by @jamesscottbrown on GitHub (Jan 5, 2026). Original GitHub issue: https://github.com/sigoden/dufs/issues/654 **Problem** The `collect_dir_entries` function doesn't handle errors when listing files in the serve path. I was using dufs to serve a directory that contained some broken symbolic links, and this was causing other files to be missing from the list of search results. This problem occurs whether or not the user specifies `--allow-symlink`, as the WalkDir iterator has `follow_links` enabled (`let mut it = WalkDir::new(&dir).follow_links(true).into_iter()`). It took me a while to realise what the problem was, as no errors are logged or returned as a response to the search query.

zhus closed this issue

2026-04-08 16:52:36 +03:00

zhus commented

2026-04-08 16:52:37 +03:00

@sigoden commented on GitHub (Jan 9, 2026):

I can't confirm the issue you're referring to.

The follow_links in WalkDir::new(&dir).follow_links(true) is not problematic; symlink issues are handled separately within the loop.

2b2c7bd5f7/src/server.rs (L1898-L1904)

@sigoden commented on GitHub (Jan 9, 2026): I can't confirm the issue you're referring to. The `follow_links` in `WalkDir::new(&dir).follow_links(true)` is not problematic; symlink issues are handled separately within the loop. https://github.com/sigoden/dufs/blob/2b2c7bd5f718b0516d3a229e91ede5c0d2e32bb3/src/server.rs#L1898-L1904

zhus commented

2026-04-08 16:52:37 +03:00

@jamesscottbrown commented on GitHub (Jan 20, 2026):

The follow_links in WalkDir::new(&dir).follow_links(true) is not problematic; symlink issues are handled separately within the loop.

Yes, this is fine - it just means that telling dufs not to follow the symlinks does not work as a work-around for the underlying problem.

The actual problem is the while let Some(Ok(entry)) = it.next(), which means the loop terminates early if it.next() encounters an error.

Here's a more explicit demonstration with a minimal program to show the problem:

src/main.rs:

use std::path::PathBuf;
use walkdir::WalkDir;


// main() is a simplification of collect_dir_entries
fn main() {
    // collect_dir_entries takes these as arguments
    let path = PathBuf::from("./test-dir");
    
    let access_paths = AccessPaths {
        paths: vec![path.clone()],
    };
    

    let mut paths: Vec<PathBuf> = vec![];
    
    for dir in access_paths.entry_paths(&path) {
        let mut it = WalkDir::new(&dir).follow_links(true).into_iter();
        it.next();
        
        while let Some(Ok(entry)) = it.next() {

            // simplified loop body 
            println!("{}", entry.path().display());
            paths.push(entry.path().to_path_buf());            
        }
    }
    
    println!("\nTotal entries found: {}", paths.len());
}

// Stubs to make this work in isolation
struct AccessPaths {
    paths: Vec<PathBuf>,
}

impl AccessPaths {
    fn entry_paths(&self, _base: &PathBuf) -> &Vec<PathBuf> {
        &self.paths
    }
}

Cargo.toml:

[package]
name = "walkdir-example"
version = "0.1.0"
edition = "2021"

[dependencies]
walkdir = "2"

Demonstration:

mkdir ./test-dir
touch ./test-dir/a
touch ./test-dir/b
touch ./test-dir/c

cargo build
cargo run

With no broken symlinks, there are no errors encountered and all files are iterated over:

❯ cargo run
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.00s
     Running `target/debug/walkdir-example`
./test-dir/a
./test-dir/c
./test-dir/b

Total entries found: 3

However, if I create a broken symlink then some are missed:

❯ ln -s  /this/is/broken ./test-dir/a1 
❯ cargo run
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.00s
     Running `target/debug/walkdir-example`
./test-dir/a

Total entries found: 1

The same thing would happen if WalkDir encountered a different type of error.

@jamesscottbrown commented on GitHub (Jan 20, 2026): > The follow_links in WalkDir::new(&dir).follow_links(true) is not problematic; symlink issues are handled separately within the loop. Yes, this is fine - it just means that telling dufs not to follow the symlinks does not work as a work-around for the underlying problem. The actual problem is the `while let Some(Ok(entry)) = it.next()`, which means the loop terminates early if `it.next()` encounters an error. Here's a more explicit demonstration with a minimal program to show the problem: `src/main.rs`: ```rust use std::path::PathBuf; use walkdir::WalkDir; // main() is a simplification of collect_dir_entries fn main() { // collect_dir_entries takes these as arguments let path = PathBuf::from("./test-dir"); let access_paths = AccessPaths { paths: vec![path.clone()], }; let mut paths: Vec<PathBuf> = vec![]; for dir in access_paths.entry_paths(&path) { let mut it = WalkDir::new(&dir).follow_links(true).into_iter(); it.next(); while let Some(Ok(entry)) = it.next() { // simplified loop body println!("{}", entry.path().display()); paths.push(entry.path().to_path_buf()); } } println!("\nTotal entries found: {}", paths.len()); } // Stubs to make this work in isolation struct AccessPaths { paths: Vec<PathBuf>, } impl AccessPaths { fn entry_paths(&self, _base: &PathBuf) -> &Vec<PathBuf> { &self.paths } } ``` Cargo.toml: ```toml [package] name = "walkdir-example" version = "0.1.0" edition = "2021" [dependencies] walkdir = "2" ``` Demonstration: ``` mkdir ./test-dir touch ./test-dir/a touch ./test-dir/b touch ./test-dir/c cargo build cargo run ``` With no broken symlinks, there are no errors encountered and all files are iterated over: ``` ❯ cargo run Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.00s Running `target/debug/walkdir-example` ./test-dir/a ./test-dir/c ./test-dir/b Total entries found: 3 ``` However, if I create a broken symlink then some are missed: ``` ❯ ln -s /this/is/broken ./test-dir/a1 ❯ cargo run Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.00s Running `target/debug/walkdir-example` ./test-dir/a Total entries found: 1 ``` The same thing would happen if WalkDir encountered a different type of error.

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: sigoden/dufs#397