remove_dir_all

Introduction

Welcome to the 0.6 release of remove_dir_all! remove_dir_all is a fast and reliable std::fs::remove_dir_all implementation for Windows, for other platforms it simply re-exports std. This release brings a lot internal updates and improvements with a complete internal refactor, switching to the 2018 edition and moving to GitHub Actions for handling CI. You can update it simply by updating your Cargo.toml.

remove_dir_all = "0.6"

Removing Directories In Parallel

The biggest change in this release is a refactor of the current the serial implementation with a Rayon based parallel implementation that is substantially faster due to the Windows IO model.

On Windows it is not enough to just recursively remove the contents of a directory and then the directory itself. Deleting does not happen instantaneously, but is delayed by IO being completed in the fs stack. Further, typical Windows machines can handle many more concurrent IOs than a single threaded application is capable of submitting: the overlapped (async) calls available do not cover the operations needed to perform directory removal.

To work around this, we use a work stealing scheduler and submit deletions concurrently with directory scanning, and delete sibling directories in parallel. This allows the slight latency of STATUS_DELETE_PENDING to only have logarithmic effect: a very deep tree will pay wall clock time for that overhead per level as the tree traverse completes, but not for every interior not as a simple recursive deletion would result in.

Earlier versions of this crate moved the contents of the directory being deleted to become siblings of base_dir, which required write access to the parent directory under all circumstances; this is no longer required - though it may be re-instated if in-use files turn out to be handled very poorly with this new threaded implementation.

There is a single small race condition where external side effects may be left: when deleting a hard linked readonly file, the syscalls required are: open, set rw, unlink (SetFileDispositionDelete), and finally set ro. A crash or power failure could lead to the loss of the readonly bit on the hardlinked inode.

Thank you so much to @rbtcollins for working on this effort, and providing the updated implementation!