ratarmount

Access large archives as a filesystem efficiently, e.g., TAR, RAR, ZIP, GZ, BZ2, XZ, ZSTD archives

MIT License

Downloads
4.1K
Stars
700
Committers
10
ratarmount - ratarmount 0.3.1

Published by mxmlnkn 8 months ago

  • Fix missing bzip2.h header in tarball on PyPI.
ratarmount - ratarmount 0.3.0

Published by mxmlnkn 8 months ago

  • Add --version option to CLI.
  • Add support for gzip compressed TAR archives.
  • Add support for bzip2 compressed TAR archives.
  • Improve SQLite index creation memory footprint and performance using
    a temporary unsorted table and then sort it once by file name.
ratarmount - ratarmount v0.2.0

Published by mxmlnkn 8 months ago

  • Add SQLite backend to improve memory usage.
  • Reduce memory usage by clearing tarfile's internal index periodically.
  • Add argument option to forward options to FUSE.
ratarmount - ratarmount 0.1.0

Published by mxmlnkn 8 months ago

First version uploaded to PyPI.

ratarmount - ratarmount 0.14.0

Published by mxmlnkn about 1 year ago

ratarmount

Added

  • Add --disable-union-mount option to present multiple archives in subfolders under the mount point.
  • Add fine-grained control over the union mount cache:
    • --union-mount-cache-max-depth
    • --union-mount-cache-max-entries
    • --union-mount-cache-timeout
  • Only use the number of cores the ratarmount process has affinity for by default.
  • Add --index-minimum-file-count with sane default (1000) to avoid creating index files for
    rather small ZIP archives.

Fixes

  • Disable auto-detection of GNU incremental TAR archives by default.
    Use --detect-gnu-incremental or better --gnu-incremental/--no-gnu-incremental
    instead.
  • Apply name change: pragzip -> rapigzip.
  • Fix ambiguous FUSE behavior that resulted in find not descending into recursively
    mounted subarchives.

ratarmountcore

Added

  • Use rapidgzip instead of indexed_gzip by default.
  • Do not parallelize gzip on slow drives because pread is slower than
    sequential read on those.
  • Enable profiling output for rapidgzip with -d 2 or greater.

Fixes

  • Do not check for GNU incremental TAR archive after an index has been loaded because
    it is only necessary for index creation. Note that this means thatSQLiteIndexedTar.isGnuIncremental
    member will remain False when an index has been loaded.
  • Test for incremental archive after loading the compression index to avoid having
    to recreate it.
  • Fix missing warning for multi-frame requirement of Zstandard archives.
  • Fix unnecessary warning about mismatching gzip-seek-point-spacing
    when loading an index for
    archives without gzip compression.
  • Change the default value of the SQLiteIndexedTar constructor argument
    gzipSeekPointSpacing from 4 MiB tp 16 MiB to make it consistent with the ratarmount CLI option.
ratarmount - ratarmount 0.13.0

Published by mxmlnkn over 1 year ago

Fixes in ratarmountcore

  • Use XDG_CACHE_HOME according to FreeDesktop as default fallback
    when the archive folder is not writable.
  • Create an SQLite index file for zip archives to speed up metadata lookup.
  • Fix the issue with folders for mounted zip files not being traversed with find.
ratarmount - ratarmount 0.12.0

Published by mxmlnkn almost 2 years ago

Fixes in ratarmount

  • Add --recursion-depth argument for more fine-grained recursion control.
  • Add support to show a joined view of split files, e.g., .001, and also support split archives.
  • Add --use-backend option to choose between multiple available backends, e.g., 'lzmaffi' and 'xz' for xz files.
  • Add support for pragzip when it has been installed and specified with '--use-backend pragzip'.
    Pragzip offers unique parallelized decompression of arbitrary gzip files. Using a 12-core processor,
    it can show speedups of 12 over gzip and speedups of 8 over pigz for sequentially decoding base64 files.
    Furthermore, similar to indexed_bzip2 it is based on a block cache and therefore can substantially speed up
    concurrent access streams as might happen when accessing multiple files through ratarmount concurrently.
    Currently, indexed_gzip will still be used to create the index in the first pass and pragzip is used
    for subsequent accesses with '--use-backend pragzip'.

Fixes in ratarmountcore

  • AutoMountLayer now takes a 'recursionDepth' argument instead of 'recursive'.
  • Fix handling of paths in zip and rar files that contain '../'.
  • Add backend prioritization option to SQLiteIndexedTar.
  • Add support for pragzip.
ratarmount - ratarmount 0.11.3

Published by mxmlnkn over 2 years ago

Fixes in ratarmount

  • Fix files with leading "./" not being deleted when using --commit-overlay
  • Fix files directly under the mount point not being deleted correctly with --write-overlay.

Fixes in ratarmountcore

  • Fix exception when trying to mount a RAR archive containing files without timestamps.
ratarmount - ratarmount 0.11.2

Published by mxmlnkn over 2 years ago

Fixes in ratarmount

  • Fix --password option not working.
ratarmount - ratarmount 0.11.1

Published by mxmlnkn over 2 years ago

Fixes in ratarmount

  • Fix compatibility of --commit-overlay with older GNU tar versions by removing the redundant --verbatim-file option.

Fixes in ratarmountcore

  • Fix duplicate mounting of uncompressed TARs inside TARs when using --recursive.
ratarmount - ratarmount 0.11.0

Published by mxmlnkn over 2 years ago

New features in ratarmount

  • Add --write-overlay option to enable write support at the mount point.
  • Add -u option for more consistent mount/unmount call signatures.
  • Add output for versions of all dependencies to --version.
  • Add support for AppImage builds which will be offered on the GitHub Releases page.
  • Add --transform-recursive-mount-point option to control the mount path of recursively mounted archives.
  • Change default for --parallelization from 1 to 0, i.e., the maximum number of cores.

Fixes in ratarmountcore

  • Fix (the last 1000) files not showing for truncated / incomplete TAR files.
  • Improve performance for gzip files significantly by using a larger buffer.
  • Do not use ParallelXZReader for single-block xz files to avoid memory issues.
  • Fix uncaught exception when a folder contains an invalid character.
  • Recursive mounting of nested uncompressed TARs did skip some files for depth > 2.
  • Relax the check for GNU incremental TAR detection because the prefix field might contain binary data.
  • Improve performance by factor ~5 and avoid storage requirements for index reading for gzip-compressed archives.
  • Improve performance by ~40% and avoid storage requirements for index writing for gzip-compressed archives.
  • Improve performance for indexing uncompressed TARs that have been appended by only analyzing the new files.
  • Fix uncompressed archives having been appended very small files (<10 KiB) were not detected as changed by default.
  • Fix problem triggered by combining --recursive and -P 0 with a recursive bzip2 archive.
ratarmount - Special AppImage Release for Ratarmount 0.10.0

Published by mxmlnkn over 2 years ago

This is a proof of concept for bundling ratarmount into a standalone AppImage. It was motivated because some systems only ship with fuse3 but ratarmount still uses fusepy, which requires fuse2. The AppImage has libfuse2 bundled in it for maximum compatiblity.