littlefs

0bbb8bc Reorganized external project links a bit
7808233 Added a link to mklittlefs
8336ecd Added a link to pico-littlefs-usb (FAT12 emulation)
11b036c Prevented unnecessary superblock rewrites if old version in superblock chain
25ee90f Clarified what is accessible at specific superblock offsets in SPEC.md
a60a986 Duplicate the superblock entry during superblock expansion

1bc1493 Tweaked on-disk config comments for consistency
01b6a47 Extended test_alloc to test inferred block_count
749a456 Fix DivideByZero exception when filesystem is completely full.

7873d81 Fixed memory leak in emubd's out-of-order write emulation
fc2aa33 Fixed issue with exhaustive + out-of-order powerloss testing
6352185 Fixed sync issue where data writes could appear before metadata writes
f2a6f45 Added out-of-order write testing to emubd
2752d8c add nim-littlefs to readme
ddbfcaa Fixed synthetic move underflows in lfs_dir_get

Itsy-bitsy teeny-tiny breaking changes:

• lfs_rename now returns LFS_ERR_NOTDIR if the source is a regular file and the destination is a directory. This better aligns with POSIX.

• Defining LFS_FILE_MAX > 2147483647 is no longer supported. It's unclear if this ever worked, and at least half the file API was unusable in this state.

What's new?

• lfs_fs_gc can now compact metadata (#913)

  This is controlled by the new optional compact_thresh configuration option. mdirs above compact_thresh will be eagerly compacted during lfs_fs_gc, making it less likely (but still possible) for compactions to occur during file writes.

  By default, compact_thresh=0, littlefs will compact mdirs > ~88% block_size. This value may change in the future.

  compact_thresh=-1 disables mdir compaction during lfs_fs_gc.

  Note that compact_thresh has no effect on mdir compaction outside of lfs_fs_gc. mdirs must compact when full.

• Added inline_max, which allows explicit user control over the size of inlined files (#914)

  Decreasing inline_max may improve some metadata-related performance around mdir traversal. But decreasing inline_max may also hurt metadata-related performance around mdir compaction and increase storage consumption. It is up to users to determine the best value per-application.

  By default, inline_max=0, uses the largest possible inline_max. This matches the previous behavior and is probably a good default for most use cases.

  inline_max=-1 disables inlined files.

• Added easier util overrides for common system functions: LFS_MALLOC, LFS_FREE, LFS_CRC (#909)

  These can now be overridden with simple defines:

  -DLFS_MALLOC=my_malloc
  -DLFS_FREE=my_free
  -DLFS_CRC=my_crc
  

  Note: Overriding LFS_CRC with a non-CRC32 checksum is discouraged. This is only intended for hardware acceleration, etc.

• Thanks to @tomscii, lfs_rename now returns LFS_ERR_NOTDIR if the source is a regular file and the destination is a directory, which better matches POSIX (#917)

• Thanks to @BrianPugh, defining overridable limits (LFS_FILE_MAX, LFS_NAME_MAX, LFS_ATTR_MAX) to incompatible values will now result in a compile error (#886)

• Relaxed lookahead buffer alignment to only be byte-aligned (#912)

• Relaxed lfs_malloc alignment requirements to only be byte-aligned (#912)

Changes

42910bc Bumped minor version to v2.9
a70870c Renamed internal functions _raw* -> *
8b8fd14 Added inline_max, to optionally limit the size of inlined files
b5cd957 Extended lfs_fs_gc to compact metadata, compact_thresh
f522ed9 Added tests over rename type errors
4f32738 Fix return value of lfs_rename()
6691718 Restricted LFS_FILE_MAX to signed 32-bits, <2^31, <=2147483647
1fefcbb Rearranged compile-time constant checks to live near lfs_init
6056767 Relaxed alignment requirements for lfs_malloc
897b571 Changed CI to just run on ubuntu-latest
b1b10c0 Relaxed lookahead buffer alignment
1f9c3c0 Reworked the block allocator so the logic is hopefully simpler
7b68441 Renamed a number of internal block-allocator fields
9a620c7 Added LFS_CRC, easier override for lfs_crc
a0c6c54 Added LFS_MALLOC/FREE, easier overrides for lfs_malloc/free
c531a5e Replace erroneous LFS_FILE_MAX upper bound 4294967296 to 4294967295
8f9427d Add value-range checks for user-definable macros

Sponsors

A special thanks to littlefs's sponsors: @nedap, @kmetabg, @fusedFET

Changes

99b84ee Update DESIGN.md, fix minor typo
e91a29d Fixed struct sizes missing from generated release notes
b9b95ab Increase threshold for superblock expansion from ~50% -> ~88% full
10bcff1 Update DESIGN.md minor typo

Contributors

A special thanks to all who proposed PRs:

• @zchen24

Sponsors

And a special thanks to littlefs's sponsors:

• @nedap
• @kmetabg
• @fusedFET

8f3f32d Added -Wmissing-prototypes
92fc780 lfs_fs_raw* functions should be static

This release brings in some useful community-driven features.

What's new?

• Thanks to @BrianPugh, littlefs no longer needs the block_count when mounting an existing filesystem (#866)

  Simply set block_count=0 in your configuration, and littlefs will automatically determine the block_count based on what's on disk. This allows you to mount a filesystem of unknown size.

• Added lfs_fs_grow which allows you to change the size of an existing filesystem, thanks to @kaetemi (#872)

  The main use case is for changing the size of a partition as storage requirements change.

  Note: littlefs does not support shrinking a filesystem, and this is unlikely to change (it's a hard problem).

• Added lfs_fs_gc which allows you to manually run the block allocator, thanks to @opilat (#875)

  Note: littlefs currently does not persist any found free blocks to disk, though this may change in the future.

  This limits the current value of lfs_fs_gc, but it may still be useful for moving the expensive block scan out of performance sensitive code paths.

Changes

f91c5bd Bumped minor version to v2.8
6b33ee5 Renamed lfs_fs_findfreeblocks -> lfs_fs_gc, tweaked documentation
63e4408 Extended alloc tests to test some properties of lfs_fs_findfreeblocks
dbe4598 Added API boilerplate for lfs_fs_findfreeblocks and consistent style
d85a0fe Move lookahead buffer offset at the first free block if such block doesn't exist move it for whole lookahead size.
b637379 Update lfs_find_free_blocks to match the latest changes.
23505fa Added lfs_fs_grow for growing the filesystem to a different block_count
2c222af Tweaked lfs_fsinfo block_size/block_count fields
127d84b Added a couple mixed/unknown block_count tests
027331b Adopted erase_size/erase_count config in test block-devices/runners
9c23329 Revert of refactor lfs_scan_* out of lfs_format
23089d5 remove previous block_count detection from lfs_format
d6098bd Add block_count and block_size to fsinfo
d6c0c6a linting
5caa83f forgot to unmount lfs in test; leaking memory
7521e0a fix newly introduced missing cleanup when an invalid superblock is found.
2ebfec7 test for failure when interpretting block count when formatting without superblock
3d0bcf4 Add test_superblocks_mount_unknown_block_count
6de3fc6 fix corruption check
df238eb Add a unit test; currently hanging on final permutation.
be68122 introduce lfs->block_count. If cfg->block_count is 0, autopopulate from superblock
6dae703 remove redundant superblock check
7328527 refactor lfs_scan_for_state_updates and lfs_scan_for_superblock out of lfs_format

5a834b6 initlize struct lfs_diskoff disk = {0} so we don't use it uninitlized in first run
d775b46 Fixed integer conversion warning from Code Composer Studio
96fb8be Fixed release script breaking if there is no previous version

This is a small release to add some features that help with the previous on-disk minor-version bump, per feedback from @rojer, @TjoBitDk, and others.

What's new?

• Added lfs_fs_stat as an analog for the POSIX stavfs (#838)

  This now provides access to the on-disk format-time configuration:

  • disk_version - on-disk version
  • name_max - configurable name limit
  • file_max - configurable name limit
  • attr_max - configurable name limit

  And may be extended in the future. Notably, the current block avail/usage is missing due to performance concerns, but can be retrieved with an explicit call to lfs_fs_size.

• Added support for writing previous on-disk minor-versions (#839)

  This feature is behind a compile-time flag, so in order to write lfs2.0 you need to compile with LFS_MULTIVERSION defined. LFS_MULTIVERSION enables an additional config option, disk_version, which currently supports three options:

  • disk_version=0 => write the most recent littlefs version
  • disk_version=0x00020001 => write lfs2.1
  • disk_version=0x00020000 => write lfs2.0 (no FCRC)

Changes

a942cdb Bumped minor version to v2.7
79cc75d Added LFS_MULTIVERSION and testing of lfs2.0 to CI
eb9af7a Added LFS_MULTIVERSION, made lfs2.0 support a compile-time option
b72c96d Added support for writing on-disk version lfs2.0
265692e Removed fsinfo.block_usage for now
08a132e Added a link to ChaN's FatFS implementation
c5fb3f1 Changed fsinfo.minor_version -> fsinfo.disk_version
8610f7c Increased context on failures for Valgrind in CI
a51be18 Removed previous-version lfsp_fs_stat checks in test_compat
a7ccc1d Promoted lfs_gstate_needssuperblock to be available in readonly builds
fdee127 Removed use of LFS_VERSION in test_compat
87bbf1d Added lfs_fs_stat for access to filesystem status/configuration

8a4ee65 Removed unnecessary sys/types.h include
6fda813 Fix build for AVR
f2bc6a8 Reclassify .toml files as .c files on GitHub

Note!

This release bumps the on-disk minor version of littlefs from lfs2.0 -> lfs2.1.

This change is backwards-compatible, but after the first write with the new version, the image on disk will no longer be mountable by older versions of littlefs.

Read-only operations will not change the on-disk version, and the on-disk version can be explicitly bumped with lfs_fs_mkconsistent if desired.

This is the first on-disk minor version bump for littlefs, and is untested waters, so do open a GitHub issue if this creates any problems.

What's new?

• FCRCs - forward-looking erase-state CRCs (#497)

  Found by @pjsg, littlefs was relying on a bad assumption about the order/atomicity of prog operations to detect incomplete metadata commits. In the case of power-failure, this could lead to littlefs progging the same location multiple times without an erase, leading to best case fixable corruption, worst case significantly reduced data retention.

  The solution implemented here is to store a checksum of the next erased prog-size in every commit. This way any incomplete commits that modify the erased state can be detected.

  In theory this could be implemented in a forward-compatible way, but littlefs currently treats all CRC tags as commit-ending CRC tags, so only updated versions of littlefs will be able to read metadata blocks containing FCRCs. This is the reason for the on-disk minor version bump. On the bright-side, future CRC-related extensions will be forward-compatible.

• A complete overhaul of the testing/bench framework (#752)

  This does not affect users, but involves a long list of needed changes to make it easier to develop/bench littlefs:

  • Tests/benches now run in parallel, and much faster thanks to shared processes.
  • Tests are more reproducible, and narrow bugs can be reproduced locally repeatedly without leaving GDB.
  • More options for power-loss testing. Exhaustive testing of all 1 and 2-deep powerlosses have been added to CI.
  • More scripts related to perf measurements, plotting, tracing, etc.
  • A make help rule to help use/document the Makefile, which is sort of the entry point for littlefs development.
  • Added a CI rule to have @geky-bot comment on each PR with CI measurements (examples here). We'll see if this is an improvement over the GitHub statuses.

  The increased power-loss coverage has already found a couple of difficult to hit bugs (eba5553 and 0b11ce0), and the bench runner will be useful for evaluating littlefs's performance moving forward.

• Added a couple more asserts to help common implementation issues (#801, #802)

• Adopted Brent's algorithm for cycle detection (#809)

  This should significantly speed up cycle detection in the case the metadata linked-list contains a cycle.

• Added lfs_fs_mkconsistent as proposed by @kasper0 (#812)

  lfs_fs_mkconsistent allows users to front-load the internal consistency checks littlefs needs to run before it can write to the filesystem.

  Conveniently this also allows a way to force littlefs to write the new on-disk minor version if that is desired.

• A number of bug fixes, thanks to @ghost, @ajaybhargav, @colin-foster-in-advantage, @rvanschoren, @hgspbs, @sosthene-nitrokey, @kasper0, @Ldd309, and others (#800, #805, #811):

  • #800 - Fixed issues with truncate around block-aligned boundaries
  • #805 - Fixed issue where seeking to end-of-directory return LFS_ERR_INVAL
  • #811 - Fixed issue where lfs_fs_deorphan may run more than needed
  • eba5553 - Fixed issue where half-orphans can hide full orphans in nested power-loss situations
  • 0b11ce0 - Fixed incorrect calculation of extra space needed in mdir blocks
  • ba1c764 - Fixed issue where deorphan could get stuck circling between two half-orpans in power-loss situations

• And a number of links to useful community projects related to littlefs, thanks to @tniessen, @yomimono, @elpiel (#572, #718, #807):

  • littlefs-disk-img-viewer
  • chamelon
  • littlefs2-rust

  If you have a littlefs-related project or a project that uses littlefs, consider opening a PR. Eventually my plan is to move these to a user-contributable wiki.

Changes

259535e Added lfs_fs_mkconsistent
94d9e09 Fixed issue where lfs_fs_deorphan may run more than needed
363a8b5 Tweaked wording of littlefs2-rust link in README.md
e43d381 chore: add littlefs2 crate to README
3e25dfc Added FCRC tags and an explanation of how FCRCs work to SPEC.md
9e28c75 Bumped minor version to v2.6 and on-disk minor version to lfs2.1
4c93600 Added ability to bump on-disk minor version
ca0da3d Added compatibility testing on pull-request to GitHub test action
116332d Added tests for forwards and backwards disk compatibility
f0cc1db Tweaked changeprefix.py to not rename dir component in paths
bf045dd Tweaked link to littlefs-disk-img-viewer to go to github repo
b33a5b3 Fixed issue where seeking to end-of-directory return LFS_ERR_INVAL
384a498 Extend dir seek tests to include seeking to end of directory
b0a4a44 Added explicit assert for minimum block size of 128 bytes
aae897f Added an assert for truthy-preserving bool conversions
e57402c Added ability to revert to inline file in lfs_file_truncate
6dc18c3 Fixed block-boundary truncate issue
d5dc487 Expanded truncate tests to test more corner cases
24795e6 Add missing iterations in tests
7b151e1 Add test scenario for truncating to a block size
ba1c764 Fixed issue where deorphan could get stuck circling between two half-orphans
d1b254d Reverted removal of 1-bit counter threaded through tags
2f26966 Continued implementation of forward-crcs, adopted new test runners
b4091c6 Switched to separate-tag encoding of forward-looking CRCs
91ad673 Cleaned up a few additional commit corner cases
52dd830 Initial implementation of forward-looking erase-state CRCs
1278ec1 Adopted Brent's algorithm for cycle detection
c2147c4 Added --gdb-pl to test.py for breaking on specific powerlosses
801cf27 Tweaked/fixed a number of small runner things after a bit of use
1f37eb5 Adopted --subplot* in plot.py
cfd4e60 Added --subplot* to plotmpl.py
2d2dd8b Added plotmpl.py --github flag to match the website's foreground/background
b0382fa Added BENCH/TEST_PRNG, replacing other ad-hoc sources of randomness
d8e7ffb Changed lfs_emubd_get* -> lfs_emubd_*
cda2f6f Changed test_runner to run with -Pnone,linear by default
9b687dd Added make benchmarks/testmarks rules
c4b3e9d A couple of script changes after CI integration
9990342 Fixed Clang testing in CI, removed override vars in Makefile
4a20934 Fixed bench workflow + changeprefix issue in prefix releases
a659c02 Added a bot-generated PR-comment with a simple status table
397aa27 Removed unnecessarily heavy RAM usage from logs in bench/test.py
65923cd Adopted script changes in GitHub Actions
387cf6f Fixed a couple corner cases in scripts when fields are empty
0b11ce0 Fixed incorrect calculation of extra space needed in mdir blocks
eba5553 Fixed hidden orphans by separating deorphan search into two passes
f89d758 Fixed test out-of-space issues with powerloss testing
6c18b4d Added a simple help rule to the Makefile
f734941 Changed default build target lfs.a -> liblfs.a
bcc88f5 A couple Makefile-related tweaks
e35e078 Renamed prefix.py -> changeprefix.py and updated to use argparse
1a07c2c A number of small script fixes/tweaks from usage
6fce9e5 Changed plotmpl.py/plot.py to not treat missing values as discontinuities
559e174 Added plotmpl.py for creating svg/png plots with matplotlib
b2a2cc9 Added teepipe.py and watch.py
3a33c37 Added perfbd.py and block device performance sampling in bench-runner
29cbafe Renamed coverage.py -> cov.py
df283ae Added recursive results to perf.py
490e1c4 Added perf.py a wrapper around Linux's perf tool for perf sampling
ca66993 Tweaked scripts to share more code, added coverage calls/hits
296c5af Renamed bench_read/prog/erased -> bench_readed/proged/erased
274222b Added some automatic sizing for field-names in scripts/runners
a2fb708 Added stddev/gmean/gstddev to summary.py
9507e62 Several tweaks to script flags
42d889e Reworked/simplified tracebd.py a bit
fb58148 Consistent handling of by/field arguments for plot.py and summary.py
7591d9c Added plot.py for in-terminal plotting
9a0e3be Added a quick trie to avoid running redundant test/bench permutations
4fe0738 Added bench.py and bench_runner.c for benchmarking
20ec0be Cleaned up a number of small tweaks in the scripts
11d6d12 Dropped namespacing of test cases
1fcd82d Made test.py output parsable by summary.py
acdea18 Made summary.py more powerful, dropped -m from size scripts
23fba40 Added option for updating a CSV file with test results
03c1a4e Added permutations and ranges to test defines
bfbe44e Dropped permutation number for full leb16-encoded defines
5a2ff17 Changed test identifier separator # -> :
c7f7094 Several tweaks to test.py and test runner
a208d84 Reworked test defines a bit to use one common array layout
91200e6 Added tracebd.py, a script for rendering block device operations
c9a6e3a Added tailpipe.py and improved redirecting test trace/log output over fifos
5279fc6 Implemented exhaustive testing of n nested powerlosses
552336e Added optional read/prog/erase delays to testbd
3f4f859 Readded support for mirror writes to a file in testbd
4689678 Added --color to test.py, fixed some terminal-clobbering issues
61455b6 Added back heuristic-based power-loss testing
01b11da Added a simple test that the block device works
a368d3a Moved emulation of erase values up into lfs_testbd
b08463f Reworked scripts/pretty_asserts.py a bit
92eee8e Removed some prefixes from Makefile variables where not necessary
d9333ec Add "chamelon" to the related projects section.
46cc6d4 Added support for annotated source in coverage.py
5b0a6d4 Reworked scripts to move field details into classes
4a7e94f Reimplemented coverage.py, using only gcov and with line+branch coverage
2b11f2b Tweaked generation of .cgi files, error code for recursion in stack.py
1616115 Fix test.py hang on ctrl-C, cleanup TODOs
4a42326 Moved test suites into custom linker section
0781f50 Ported tests to new framework
d679fbb In ./scripts/test.py, readded external commands, tweaked subprocesses
5a572ce Reworked how test defines are implemented to support recursion
be0e6ad More progress toward test-runner feature parity
4962829 Continued progress toward feature parity with new test-runner
5ee4b05 Misc test-runner improvements
5812d2b Reworked how multi-layered defines work in the test-runner
6443693 Putting together rewritten test.py script
92a600a Added trace and persist flags to test_runner
9281ce2 More test_runner progress
4b0aa62 Some more minor improvements to the test_runner
d683f1c Reintroduced test-defines into the new test_runner
56a9903 Created new test_runner.c and test_.py
3ae87f4 Add littlefs-disk-img-viewer to README

740d9ac Fix readme Mbed link
d08f949 Fixed lfs_dir_fetchmatch not propogating bd errors correctly in one case
9e965a8 lfs_util: Fix endiannes conversion when LFS_NO_INTRINSICS is set
eb9f4d5 Fix invalid block size reporting.
a25681b Improve lfs_file_close usage description
47914b9 Fixed self-assign warnings discovered by clang
30175de Remove -Wshadow -Wjump-misses-init -Wundef
2374762 Added clang build step to CI
a405c32 lfs_filebd_sync: fix compilation on Windows
9af63b3 Fix buffer overflow in tests when using a large block size
6c720dc Fix unused function warning with LFS_NO_MALLOC

What's new?

• Removed all recursion (#658)

  On paper, littlefs promises O(1) RAM consumption, however in practice, this hasn't quite been true due to the use of recursion in several parts of the implementation. 2 instances of recursion were bounded at runtime, but 1 instance could be unbounded, though this would require many block relocations to all occur simultaneously.

  These pieces have now been rewritten to strictly avoid recursion, at the cost of some code size (<=5.1%). This means littlefs now lives up to its promise of bounded RAM, and the total RAM usage can be easily evaluated using static tooling.

• Added several new scripts for measuring RAM usage, including detection of future recursion (#658)

  These have also been integrated into CI, so data, stack, and struct usage will now be reported with all CI jobs.

• Several corner-case performance improvements thanks to @invoxiaamo and @robekras:

  • lfs_dir_traverse no longer exhibits a nasty O(n^3) performance spike when a rename occurs at the same time a directory is being compacted (#666)

    @invoxiaamo notes a reduction in lfs_dir_traverse calls by 97.8% (3248 vs 148393), with a number of other proposal to improve this with additional RAM caching in #621.

    This change also reduces stack consumption of lfs_dir_traverse by ~25%.

  • lfs_file_seek now avoids flushing when seeking inside the cache while reading (#632)

    @robekras notes a ~90% (2 sec vs 20 sec) speedup when loading a sparse binary font file into LVGL.

• Several improvements to API robustness thanks to @colin-foster-in-advantage, @nnayo, @Johnxjj, and @lmapii:

  • Attempting to mount a littlefs image with a block_size or block_count that does not match the configuration will now error (#584)

  • lfs_file_open is now omitted when building with LFS_NO_MALLOC, previously this would always be a runtime error (#614)

  • Seeking to a negative offset is now an error, instead of underflowing (#630)

  • lfs_file_open now returns LFS_ERR_NAMETOOLONG if it can't fit the name + file metadata into a metadata block, previously this would return LFS_ERR_NOSPC (#638)

• Thanks to @m8ddin, lfs_filebd now works on Windows (#643)

• The copyright notice has been appended to reflect ownership changing to "the littlefs authors" (#657)

Changes

148e312 Bumped minor version to v2.5
abbfe8e Reduced lfs_dir_traverse's explicit stack to 3 frames
1e038c8 Fixes to use lfs_filebd on windows platforms
c2fa1bb Optimization of the rename case.
3b62ec1 Updated error handling for NOSPC
b898977 Set the limit, the cursor cannot be set to a negative number
cf274e6 Squash of CR changes
425dc81 Modified robekras's optimization to avoid flush for all seeks in cache
a6f01b7 Update lfs.c
2db5dc8 Update copyright notice
8109f28 Removed recursion from lfs_dir_traverse
fedf646 Removed recursion in file read/writes
84da4c0 Removed recursion from commit/relocate code path
554e4b1 Fixed Popen deadlock issue in test.py
fe8f3d4 Changed./scripts/struct.py to organize by header file
316b019 In CI, determine loop devices dynamically to avoid conflicts with Ubuntu snaps
8475c80 Limit ./scripts/structs.py to report structs in local .h files
563af5f Cleaned up make clean
3b495ba Fixed spurious CI failure caused by multiple writers to .o files
e4adefd Fixed spurious encoding error
9d54603 Added new scripts to CI results
7ea2b51 A few more tweaks to scripts
55b3c53 Added ./script/summary.py
eb8be9f Some improvements to size scripts
50ad2ad Added make *-diff rules, quick commands to compare sizes
0a2ff3b Added scripts/structs.py for getting sizes of structs
d7582ef Changed script's CSV formats to allow for merging different measurements
f4c7af7 Added scripts/stack.py for viewing stack usage
20c58dc Added coverage-sort to scripts/coverage.py
f5286ab Added scripts/calls.py for viewing the callgraph directly
2cdabe8 Split out scripts/code.py into scripts/code.py and scripts/data.py
b045436 Added size-sort options to scripts/code.py
e334983 don't use lfs_file_open() when LFS_NO_MALLOC is set
487df12 Fail when block_size doesn't match config
3efb8e4 Fail mount when the block size changes

9c7e232 Fixed missing definition of lfs_cache_drop in readonly mode
03f088b Tweaked lfs_file_flush to still flush caches when build under LFS_READONLY
e955b9f Fix lfs_file_seek doesn't update cache properties correctly in readonly mode. Invalidate cache to fix it.
1363c9f fix bug:lfs_alloc will alloc one block repeatedly in multiple split
5bc682a Typo
1877c40 Indent sub-portions of tag fields
e29e7ae Specify unit of the size members of the lfs_config struct
4977fa0 Fix spelling errors
fdda3b4 Always zero rambd buffer before first use
fb2c311 Fix compiler warnings when LFS_READONLY defined

1e423ba Fix link to test block device
3bee4d9 scripts/test.py: Fix infinite busy loops on macOS

What's new?

• Graceful LFS_ASSERT (#514)

  Thanks to @mon, @davidefer, and everyone else who provided feedback, littlefs now supports overriding LFS_ASSERT with a macro that can return an error instead of halting.

  This doesn't change the behavior LFS_ASSERT itself, but just means littlefs only uses LFS_ASSERT in functions that return a signed error code. This is an internal rule littlefs will follow, and thanks to @mon we now have a CI job to prevent breaking this behavior. Outside of littlefs, you're free to use LFS_ASSERT in non-erroring functions, you'll just lose the ability to override LFS_ASSERT in this way.

• New metadata_max config (#502)

  A great idea from @mon, metadata_max provides a workaround for performance issues littlefs faces with large block sizes (https://github.com/littlefs-project/littlefs/issues/203).

  metadata_max reduces the performance hit of metadata compaction by artificially limiting how much of a metadata block we use. This improves performance since we there is less metadata to compact, but does mean we waste whatever space in the block we don't use. It's interesting tradeoff since the devices where block sizes are large enough for this to be an issue (NAND) also have enough storage to throw at this problem. It's not the best solution, but may make it possible to get past some of these performance bottlenecks.

  @mon notes a 10x speedup in directory traversal/file deletion and a 2x-4x speed up in file reads on a 128KiB NAND block device (MT29F) with metadata_max set to 4KiB.

• Adopt GitHub Actions for CI, and a number of script/Makefile improvements (#516)

  Since this is primarily for development and doesn't affect most users, I will just point to the PR notes in #516.

  If everything works, you will probably already see the new code size/coverage measurements automatically appended to these release notes.

  One of the more interesting things to note is the addition of a post-release process to automatically create PRs against dependent repos. This should help keep dependents such as littlefs-fuse no longer constantly-out-of-date. If you want to add a repo to receive automatic PRs feel free to create a PR against the littlefs.post-release repo.

• Thanks to @embeddedt, lfs_file_seek has a new optimization to avoid flushing caches when seeking to the same position (#511)

• littlefs now performs a sanity check in lfs_format after compacting the superblock, instead of before (#508)

Changes

3d4e4f2 Bumped minor version to v2.4
c0cc0a4 Enabled overriding of LFS_ASSERT/TRACE/DEBUG/etc
c9eed1f Add test to ensure asserts can return
e7e4b35 lfs_fs_preporphans ret int for graceful LFS_ASSERT
21488d9 Fixed incorrect documentation in test.py
10a0883 Moved lfs_mdir_isopen behind LFS_NO_ASSERT
47d6b2f Removed unnecessary truncate condition thanks to new seek optimization
745d98c Fixed lfs_file_truncate issue where internal state may not be flushed
3216b07 Use lfs_file_rawsize to calculate LFS_SEEK_END position
6592719 Removed .travis.yml
c911061 Added post-release script, cleaned up workflows
104d651 Reduced build sources to just the core littlefs
6d3e4ac Brought over the release workflow
9d65460 Fixed a recompilation issue in CI, tweaked coverage.py a bit more
b84fb6b Added BUILDDIR, a bit of script reworking
887f366 Switched to lcov for coverage collection, greatly simplified coverage.py
eeeceb9 Added coverage.py, and optional coverage info to test.py
b2235e9 Added GitHub workflows to run tests
6bb4043 Skip flushing file if lfs_file_rawseek() doesn't change position
2b80453 Moved sanity check in lfs_format after compaction
d804c2d Added scripts/code_size.py, for more in-depth code-size reporting
37f4de2 Remove inline_files_max and lfs_t entry for metadata_max
6b16daf Add metadata_max and inline_file_max to config

What's new?

Thanks to @maximevince, @renesas-billgesner, and @roykuper13, we now have two impactful compile-time options for littlefs!

• LFS_READONLY

  Define LFS_READONLY to enable read-only mode. This will limit littlefs to functions that do not modify the disk, but will reduce the code cost to ~1/3. This is a great addition for bootloaders or other applications where a small, read-only storage blob is useful.

• LFS_THREADSAFE

  Define LFS_THREADSAFE to enable thread-safe mode. Note this needs the lock and unlock callbacks in the lfs_config struct.

  When defined, littlefs will call the related lock and unlock callbacks when necessary to synchronize the filesystem. Allowing easy integration into an RTOS without needing an OS or wrapper layer.

In addition, this release brings in a number of improvements:

• More strict user asserts on common file/dir open mistakes thanks to @NoahGorny (https://github.com/littlefs-project/littlefs/pull/443)
• More efficient, 8-byte lfs_bd_cmp thanks to @gmpy (see https://github.com/littlefs-project/littlefs/pull/395 for measurements)
• A number of wear-leveling fixes thanks to @guiserle and @gtaska (https://github.com/littlefs-project/littlefs/pull/487)
• Fixes for allocation-eviction math thanks to @tim-nordell-nimbelink (https://github.com/littlefs-project/littlefs/pull/490)

Here are the current code costs for the different littlefs configurations in CI:

Changes

6a70127 Renamed internal lfs_*raw -> lfs_raw* functions
288a5cb Bumped minor version to v2.3
2bb5234 Moved lfs_mlist_isopen checks into the API wrappers
7388b29 Deprecate LFS_F_OPENED and use lfs_mlist_isused instead
a99a93f Added thread-safe build+size reporting to CI
45afded Moved LFS_TRACE calls to API wrapper functions
00a9ba7 Tweaked thread-safe implementation
fc6988c make raw functions static. formatting tweaks
d0f055d Squash of thread-safe PR cleanup
2efebf8 Added read-only build+size reporting to CI
754b4c3 Squash of LFS_READONLY cleanup
008ebc3 Add lfs_mlist_append/remove helper
b8dcf10 Changed lfs_dir_alloc to maximize block cycles for new metadata pairs
0aba71d Fixed single unchecked bit during commit verification
0ea2871 Fixed typo in scripts/readtree.py
d04c139 Fixed allocation-eviction issue when erase state is multiple of block_cycles+1
f215027 Switched to CRC as seed collection function instead of xor
1ae4b36 Removed unnecessary randomization of offsets in lfs_alloc_reset
480cdd9 Fixed incorrect modulus in lfs_alloc_reset
6303558 Use LFS_O_RDWR instead of magic number in lfs_file_* asserts
4bd653d Assert that file/dir struct is not reused in lfs_file_opencfg/lfs_dir_open
8e6826c Add LFS_READYONLY define, to allow smaller builds providing read-only mode
10ac6b9 add thread safe wrappers
87a2cb0 Fix assert
6d0ec5e Added littlefs-python to the related projects section
64f70f5 lfs_bd_cmp() compares more bytes at one time

5a9f38d Remove -Wno-missing-field-initializers
1b033e9 Fix -Wmissing-field-initializers

What's new?

This release brings in a revamped testing framework intended to improve the rate of bug isolation and fixes. There are new runtime contexts for tests such as simulated power-loss and Valgrind based memory checking, and additional scripts for printing out metadata to debug the filesystem (readtree.py/readmdir.py/readblock.py in the scripts directory). This also comes with a large number of bug fixes as detailed in https://github.com/ARMmbed/littlefs/pull/372.

More details on this can be found in the overly wordy https://github.com/ARMmbed/littlefs/pull/372.

Other changes:

• Introduced rambd and filebd, two block devices that replace emubd for simulating block devices locally
• Changed asserts on corrupted filesystem to return errors, allowing additional user recover
• A number of quality-of-life improvements to logging statements
• Reenabled cache bypassing if the filesystem is handling data that is aligned to the block device's requirements
• Did I mention a large number of bug fixes? There's a lot of bugs fixed in this release, see https://github.com/ARMmbed/littlefs/pull/372

Changes

6372f51 Allow debug prints without VA_ARGS
6622f3d Bumped minor version to v2.2
5137e4b Last minute tweaks to debug scripts
ff84902 Moved out block device tracing into separate define
f9dbec3 Added test case catching issues with errors during a lookahead scan
f17d3d7 Minor cleanup
5e5b5d8 (chore) updates from PR, we decided not to move forward with changing v1 code since it can be risky. Let's improve the future! Also renamed and moved around a the lookahead free / reset function
d498b9f (bugfix) adding line function to clear out all the global 'free' information so that we can reset it after a failed traversal
4677421 Added "evil" tests and detecion/recovery from bad pointers and infinite loops
cb26157 Change assert to runtime check.
a7dfae4 Minor tweaks to debugging scripts, fixed explode_asserts.py off-by-1
50fe8ae Renamed test_format -> test_superblocks, tweaked superblock tests
0990296 Limited byte-level tests to native testing due to time
d04b077 Fixed minor things to get CI passing again
c7987a3 Restructured .travis.yml to span more jobs
dcae185 Fixed typo in LFS_MKTAG_IF_ELSE
f4b17b3 Added test.py support for tmpfs-backed disks
9f546f1 Updated .travis.yml and added additional geometry constraints
b69cf89 Fixed CRC check when prog_size causes multiple CRCs per commit
02c84ac Cleaned up dependent fixes on branch
6530cb3 Fixed lfs_fs_size doubling metadata-pairs
fe957de Fixed broken wear-leveling when block_cycles = 2n-1
6a55084 Modified readmdir/readtree to make reading non-truncated data easier
f9c2fd9 Removed file outlining on ENOSPC in lfs_file_sync
44d7112 Fixed tests/.toml. in .gitignore
77e3078 Added/fixed tests for noop writes (where bd error can't be trusted)
517d341 Fixed more bugs, mostly related to ENOSPC on different geometries
4fb1883 Update SPEC.md
c8e9a64 Indicate C99 standard as target for LittleFS code
aab6aa0 Cleaned up test script and directory naming
52ef0c1 Fixed a crazy consistency issue in test.py
b9d0695 Rewrote explode_asserts.py to be more efficient
a5d614f Added tests for power-cycled-relocations and fixed the bugs that fell out
f4b6a6b Fixed issues with neighbor updates during moves
9453ebd Added/improved disk-reading debug scripts
fb65057 Restructured block devices again for better test exploitation
ecc2857 Migrated bad-block tests
5181ce6 Migrated the first of the tests with internal knowledge
b06ce54 Migrated the bulk of the feature-specific tests
1d2688a Migrated test_files, test_dirs, test_format suites to new framework
eeaf536 Replaced emubd with rambd and filebd
626006a Fix incorrect comment on lfs_npw2
53d2b02 Added reentrant and gdb testing mechanisms to test framework
ed8341e Reworked permutation generation in test framework and cleanup
f42e007 Created initial implementation of revamped test.py
5a12c44 Revert "Don't bypass cache in lfs_cache_prog() and lfs_cache_read()"

ce2c01f Fixed lfs_dir_fetchmatch not understanding overwritten tags
0197b18 Fixed issue with superblock breaking lfs_dir_seek
d6dc728 Fixed some issues in lfs_migrate
aeff2a2 Stop wear-leveling during migration
aae22c8 Fixed issue with directories falling out of date after block relocation
60e67ae Fixed implicit change-of-sign warning in lfs_dir_fetch
64dedee prepare upstream bugfix of lfs
5925db4 Fix some LFS_TRACE format specifiers
ab56dc5 README: fix incorrect description