moosefs

MooseFS Distributed Storage – OpenΒ Source, Petabyte, Fault-Tolerant, Highly Performing, Scalable Network Distributed File System / Software-Defined Storage

GPL-2.0 License

Stars
1.7K
Committers
14

Bot releases are hidden (Show)

moosefs - MooseFS 3.0.118 Latest Release

Published by pkonopelko 2 months ago

Dear MooseFS Users,

We are happy to announce that MooseFS 3.0.118 is released! πŸŽ‰

This is mostly a bugfix release. It also introduces a few improvements and new features, including:

  • MooseFS is now cross-build friendly,
  • New moosefs man page.

We would like to say big thanks to all the contributors who helped us making this release, especially (alphabetically): @asyslinux, @tianon, @xctan. Thank you for your valuable time, input, and help in making MooseFS better!
(If I forgot to mention somebody, I am sorry for that and please definitely let me know!)

Please find a complete list of changes included in this release below:

  • (all) regenerated autoconf files (issue #525)
  • (scripts) added simple class that replaces deprecated cgi and cgitb (issue #567)
  • (bdev) fixed unmap protocol (issue #551)
  • (cgi) added protections against javascript injection
  • (all) moved build aux files to separate folder build-aux and made moosefs cross-build friendly
  • (all) housekeeping changes (year, regenerated with new autotools, silenced some compiler warnings)
  • (all) updated README and README.md files
  • (man) added new man page moosefs

We recommend to update your MooseFS clusters to this version.

Thank you,
Piotr / MooseFS Team

moosefs - MooseFS 3.0.117

Published by pkonopelko over 1 year ago

Dear MooseFS Users,

We are happy to announce that MooseFS 3.0.117 is released! πŸŽ‰

This is mostly a bugfix release. It also introduces a few improvements and new features, including:

  • Displaying an error message when a config file for Master Server or Chunkserver cannot be loaded,
  • An option to ignore locks for MooseFS Block Device (mfsbdev / nbd),
  • A change of default value for ATIME_MODE to 2 (similar to relatime).

We would like to say big thanks to all the contributors who helped us making this release, especially (alphabetically): @alexcrow1974, @alpharde, @borkd, @deltabweb, @eleaner, @footlooseboss, @inkdot7, @jkiebzak, @jSML4ThWwBID69YC, @nickb937, @njaard, @onlyjob, @struthio, @tianon, @zcalusic, @Zorlin. Thank you for your valuable time, input, and help in making MooseFS better!

(If I forgot to mention somebody, I am sorry for that and please definitely let me know!)

Please find a complete list of changes included in this release below:

  • (master) fixed handling abnormal operation status (issue #456)
  • (master+cs) added error message when a config file can't be loaded (issue #457)
  • (client) fixed invalidating directory cache after unlink, rmdir and rename
  • (cs) fixed cond variables leaking
  • (all) silence silly compiler warnings
  • (cgi+cli) fixed division by zero when scount is zero in arch mode (no chunkservers – issue #412)
  • (client) fixed freeing atime/mtime data structure
  • (client) added displaying build number
  • (master) silenced more wrong compiler warnings
  • (nbd) added option to ignore locks (issue #353)
  • (master) fixed file mode in changelog (problem with mask when parent has default ACLs)
  • (master) reduced function stack frame usage (slow down due to use -fstack-clash-protection by some OS'es)
  • (master) fixed handling .. and . in fs_lookup (issue #473)
  • (all) using SIGUSR1 instead of SIGINFO even on system where SIGINFO is defined
  • (master) changed default ATIME_MODE to 2 (issue #478)
  • (master) fixed atime/mtime/ctime preservation during snapshot (in HA environment upgrade may cause one time desync; issue #489)
  • (client) fixed race condition between client and kernel in retrieving open file list (FreeBSD only)
  • (master) fixed bug in sorting chunkservers during selection for chunk creation (issue #481)

We recommend to update your MooseFS clusters to this version.

Best regards,
Piotr / MooseFS Team

moosefs - MooseFS 3.0.116

Published by pkonopelko about 3 years ago

Dear MooseFS Users,

We are happy to announce that MooseFS 3.0.116 is released! πŸŽ‰

This is mostly a bugfix release. It also introduces a few improvements and new features, including:

  • Possibility to set symlink cache timeout (-o mfssymlinkcacheto mounting parameter),
  • New INODE_REUSE_DELAY config option for defining inode reuse delay in Master Server (in mfsmaster.cfg),
  • Allowing servers in rebalance state to be used in replication,
  • Protection against listing too big directory.

We would like to say big thanks to all the contributors who helped us making this release, especially (alphabetically): @BillKen, @Eronana, @jSML4ThWwBID69YC, @onlyjob, @stevew-Purdue, @struthio, @tianon. Thank you for your valuable time, input, and help in making MooseFS better!

(If I forgot to mention somebody, I am sorry for that and please definitely let me know!)

Please find a complete list of changes included in this release below:

  • (check) making clock test less prone to a single failure,
  • (master) added protection against listing too big directory,
  • (master) servers in rebalance state are now allowed to be used in replication,
  • (cs) changed handling load+hlstatus packet,
  • (mount) added symlink cache timeout,
  • (master) added option to define inode reuse delay,
  • (cgi+cli) fixed division by zero when scount is zero (no chunkservers – issue #412),
  • (master) added inode virtual length for append mode,
  • (mount) moved rwlock from descriptor to inode (fixes append issues),
  • (cgi) changed method of charts selection in charts comparison,
  • (cgi) added message to function that handles exceptions,
  • (cgi) added using htmlentities function for various strings,
  • (mount) removed old osxfuse patch (doesn't work with current osxfuse and is no longer needed),
  • (master) fixed default paths for mfsexports.cfg and mfstopology.cfg,
  • (master) fixed restore mode (missing cache initialization – issue #432),
  • (mount) added parents of CWDs (current working directories) to sustained inodes (workaround for FreeBSD problem – issue #362),
  • (all) fixed spelling,
  • (master) fixed handling copy sgid for directories on follower and during restore from changelog,
  • (master) fixed changelog order between unlock and close file or session,
  • (cgi) fixed using & in javascript URLs (issue #439),
  • (cgiserv) fixed error/redirect responses (Pull Request #443 by @Eronana),
  • (master+cs) fixed overflows in cgi charts,
  • (cs) fixed crc data in info packets (*_CHUNK_CHECKSUM, *_CHUNK_CHECKSUM_TAB).

We recommend to update your MooseFS clusters to this version.

Best regards,
Piotr / MooseFS Team

moosefs - MooseFS 3.0.115

Published by pkonopelko about 4 years ago

Dear MooseFS Users,

We are happy to announce that MooseFS 3.0.115 is now stable!

This is a bugfix release. We would like to say big thanks to all the contributors who helped us making this release, especially (alphabetically): @borkd, @eleaner, @richarson. Thank you for your valuable time, input, and help in making MooseFS better!

(If I forgot to mention somebody, I am sorry and please let me know!)

Please find a complete list of fixes and new features that this release includes, below:

  • (cs) fixed chunk locking issues in chunkserver (lock timeout, missing release etc.)
  • (all) changed to hex all chunkid logged to syslog (only in changelog it is left in decimal format)
  • (cs) changed port number from hex to dec in replicator module
  • (mount) added printing open/create mode to the .oplog
  • (mount) fixed opening file after using lookup cache
  • (master) added ignoring empty filenames

We recommend to update your MooseFS cluster(s) to this version.

Best regards,
Piotr / MooseFS Team

moosefs - MooseFS 3.0.114

Published by pkonopelko about 4 years ago

Dear MooseFS Users,

We are happy to announce that MooseFS 3.0.114 is now stable!

This is mostly a bugfix release, however it also introduces a few improvements, including:

  • Improved internal rebalance algorithm in Chunkserver,
  • More information reported in .params special file on MooseFS Client side,
  • Changes in job queue policy in Chunkserver,
  • Changed time format in MooseFS CGI Monitoring Interface,
  • Improvements related to chunks database file (.chunkdb) on Chunkserver (logging, write error detection).

We would like to say big thanks to all the contributors who helped us making this release, especially (alphabetically):
@BillKen, @jSML4ThWwBID69YC, @kopaczew, @onlyjob, @uu.
Thank you for your valuable time, input, and help with making MooseFS better!

(If I forgot to mention somebody, I am sorry and please let me know!)

Please find a complete list of fixes and new features that this release includes, below:

  • (mfsio/nbd) fixed compile issue on systems without LOCK_* constants (issue #366)
  • (mount) fixed truncating files open with O_APPEND flag (bug introduced in 3.0.113, issue #368)
  • (cs) added disk rebalance when all designated source disks are already empty (issue #364)
  • (mount) added additional info to .params file (versions of MooseFS, libfuse etc.)
  • (master+cs) added check against EACCES after lockf (issue #369)
  • (master) fixed access checks in snapshot
  • (cgi+cli) removed using supervisor (only available in Pro version)
  • (cs) fixed deadlock condition on mutexes folderlock and hashlock (exists since 3.0.110)
  • (mount) fixed handling .params file
  • (tools) fixed overflow protection in number parsing function
  • (cs) fixed reporting to master duplicates with newer version
  • (cs) changed job queue policy (some tasks are not limited now)
  • (cli) fixed help and manpage
  • (master) fixed reaction to status NOTDONE received from chunkservers
  • (master) fixed syncing and closing negative file descriptor in bgsaver
  • (mount+master) fixed handling keepcache and direct flags (related to issue #374)
  • (mount) fixed compiling on systems with old fuse library
  • (cgi) changed time format (issue #197)
  • (cs) added logging info when .chunkdb is not written to disk
  • (master) fixed alphabetical order of commands detected in changelog
  • (cs) changed queue limit to max workers (limited dynamically)
  • (cs) fixed automatic chunkserver removal in master
  • (master) added meta version increment in chunks_set_version
  • (cs) added error detection during writing .chunkdb
  • (all) added build id to what strings
  • (all) added missing idstr.h and rearranging source lists in Makefiles
  • (nbd+cgiserv) added chdir("/")
  • (freebsd) updated rc.d master script
  • (man) updated mfsmetadump manpage
  • (man) fixed formatting in mfsmount manpage

Best regards,
Piotr / MooseFS Team

moosefs - MooseFS 3.0.113

Published by pkonopelko over 4 years ago

Dear MooseFS Users,

We are happy to announce that MooseFS 3.0.113 is now stable!

This is both bugfix and feature release. Apart from fixing the issues that were recently found, it introduces:

  • Support for new extra attributes: immutable, appendonly, undeletable.
    Extra attributes are MooseFS-specific feature that allows you to add / manage a small portion of metadata to files on MooseFS. There are several extra attributes and they change file system behaviour (apart from new ones mentioned above there are several more – please refer to man mfseattr for more information).
  • Topology grouping when new chunks are about to be created,
  • Readonly mode and locking in MooseFS Block Device,
  • Changing subfolder during internal rebalance,
  • Official labels format defined in chunkserver configuration,

and several more features.

This release improves Chunkserver a lot (registering chunks to Master Server, reporting damaged chunks, internal rebalance, reloading, calculating size limits, handling inode limits on underlying file systems). It also adds new and improves existing workarounds to issues related to OS's kernels and improves directory entry invalidator introduced in MooseFS 3.0.112.

We would like to thank all the contributors who helped us making this release, especially (alphabetically):
@borkd, @eleaner, @jSML4ThWwBID69YC, @onlyjob, @rafamiga, @tamas-ac.
Thank you for your valuable time, input, and help with making MooseFS better!

(If I forgot to mention somebody, I am sorry and please let me know!)

Please find a complete list of fixes and new features that this release includes, below:

  • (master) removed unnecessary debug syslog messages
  • (check) increased usleep tolerance due to some operating systems (issue #351)
  • (master) changed condition that decides if master should wait for more chunks during I/O
  • (mount) delayed setting channel for fuse notifications (very rare segfault in libfuse2 during init)
  • (client+master) changed conditions for ancestor test in getattr (issue #350)
  • (cs) added parameter for number of chunks to be send in single register packet
  • (cs) fixed reporting damaged chunks in testing function (issue #352)
  • (cs) removed some unnecessary damaged chunk notifications
  • (client) changed open test in setattr (related to issue #350)
  • (master) fixed handling truncate for open files (related to issue #350)
  • (master) added uid mapping in setfacl function
  • (master+client) added support for atomic truncate with open
  • (master+client) fixed keep cache conditions
  • (cs) introduced official label format defined in chunkserver configuration
  • (nbd) added readonly mode and locking (issue #353)
  • (cs) fixed condition in choosing disks for internal rebalance
  • (mount) added workaround in access for a bug in FreeBSD kernel (issue #354)
  • (cs) fixed master reconnection conditions (reload usually shouldn't cause reconnection)
  • (freebsd) fixed FreeBSD port (makeports.sh)
  • (master) changed disk removal detection algorithm (issue #356)
  • (cs) fixed calculating size limits
  • (cs) added handling inode limits on local filesystems on chunkservers (issue #358)
  • (master) added topology grouping when new chunks are about to be created (prefer closer servers)
  • (master) added more error messages to bgsaver
  • (cs) added changing subfolder during internal rebalance (related to issue #326)
  • (master+cs) added optional logging of long function execution times
  • (all) fixes of small errors found by static code analysers
  • (mount) turn off dentry invalidator for Linux kernels >= 4.19 (related to issue #357)
  • (master) changed condition that checks timestamp in changelogs (less prone to small clock differences)
  • (daemons) added time refresh function (for future use)
  • (cgi+cli) changed mark for removal state name UNKNOWN –> PENDING (related to issue #359)
  • (client) decreased max IDLE time in writer module from 1s to 0.1s
  • (client) update inode in dentry invalidator
  • (tools) fixed mfscopyeattr, mfsseteattr and mfs*archive tools
  • (master+client) added support for new eattrs: immutable, appendonly, undeletable (setting new undeletable extra attribute appropriately can help with issues like #357)
  • (client) silenced kern.proc.filedesc syslog messages on FreeBSD (issue #360)

Best regards,
Piotr / MooseFS Team

moosefs - MooseFS 3.0.112

Published by pkonopelko over 4 years ago

Dear MooseFS Users,

MooseFS 3.0.112 is now stable!

This release is a bugfix and feature release. It introduces support for disabling individual filesystem commands in Master Server (configurable in mfsexports.cfg) and respecting them on the Client side (issue #344). It also adds support for STRICT and LOOSE mode in KEEP and ARCHIVE (issue #249). This release fixes various other issues too, including a workaround for Linux kernel bug related to EBUSY issue by adding directory entry invalidator (issue #98).

We would like to thank all the contributors who helped us making this release, especially (alphabetically):
@borkd, @Damien9527, @linux-ops, @MonkeyFang, @onlyjob, @pault28, @tnktls.
Thank you for all your valuable input, help and making MooseFS better!

(If I forgot to mention somebody, I am very sorry and please let me know!)

Please find a complete list of fixes and new features that this release includes below:

  • (cs) silence stupid compiler warning
  • (client) fixed handling LOCKED and EAGAIN status in readdata
  • (client) added handle info to .oplog messages
  • (all) added support for disabling individual filesystem commands in mfsexports.cfg
  • (client) added handling read/write/readdir disables on client side (better error messages)
  • (client) added session paramaters to .params file
  • (tools) fixed packet reallocation error
  • (master) changed syslog message for locked chunks
  • (client) added dentry invalidator (needed in Linux with kernel < 4.19 – EBUSY issue)
  • (master) fixed memory leaks in xattr and posixacl modules
  • (cs) when chunk can't be located always send to master info about lost chunk
  • (supervisor, master, cs) fixed buffer overrun in mastersupervisor code (HA only, intr. in 3.0.107)
  • (master) changed algorithm of reusing csid in chunk module (adding released csid to the end of free list)
  • (all) added support for STRICT and LOOSE mode in KEEP and ARCHIVE
  • (master) fixed segfault during appending chunks of file with positive length and no chunks
  • (metadump) fixed bugs introduced in version 3.0.106
  • (master) added protection between never-ending desyncing between newer LEADER and older FOLLOWER (HA only)
  • (master) fixed slices with to set to 0 in mfsappendchunks
  • (master) removed starting protection time from client communication module
  • (man) fixed typo in mfsappendchunks man page

Best regards,
Piotr / MooseFS Team

moosefs - MooseFS 3.0.111

Published by pkonopelko over 4 years ago

Dear MooseFS Users,

MooseFS 3.0.111 is now stable! This release adds protections against segfaults in MooseFS Chunkserver module (issue #342).

We would like to thank @onlyjob and @uu for reporting the issue. Thank you for all your valuable input, help and making MooseFS better!

Please find a complete list of fixes and new features that this release includes below:

  • (cs) added protections against segfaults (issue #342)
  • (cs) removed unnecessary error messages

Best regards,
Piotr / MooseFS Team

moosefs - MooseFS 3.0.110

Published by pkonopelko over 4 years ago

Dear MooseFS Users,

MooseFS 3.0.110 is now stable! This release is a bugfix release mainly. It addresses issues related to handling duplicated chunks, placing chunks in subfolders on Chunkservers, internal rebalance, fixes FreeBSD-related issues and adds a few other minor fixes.

This release also introduces several new config options, new logo and favicon in MFS CGI interface and possibility to dump non-printable characters in metadata dumper.

We would like to thank all the contributors who helped us making this release, especially (alphabetically): @borkd, @jSML4ThWwBID69YC, @onlyjob and @smurfix. Thank you for all your valuable input, help and making MooseFS better!

(If I forgot to mention somebody, I am very sorry and please let me know!)

Please find a complete list of fixes and new features that this release includes below:

  • (cs) fixed bug that may lead to creating much more chunks in one subfolder (issues #319, #326)
  • (cs) added option for controlling subdirectory selection algorithm for newly created chunks (issues #319, #326)
  • (cs) prevent potential dumping deleted chunks to .chunkdb
  • (client) added options for setting behaviour when master returns unavailable chunks or no space status (issue #322)
  • (cgi) fixed logo and icon (file type for favicon.ico, change to current logotype – issue #328)
  • (master) turning off sending NOPs to unregistered clients (matoml module)
  • (cs) fixed reporting wrong number of hours in duplicates removal message
  • (cs) redesigned duplicates removal procedure to do it in groups of 100 files (issue #327)
  • (cs) introduce mypread and mypwrite macro (code cleaning)
  • (cs) added check for changed CRC before every chunk file close (just in case)
  • (cs) never use open chunk as a chunk duplicate that needs to be deleted
  • (cs) never use disk with pending duplicates as destination disk during internal rebalance (issue #327)
  • (cs) fixed condition in internal rebalance procedure (issue #327)
  • (cs) added option for setting duplicates retention time in hours (issue #333)
  • (cs) fixed memory leak in removing disk with duplicates
  • (cs) removed unnecessary attribute reading when chunk duplicates are detected (issue #327)
  • (cs) changed allocation of duplicated chunks from malloc to mmap, decreased memory usage by duplicates
  • (client) reading in O_WRONLY no longer causes EACCES (might be related to issue #334)
  • (metadump) printing dots for non-printable characters made optional (issue #332)

Best regards,
Piotr / MooseFS Team

moosefs - MooseFS 3.0.109

Published by pkonopelko almost 5 years ago

Dear MooseFS Users,

MooseFS 3.0.109 is now stable! This release fixes one minor bug in MooseFS CGI Server introduced in 3.0.108 and adds a possibility to allow starting MooseFS Chunkserver with invalid entries in mfshdd.cfg.

We would like to thank all the contributors who helped us making this release. Thank you for all your help!

Please find the list of fixes and new features that this release includes below:

  • (mfscgiserv) fixed handling redirects (bug introduced in 3.0.108)
  • (cs) added allowance for starting with invalid entries in mfshdd.cfg

Best regards,
Piotr / MooseFS Team

moosefs - MooseFS 3.0.108

Published by pkonopelko almost 5 years ago

Dear MooseFS Users,
MooseFS 3.0.108 is now stable! This release is a bugfix release.

We would like to thank all the contributors who helped us making this release, especially (alphabetically): @onlyjob, @Programie, @xandrus. Thank you for all your help!

(If I forgot to mention somebody, I am very sorry and please let me know!)

Please find the list of fixes that this release includes below:

  • (cli+cgi) fixed bug related to masterinfolist
  • (master) fixed sign in reporting time difference
  • (cgiserv) added content-type: text/plain to cgi outputs with errors
  • (master) fixed rmdir packet size

Best regards,
Piotr / MooseFS Team

moosefs - MooseFS 3.0.107

Published by pkonopelko almost 5 years ago

Dear MooseFS Users,
Long-awaited MooseFS 3.0.107 is now stable!

This release is mainly a bugfix release (and includes a lot of fixes). We strongly recommend to upgrade to this latest version! This release also introduces some new features, including support for libfuse3.

We would like to thank all the contributors who helped us making this release, especially (alphabetically): @eleaner, @hradec, @itongqingbo, @njaard, @onlyjob, @steffenb7333. Thank you for all your help!

(If I forgot to mention somebody, I am very sorry and please let me know!)

Please find the list of fixes and new features that this release includes below:

  • (master) added NOP command to changelog
  • (master+supervisor) added time check
  • (cs) fixed behaviour when job queue is full
  • (cli+cgi) added time data to master servers
  • (cs) fixed NOP packets ping-pong
  • (client) fixed NOP flooding in write module
  • (cs) fixed race condition in bgjobs module (jobhash)
  • (cs+mount) added new status returned to mount when queue is full and support for it in client module
  • (mount) added initial support for libfuse3 (related to issue #302)
  • (mount) added support for readdirplus (related to issue #302)
  • (mount) workaround for bug in fuse (wrong lock_owner in flush – issue #305)
  • (mount) fixed deadlock in readdata.c (closes issue #306)
  • (mount+master) fixed few small issues related to POSIX compliance (checked by pjd fstest)
  • (all) changed preferred C standard from C99 to C11 and added tests for atomics
  • (mount) move read/write structures initialization from open to read/write
  • (mount,mfsio) added new inode->fleng map
  • (configure) added option disable_all, added summary at the end
  • (freebsd) detecting new fuse version in the kernel and adjusting behaviour
  • (tools) better handling of unrecognized options
  • (master) better hard links handling
  • (master) added limit for number of hardlinks allowed for one object
  • (master) added option for automatic removal of unused chunkservers (issue #247)
  • (metadirinfo) fixed segfault when output filename is not specified
  • (chunktool) added file name fixing option (issue #276)
  • (all) changed chunk replication and deletion charts into two-color charts (successful/unsuccessful – issue #273)
  • (all) added new charts with number of chunk creations and internal changes (also successful and unsuccessful)
  • (master) added ignoring chunk with id zero sent from chunkserver
  • (metadump) added option for dumping parts of metadata
  • (master) added more logs to background data write subprocess (related to issue #280)
  • (master) fixed LEADER/FOLLOWER desync on SETCHUNK command (pro only)
  • (cli+cgi) fixed shebang in python scripts (preference for python3 in configure)
  • (master) changed pipe in bgsaver to nonblocking mode
  • (master) added check if the background saving process is alive
  • (master) added lockfile for background saving process
  • (master) added protection against time going back in the free inodes list
  • (master) added protection in FOLLOWER against not keeping up with the LEADER (pro only)
  • (master) changed condition for removing chunk from data structures (all chunkservers have to be connected)
  • (master) added restore run mode (works similar to mfsmaster -a but without running daemon)
  • (master) take write replication counters into account in server ordering for new chunk creations
  • (master) changed calculating replication limits in chunk rebalance procedure (using actual usage difference)

Best regards,
Piotr / MooseFS Team

moosefs - MooseFS 3.0.105

Published by pkonopelko over 5 years ago

MooseFS 3.0.105 is now stable! This release is mainly a bugfix release, however it also introduces a new feature of specifying subfolder for Block Devices on MooseFS.

We would like to thank all the contributors who helped us making this release, especially (alphabetically): @dragon2611, @eleaner, @jkiebzak, @onlyjob, @xandrus. Thank you for all your help!

(If I forgot to mention somebody, I am very sorry and please let me know!)

Please find the list of fixes this release includes below:

  • (nbd) fixed segfault when filename not specified
  • (client) fixed size of waiting threads counter (very rare deadlock in write module)
  • (cli) fixed status returned in case of master name that can't be resolved (issue #238)
  • (master) improved readability of some functions (related to issue #241)
  • (nbd) added option for specifying subfolder on MFS for block devices (issue #252)
  • (netdump) fixed using deprecated function pcap_lookupdev
  • (cli) fixed 'free space' in plain mode

Best regards,
Piotr / MooseFS Team

moosefs - MooseFS 3.0.104

Published by pkonopelko over 5 years ago

MooseFS 3.0.104 is now stable! This release includes both bugfixes and new features.

We would like to thank all the contributors who helped us making this release, especially (alphabetically): @borkd, @dragon2611, @nickb937, @njaard, @onlyjob. Thank you for all your help!

(If I forgot to mention somebody, I am very sorry and please let me know!)

Please find the list of fixes this release includes below:

  • (master) fixed taking into account labels in classes 1-9
  • (master) fixed recovering userperm and otherperm in ACL (in practice it doesn't matter – mainly cosmetics)
  • (cli+cgi) fixed python3 compatibility issue (casting to int for %X format)
  • (master) increased mmap allocation block size (Linux kernel issue – too many mmaps leads to memory not being properly released by the kernel)
  • (client) fixed handling internal packets with size zero (client disconnection during chunkserver registration)
  • (client) fixed symlink cache handling (potential race condition)
  • (master) added global option for preventing using same ip or same rack for more than one chunk copy
  • (client) added ignoring _netdev option – sometimes needed in case of mounting from fstab
  • (master) added defining rackid as string paths in mfstopology.cfg (introduced optional higher distances between servers)
  • (master) fixed desync on adding damaged chunk
  • (master) added changelog save mode (background, async and sync)
  • (all) fixed issues reported by cppcheck (fixes #179)
  • (cli+cgi) fixed issues reported by pylint (fixes #180)
  • (cs) added 'damaged' flag to chunk (prevents syslog flooding – issue #192)
  • (metatools) fixed conflict with builtin function humanize_number (NetBSD)
  • (master) better handling chunks with pending high-priority jobs
  • (master) long loop detection (log info whenever time between polls is longer than 5 seconds)
  • (master) fixed condition that checks if thare are all chunkservers and all chunks available (and using it for detecting ENOSPC condition – issue #201)
  • (mount) added .params pseudofile with mfsmount working parameters (such as cache sizes, timeouts etc.)
  • (cs) fixed removing chunks from data structures when marking disk as 'damaged' (issue #207)
  • (cs) fixed handling 'damaged' disks with chunks on reload (issue #207)
  • (cs) unlink duplicates week after detection (instead of removing them instantly – related to issue #207)
  • (tools) added displaying number of blocks in mfsfileinfo
  • (master) added error messages for 'fork' errors
  • (cgi) fixed handling encodings (issue #206 and #208)
  • (cgi) added timestamp (issue #197)
  • (cgi) fixed chart.cgi url (issue #200)
  • (cs) fixed crc values for non existing blocks
  • (mount) fixed changing processname
  • (supervisor) added reloading config parameters on SIGHUP (aka 'reload' – 'pro' version only)
  • (cs) fixed exiting protocol
  • (cs) added internal rebalance chart

Best regards,
Piotr / MooseFS Team

moosefs - MooseFS 3.0.103

Published by pkonopelko almost 6 years ago

Dear MooseFS Users,

MooseFS 3.0.103 is now stable! This release includes both bugfixes and new features.

Block Device

The biggest and most awaited feature introduced in this version is support for Block Storage which allows you to skip FUSE layer completely and give a block device instead of a file system layer to the OS on the Client using mbfsbdev instead of mfsmount.

See more in man mfsbdev.

This is kind of "feature release" of Block Device - we may improve it or its management in the future. You are very welcome to test it and report any issues by raising a GitHub issue. Your reports will help to make this feature and MooseFS better! πŸ‘ :)

MooseFS Meta Dirinfo

The second new big feature is mfsmetadirinfo. It is so-called "offline" implementation of well-known mfsdirinfo -p and similarly allows you to count real space usage of a particular directory precisely, with taking snapshots into consideration. The tool is faster than mfsdirinfo -p and you can use it on your Master Server. The tool operates on a dumped metadata file (e.g. metadata.mfs.back or metadata.mfs).

See more in man mfsmetadirinfo.

Extremely fast Pseudo-random Generator

This release also includes extremely fast (>1GB/s) pseudo-random generator visible as a virtual file .random in root directory (not listed by ls). You can use it for testing purposes, similarly as you use e.g. /dev/urandom.

New Chunk testing routine in MB/s

MooseFS constantly checks Chunks' checksums in background in order to prevent silent data corruption. This version gives you ability to define speed in MB/s of this checking. It is especially important for big clusters with huge number of small files where definition in chunks per second could reflect in a very long time of whole cluster checking (like years).

Complete list of features and fixes in MooseFS 3.0.103

Please find the list of fixes this release includes below:

  • (bdev) new module for Linux - dedicated block device
  • (metatools) added new metatool mfsmetadirinfo - offline version of mfsdirinfo -p
  • (mount) added extremely fast (>1GB/s) pseudo-random generator visible as a virtual file .random in root directory (not listed by ls)
  • (cs) changed chunk testing routine (new definition is in MB/s per disk)
  • (tools) changed data structure used by mfsdirinfo -p (faster, less memory usage)
  • (cs) added optional high speed internal rebalancing
  • (tools) network code refactoring (less code, reconnection to master on error)
  • (master+client) added space reservation for superuser
  • (daemons) added optional processname_set function
  • (daemons) fixed waitpid condition
  • (cs) added data precaching (posix_fadcvise(WILLNEED)) before reading chunk data
  • (master) remove chunk copy from structures (instead of leaving as INVALID) when replication returned error
  • (master) close listen sockets after fork (metadata save in background)
  • (master) added subprocess for saving downloaded metadata in background (pro only)
  • (master) fixed mfsrmsnapshot issue (SNAPSHOT flag not set in directories - intr. in 3.0.101)
  • (mount) added option to load master password from given file
  • (cs) proper disconnection from master during reload (maintenance mode during reload)

Best regards,
Piotr / MooseFS Team

moosefs - MooseFS 3.0.101

Published by pkonopelko about 6 years ago

Dear MooseFS Users,

MooseFS 3.0.101 is stable now! Please find the list of fixes this release includes below:

  • (master) make under goal replications respect topology (patch made by tclh123)
  • (master) fix inforec undergoal (patch made by tclh123)
  • (cgiserv) fixed handling connection close during data sending (leaving active pipes which caused leaving mfs.cgi subprocesses)
  • (mount) decreased memory usage (fixed handling buffers allocated per thread)
  • (all) setting correct errno when read returns 0 (in sockets.c)
  • (mount) fixed double free of plock_data structure
  • (master) fixed inconsistency in calculating realsize when storage class definition has changed
  • (master+mount) fixed handling read-only filesystems (ro flag in exports and lookup-open lflags)
  • (all) fixed differences in defaults between source, config and manpages
  • (all) added nobreak macro to explicitly inform compiler that intentionally there is no break before following case
  • (mount) added mfssuid and mfsdev options to mount (allows mounting with suid/dev options using system mount on Linux)
  • (cs) added using posix_fadvise(...DONTNEED) for tested chunks
  • (master) fixed HA bug (uninitialized variable might cause everlasting ELECT - PRO only)
  • (cs) fixed metaid handling
  • (master) added clearing SNAPHSOT eattr on file/directory modifications
  • (master) added optional same rack preference for destination server during undergoal replications
  • (mount+tools) increased masterproxy timeout from 1s to 20s

Best regards,
Piotr / MooseFS Team

moosefs - MooseFS 3.0.100

Published by pkonopelko over 6 years ago

Dear MooseFS Users,

MooseFS 3.0.100 is stable now! Please find the list of fixes this release includes below:

  • (cs) fixed rare segfault during chunk sending to master after disconnection
  • (mount) added thread for cleaning released files with delay
  • (mount) added assertion for lru cache (sustained open files)
  • (master+mount) added chunks data cache invalidation after chunkserver disconnection and reconnection
  • (mount) changed (lowered) number of connection retries (depends on I/O retry counter)

Best regards,
Piotr / MooseFS Team

moosefs - MooseFS 3.0.99

Published by pkonopelko almost 7 years ago

Dear MooseFS Users,

MooseFS 3.0.99 is stable now! Please find the list of fixes this release includes below:

  • (master) disconnect all clients when exports has changed
  • (mount) do not show message about 'negative travel time' when the measured time is zero (happens in virtual machines)
  • (all) fixed get16bit function on big-endian machines

Best regards,
Peter / MooseFS Team

moosefs - MooseFS 3.0.98

Published by pkonopelko almost 7 years ago

Dear MooseFS Users,

MooseFS 3.0.98 is stable now! Please find the list of fixes and new features this release includes below:

  • (mount) changed default cache mode on OS X from 'direct' to 'auto' due to problems with mmap
  • (cgiserv) added seteuid to 'mfs' or 'nobody'
  • (cs+tools) use readdir instead of readdir_r on glibc 2.24+
  • (debian) added support for systemd in debian packages
  • (all) fixed library dependencies (for libpcap,libz and libm)
  • (mount) fixed race between defered 'open' and 'locks'
  • (master) more chunk debugs and better handling chunk status change
  • (cs) fixed handling mark for removal (after reload sometimes cs could send chunks marked for removal as normal chunks)
  • (master) added file with metadata checksums (for future use)
  • (master) fixed way of sending gids to changelog/follower
  • (master) added inode checksum for some changes sent to changelog/follower
  • (metadump) added '0x' prefix for fileds dumped as hex numbers
  • (master) fixed ACL mask synchronization issues (restoring from changelog/leader->follower)

Best regards,
Peter / MooseFS Team

moosefs - MooseFS 3.0.97

Published by pkonopelko about 7 years ago

Dear MooseFS Users,

MooseFS 3.0.97 is stable now! Please find the list of fixes this release includes below:

  • (tools) fixed trash/sustained size desynchronization after file length changed by chunk creation
  • (mount) fixed handling invalid arguments in posix lock command

Best regards,
Peter / MooseFS Team