seaweedfs

• Support atomic renaming! Works well with Hadoop!
• weed master
  • redirect to leader for default admin UI, fix #898
• weed backup
  • efficient delta backup, fix #399
• weed filer
  • detect mime type better
• Added weed shell with the following commands:

  collection.list               	# list all collections
  fs.du                         	# show disk usage
  fs.ls                         	# list all files under a directory
  fs.tree                       	# recursively list all files under a directory
  volume.fix.replication        	# add replicas to volumes that are missing replicas
  volume.list                   	# list all volumes

• weed shell add a few real commands
  • list volumes
  • list collections
  • file system disk usage
  • fix under-replicated volumes
• weed filer:
  • fix gRpc port to port + 10000
  • notification: add Go CDK pubsub support (https://github.com/chrislusf/seaweedfs/pull/895 by @jba from Google)
• weed benchmark
  • add replication flag (https://github.com/chrislusf/seaweedfs/pull/893 by @PapaYofen from SenseTime)
• weed master
  • add option to disable http, only uses gRpc
• miscellaneous bugs: #886

• weed master
  • add back "/cluster/status" (fix #870)
  • add option to disable http operations
• weed server
  • fix option error coming from moving raft to grpc (fix #872)
• weed filer
  • support windows backslash file path separator (fix #868)
  • UI: synchronous file uploads (fix #807)
• weed volume
  • JWT checks for batch assigned file ids (fix #867)
• weed mount
  • add option to disable "allow others" (fix #877)
• weed s3
  • add namespace in XML responses (fix #863)

• Add JWT based access control. The "white-list" based access control will be removed in later releases. See https://github.com/chrislusf/seaweedfs/wiki/Security-Overview#jwt-based-access-control
• Support mutual TLS on all gRPC connections.
• weed master
  • fix possible split brain

• weed mount
  • set file attributes, fix #847
• weed volume
  • report 404 for deleted files with memory needle map, fix #850

• weed mount
  • misc performance and memory usage improvements
• weed s3
  • fix list buckets
  • fix s3cmd put
• weed volume
  • read waits when volume is committing compaction
  • fix volume writes error introduced from 1.19 27093bc
• weed master
  • Speed up heartbeat to new master leader 41fe31b
  • Change to separate gRpc port due to https://github.com/soheilhy/cmux/issues/64
• weed filer
  • support TTL with Cassandra and Redis filer store

• weed mount
  • keep mount point existing permissions fix https://github.com/chrislusf/seaweedfs/issues/806
  • reduce memory footprint by switching to https://github.com/seaweedfs/fuse , related to https://github.com/chrislusf/seaweedfs/issues/805
  • improve performance by caching, 60% faster in 32 threads mode, ~ 400% faster in single thread mode.

• weed mount
  • support soft links
  • Performance improvements
• weed filer.copy
  • limit to copying to a folder

A Christmas Release! Happy Holidays!

• weed filer, volume, HCFS
  • report original file size
• Hadoop client (HCFS)
  • fix #793 first-level directory creation
  • fix #797 list directory
• weed volume
  • enhance on detecting gzippable content
  • avoid ".gz" auto decompression
  • fix #793 clean up ".idx" files
• weed backup
  • pull changes via gRpc streaming
  • fix problem with compact map iterating

• weed filer
  • paginate through large folders
• weed volume
  • reduce needle map memory usage by 25%
• weed mount
  • paginate through list results https://github.com/chrislusf/seaweedfs/issues/787
  • paginate when "rm" and "mv"
• Java
  • Hadoop compatible file system 1.03
    • fix bug on mkdirs
  • Seaweed-client 1.03
    • adding more functions, refactored from seaweed-hadoop-client
    • paginate through large folders

• Add Hadoop Compatible File System! Run your hadoop jobs on SeaweedFS! https://github.com/chrislusf/seaweedfs/wiki/Hadoop-Compatible-File-System
• weed filer
  • default to leveldb store in current directory if no filer.yaml defined
• weed mount
  • use gzip to transmit data if possible

• statically build executables, https://github.com/chrislusf/seaweedfs/issues/775
• weed master
  • "/submit" handler add "count" parameter to allocate multiple file ids
• weed volume
  • default "-images.fix.orientation" to false
• weed filer
  • avoid a directory and a file with the same name overwriting each other https://github.com/chrislusf/seaweedfs/issues/777
• weed mount
  • add support for "df" to display free disk space

• weed mount
  • avoid memory leak for FUSE
  • improved performance
  • avoid possible file handle conflicts
  • much improved "ls -al" performance with O(1) metadata fetching
  • fix corrupted file with missing sections when out-of-order writes on Linux https://github.com/chrislusf/seaweedfs/issues/774
• weed s3
  • fix get object error

• Volume Server
  • Fix vacuum problem on windows
• Filer
  • Configuration separate into filer.toml and notification.toml
• Replicate
  • Support bootstrapping to replicate existing files
  • Support BackBlaze B2
  • Support message queue AWS SQS, Google Pub/Sub

• weed mount
  • Drastically improve file IO performance
• weed volume
  • Added gRpc port, which is volume port(default to 8080) + 10000, default to 18080
  • use gRpc to replace batch delete http API "/delete"
• weed master
  • use gRpc to replace batch volume id lookup http API "/vol/lookup"

• Filer
  • Filer replicates to Amazon S3, Google Cloud Storage, Azure.
  • Increase filer directory operation concurrency, by skipping an unnecessary deduplication error.
• minor
  • Embed images, javascripts into servers, avoid external http resources.
  • Fix profiling bug.
  • Fix UI issue when filer is empty.

• weed s3
  • Add Amazon S3 Multipart Upload APIs
  • Fix bug with buckets when redirecting reads to volume server.
• weed mount
  • Fix bug when write intervals are not continuous
  • Default directory list limit to a large number 100K
  • Default replication to empty, so filer can decide the replication.
  • Fix "du" on linux.
  • Fix reading empty file on linux.
• weed filer
  • Default directory list limit to a large number 100K
• weed filer.replicate
  • Add this feature to asynchronously replicate from one filer to anther filer
• weed volume
  • when leveldb or boltdb is used, remove old indexes when committing the compaction.
• other commands
  • Fix "weed backup" error when rerunning

• fix large file copying error with "weed mount"
• fix file copying permission setting

• Fix a possible data racing problem.
• Fix weed s3 upload file support for S3 v4 API.

AWS S3 Multipart upload is still working in progress.

• Fix bug when starting with old filer.toml
• Add filer.export to copy meta data between different stores.
• Fix bug with filer on Redis