seaweedfs

• FUSE mount
  • Wait for master
  • fix a nil pointer error
• Volume Server
  • Automatic detect IP address if not set
• Shell
  • Simplify CLI option for filer

• FUSE mount
  • Fix data write issue under high concurrency
• Filer
  • Add back support for If-None-Match and If-Modified-Since
• Minor
  • Fix filer.toml settings for S3 buckets folder
  • Filer replication adds endpoint for in house S3 storage

• Volume
  • volume.fsck also checks erasure encoded volumes
  • volume.fix.replication supports complicated replications
  • volume copying adds cleaning up on failed attempts
  • able to resize image uploaded via filer
• Fuse Mount
  • Fail early if cannot connect to the filer
• Filer
  • Fail early if master address is wrong
  • Increase gRPC message size to support TB-sized files with default 4MB chunk size.

• FUSE mount
  • Speed up (~4.8x) with new chunk caches, Opimization 1, Optimization 2
• Hadoop compatible file system
  • Speed up reads with new chunk read cache

• weed shell
  • Added "volume.fsck" command to detect and purge orphan data not referenced by Filer
• Volume
  • Added "-max=0" option to automatically configure max volume count on linux
• FUSE Mount, WebDAV
  • Faster buffered read without repeated http requests
  • FUSE mount rename correctly across folders
• Docker
  • Optimize volume size and count configurations.

• Filer
  • Add back image resizing capability
  • Fix where deletion can be missed under high concurrency. The bug is introduced in 1.57, and impact systems with frequent updates or deletions.
• S3 API
  • Fix DeleteMultipleObjectsHandler
• Minor
  • Fix master UI connection count reporting

• Volume
  • sync the dat file right before compaction
• Tiered Storage
  • Add option to configure S3 endpoint to support private S3 implementation
• weed upload
  • add usePublicUrl option to copy to volume server by its public url

This release is mostly bug fixes.

• Volume
  • correctly report readonly status after compaction
  • redirect writes to the right volume servers if volume is moved
  • consistent etag for replicated writes
• Filer
  • server shutdown closes the filer store

• Volume
  • properly handle read only volumes after compaction
  • vacuum uses pre-allocate variable
  • vacuum follows compactionBytePerSecond
  • Fix a nil pointer problem with replicated writes.
• HDFS compatible
  • Released HDFS client v1.25 to support encrypted data storage
• Filer
  • Postgres store skips empty password

• Filer
  • Add TTL support for all filer stores
  • Filer fix ttl parsing
• Volume Server
  • Fix memory leak during volume compaction. The bug was introduced since 1.48. It shows up if there are lots of compactions.
  • Fix compaction error on windows

• Filer
  • Add TTL support for all filer stores
  • Filer fix ttl parsing
• Volume Server
  • Fix memory leak during volume compaction. The bug was introduced since 1.48. It shows up if there are lots of compactions.

• Kubernetes

  • @LazyDBA247-Anyvision Added Helm Charts for Kubernetes! #1216

• Filer

  • Automatically create collections for folders under /buckets

• Volume

  • Weighted volume allocation #1219 by @song-zhang

• Experimental

  • Added option to encrypt data on volume servers

• Bugs

  • Filer
    • fix reported uploaded file size
    • fix listing for TTL on Redis store
  • Weed mount
    • fix nil during entry lookup #1221 #1223

• Filer
  • dynamically create buckets under /buckets folder
• S3 API
  • Implemented DeleteMultipleObjects API
• Mount
  • Able to handle large amount of files. Previously gRPC context was resetting some requests.
  • Able to run outside of docker cluster.
• Volume Server
  • Fix readonly status reporting
  • Fix compaction on windows
• Shell
  • Add commands to create/delete/list buckets

• S3 API
  • Access control limited by bucket
  • Handle nil during edge cases
• Filer
  • Avoid possible timeouts for frequent updates and deletions
• Shell
  • volume.balance ensures dc and rack does not change for replicated volumes
• Volume Server
  • Move server stats to gRpc.

• S3 API Server
  • Add S3 Authentication
• Volume Server
  • Avoid segment fault, fixing https://github.com/chrislusf/seaweedfs/pull/1198

Two small but important fixes.

• Filer Server
  • Recursively batch delete file chunks, fix https://github.com/chrislusf/seaweedfs/issues/1197
• Volume Server
  • Return 204 for unchanged file uploads, fix #1196

• Volume Server
  • Reduce memory usage when generating read only volumes, avoiding OutOfMemory
  • Sync index file before closing
• Master server
  • Add configurable volume growth toml setting
• Filer
  • Upload chunked file in streams, reduce memory usage
• Shell
  • add volume.configure.replication to change volume replication

• S3
  • Add S3 copy
• Mount
  • More robust with retries and large file writes
• Volume
  • Support wav file compression

This release also added a feature to overwrite TOML file configurations with environment variables, making it easier to dynamically pass variables, such as database password.

• volume
  • better compaction: avoid sharing volume dat file handle(#1184)
• filer
  • file creation follows O_EXCL flag, to stop creation if the file already exists
• mount
  • misc fixes to make git clone work (#1182)
• docker
  • small (34MB) docker image with latest changes docker pull chrislusf/seaweedfs:dev

Fixed miscellaneous weed mount problems.