seaweedfs

• Fix ElasticSearch startup error. #1452

• Filer
  • Active-Active or Active-Passive Filer Store replication!
  • New ElasticSearch backend as filer store #1446 #1449
  • GRPC API supports prefixed query #1431
• Volume Server
  • Check disk space before compaction #1440
• FUSE Mount
  • Map user id and group id between local OS and filer.

• Filer
  • Ensure http connection reuse.
• S3
  • Avoid possible completeMultipartUpload re-tries that can overwrite uploaded content.
  • List bucket permission change from admin to read #1430.
• Volume Server
  • Anonymous read access to status handler.
  • Add option preStopSeconds to drain volume server traffic before shutdown #1433.
  • Fix EC error if multiple disks are configured in one volume server.
• Mount
  • Fix issue found when running UrBackup with SeaweedFS FUSE

• FUSE mount
  • many tests and changes for writing and reading random access files
• S3
  • fix listObjectsV2 response format #1426
• Filer
  • SQL store insert falls back to update #1429
• Volume server
  • Able to read last deleted file id with URL params readDeleted=true

• Support reading and writing random access files.
• FUSE mount
  • works correctly with random access files.
  • miscellaneous changes for renaming, set attributes, locking, etc.
• Filer
  • fixes for stale directories when using multiple filers.
  • add file upload retries for edge cases.
• Hadoop 1.4.6
  • remove unnecessary jars for better compatibility.
  • read random access files.
• S3
  • ListObjects and listObjectsV2 supports sub directories #1418

• Hadoop Compatible File System 1.44
  • Improve read and write performance with better caching, GC, read ahead, compression.
• S3
  • Control access for anonymous users
• Volume
  • Report Content-MD5 in response header
• Filer
  • Auto chunking for PUT requests, similar to existing POST requests handling.
  • Stores md5 metadata for files uploaded through filer
  • Report Content-MD5 in response header
• FUSE Mount
  • report EEXIST when deleting non-empty folders

• FUSE mount
  • changes to make mongodb to run on SeaweedFS mount (#1398 #1400)
• Hadoop Compatible File System
  • 1.4.3 for faster reads by caching volume id locations and buffering.
• S3
  • Add support for requests
  • Remove directory listing size limit
  • fix s3api auth bug
  • fix s3api copy object handler SerializationError
• Volume server
  • HEAD response sets Content-Disposition

• Overall
  • Support super large files
• Hadoop
  • Read/Write performance improvements.
  • Large file handling.
  • Reduce memory overhead.
  • Better support for Hadoop 3.
• Minor
  • command line allows tilde in directory parameter

• Replication
  • Support active-active filer replication.
• Volume Server
  • Default to minimum 1% free disk space.
  • (minor) Default the number of volumes in a volume server to 8. Adding one for logs in case all 7 volumes are allocated.
• FUSE mount
  • Switch to async meta data caching.
• Released Hadoop Compatible File System 1.3.2
  • Reuse HTTP connections.
• Docker
  • Dockerfile can support multiple architecture. #1377
• Bug
  • Fix missing metadata notification for folder deletion.

• S3 gateway
  • Fix wrong md5 validation during PUT. https://github.com/chrislusf/seaweedfs/issues/1374

• Volume Server
  • Fix logic to detect mime type and gzip accordingly.
  • Remove fixJpgOrientation feature
  • Add default minFreeSpacePercent #1366
  • Add optional MD5 verification of uploaded data #1371
  • Erasure Coding generate ec shard first, before generating the ec index file, in case of possible EC encoding interruption.
• S3 API gateway
  • DeleteObject API also support deleting a folder recursively
• Miscellaneous changes
  • Master
    • Always clear up previous master meta data when startup
  • FUSE mount
    • Prevent concurrent modification eror

• Released SeaweedFS CSI Driver! https://github.com/seaweedfs/seaweedfs-csi-driver
• FUSE Mount
  • Lazy metadata syncrhonization. (fix #1322)
• Hadoop Compatible File System
  • Fixed a concurrency issue when running HBase on SeaweedFS
• Volume
  • Add a hard limit to ensure volume file correctness (7ccc31853cac1833a3a1372aac2327e744a7c5ba)
• Minor
  • S3 return ErrNoSuchKey if not found
  • Fix issues with Erasure Coding encode/decode

• Volume Server
  • Add a fix for possible dead lock when volume is over 32GB.
• FUSE Mount
  • fix possible file reading EOF bug introduced from 1.80.

• Filer
  • default leveldb store add auto recovery, report correct error
• Hadoop
  • HBase can work properly
• Volume
  • On windows, support -max=0 to auto set volume max count
  • Able to set volume readonly if below a configurable free disk space percentage
• Master
  • Remove option to change CLI option -pulseSeconds
• WebDAV
  • Fix an error when file size is less than 512 and content type is not detectable by file name extension due to/etc/mime.types is not installed.

Fix compilation problem with 8000GB volume version.

• Added Message Broker!
• Faster fsync writes! (#1299)
• Filer
  • Fix directory deletion in MongoDB (#1303)
• FUSE mount
  • Atomic local cache updates
  • Avoid possible directory deletion error
• Master
  • Prevent race condition with possible transient volume server disconnection.
• Minor
  • Fix Prometheus verbose log
  • S3 : print out time in UTC format (fix #1297)
  • Benchmark randomized the content
  • Benchmark add fsync option
  • Filer faster image resizing
  • Filer copy HDFS user name and group names

• FUSE Mount
  • Default to use local meta data. This brings great performance improvements.
• Master and Shell
  • Add lock/unlock for shell scripts, to exclusively manage the cluster. Existing volume operations need to lock first before continuing.
  • Support semicolon separated multiple commands in one line.
• Hadoop Compatible File System
  • Add 1.2.7 release to fix HBase related issues.
    • seaweedfs-hadoop2-client-1.2.7.jar
    • seaweedfs-hadoop3-client-1.2.7.jar

• FUSE mount
  • Meta data caching works under high concurrency
• Master
  • Temporarily disable the exclusive weed shell restriction.

• FUSE mount
  • Added asynchronous meta data caching
  • Added tiered chunk caching
• Filer
  • Filer store supports Mongodb by @bukton123 (https://github.com/chrislusf/seaweedfs/pull/1280)
  • Add gRPC API to append to a file
• New work waiting for comments!
  • Add pub/sub feature with examples at https://github.com/seaweedfs/messaging-client-go

• FUSE Mount
  • Add on disk caching
• Volume Server
  • Add optional fsync on writes.
• S3
  • Add fsync buckets
• Filer
  • Add redis2 for faster large directory listing
• Shell
  • Only allow one weed shell connected to manage the cluster
• Replication
  • Fix replicating via Amazon SQS