seaweedfs

• Filer
  • Small files can be cached directly to filer store, avoiding additional http read to volume server. See -cacheToFilerLimit option.
• WebDAV
  • Much improved WebDAV write performance.

• Volume Server
  • Support -dir.idx option for separate index file storage, which can be on faster SSDs.
  • Detect volume data file disk errors, and reported them as deleted.
• Filer
  • Only retry broken network calls, but not normal errors. https://github.com/chrislusf/seaweedfs/issues/1639
• S3
  • Remove empty folders. #1637

• S3
  • Deprecating "filer.options.buckets_fsync" in filer.toml. Use fs.configure to configure path specific options for the bucket specific options.
  • Avoid nil when response has error. #1622
• Filer
  • fs.configure adds configurable volume growth
• Hadoop Compatible File System 1.5.6
  • add replication parameter fs.seaweed.replication
  • fix ordering in FileSystem.close()
  • One independent benchmark with HDFS showed comparable results, with much faster reads.
• Volume
  • Fix volume.balance wrong volume placement #1626

• Shell
  • Add fs.configure command to configure path-specific setting for replication/collection/ttl/fsync.
• S3
  • Add bucket owner attribute and use it for permission checking. #1612 #1613 #1616
  • Fix multipart copy operation. #1609
• Filer
  • Read from volumes in preferred data center #1610
• Volume
  • Retry-able http POST operations. #1592
• Hadoop Compatible File System 1.5.4
  • Add volume id cache to drastically improve read performance. a2962604
• Java
  • Add example to watch a folder 3e362451d226d9e19b4b652a02926dedc02f6cf9

Last release has a small issue with weed version. It does not affect actual usage.

• Filer
  • Add API to tag a file.
• Some tiny fixes:
  • adjust verbose logs for disk size checking outputs
  • adjust logic to pick the old volumes when purging over-replicated volumes.

• Hadoop Compatible File System 1.5.3
  • Close file system cleanly. https://github.com/chrislusf/seaweedfs/commit/c709059b690e2e5c70b262f4c884bb784cd1781a
• Filer
  • Write request headers with Seaweed- prefix will be stored as metadata.
• Volume Server
  • Detect max volume count changes based on disk usage #1594
• S3
  • Avoid blank response for 404

• Mount
  • Add retries to all gRPC operations. #1589
  • Add CLI option to limit concurrent writers
• S3
  • Return x-amz-tag-count header in GET requests #1566
  • Support X-Amz-Storage-Class in request header.
  • Support X-Amz-Tagging in request header.
  • Support X-Amz-Meta- in request header.
• Filer
  • Add CORS support
• Volume Server
  • Add CORS support
  • Trim out unreachable content left from abrupt shutdown.
  • Add special handling for content outside of 32GB.
• Shell
  • Add volume.mark -readonly #1574
• Fixes
  • weed filer.copy had filename as the directory.
  • Add back upload retries.

• Mount
  • A better way to invalidate in-memory cache. Previous fix could be slow for large directories. This release is just to prevent possible performance degradation.

• Hadoop
  • Hadoop-Compatible-File-System 1.5.1 reports block size correctly.
• Mount
  • Fix reading stale content on replicated changes.
  • Performance improvements: async file copying, less memory usage.
  • Fix sporadic read errors #1541
• S3
  • Report error to s3 when updating an object but it is already a directory. #1545
  • Multiple domain names for multiple buckets. #1553
• Docker
  • Allow custom command arguments #1547
• Filer
  • Support default rack

• FUSE Mount
  • Parallel write requests with 4x speed increase for large files.
  • Fix empty folder when weed mount -filer.path=... parameter. #1528
  • Add retry for directory listing with filer #1530
• Filer
  • Fix weed filer.sync with MySql, Postgres and Cassandra stores.
  • Fix support for streaming range requests.
• WebDAV
  • Fix dead loop when files smaller than 512 and unknown extenstion. #1344
• Filer
  • Avoid early disconnection when reading large files.
• Volume
  • Add metrics for read only volumes.
• S3
  • Avoid duplicated buckets.

• Breaking Changes!
  • S3 Added support for "List" action. Now the previous "Read" permission is split into "Read" and "List". The "Read" permission previously allow reading and listing, but now can only read one file. So your s3 config file should add the "List" action if desired.
  • weed mount -outsideContainerClusterMode now uses volume server publicUrl.
• Filer, Mount, Hadoop
  • Support retrying to read from alternative locations.
  • Exponential backoff when retrying for failed reads.
• Mount
  • Mount adds configurable read wait time option -readRetryTime
  • Fix a race condition when recursively deleting large folders #1514
• Filer
  • Adds a QR code in the file listing page, to make it easy to transfer files with mobile devices.
• Hadoop 1.5.0
  • Fix reading ORC file tail.
  • Fix error reporting when creating a file or folder.
• S3
  • Adds "List" and "Tagging" actions for configuring access.
    Bug fixes:
  • Volume: avoid hanging when stopping a volume server
  • Master: check peers for existing leader before starting a leader election #1509

• FUSE mount
  • Increase large file read performance with prefetching, avoid duplicated reads and fix caching bugs.
• Master
  • Add "resumeState" option to avoid transient state loss if all masters are restarted at the same time.
• S3 API
  • Added object tagging.
    • GetObjectTagging
    • PutObjectTagging
    • DeleteObjectTagging

• Volume Server
  • Avoids dead lock when deleting volumes. #1501 introduced since version 1.93
• Filer
  • Replication uses master default value if not specified
• S3
  • Metrics adds http status code #1502

• FUSE Mount
  • Support Hard Link!
• Shell
  • Consistently use flags for command options.
• Hadoop on SeaweedFS
  • Spark cannot create empty file #1494
• Filer
  • Ensure file names are valid UTF-8 #1493
  • Cassandra store adds authentication #1490
  • Enable file extended properties
• S3
  • Can be started together with Filer with weed filer -s3
  • Fix get object metadata behavior #1491
  • Handle response-content-encoding in the header #1487
  • Fix listing multipart uploads
• Volume
  • Replicated volume can be writable after compaction #1485
• Master
  • Fix a deadlock with KeepConnected and SendHeartbeat #1475
• Other
  • Add Prometheus http port #1489
  • weed upload respects -usePublicUrl for large files #1492

• HDFS 1.4.7
  • Add fs.seaweed.buffer.size option. Dramatically improved performance!
• Shell
  • Add volumeServer.leave command
  • Add volumeServer.evacuate command
• S3 API Gateway
  • Add support for PostPolicy API #1426
  • Emit metrics to prometheus gateway.
  • Updated grafana dashboard for Seaweed adds S3 metrics and possible cost if run in AWS.
• Filer
  • Add capability to create an empty directory by http POST #1161
  • gRPC server also supports ip.bind
  • Mongodb update bug #1471
• Docker
  • Add image for large disk version. docker pull chrislusf/seaweedfs:2.00_large_disk
• WebDAV
  • Support files larger than 2GB #1468
• Volume
  • load configuration from master at the start, to support cloud tier in weed server running mode #1469

• Filer
  • Fix postgres failed to start #1465
  • Fix a problem where mysql/postgres would insert and then update the same record immediately again.
• Volume Server
  • Revert back the "read deleted entry" feature introduced in 1.96. It caused problems when checking .dat file integrity.

• Feature
  • Balance volumes by ratio of fullness.
• Bug Fixes
  • Volume Server: validate volume correctness if last entry is a deletion
  • Filer Store: etcd store avoid read with nil option #1463
  • S3 API: fix ListParts output xml format #1461
• Minor
  • Reduce -preStopSeconds default wait time from 30s to 10s

• Volume Server
  • Fix an issue where volumes are wrongly considered readonly if the last change is a deletion.

• Fix a bug introduced in 1.95 where lock in weed shell always fail on the first attempt.

• Filer
  • Cross Cluster Active-Active Asynchronous Replication
  • ElasticSearch filer store adds more options. #1455
• Helm Charts upgrades #1456
• Shell
  • volume.fix.replication also purge over replicated volumes
• Bugs
  • Close http response in order to reuse connections.
  • Properly cancel context for streaming grpc.