This is the second beta release for JuiceFS v1.2. Since beta1 it has 84 commits from 14 contributors, thanks to @zhoucheng361 @zhijian-pro @CaitinChen @SandyXSD @jiefenghuang @yuhr123 @polyrabbit @davies @tangyoupeng @kyungwan-nam @testwill @chnliyong @JoyLiuc @xiaogaozi !

New

• cmd/mount: add a new option max-stage-write to enable concurrent write to both stage files and object storage in writeback mode (#4743)

Changed

• cmd/mount: support graceful restart on macOS (#4792, #4823)
• cmd/dump: use a temporary file during dumping (#4767)
• meta: remove tail zero slice when compacting a chunk (#4789)
• chunk: improve the management of failed cache disks (#4749, #4780, #4784, #4871, #4848)
• chunk: delete leaked objects if it's already deleted by other goroutines (#4748)
• hadoop: support using ; to separate values for juicefs.users and juicefs.group (#4724)

Bugfix

• cmd/mount: fix FUSE option for ACL (#4787)
• cmd/mount: fix duplicated FUSE service (#4773)
• cmd/gateway: add rw locks to lock multiple requests within a single machine (#4845)
• cmd/gateway: fix empty bucket display exception (#4875)
• cmd/clone: set the attr should be done after checking permissions (#4745)
• cmd/clone: fix source mode check (#4856)
• cmd/debug: add a timeout when collecting the information (#4824)
• cmd/debug: fix the collection in dual-mount-processes mode (#4816)
• meta: fix the error message in quota that always shows inode 0 (#4741)
• meta/redis: fix some transactions that are not wrapped by multi/exec (#4801)
• meta/postgreSQL: fix the address parse (#4731)
• vfs: update data of internal files only when it's empty (#4770)
• vfs: dump the unread portion of the access log (#4855)
• vfs: fix the race of getting file length (#4876)
• hadoop: fix the value of upload/download limit (#4813)

This is the first beta release for JuiceFS v1.2. Since v1.1 it has 354 commits from 41 contributors, thanks to @zhijian-pro @davies @zhoucheng361 @SandyXSD @jiefenghuang @polyrabbit @CaitinChen @yuhr123 @Hexilee @tangyoupeng @YunhuiChen @eryugey @timfeirg @Suave @solracsf @kyungwan-nam @xiaogaozi @1ambd4 @stefanb @XDTD @occupyhabit @menwenjun @li1213987842 @testwill @fengleng @deferdeter @camcui @uvletter @T-TRz879 @yuchen-sun @TechQuery @reAsOn2010 @mtnk2005 @kanemathers @JoJossd @sjp00556 @CodingPoeta @abserari @AChangFeng @feeyman @l0wl3vel !

Highlights

• Supports graceful restart for mount process (#4392, #4450, #4487, #4499, #4509, #4511, #4524, #4547, #4555)
• Supports POSIX ACL (#4421, #4437, #4443, #4490, #4503, #4518, #4522, #4523, #4516, #4551, #4640)
• Supports Identity and Access Management (IAM) and events notify for gateway (#4030, #4593)

New

• Added a new command compact to trigger compaction on a specified path (#4337)
• Supports using NFS as object storage (#3823, #4101, #4105, #4111, #4147, #4163, #4718)
• Supports using Dragonfly as object storage (#4057)
• Supports using Bunny as object storage (#4719)
• cmd: add a new option --log-id to append user-specified ID in log (#4040)
• cmd/format: add a new option --enable-acl to enable POSIX ACL (#4421, #4437, #4443, #4490, #4503, #4518, #4522, #4523, #4516, #4551, #4640)
• cmd/mount: add a new option --skip-dir-mtime to eliminate burst mtime updates of a directory (#4189)
• cmd/mount: add a new option --upload-hours to limit the period for background uploads (#4250)
• cmd/mount: add a new option --cache-expire to set expired time for cached blocks (#4255, #4365, #4434)
• cmd/mount: add a new option --custom-labels to set labels for metrics (#4312, #4612)
• cmd/mount: add a new environment variable JFS_MOUNT_TIMEOUT to adjust timeout for checking mount point ready (#4219)
• cmd/mount: supports restarting mount process gracefully (#4392, #4450, #4487, #4499, #4509, #4511, #4524, #4547, #4555)
• cmd/gateway: supports IAM and events notify (#4030, #4593, #4622)
• cmd/gateway&webdav: add a new option --background to support daemon running (#3993, #4631, #4708)
• cmd/gateway: add a new option --domain to support virtual-host-style requests (#4462)
• cmd/warmup: add a new option --evict to manually evict cached blocks (#4370)
• cmd/warmup: and a new option --check to check the ratio of already warmed blocks (#4370)
• cmd/dump: add a new option --fast=false to reduce memory usage with a slower dump process (#4253)
• cmd/dump: add a new option --skip-trash to skip the trash directory when backing up metadata (#4479, #4563)
• cmd/dump: add a new option --threads to adjust speed for dump (#4581)
• cmd/sync: add a new option --inpace to put file in-place, eliminating the rename operation (#4104)
• cmd/sync: supports two stars ** and three stars *** to match all files recursively (#4422, #4448, #4460, #4468, #4474)
• cmd/sync: add a new option --match-full-path to match the filter against full path (#4492)
• cmd/sync: add p8s metrics to better monitor the sync progress (#4119)
• meta: supports Open File Description (OFD) lock (#4083)
• chunk: disk cache switch to consistent hash and add bad disk detection (#4543)
• object: add query string to disable md5 and crc verification when s3 compatible storage does not support them (#4606)
• object: add a new environment variable JFS_S3_VHOST_STYLE=1 to enable vhost style (#4697)
• hadoop: add push-labels option to set custom labels for metrics (#4312)

Changed

• cmd: support human friendly values for options (#4246, #4558, #4567, #4592, #4641, #4671)
• cmd/mount: only use writeback_cache to enable fuse writeback (#4632)
• cmd/gateway: judge loop symbolic links rather than hard-code deep limit (#4045)
• cmd/gateway: create MinioMetaBucket when it doesn't exist (#4383)
• cmd/gateway: create .minio.sys directory even if multi-buckets are not enabled (#4367)
• cmd/sync: limit the maximum memory size for multiple uploads (#4135)
• cmd/sync: speed up sync for huge file (#4456)
• cmd/sync: use larger io-size when writing to backends (#4546)
• cmd/sync: skip listing the destination in force mode (#4602)
• cmd/sync: use HEAD instead of LIST for a single object (#4623)
• meta: add mount time in session info (#3987)
• meta: export meta backup metrics (#4480)
• meta: force compaction when there are too many slices in one chunk (#4573)
• meta/tkv & meta/sql: speedup the dump process using multiple threads (#4501)
• meta/redis: reduce the memory usage when dumping large directories (#4481)
• meta/badger: upgrade to v4 (#4564)
• chunk: prefetch the whole block only on successful partial read (#4658)
• vfs: fuse ops metric supports showing method label (#4554, #4560)
• vfs: add errno label to fuse ops metric (#4656)
• object/tos: adjust min part size from 5 MiB to 4 MiB (#4688)
• object/ceph: allow delete when ceph cluster is full (#4131)
• object/ceph: speedup list for ceph (#4218)
• object/ceph: set default path for admin socket and log file (#4608)
• hadoop: support human friendly values for options (#4655)

Bugfix

• cmd/gateway: fix ListMultipartUploads sorting rule and support delimiter (#4297)
• cmd/gateway: fix HEAD directory returns 200 but expect 404 (#4486)
• cmd/gateway: fix the issue that memory is not released immediately after object downloaded (#4507)
• cmd/sync: fix the issue that empty files are always synchronized (#4223)
• cmd/sync: execute chmod after chown to avoid setuid/setgid failure (#4418)
• cmd/sync: fix the issue that setuid/setgid/sticky bits are not set when using jfs protocol (#4419)
• meta: fix the issue that df may see negative values (#4591)
• meta/sql: use upsert in Write() to avoid possible deadlock (#4529)
• vfs: fix the issue that file entry may still be returned if it's deleted during readdir (#4453)
• vfs: fix the issue that file length may not be updated immediately after fallocate & copyfilerange (#4427)
• hadoop: make libjfs singleton to avoid possible jnr weakhashmap infinity loop (#4407)
• hadoop: fix guid lookupGroup (#4532)
• hadoop: use auto-increment fs handle to avoid possible conflict (#4411)
• hadoop: fix the issue that dfsio does not work because mapreduce.output.fileoutputformat.compress is true (#4646)

This is the second patch release for JuiceFS v1.1. It has 16 commits from 6 contributors, thanks to @zhijian-pro @davies @SandyXSD @tangyoupeng @mtnk2005 @AChangFeng !

Changed

• Build(deps): bump golang.org/x/crypto from 0.14.0 to 0.17.0 (#4287)

Bugfix

• cmd/sync: fix the issue that name of the temporary file may be too long (#4215)
• cmd/sync: fix the issue that password of source is not removed (#4390)
• cmd/sync: fix symlink when using jfs:// schema (#4286)
• cmd/bench: fix the issue that the final result of progress bar is not correct (#4388)
• meta: trigger compaction more aggresively when there are too many slices (#4309)
• meta: fix the issue that truncated slices may never be compacted (#4320)
• meta/redis: use TxPipelined() instead of Pipelined() (#4264)
• object: fix the issue that sometimes checksum is not verified (#4213)
• object/obs: don't decode the object key in list result (#4331)
• object/cos: fix multipart upload with long key (#4258)
• object/ceph: fix the issue that list with many threads may panic (#4216)
• hadoop: fix guid update (#4299)
• windows: fix the issue that truncate may return syscall.EBADF (#4238)

This is the seventh patch release for JuiceFS v1.0. It has 12 commits from 5 contributors, thanks to @davies @SandyXSD @zhijian-pro @tangyoupeng @mtnk2005 !

Changed

• Build(deps): bump golang.org/x/crypto from 0.14.0 to 0.17.0 (#4287)

Bugfix

• cmd/sync: fix the issue that name of the temporary file may be too long (#4215)
• cmd/sync: fix the issue that password of source is not removed (#4390)
• meta: trigger compaction more aggresively when there are too many slices (#4309)
• meta: fix the issue that truncated slices may never be compacted (#4320)
• meta/redis: use TxPipelined() instead of Pipelined() (#4264)
• object/obs: don't decode the object key in list result (#4331)
• object/cos: fix multipart upload with long key (#4258)
• object/ceph: fix the issue that list with many threads may panic (#4216)
• hadoop: fix guid update (#4299)

This is the first patch release for JuiceFS v1.1. It has 17 commits from 6 contributors, thanks to @zhijian-pro @davies @Hexilee @SandyXSD @tangyoupeng @fengleng !

Changed

• cmd/quota: set limitation before calculating the used stats (#4158)
• meta: increase inodeBatch & sliceIdBatch (#4188)
• meta: skip cleaning session when sid is 0 (#4133)
• deps: upgrade golang/x/net to 0.17 (#4136)

Bugfix

• cmd/gateway: create the parent directory of the target on demand when copying an object (#4058)
• cmd/restore: fix the issue that dirStats and quota are not properly set (#4095)
• cmd/sync: fix copying times, owner and mode of symlinks (#4151)
• cmd/sync: fix the issue that checksum are wrongly calculated for symlinks (#4082)
• cmd & object: fix incorrect paths in Windows (#4138)
• meta: allow flock converting current exclusive lock to a shared lock (#4179)
• meta: fix possible password leak when it contains the @ symbol (#4071)
• meta/mysql: don't escape the mysql password (#4069)
• meta/tkv: fix offset overflow in CopyFileRange (#4130)
• meta/tkv: fix the issue that info command may wrongly fail on read-only mountpoint (#4085)
• object: skip listing directories with no permission (#4125)
• object/s3: fix nil when StorageClass is missing from List (#4183)
• hadoop: fix guid refresh by avoid using closed fs (#4180)

This is the sixth patch release for JuiceFS v1.0. It has 8 commits from 4 contributors, thanks to @zhijian-pro @SandyXSD @fengleng @davies !

Changed

• meta: increase inodeBatch & sliceIdBatch (#4188)
• deps: upgrade golang/x/net to 0.17 (#4136)

Bugfix

• cmd/gateway: create the parent directory of the target on demand when copying an object (#4058)
• meta: allow flock converting current exclusive lock to a shared lock (#4179)
• meta: fix possible password leak when it contains the @ symbol (#4071)
• meta/mysql: don't escape the mysql password (#4069)
• meta/tkv: fix offset overflow in CopyFileRange (#4130)
• object: skip listing directories with no permission (#4125)

This is the fifth patch release for JuiceFS v1.0. It has 24 commits from 5 contributors, thanks to @zhijian-pro @SandyXSD @davies @Hexilee @eryugey !

Changed

• meta/redis: speed up truncate for large files (#3989)
• object/sftp: try public keys before interactive password (#3886)

Bugfix

• cmd/format: fix parsing the parameter named session-token (#3519)
• cmd/gateway: fix the issue that List may fail if there's a symlink pointing to a file in another file system (#3879)
• cmd/sync: fix the issue that arguments may be changed unexpectedly in cluster mode (#3516)
• meta: fix the issue that zero field of Format not applied in refreshSession (#4033)
• meta: fix the issue that file attribute may be overwritten in some cases (#3766)
• meta: delete hardlinks directly if there's already same entry in the trash (#3706)
• meta/sql: remove lock on edge to avoid deadlock (#3677)
• vfs: fix the issue that error from meta.Truncate is ignored (#3596)
• vfs: fix the deadlock under concurrent truncate and releases (#3457)
• object/ceph: fix the error handling to range get and get non-exist object (#3855)
• object/sftp: fix the issue that sftp schema is not properly parsed (#3847)
• object/gs: remove the invalid storage class setting (#3536)
• metrics: fix the calculation method of stageBlockBytes (#3462)
• hadoop: fix the issue that the generated uid/gid may overflow int32 when using SQL metadata engines (#3680)

This is the second stable release of JuiceFS, and is an LTS version as well that will be maintained for 24 months.

Starting from v1.0 it has 876 commits from 62 contributors, thanks to @SandyXSD @zhijian-pro @davies @zhoucheng361 @Hexilee @yuhr123 @tangyoupeng @timfeirg @sanwan @xiaogaozi @Suave @solracsf @tcodehuber @zyfjeff @kyungwan-nam @zwwhdls @Ziy1-Tan @pigletfly @qingyunha @rfyiamcool @201341 @rayw000 @shichanglin5 @AIXjing @Vigor-jpg @baifachuan @yongxie09 @tisonkun @neocxf @molei20021 @luohy15 @leoliulei @jomenxiao @divinerapier @cuishuang @cmmp6 @xiedeyantu @xyb @Karitham @aronsky @yqlbu @JoyLiuc @sjp00556 @shjwudp @jzcruiser @hamidzr @ggicci @gdams @eryugey @guaneryu @Erisa @p1u3o @CodeRusher @CaitinChen @dugusword @Arvintian @AChangFeng @T-TRz879 @arnoldberlin @CaitinChen @leemingzixxoo @wzhallright !

Highlights

• Supports setting quotas on directory level
• Supports fast copying a tree with clone command
• Supports using FoundationDB as metadata engine
• Supports using GlusterFS as object storage

New

• Added a new command quota to manage directory quotas (#3268, #3371, #3374, etc.)
• Added a new command clone to fast clone a directory tree (#3279, #3366, #3369, etc.)
• Added a new command debug to collect a bunch of client information used for diagnosing problems (#2360, #2815, #2857, etc.)
• Added a new command restore to batch restore files in trash (#3657)
• Added a new command summary to show tree summary under a directory (#3381)
• Supports using FoundationDB as metadata engine (#2581, #2824)
• Supports using GlusterFS as object storage (#3840, #3869)
• Supports using GETFLAGS & SETFLAGS in ioctl (#2346, #2859)
• Supports using jfs:// as schema for the sync command (#3161)
• cmd/mount: add a new option --update-fstab to add/update fstab (#2462, #2997)
• cmd/mount: add a new option --verify-cache-checksum to adjust the checksum strategy for cache files (#2626)
• cmd/mount: add a new option --cache-scan-interval to customize the interval for large cache directories (#2692)
• cmd/mount: add a new option --skip-dir-nlink to reduce transaction conflicts in certain cases (#3285, #3322)
• cmd/mount: add a new option --cache-eviction to specify the eviction policy (#3478)
• cmd/mount: add a new option --prefix-internal to prefix internal files by .jfs (#3567)
• cmd/mount: add a new option --root-squash to map root user to an unprivileged one (#3575, #3579)
• cmd/mount: add a new option --atime-mode to specify the atime update policy (#3521, #3585, #3591)
• cmd/umount: add a new option --flush to wait for all staging chunks to be flushed (#3224)
• cmd/webdav: add new options --cert-file and --key-file to support basic authentication and TLS (#3146, #3156)
• cmd/format: add a new option --encrypt-algo to specify the encryption algorithm (aes256 or chacha20) for data objects (#2330)
• cmd/format&mount: add a new option --storage-class to specify the storage class used for the object storage (#3534, #3588)
• cmd/config: add a new option --dir-stats to enable constantly track of directory usage (#3257, #3276, #3277, etc.)
• cmd/config: add new options --upload-limit and --download-limit to set default bandwidth limits for the volume (#3574, #3589)
• cmd/fsck: add a new option --repair to repaire broken directories (#2654, #2785)
• cmd/dump: add a new option --keep-secret-key to keep secret key intact during dumping (#2699)
• cmd/status: add a new option --more to show information about trash files/slices and pending deleted files/slices (#2977, #3085, #3145)
• cmd/sync: add a new option --manager-addr to customize the IP and port of the manager node (#3912)
• cmd/sync: add a new option --existing to skip creating new files on the destination (#3566)
• cmd/sync: add a new option --ignore-existing to skip updating files that already exist on the destination (#3566)
• meta/redis: support specifying tls-cert-file, tls-key-file and tls-ca-cert-file in the META-URL query string (#3388, #3403)
• meta/postgres: support specifying one schema by search_path in the META-URL query string to replace the default one (#2663)
• chunk: add upload delay metrics (#2713)
• object: support listing with delimiter (#2813)
• object: add the Copy interface to copy an object within the object storage (#3581)
• object/hdfs: support configuring the number of replications by dfs.replication (#2680)
• object/hdfs: support more formats of bucket path (#3517, #3528, #3577, #3713)
• hadoop: add an option no-session to disable session management (#3750)

Changed

• cmd/format: add default MinClientVersion to prevent old clients (#3871)
• cmd/mount: don't mount again if the mount point is already mounted by the same juicefs volume (#2979, #3059)
• cmd/mount: allow setting heartbeat to 0 (used for embedded metadata engines) (#3471)
• cmd/mount: make the format configuration reloadable without re-mount (#2994)
• cmd/mount: remove gops agent (#2884)
• cmd/status: show detailed plock records for sessions (#2627)
• cmd/status: add IP addresses to the session info (#3464)
• cmd/info: show locks of a certain file (#3001)
• cmd/info: support fast mode to get recursive stats of a directory more quickly (#3296, #3317, #3340, #3423)
• cmd/gc: support cleaning expired trash files, delfiles and delslices (#3022)
• cmd/fsck: ignore chunks of deleted files (#3832)
• cmd/dump&load: support dumping to or loading from a gzip compressed JSON file (#2908)
• cmd/wamrup: warm up metadata as well when open cache is enabled (#3185)
• cmd/rmr: skip trash directories when removing entries (#3179)
• cmd/sync: read large file from sftp in parallel (#3933)
• cmd/sync: list file systems in parallel (#3769, #3782)
• meta/redis: skip setting maxmemory_policy if this field does not exist (#3888)
• meta/sql: add index to column refs in sliceRef to speed up the doCleanupSlices function (#3842)
• chunk: disable refreshCacheKeys if interval < 0 (#2865)
• object/hdfs: support using base64 encoded keytab for hdfs (#3818, #3874)
• object/sftp: try public keys before interactive password (#3886)
• utils: write syslog in background (#3967)
• hadoop: reduce memory usage (#2580, #2592, #2620)
• hadoop: treat user 'root' as an unprivileged user (#3213)
• hadoop: print go log messages to the Java log (#3420)

Bugfix

• cmd: fix the issue that some commands (e.g. info) may fail if internal data is very large (#3850)
• cmd: set the correct GOMAXPROCS value when running inside container (#2935)
• cmd/mount: fix the issue that go-fuse log may not be printed when running in daemon mode (#3206)
• cmd/gateway: fix the issue that List may fail if there's a symlink pointing to a file in another file system (#3879)
• cmd/sync: fix the issue that for jfs schema customized endpoint is not passed to the worker (#3922)
• cmd/sync: fix the issue that --links does not work for symlinks pointing to a directory (#3942)
• cmd/sync: fix the issue that umask is not correctly applied (#3784)
• cmd/sync: fix the issue that environments are not properly shared in cluster mode (#3254)
• meta: fix the issue that baseMeta.fmt is not atomically updated (#3956)
• meta: fix the issue that files may be created under a directory that is already in trash (#3864)
• meta: fix the issue that mtime is not correctly updated during writing (#3552)
• meta: fix the issue that file attribute may be overwritten in some cases (#3766)
• meta/redis: pass all gids to the LUA script to check permission (#3902)
• meta/tkv: fix the issue that attr may be nil during dumping if the inode key is missing (#3833)
• meta/memkv: fix the conflict between scan and write operations (#3838)
• vfs: fix ioctl result and newly support the FS_IOC_FSGETXATTR command (#3936)
• vfs: fix the deadlock under concurrent truncate and release (#3457)
• fs: limit the maximum resolve depth for symlinks (#3896)
• object/sql: fix the issue that MySQL may not properly handle some special characters (#3299, #3303)
• object/ceph: fix the error handling to put empty file, range get and get non-exist object (#3835, #3855)
• metrics: fix the issue that sometimes object metrics may be missing (#3853)
• hadoop: fix the method to obtain jarPath on Windows (#3907)
• hadoop: fix the issue that the generated uid/gid may overflow int32 when using SQL metadata engines (#3680)

Note: all bug fixes in v1.0.2, v1.0.3 and v1.0.4 are also included in this release.

This is the first release candidate for JuiceFS v1.1. It has 55 commits from 14 contributors, thanks to @zhoucheng361 @zhijian-pro @CaitinChen @JoyLiuc @Hexilee @SandyXSD @timfeirg @yuhr123 @davies @tangyoupeng @chenjie4255 @leemingzixxoo @p1u3o @xiaogaozi !

New

• cmd/sync: add a new option --manager-addr to customize the IP and port of the manager node (#3912)

Changed

• cmd/sync: read large file from sftp in parallel (#3933)
• meta/redis: skip setting maxmemory_policy if this field does not exist (#3888)
• utils: write syslog in background (#3967)

Bugfix

• cmd/sync: fix the issue that for jfs schema customized endpoint is not passed to the worker (#3922)
• cmd/sync: fix the issue that --links does not work for symlinks pointing to a directory (#3942)
• meta: add lock for baseMeta.fmt (#3956)
• meta/redis: pass all gids to the LUA script to check permission (#3902)
• vfs: fix ioctl result and newly support the FS_IOC_FSGETXATTR command (#3936)
• fs: limit the maximum resolve depth for symlinks (#3896)
• object/gluster: fix the issue that List may return wrong result for symlinks (#3950)
• hadoop: fix the method to obtain jarPath on Windows (#3907)

This is the second beta release for JuiceFS v1.1. Since beta1 it has 62 commits from 12 contributors, thanks to @SandyXSD @zhijian-pro @zhoucheng361 @davies @Hexilee @tangyoupeng @timfeirg @leoliulei @kyungwan-nam @T-TRz879 @solracsf @arnoldberlin !

New

• Supports using Gluster as object storage (#3840, #3869)

Changed

• cmd/format: add default MinClientVersion to prevent old clients (#3871)
• cmd/fsck: ignore chunks of deleted files (#3832)
• cmd/fsck: ignore dirStats if it's disabled (#3831)
• meta: don't update quota in rename if both src & dst are under the same quota (#3872)
• meta/sql: add index to column refs in sliceRef to speed up the doCleanupSlices function (#3842)
• object/hdfs: support using base64 encoded keytab for hdfs (#3818, #3874)
• object/sftp: try public keys before interactive password (#3886)

Bugfix

• cmd: fix the issue that some commands (e.g. warmup) fail when used on binded mount points (#3858)
• cmd: fix the issue that some commands (e.g. info) may fail if internal data is very large (#3850)
• cmd/gateway: fix the issue that List may fail if there's a symlink pointing to a file in another file system (#3879)
• cmd/sync: fix the issue that umask is not correctly applied (#3784)
• meta: fix the issue that files may be created under a directory that is already in trash (#3864)
• meta: fix the issue that atime is updated too frequently in relatime mode if opencache is disabled (#3819)
• meta/tkv: fix the issue that attr may be nil during dumping if the inode key is missing (#3833)
• meta/memkv: fix the conflict between scan and write operations (#3838)
• object/ceph: fix the error handling to put empty file, range get and get non-exist object (#3835, #3855)
• metrics: fix the issue that sometimes object metrics may be missing (#3853)

This is the first beta release for JuiceFS v1.1. Since v1.0 it has 726 commits from 57 contributors, thanks to @zhijian-pro @SandyXSD @zhoucheng361 @davies @Hexilee @yuhr123 @tangyoupeng @timfeirg @sanwan @xiaogaozi @Suave @solracsf @tcodehuber @zyfjeff @kyungwan-nam @zwwhdls @Ziy1-Tan @pigletfly @qingyunha @rfyiamcool @201341 @rayw000 @shichanglin5 @AIXjing @Vigor-jpg @baifachuan @yongxie09 @tisonkun @neocxf @molei20021 @luohy15 @leoliulei @jomenxiao @divinerapier @cuishuang @cmmp6 @xiedeyantu @xyb @Karitham @aronsky @yqlbu @JoyLiuc @sjp00556 @shjwudp @jzcruiser @hamidzr @ggicci @gdams @eryugey @guaneryu @Erisa @p1u3o @CodeRusher @CaitinChen @dugusword @Arvintian @AChangFeng !

Highlights

• Supports setting quotas on directory level
• Supports fast copying a tree with clone command
• Supports using FoundationDB as metadata engine

New

• Added a new command quota to manage directory quotas (#3268, #3371, #3374, etc.)
• Added a new command clone to fast clone a directory tree (#3279, #3366, #3369, etc.)
• Added a new command debug to collect a bunch of client information used for diagnosing problems (#2360, #2815, #2857, etc.)
• Added a new command restore to batch restore files in trash (#3657)
• Added a new command summary to show tree summary under a directory (#3381)
• Supports using FoundationDB as metadata engine (#2581, #2824)
• Supports using GETFLAGS & SETFLAGS in ioctl (#2346, #2859)
• Supports using jfs:// as schema for the sync command (#3161)
• cmd/mount: add a new option --update-fstab to add/update fstab (#2462, #2997)
• cmd/mount: add a new option --verify-cache-checksum to adjust the checksum strategy for cache files (#2626)
• cmd/mount: add a new option --cache-scan-interval to customize the interval for large cache directories (#2692)
• cmd/mount: add a new option --skip-dir-nlink to reduce transaction conflicts in certain cases (#3285, #3322)
• cmd/mount: add a new option --cache-eviction to specify the eviction policy (#3478)
• cmd/mount: add a new option --prefix-internal to prefix internal files by .jfs (#3567)
• cmd/mount: add a new option --root-squash to map root user to an unprivileged one (#3575, #3579)
• cmd/mount: add a new option --atime-mode to specify the atime update policy (#3521, #3585, #3591)
• cmd/umount: add a new option --flush to wait for all staging chunks to be flushed (#3224)
• cmd/webdav: add new options --cert-file and --key-file to support basic authentication and TLS (#3146, #3156)
• cmd/format: add a new option --encrypt-algo to specify the encryption algorithm (aes256 or chacha20) for data objects (#2330)
• cmd/format&mount: add a new option --storage-class to specify the storage class used for the object storage (#3534, #3588)
• cmd/config: add a new option --dirStat to enable constantly track of directory usage (#3257, #3276, #3277, etc.)
• cmd/config: add new options --upload-limit and --download-limit to set default bandwidth limits for the volume (#3574, #3589)
• cmd/fsck: add a new option --repair to repaire broken directories (#2654, #2785)
• cmd/dump: add a new option --keep-secret-key to keep secret key intact during dumping (#2699)
• cmd/status: add a new option --more to show information about trash files/slices and pending deleted files/slices (#2977, #3085, #3145)
• cmd/sync: add a new option --existing to skip creating new files on the destination (#3566)
• cmd/sync: add a new option --ignore-existing to skip updating files that already exist on the destination (#3566)
• meta/redis: support specifying tls-cert-file, tls-key-file and tls-ca-cert-file in the META-URL query string (#3388, #3403)
• meta/postgres: support specifying one schema by search_path in the META-URL query string to replace the default one (#2663)
• chunk: add upload delay metrics (#2713)
• object: support listing with delimiter (#2813)
• object: add the Copy interface to copy an object within the object storage (#3581)
• object/hdfs: support configuring the number of replications by dfs.replication (#2680)
• object/hdfs: support more formats of bucket path (#3517, #3528, #3577, #3713)
• hadoop: add an option no-session to disable session management (#3750)

Changed

• cmd/mount: don't mount again if the mount point is already mounted by the same juicefs volume (#2979, #3059)
• cmd/mount: allow setting heartbeat to 0 (used for embedded metadata engines) (#3471)
• cmd/mount: make the format configuration reloadable without re-mount (#2994)
• cmd/mount: remove gops agent (#2884)
• cmd/status: show detailed plock records for sessions (#2627)
• cmd/status: add IP addresses to the session info (#3464)
• cmd/info: show locks of a certain file (#3001)
• cmd/info: support fast mode to get recursive stats of a directory more quickly (#3296, #3317, #3340, #3423)
• cmd/gc: support cleaning expired trash files, delfiles and delslices (#3022)
• cmd/dump&load: support dumping to or loading from a gzip compressed JSON file (#2908)
• cmd/wamrup: warm up metadata as well when open cache is enabled (#3185)
• cmd/rmr: skip trash directories when removing entries (#3179)
• cmd/sync: list file systems in parallel (#3769, #3782)
• chunk: disable refreshCacheKeys if interval < 0 (#2865)
• hadoop: reduce memory usage (#2580, #2592, #2620)
• hadoop: treat user 'root' as an unprivileged user (#3213)
• hadoop: print go log messages to the Java log (#3420)

Bugfix

• cmd: set the correct GOMAXPROCS value when running inside container (#2935)
• cmd/mount: fix the issue that go-fuse log may not be printed when running in daemon mode (#3206)
• cmd/sync: fix the issue that environments are not properly shared in cluster mode (#3254)
• meta: fix the issue that mtime is not correctly updated during writing (#3552)
• meta: fix the issue that file attribute may be overwritten in some cases (#3766)
• vfs: fix the deadlock under concurrent truncate and release (#3457)
• object/sql: fix the issue that MySQL may not properly handle some special characters (#3299, #3303)
• hadoop: fix the issue that the generated uid/gid may overflow int32 when using SQL metadata engines (#3680)

Note: all bug fixes in v1.0.2, v1.0.3 and v1.0.4 are also included in this release.

This is the fourth patch release for JuiceFS v1.0. It has 33 commits from 9 contributors, thanks to @zhijian-pro @SandyXSD @davies @Hexilee @tangyoupeng @molei20021 @jomenxiao @shichanglin5 @p1u3o !

Changed

• cmd/format: remove the testing directory after test (#3418)
• cmd/load: support loading metadata from an encrypted file (#3311)
• meta: recreate the client session with info if it was cleaned before (#3190, #3197)
• meta: use 'max-deletes' to control the number of background clean-up workers as well (#3227, #3401)
• meta/tikv: add background GC worker for TiKV (#3262, #3432)
• chunk: deleting an object should be idempotent (#3171)
• object: set encoding type when listing objects (#3244, #3269)
• object/s3: support using OVH region (#3133)
• object/s3: optionally turn off 'S3Disable100Continue' via url query parameter (#3228)
• deps: upgrade go version to 1.18 (#3135)
• deps: upgrade BadgerDB to v3.2103.5 (#3309)
• deps: upgrade golang.org/x/net to v0.7.0 (#3350)

Bugfix

• cmd/sync: fix the issue that password is displayed in process title (#3256, #3258)
• cmd/sync: fix the issue that sync fails when the source file is growing (#3405)
• cmd/gateway: fix the issue that a folder object is not properly detected when using s3fs on gateway (#3378)
• meta/redis: fix the issue that Redis sentinel with TLS fails hostname verification (#3194)
• meta/redis: fix the issue that client may panic on doRename when a node key is lost (#3266)
• meta/redis: fix the issue that keys may be unwatched unexpectedly in copy_file_range (#3400)
• meta/badger: fix the issue that scanKeysRange ignores begin/end (#3138)
• chunk: fix the issue that staging files may be uploaded for many times (#3157)
• vfs: fix the issue that large response may not be correct when reading from .control (#3170)
• fs: fix the issue that attribute cache is not invalidated after calling Utime (#3137)
• object/obs: fix the issue that error is not properly handled when using OBS encrypted bucket (#3199)
• object/obs&tos: fix the issue that error is not properly handled when calling RangeGet (#3270)
• object/b2: fix the issue that error is not properly handled when getting info of an empty object (#3274)
• object/sftp: fix the issue that the size of a symlinked file is not correct (#3426)
• hadoop: fix the issue that symlink is not followed as expected (091a7bff, #3165)

This is the third patch release for JuiceFS v1.0. It has 35 commits from 9 contributors, thanks to @zhijian-pro @SandyXSD @davies @tangyoupeng @Hexilee @baifachuan @neocxf @cmmp6 @zhoucheng361 !

New

• Support TOS as object storage (#2909, #2912, #2914)

Changed

• cmd: increase default value of max-deletes from 2 to 10 (#3080)
• cmd/mount: support customizing permission of cache files (#2976)
• meta: use an independent lock for session so it won't affect workloads (#3000)
• meta: limit cleanupDelayedSlices by time, which may speed up the cleanup (#3079)
• meta: force delete file data when cleaning up trash (#3119)
• meta/sql: set global lock for sqlite3 transactions to reduce conflict (#3046)
• chunk/cache: cleanup cached objects during scaning (#3113)
• object: increase buffer size from 4KB to 32KB, which can reduce CPU utilization for syscalls (#3072)
• object/encrypt: support PEM in PKCS8 (#3065)
• object/bos: support user choosing http or https (#2978)
• object/ks3: support both public and private regions (#3104)
• fuse: reduce number of max readers to save CPU utilization for hosts with many CPU cores (#3069)
• hadoop: improve the read performance when buffer size < 128 KiB (#3004)
• hadoop: add timeout for pushgateway (#3044)
• deps: upgrade tikv client-go to v2.0.2 (#2947)

Bugfix

• cmd/objbench: fix the issue that latency is wrongly calculated (#3100)
• cmd/gateway: fix the issue than HEAD a non-existent directory returns 200 (#2955)
• meta: fix the issue that client may crash when the value of slices is corrupt (#2876)
• meta: fix the issue that password may be exposed when meta engine is not available (#3003)
• meta: fix the issue that cleanup trash may run infinitely when there is a bad entry (#3032)
• meta/redis: fix the issue that slice may leak when deleting chunk (#2879)
• meta/sql: do not use the number of affected rows returned by update (#2986)
• object/sftp: fix the issue that special characters in username and password are not properly handled (#2981)
• object/s3: fix the issue that endpoint is wrongly parsed for oracle cloud (#3075)
• object/obs: fix the issue that etag is not correct when obs enables encrypted (#3098)
• hadoop: fix the issue that the running process may crash if the local juicefs-hadoop.jar is deleted (#2906)
• hadoop: fix the issue that thread may leak because emptier filesystem is not properly closed (#3090)

This is the first patch release (v1.0.1 is broken) for JuiceFS v1.0. It has 32 commits from 10 contributors, thanks to @davies @zhijian-pro @tangyoupeng @SandyXSD @timfeirg @201341 @pigletfly @dugusword @Arvintian @zwwhdls !

Changed

• cmd/sync: sync uid/gid even if the user/group does not exist (#2502)
• meta/badger: improve the performance to reset the database (#2811)
• vfs: flush data and close fd even if the close is interrupted (#2745)
• object/minio: support setting MinIO region via an environment variable (#2673)
• object/ks3: support private region for KS3 (#2812)
• gateway: support specifying umask for directories (#2445)
• hadoop: support nest juicefs-hadoop.jar in spring-boot (#2756)

Bugfix

• cmd/format: fix the issue that schema in bucket address is not properly handled (#2749)
• cmd/objbench: fix the issue that bucket is not automatically created when objbench is run with skip-functional-tests (#2773)
• cmd/sync: fix the issue that '\' is wrongly replaced for non-windows system (#2778)
• meta/pg: fix the issue that PostgreSQL does not recognize Unix domain socket properly (#2607)
• chunk/cache: fix the issue that client may crash if there are invalid files in the cache dir (#2686)
• vfs: fix the issue on Linux that entries are still visible after rmr finishes (#2776, #2838)
• vfs/backup: fix the issue that metadata backup may fail because the client cannot access /tmp (#2707)
• utils: fix the issue on Windows that logger prints color control codes (but no color) (#2567)
• compress: fix the issue that client may crash if input is empty for LZ4 decompressing (#2499)
• gateway: fix the issue that file descriptors may leak in gateway (#2661)
• hadoop: fix the issue that trash emptier may fail because of no permission (#2512)
• hadoop: fix the issue that relative paths are not properly qualified (#2696, #2709)
• hadoop: fix the issue that juicefs.umask is not properly applied when creating parent dir (#2833)
• arch: fix the issue on 32bits system that client may crash because of unaligned atomic operation (#2703)
• arch: fix the issue on 32bits system that readAheadTotal may overflow (#2726)

This is the first stable release of JuiceFS, and is an LTS version that will be maintained for 24 months.

Starting from v1.0.0-rc3 it has 73 commits from 13 contributors, thanks to @SandyXSD @zhijian-pro @xiaogaozi @zhoucheng361 @rayw000 @tangyoupeng @AIXjing @sanwan @davies @yuhr123 @timfeirg @201341 @solracsf !

New

• object/redis: support redis sentinel & cluster mode (#2368, #2383)

Changed

• cmd/gc: no warning log if no --delete specified (#2476)
• cmd/load: reset root inode to 1 if loading from a subdir (#2389)
• meta: check UUID and metadata version after setting reloaded (#2416, #2420)
• meta: reset variables at the beginning of transactions (#2402, #2409)
• meta refactor: distinguish Slice from Chunk (#2397)
• fuse: support CAP_EXPORT_SUPPORT flag (#2382)
• util: progressbar total equals current max (#2377)

Bugfix

• cmd/profile: ignore error of Scanln (#2400)
• meta: fix potential overflowed size in Fallocate (#2403)
• chunk: cleanup cache if it's added after removed (#2427)
• hadoop: do not use local uid/gid if global user/group is specified (#2433)
• hadoop: update guid and clean old ones (#2407)
• hadoop: fix make in mac m1 (#2408)

JuiceFS v1.0.0-rc3 is the third release candidate for v1.0. It has 35 commits from 10 contributors, thanks to @zhijian-pro @SandyXSD @davies @tangyoupeng @sanwan @xiaogaozi @chenhaifengkeda @zhoucheng361 @201341 @Suave !

New

• Supports using unix socket for Redis (#2319)

Changed

• cmd/info: print objects for files, and add --raw option for slices (#2316)
• fuse/context: ignore interrupt within one second (#2324)

Bugfix

• cmd/info: support get trash info (#2363)
• cmd/bench: fix bench display (#2322)
• cmd/objbench: fix skip functional test (#2341)
• meta/redis: fix unlock on not-existed lock (#2325)
• object: fix skip tls verify (#2317)
• object/ks3: use virtual hosted-style for ks3 (#2349)
• object/eos: using the Path style addressing in eos (#2344)
• object/ceph: fix the arguments of register function (#2306)
• vfs&fuse: Fix stale attribute cache in kernel (#2336)
• hadoop: checksum fix for files with size close to blockSize (#2333)

JuiceFS v1.0.0-rc2 is the second release candidate for v1.0. It has 40 commits from 10 contributors, thanks to @davies @zhijian-pro @sanwan @zhoucheng361 @tangyoupeng @SandyXSD @201341 @chnliyong @solracsf @xiaogaozi !

New

• Supports using session token for object storage (#2261)

Changed

• cmd/info: make output more human friendly (#2303)
• meta: optimize output of format structure (#2250)
• meta/sql: support database without read-only transaction (#2259)
• object/webdav: replace webdav client (#2288)
• SDK: only release sdk jar (#2289)

Bugfix

• cmd: --backup-meta could be 0 to disable backup (#2275)
• cmd: fix concurrent rmr/warmup in mac (#2265)
• cmd/fsck: fix object name for Head method (#2281)
• cmd/info: Fix repeat path info when inode is 1 (#2294)
• meta: fix GetPaths for mount points with subdir specified (#2298)
• meta: don't treat trash as dangling inode (#2296)
• meta: fix index underflow in 32bit arch (#2239)
• meta/tkv: use absolute path for badgerDB as metadata engine (#2256)
• object/azure: fix azure list api and incorrect bucket URL (#2263)
• metrics: use separate metrics per volume (#2253)

JuiceFS v1.0.0-rc1 is the first release candidate for v1.0. It has 184 commits from 17 contributors, thanks to @davies @zhoucheng361 @SandyXSD @zhijian-pro @sanwan @xiaogaozi @tangyoupeng @solracsf @showjason @rayw000 @AIXjing @helix-loop @Suave @zhouaoe @chnliyong @yuhr123 @liufuyang !

Highlights

• Dumping metadata from Redis has been improved, massively reducing the memory required to below 1/20 of that before. It also relieves the memory spike of a client when doing metadata backup. Dumping metadata from SQL and TiKV within a single transaction to ensure consistency.
• Loading metadata to all engines has been improved as well. Instead of loading the whole dumped file in one step, JuiceFS will now read it in a stream, and simultaneously import metadata to the engine. This saves a lot of memory when the dumped file is huge.
• Improved stability for SQL engine under heavy workload.
• Added a new command juicefs objbench that can be used to run basic function tests and benchmarks on object storage, making sure it works as expected.

New

• Supports using SQL databases, etcd as data storage (#2003, #2009)
• Supports finding all paths of an inode with juicefs info command (#2058, #2161, #2193)
• Supports using Pyroscope to record JuiceFS profiling (#1952)
• Added progress bar for juicefs rmr and juicefs warmup commands (#2197)
• Added a new command juicefs objbench to run basic benchmarks on object storage (#2055, #2091)
• Added a new command juicefs version to print version, as an alternative to --version (#2229)

Changed

• cmd: check the range of parameters (#2195)
• cmd: eliminate panic which is triggered by missing argument (#2183)
• cmd/mount: warn about the behavior of mounting the same directory multiple times (#2141)
• cmd/warmup: support warmup from inside of a container (#2056)
• meta: add delayed slice only when chunkid > 0 (#2231)
• meta: speed up and reduce memory for loading metadata (#2142, #2148)
• meta: add a pessimistic lock to reduce conflicted transactions in the database (#2111)
• meta: limit the number of scanned files in cleanup (#2157)
• meta: limit number of files when cleanup trash (#2061)
• meta: limit the number of coroutines to delete file data (#2042)
• meta: log last error if a transaction has been ever restarted (#2172)
• meta: log session info when cleaning up a stale one (#2045)
• meta: skip updating mtime/ctime of the parent if it's updated recently (#1960)
• meta/redis: check config 'maxmemory-policy' (#2059)
• meta/redis: Speedup dump for Redis and reduce memory usage (#2156)
• meta/tkv: Speedup dump for kv storage (#2140)
• meta/tkv: dump metadata using snapshot (#1961)
• meta/tkv: use scanRange to get delayed slices (#2057)
• meta/sql: dump metadata in a single transaction (#2131)
• chunk/store: keep cache after uploading staging blocks (#2168)
• object: reload the configuration for data storage (#1995)
• object/sftp: load default private keys for sftp (#2014)
• object/oss: add user agent for oss #1992 (#1993)
• vfs: hide .control from readdir (#1998)
• gateway: clean up expired temporary files (#2082)
• SDK: package amd64 and arm64 libjfs (#2198)
• SDK: don't reuse fd in Java SDK (#2122)
• Dependency: upgrade coredns for CVE-2019-19794 (#2190)
• Dependency: upgrade azblob sdk (#1962)
• meta: keep valid utf8 in dumped JSON (#1973)
• SDK: mvn shade some dependency to avoid class conflict (#2035)
• meta: truncate trash entry name if it's too long (#2049)
• meta/sql: use repeatable-read for transaction (#2128)

Bugfix

• cmd: fix not showing arguments for commands without META-URL (#2158)
• cmd/sync: fix sync lost file (#2106)
• cmd/warmup: fix warmup on read-only mount point (#2108)
• meta: skip updating sliceRef if id is 0 (#2096)
• meta: fix update xattr with the same value (#2078)
• meta/redis: handle lua result from Redis v7.0+ (#2221)
• meta/sql: fix dump with more than 10000 files (#2134)
• meta/sql: one transaction in SQLite at a time (#2126)
• meta/sql: fix rename with Postgres with repeatable read (#2109)
• meta/sql: fix deadlock in PG (#2104)
• meta/sql: ignore error about duplicated index (#2087)
• meta/sql: read database inside transaction (#2073, #2086)
• meta/sql: retry transaction on duplicated entry and concurrent update (#2077)
• meta/sql: fix the deadlock in rename (#2067)
• meta/sql: retry transaction for duplicate key in PG (#2071)
• meta/sql: fix update query in SQL transaction (#2024)
• meta/tkv: fix value of delSliceKey (#2054)
• meta/tkv: upgrade TiKV client to 2.0.1 to fix nil connection (#2050)
• chunk/store: fix stats of cached space in writeback mode (#2227)
• object: delete should be idempotent (#2034)
• object/file: Head of file should return File (#2133)
• object/s3: check prefix and marker with returned keys from S3 (#2040)
• object/prefix: fix with prefix returning nil error for unsupported ops (#2021)
• object/sftp: fix auth of sftp with multiple keys (#2186)
• object/sftp: fix prefix of sftp, support ssh-agent (#1954)
• vfs/backup: skip cleanup if list failed (#2044)
• SDK: handle atomic rename exception (#2192)

JuiceFS v1.0.0-beta3 is the third beta release for v1.0. It has 247 commits from 22 contributors, thanks to @SandyXSD @zhoucheng361 @davies @zhijian-pro @yuhr123 @sanwan @AIXjing @rayw000 @xiaogaozi @Suave @showjason @tangyoupeng @201341 @solracsf @guo-sj @chnliyong @DeanThompson @zwwhdls @wph95 @lidaohang @sjp00556 @DEvmIb !

Highlights

• Supports etcd as a new metadata engine. It can be a handy choice when you only need a small volume but cares more about the data availability and persistence.
• Supports Redis Cluster and other compatible services (Amazon MemoryDB for Redis) as metadata engines.
• When using SQL metadata engines, file names not encoded by UTF-8 can now be properly handled after manual modification to the table schema, see details.
• A new session management format is introduced. Old clients are unable to detect sessions with version 1.0.0-beta3 or higher via juicefs status or juicefs destroy command, see details.
• If trash is enabled, compacted slices are kept as well in case they are needed to recover file data. These slices will be cleaned up automatically after trash-days, and can be deleted manually via juicefs gc command.
• A lot of improvements have been made to juicefs sync command.
• A lot of protection checks against unintentional misuse have been added.

New

• Supports etcd as metadata engine (#1638)
• Supports Redis in cluster mode while using one slot for each file system (#1696)
• Supports handling file names not encoded by UTF-8 for SQL metadata engines (#1762)
• Supports TLS when using TiKV as metadata engine or object storage (#1653, #1778)
• Supports Oracle Object Storage as data storage (#1516)
• Supports setting umask for S3 Gateway (#1537)
• Java SDK now supports pushing metrics to Graphite (#1586)
• Added a new option --heartbeat for the mount command to adjust heartbeat interval (#1591, #1865)
• Added many improvements for sync command to make it more handy (#1554, #1619, #1651, #1836, #1897, #1901)
• Added a new option --hash-prefix for the format command to add a hashed prefix for objects (#1657)
• Added a new client option --storage to allow customized storage type (#1912)

Changed

• compacted slices will be kept for trash-days if trash is enabled (#1790)
• cmd: support using integer for duration flags (#1796)
• cmd: use homedir as default working directory for non-root users (#1869)
• cmd/format: create a uuid object in the target bucket (#1548)
• cmd/dump&load: dump load behavior supports non-ascii characters (#1691)
• cmd/dump: omit empty value in dumped JSON (#1676)
• cmd/dump: remove secret key (#1569)
• meta: encrypt the secret-key and encrypt-key in setting (#1562)
• meta: create subdir automatically (#1712)
• meta: specify the format field preventing update (#1776)
• meta: escape meta password from env (#1879)
• meta/redis: check redis version (#1584)
• meta/redis: use smaller retry backoff in sentinel mode (#1620)
• meta/redis: retry transaction for connection error or EXECABORT (#1637)
• meta/sql: retry transaction after too many connections (#1876)
• meta/sql: add primary key for all tables (#1913, #1919)
• meta&chunk: Set max retries of meta & chunk according to the config io-retries (#1713, #1800)
• chunk: limit number of upload goroutines (#1625)
• chunk/store: limit max retry for async upload as well (#1673)
• object/obs: Verify Etag from OBS (#1715)
• object/redis: implement listAll api for redis (#1777)
• fuse: automatically add ro option if mount with --read-only (#1661)
• vfs/backup: reduce the limit for skipping backup (#1659)
• sync: reduce memory allocation when write into files (#1644)
• SDK: use uint32 for uid,gid (#1648)
• SDK: handle removeXAttr return code (#1775)
• Dependency: switch to Go 1.17 (#1594)
• Dependency: fix jwt replace (#1534)
• Dependency: upgrade golang-cross version to v1.17.8 (#1539)
• Dependency: upgrade tikv to v2.0.0 (#1643)
• Dependency: reduce dep from minio (#1645)
• Dependency: upgrade gjson to 1.9.3 (#1647)
• Dependency: upgrade sdk for object storage (#1665)
• Dependency: upgrade qiniu sdk (#1697)

Bugfix

• cmd/format: fix setting quota (#1658)
• cmd/mount: fix parsing of cache dir (#1758)
• cmd/warmup: fix handling of relative paths (#1735)
• cmd/sync: fix sync command not working when destination is webdav (#1636)
• cmd/gateway: fix s3 gateway DeleteObjects panic (#1527)
• meta: forbid empty name for dentry (#1687)
• meta: lock counters when loading entries (#1703)
• meta: fix snap not released if error occurs when dumping meta (#1669)
• meta: don't update parent attribute if it's a trash directory (#1580)
• meta/redis: fix loading large directory into Redis (#1858)
• meta/redis: update used space/inodes in memory whenever needed (#1573)
• meta/sql: use upsert to update xattr for PG (#1825)
• meta/sql: split insert batch (#1831)
• meta/sql: fix wrong result from scanning SQL database (#1854)
• chunk/cache: Fix read disk_cache not existed remove cache key (#1677)
• object: fallback to List only if ListAll is not supported (#1623)
• object/b2: check returned bucket from B2 (#1745)
• object/encrypt: fix parse rsa from pem (#1724)
• object/encrypt: Add JFS_RSA_PASSPHRASE environment variable prompt information (#1706)
• object/sharding: fix ListAll returning invalid objects (#1616)
• object/ceph: fix listAll hangs if there are many objects (#1891)
• vfs: write control file asynchronously (#1747)
• vfs: fix getlk in access log (#1788)
• sync: Fix copied and copiedBytes (#1801)
• utils: fix the problem that the progress bar loses the log (#1756)
• SDK: rename libjfs atomic (#1939)