lotus

Reference implementation of the Filecoin protocol, written in Go

OTHER License

Stars
2.8K
Committers
268
View Code on GitHub Visit Website Twitter
Ecosystems: IPFS, Go

Bot releases are hidden (Show)

lotus - v1.23.3-rc2

Published by arajasek over 1 year ago

This is the second release candidate for the upcoming feature release v1.23.3 of Lotus. This feature release includes numerous improvements and enhancements for node operators, ETH RPC-providers and storage providers.

Highlights

New features

For a full changelog, see here.

lotus - v1.23.3-rc1

Published by arajasek over 1 year ago

This is the first candidate for the upcoming feature release v1.23.3 of Lotus. This feature release includes numerous improvements and enhancements for node operators, ETH RPC-providers and storage providers.

Highlights

New features

For a full changelog, see here.

lotus - v1.22.2

Published by shrenujbansal over 1 year ago

v1.22.2 / 2023-06-28

This is a patch release on top of 1.22.1 containing the fix for https://github.com/filecoin-project/lotus/issues/10906
This fixes the syncing issue seen by many node operators/SPs, usually when performing actions which would result in msgs staying in their mpool for longer periods of time (ex. PSD) resulting in these msgs being republished multiple times and possibly lowering your peer scores. Please refer to the above issue for more details.
We'd recommend everyone to accept this fix to better overall network health

lotus - v1.23.2

Published by shrenujbansal over 1 year ago

v1.23.2 / 2023-06-28

This is a patch release on top of 1.23.1 containing the fix for https://github.com/filecoin-project/lotus/issues/10906
This fixes the syncing issue seen by many node operators/SPs, usually when performing actions which would result in msgs staying in their mpool for longer periods of time (ex. PSD) resulting in these msgs being republished multiple times and possibly lowering your peer scores. Please refer to the above issue for more details.
We'd recommend everyone to accept this fix to better overall network health

Improvements

  • fix: pubsub: do not treat ErrExistingNonce as Reject
lotus - v1.23.1

Published by shrenujbansal over 1 year ago

v1.23.1 / 2023-06-20

This is an optional feature release of Lotus. This feature release includes numerous improvements and enhancements for node operators, ETH RPC-providers and storage providers.

☢️ Upgrade Warnings ☢️

If you are upgrading to this release candidate from Lotus v1.22.1, please make sure to read the upgrade warnings section in the v1.23.0 release first.

  • Storage providers: The Lotus-Miner legacy-markets has been disbled by default in this feature release and will be removed in the near term future. Users are adviced to migrate to Boost or other SP markets systems.

Highlights

🛣 Execution Lanes 🛣
This feature release introduces VM Execution Lanes! Execution lanes efficiently divide the workload between system processes (chainsync) and RPC requests. This way syncing the chain will not be at the mercy of responding to users' requests and RPC providers nodes should have less problems catching up.

To take advantage of VM Execution Lanes, you need to set up two environment variables:

  • LOTUS_FVM_CONCURRENCY - read more about how this value should be set to here
  • LOTUS_FVM_CONCURRENCY_RESERVED = 4

🧱 Aggregation / Batching fixes 🔨

Numerous aggregation and batching fixes has been included in the feature release. Large ProveCommitAggregate and PreCommitBatching messages that exceeds the block limit will now automatically be split into smaller messages when sent to the chain.

Additionally we have added a new feature that staggers the amount of ProveCommit messages sent simulatanously to the chain if a storage provider has been aggregating many sectors in ProveCommitAggregate message, but at the time of publishing the BaseFee is below the aggregation threshold. This stagger feature prevents issues where some of the ProveCommit messages fail with the SysErrorOutOfGas message. You can tweak how many messages will be staggered per epoch by changing MaxSectorProveCommitsSubmittedPerEpoch in the sealing section of the config.toml file.

NB: While these fixes are great for the reliability of aggregation and batching on the Lotus side, it has been uncovered that aggregating ProveCommit messages for sectors containing verified deals are currently more expensive then single messages due to an issue on the actors side. We therefore do not reccomend our users to aggregate ProveCommit messages when doing verified deals until that issue has been resolved. You can follow the discussion on resolving the issue on the actors side here.

Unsealing CLI/API

This feature release adds a dedicated lotus-miner sectors unseal command and API, allowing you to unseal specific sealed sectors easily.

New features

  • feat: VM Execution Lanes (filecoin-project/lotus#10551)
    • Adds VM exections lanes, efficiently dividing the workload between system processes and RPC-requests.
  • Add API and CLI to unseal sector (#10626) (filecoin-project/lotus#10626)
    • Adds lotus-miner sectors unseal cmd, and a API-method to unseal a sector.
  • feat: sealing: Split PCA/PCB batches if gas used exceeds block limit (filecoin-project/lotus#10647)
    • Splits ProveCommitAggregate and PreCommitBatch messages into multiple messages if the message exceeds the block limit.
  • Add feature to stagger sector prove commit submission (#10543) (filecoin-project/lotus#10543)
    • Staggers the amount of ProveCommit messages sent simultanously if a storage provider has been aggregating many message, but at the moment of publishing the BaseFee is below the threshold for aggregation to prevent unwanted SysErrorOutOfGas issues.
  • Set default for MaxSectorProveCommitsSubmittedPerEpoch (filecoin-project/lotus#10728)
    • Sets the default amount of ProveCommits submitted per epoch to 20.
  • feat: worker: Ensure tempdir exists (#10433) (filecoin-project/lotus#10433)
    • Ensures that a temporary directory exists on start of a lotus-worker with a custom TMPDIR set.
  • feat: sync: harden chain sync (#10756) (filecoin-project/lotus#10756)
  • feat: populate the index on snapshot import (filecoin-project/lotus#10556)
  • feat:chain: Message Index (HIGHLY EXPERIMENTAL) (filecoin-project/lotus#10452)
    • MVP of a message index that allows us to accelrate StateSearchMessage and related functionality, and eventually accelerate critical chain calls (follow up).
  • feat: Add small cache to execution traces (filecoin-project/lotus#10517)
  • feat: shed: incoming block-sub chainwatch tool (filecoin-project/lotus#10513)

Improvements

Dependencies

Others

Contributors

Contributor Commits Lines ± Files Changed
vyzo 70 +1990/-429 135
Alfonso de la Rocha 25 +814/-299 56
Steven Allen 14 +125/-539 28
Shrenuj Bansal 13 +482/-138 52
Aayush 17 +317/-301 90
Łukasz Magiera 13 +564/-26 16
Jennifer Wang 7 +401/-140 10
Fridrik Asmundsson 14 +315/-84 20
Jorropo 2 +139/-137 74
Mikers 6 +114/-43 14
Hector Sanjuan 5 +92/-44 5
Ales Dumikau 1 +117/-0 10
Mike Seiler 4 +51/-51 6
zenground0 6 +33/-25 8
Phi 8 +32/-10 10
Aayush Rajasekaran 1 +1/-32 2
Ian Davis 2 +7/-10 3
Marcel Telka 1 +5/-7 1
ychiao 1 +8/-3 2
jennijuju 1 +4/-4 8
adlrocha 2 +2/-2 2
Jiaying Wang 1 +0/-4 1
ZenGround0 1 +2/-1 2
Zeng Li 1 +1/-1 1
lotus - v1.23.1-rc4

Published by shrenujbansal over 1 year ago

This is the 4th release candidate of the upcoming optional feature release of Lotus v1.23.1.

☢️ Upgrade Warnings ☢️

  • If you are upgrading to this release candidate from Lotus v1.22.1, please make sure to read the upgrade warnings section in the v1.23.0 release first.

New features

Improvements

Dependencies

Others

Contributors

Contributor Commits Lines ± Files Changed
vyzo 70 +1990/-429 135
Alfonso de la Rocha 25 +814/-299 56
Steven Allen 14 +125/-539 28
Shrenuj Bansal 13 +482/-138 52
Aayush 17 +317/-301 90
Łukasz Magiera 13 +564/-26 16
Jennifer Wang 7 +401/-140 10
Fridrik Asmundsson 14 +315/-84 20
Jorropo 2 +139/-137 74
Mikers 6 +114/-43 14
Hector Sanjuan 5 +92/-44 5
Ales Dumikau 1 +117/-0 10
Mike Seiler 4 +51/-51 6
zenground0 6 +33/-25 8
Phi 8 +32/-10 10
Aayush Rajasekaran 1 +1/-32 2
Ian Davis 2 +7/-10 3
Marcel Telka 1 +5/-7 1
ychiao 1 +8/-3 2
jennijuju 1 +4/-4 8
adlrocha 2 +2/-2 2
Jiaying Wang 1 +0/-4 1
ZenGround0 1 +2/-1 2
Zeng Li 1 +1/-1 1
lotus - v1.23.1-rc3 / 2023-05-31

Published by shrenujbansal over 1 year ago

This is the 3rd release candidate of the upcoming optional feature release of Lotus v1.23.1.

☢️ Upgrade Warnings ☢️

  • If you are upgrading to this release candidate from Lotus v1.22.1, please make sure to read the upgrade warnings section in the v1.23.0 release first.

New features

Improvements

Dependencies

Others

Contributors

Contributor Commits Lines ± Files Changed
vyzo 70 +1990/-429 135
Alfonso de la Rocha 25 +814/-299 56
Steven Allen 14 +125/-539 28
Shrenuj Bansal 13 +482/-138 52
Aayush 17 +317/-301 90
Łukasz Magiera 13 +564/-26 16
Jennifer Wang 7 +401/-140 10
Fridrik Asmundsson 14 +315/-84 20
Jorropo 2 +139/-137 74
Mikers 6 +114/-43 14
Hector Sanjuan 5 +92/-44 5
Ales Dumikau 1 +117/-0 10
Mike Seiler 4 +51/-51 6
zenground0 6 +33/-25 8
Phi 8 +32/-10 10
Aayush Rajasekaran 1 +1/-32 2
Ian Davis 2 +7/-10 3
Marcel Telka 1 +5/-7 1
ychiao 1 +8/-3 2
jennijuju 1 +4/-4 8
adlrocha 2 +2/-2 2
Jiaying Wang 1 +0/-4 1
ZenGround0 1 +2/-1 2
Zeng Li 1 +1/-1 1
lotus - v1.23.1-rc2

Published by shrenujbansal over 1 year ago

v1.23.1-rc2 / 2023-05-10

This is the second release candidate of the upcoming optional feature release of Lotus v1.23.1.

☢️ Upgrade Warnings ☢️

  • If you are upgrading to this release candidate from Lotus v1.22.1, please make sure to read the upgrade warnings section in the v1.23.0 release first.

New features

Improvements

Dependencies

Others

Contributors

Contributor Commits Lines ± Files Changed
vyzo 70 +1990/-429 135
Alfonso de la Rocha 25 +814/-299 56
Steven Allen 14 +125/-539 28
Shrenuj Bansal 13 +482/-138 52
Aayush 17 +317/-301 90
Łukasz Magiera 13 +564/-26 16
Jennifer Wang 7 +401/-140 10
Fridrik Asmundsson 14 +315/-84 20
Jorropo 2 +139/-137 74
Mikers 6 +114/-43 14
Hector Sanjuan 5 +92/-44 5
Ales Dumikau 1 +117/-0 10
Mike Seiler 4 +51/-51 6
zenground0 6 +33/-25 8
Phi 8 +32/-10 10
Aayush Rajasekaran 1 +1/-32 2
Ian Davis 2 +7/-10 3
Marcel Telka 1 +5/-7 1
ychiao 1 +8/-3 2
jennijuju 1 +4/-4 8
adlrocha 2 +2/-2 2
Jiaying Wang 1 +0/-4 1
ZenGround0 1 +2/-1 2
Zeng Li 1 +1/-1 1
lotus - v1.23.1-rc1

Published by shrenujbansal over 1 year ago

This is the first release candidate of the upcoming optional feature release of Lotus v1.23.1.

☢️ Upgrade Warnings ☢️

  • If you are upgrading to this release candidate from Lotus v1.22.1, please make sure to read the upgrade warnings section in the v1.23.0 release first.

New features

Improvements

Dependencies

Others

Contributors

Contributor Commits Lines ± Files Changed
vyzo 70 +1990/-429 135
Alfonso de la Rocha 25 +814/-299 56
Steven Allen 14 +125/-539 28
Shrenuj Bansal 13 +482/-138 52
Aayush 17 +317/-301 90
Łukasz Magiera 13 +564/-26 16
Jennifer Wang 7 +401/-140 10
Fridrik Asmundsson 14 +315/-84 20
Jorropo 2 +139/-137 74
Mikers 6 +114/-43 14
Hector Sanjuan 5 +92/-44 5
Ales Dumikau 1 +117/-0 10
Mike Seiler 4 +51/-51 6
zenground0 6 +33/-25 8
Phi 8 +32/-10 10
Aayush Rajasekaran 1 +1/-32 2
Ian Davis 2 +7/-10 3
Marcel Telka 1 +5/-7 1
ychiao 1 +8/-3 2
jennijuju 1 +4/-4 8
adlrocha 2 +2/-2 2
Jiaying Wang 1 +0/-4 1
ZenGround0 1 +2/-1 2
Zeng Li 1 +1/-1 1
lotus - v1.23.0

Published by ZenGround0 over 1 year ago

v1.23.0 / 2023-04-21

This is the stable feature release for the upcoming MANDATORY network upgrade at 2023-04-27T13:00:00Z, epoch 2809800. This feature release delivers the nv19 Lighting and nv20 Thunder network upgrade for mainnet, and includes numerous improvements and enhancements for node operators, ETH RPC-providers and storage providers.

☢️ Upgrade Warnings ☢️

Please read carefully through the upgrade warnings section if you are upgrading from a v1.20.X release, or the v1.22.0 release. If you are upgrading from a v1.21.0-rcX these warnings should be familiar to you.

  • Starting from this release, the SplitStore feature is automatically activated on new nodes. However, for existing Lotus users, you need to explicitly configure SplitStore by uncommenting the EnableSplitstore option in your config.toml file. To enable SplitStore, set EnableSplitstore=true, and to disable it, set EnableSplitstore=false. It's important to note that your Lotus node will not start unless this configuration is properly set. Set it to false if you are running a full archival node!
  • This feature release requires a minimum Go version of v1.19.7 or higher to successfully build Lotus. Additionally, Go version v1.20 and higher is now also supported.
  • Storage Providers: The proofs libraries now have CUDA enabled by default, which requires you to install (CUDA)[https://lotus.filecoin.io/tutorials/lotus-miner/cuda/] if you haven't already done so. If you prefer to use OpenCL on your GPUs instead, you can use the FFI_USE_OPENCL=1 flag when building from source. On the other hand, if you want to disable GPUs altogether, you can use the FFI_USE_GPU=0 environment variable when building from source.
  • Storage Providers: The lotus-miner sectors extend command has been refactored to the functionality of lotus-miner sectors renew.
  • Exchanges/Node operators/RPC-providers:: Execution traces (returned from lotus state exec-trace, lotus state replay, etc.), has changed to account for changes introduced by the by the FVM. Please make sure to read the Execution trace format change section carefully, as these are interface breaking changes
  • Syncing issues: If you have been struggling with syncing issues in normal operations you can try to adjust the amount of threads used for more concurrent FMV execution through via the LOTUS_FVM_CONCURRENCY enviroment variable. It is set to 4 threads by default. Recommended formula for concurrency == YOUR_RAM/4 , but max during a network upgrade is 24. If you are a Storage Provider and are pushing many messages within a short period of time, exporting LOTUS_SKIP_APPLY_TS_MESSAGE_CALL_WITH_GAS=1 will also help with keeping in sync.
  • Catching up from a Snapshot: Users have noticed that catching up sync from a snapshot is taking a lot longer these day. This is largely related to the built-in market actor consuming a lot of computational demand for block validation. A FIP for a short-term mitigation for this is currently in Last Call and will be included network version 19 upgrade if accepted. You can read the FIP here.

Highlights

Execution Trace Format Changes

Execution traces (returned from lotus state exec-trace, lotus state replay, etc.), has changed to account for changes introduced by the FVM. Specifically:

  • The Msg field no longer matches the Filecoin message format as many of the message fields didn't make sense in on-chain sub-calls. Instead, it now has the fields To, From, Value, Method, Params, and ParamsCodec where ParamsCodec is a new field indicating the IPLD codec of the parameters.
    • Importantly, the Msg.CID field has been removed. This field is still present in top-level invocation results, just not inside the execution trace itself.
  • The MsgRct field no longer includes a GasUsed field and now has a ReturnCodec field to indicating the IPLD codec of the return value.
  • The Error and Duration fields have been removed as these are not set by the FVM. The top-level message "invocation result" retains the Error and Duration fields, they've only been removed from the trace itself.
  • Gas Charges no longer include "virtual" gas fields (those starting with v...) or source location information (loc) as neither field is set by the FVM.

A note on "codecs": FVM parameters and return values are IPLD blocks where the "codec" specifies the data encoding. The codec will generally be one of:

  • 0x51, 0x71 - CBOR or DagCBOR. You should generally treat these as equivalent.
  • 0x55 - Raw bytes.
  • 0x00 - Nothing. If the codec is 0x00, the parameter and/or return value should be empty and should be treated as "void" (not specified).
{
  "Msg": {
    "Version": 0,
    "To": "f01234",
    "From": "f04321",
    "Nonce": 1,
    "Value": "0",
    "GasLimit": 0,
    "GasFeeCap": "1234",
    "GasPremium": "1234",
    "Method": 42,
    "Params": "<base64-data-or-null>",
    "CID": {
        "/": "bafyxyz....."
    },
  },
  "MsgRct": {
    "ExitCode": 0,
    "Return": "<base64-data-or-null>",
    "GasUsed": 12345,
  },
  "Error": "",
  "Duration": 568191845,
  "GasCharges": [
    {
      "Name": "OnMethodInvocation",
      "loc": null,
      "tg": 23856,
      "cg": 23856,
      "sg": 0,
      "vtg": 0,
      "vcg": 0,
      "vsg": 0,
      "tt": 0
    },
    {
      "Name": "wasm_exec",
      "loc": null,
      "tg": 1764,
      "cg": 1764,
      "sg": 0,
      "vtg": 0,
      "vcg": 0,
      "vsg": 0,
      "tt": 0
    },
    {
      "Name": "OnSyscall",
      "loc": null,
      "tg": 14000,
      "cg": 14000,
      "sg": 0,
      "vtg": 0,
      "vcg": 0,
      "vsg": 0,
      "tt": 0
    },
  ],
  "Subcalls": [
    {
      "Msg": { },
      "MsgRct": { },
      "Error": "",
      "Duration": 1235,
      "GasCharges": [],
      "Subcalls": [],
    },
  ]
}

{
  "Msg": {
    "To": "f01234",
    "From": "f04321",
    "Value": "0",
    "Method": 42,
    "Params": "<base64-data-or-null>",
    "ParamsCodec": 81
  },
  "MsgRct": {
    "ExitCode": 0,
    "Return": "<base64-data-or-null>",
    "ReturnCodec": 81
  },
  "GasCharges": [
    {
      "Name": "OnMethodInvocation",
      "loc": null,
      "tg": 23856,
      "cg": 23856,
      "tt": 0
    },
    {
      "Name": "wasm_exec",
      "loc": null,
      "tg": 1764,
      "cg": 1764,
      "sg": 0,
      "tt": 0
    },
    {
      "Name": "OnSyscall",
      "loc": null,
      "tg": 14000,
      "cg": 14000,
      "sg": 0,
      "tt": 0
    },
  ],
  "Subcalls": [
    {
      "Msg": { },
      "MsgRct": { },
      "GasCharges": [],
      "Subcalls": [],
    },
  ]
}

SplitStore

This feature release introduces numerous improvements and fixes to tackle SplitStore related issues that has been reported. With this feature release SplitStore is automatically activated by default on new nodes. However, for existing Lotus users, you need to explicitly configure SplitStore by uncommenting the EnableSplitstore option in your config.toml file. To enable SplitStore, set EnableSplitstore=true, and to disable it, set EnableSplitstore=false. It's important to note that your Lotus node will not start unless this configuration is properly set. Set it to false if you are running a full archival node!

SplitStore also has some new configuration settings that you can set in your config.toml file:

  • HotstoreMaxSpaceTarget suggests the max allowed space (in bytes) the hotstore can take.
  • HotstoreMaxSpaceThreshold a moving GC will be triggered when total moving size exceeds this threshold (in bytes).
  • HotstoreMaxSpaceSafetyBuffer a safety buffer to prevent moving GC from an overflowing disk.

The SplitStore also has two new commands:

  • lotus chain prune hot is a much less resource-intensive GC and is best suited for situations where you don't have the spare disk space for a full GC.
  • lotus chain prune hot-moving will run a full moving garbage collection of the hotstore. This commands create a new hotstore before deleting the old one so you need working room in the hotstore directory. The current size of a fully GC'd hotstore is around 295 GiB so you need to make sure you have at least that available.

You can read more about the new SplitStore commands in the documentation.

RPC API improvements

This feature release includes all the RPC API improvements made in the Lotus v1.20.x patch releases. It includes an updated FFI that sets the FVM parallelism to 4 by default.

Node operators with higher memory specs can experiment with setting LOTUS_FVM_CONCURRENCY to higher values, up to 48, to allow for more concurrent FVM execution.

Experimental scheduler assigners

In this release there are four new expirmental scheduler assigners:

  • The experiment-spread-qcount - similar to the spread assigner but also takes into account task counts which are in running/preparing/queued states.
  • The experiment-spread-tasks - similar to the spread assigner, but counts running tasks on a per-task-type basis
  • The experiment-spread-tasks-qcount - similar to the spread assigner, but also takes into account task counts which are in running/preparing/queued states, as well as counting running tasks on a per-task-type basis. Check the results for this assigner on (storage-only lotus-workers here).
  • The experiment-random - In each schedule loop the assinger figures a set of all workers which can handle the task and then picks a random one. Check the results for this assigner on (storage-only lotus-workers here).

Graceful shutdown of lotus-workers
We have cleaned up some commands in the lotus-worker to make it less confusing how to gracefully shutting down a lotus-worker while there are incoming sealing tasks in the pipeline. To shut down a lotus-worker gracefully:

  1. lotus-worker tasks disable --all and wait for the worker to finish processing its current tasks.
  2. lotus-worker stop to detach it and do maintenance/upgrades.

CLI speedups

The lotus-miner sector list is now running in parallel - which should speed up the process from anywhere between 2x-10x+. You can tune it additionally with the check-parallelism option in the command. The Lotus-Miner info command also has a large speed improvement, as calls to the lotus legacy market has been removed.

New features

  • feat: splitstore: Pause compaction when out of sync (filecoin-project/lotus/#10641)
    • Pause the SplitStore compaction if the node is out of sync. Resumes the compation when its back in sync.
  • feat: splitstore: limit moving gc threads (#10621) (filecoin-project/lotus/#10621)
    • Makes moving gc less likely to cause node falling out of sync.
  • feat: splitstore: Update config default value (#10605) (filecoin-project/lotus/#10605)
    • Sets Splitstore HotStoreMaxSpaceTarget config to 650GB as default
  • feat: splitstore: Splitstore enabled by default (#10429) (filecoin-project/lotus#10429)
    • Enables SplitStore by default on new Lotus nodes. Existing Lotus users need to explicitly configure
  • feat: splitstore: Configure max space used by hotstore and GC makes best effort to respect (filecoin-project/lotus#10391)
    • Adds three new configs for setting the maximum allowed space the hotstore can take.
  • feat: splitstore: Badger GC of hotstore command (filecoin-project/lotus#10387)
    • Adds a lotus chain prune hot command, to run the garbage collection of the hotstore in a user driven way.
  • feat: sched: Assigner experiments (filecoin-project/lotus#10356)
    • Introduces experimental scheduler assigners that works better for setups that uses storage-only lotus-workers.
  • fix: wdpost: disabled post worker handling (filecoin-project/lotus#10394)
    • Improved scheduling logic for Proof-of-SpaceTime workers.
  • feat: cli: list claims and remove expired claims (filecoin-project/lotus#9875)
    • Adds a command to list claims made by a provider lotus filplus list-claims. And lotus filplus remove-expired-claims to remove expired claims.
  • feat: cli: make sectors list much faster (filecoin-project/lotus#10202)
    • Makes lotus-miner sector list checks run in parallel.
  • feat: cli: Add an EVM command to fetch a contract's bytecode (filecoin-project/lotus#10443)
    • Adds an lotus evm bytecode command to fetch a contract's bytecode.
  • feat: mempool: Reduce minimum replace fee from 1.25x to 1.1x (#10416) (filecoin-project/lotus#10416)
    • Reduces replacement message fee logic to help include update message replacements from developers using Ethereum tools like MetaMask.
  • feat: update renew-sectors with FIP-0045 logic (filecoin-project/lotus#10328)
    • Updates the lotus-miner sectors extend with FIP-0045 logic to include the ability to drop claims and set the maximum number of messages contained in a message.
  • feat: IPC: Abstract common consensus functions and consensus interface (filecoin-project/lotus#9481)
    • Add eudico's consensus interface to Lotus and implement EC behind that interface. This abstraction is the stepping-stone for Mir's integration.
  • fix: worker: add all tasks flag (filecoin-project/lotus#10232)
    • Adds an all flag for the lotus-worker tasks enable/disable cmds.
  • feat:shed:add cid to cbor serialization command (filecoin-project/lotus#10032)
    • Adds two lotus-shed commands, lotus-shed cid bytes and lotus-shed cid cbor to serialize cid to cbor and cid to bytes.
  • feat: add toolshed commands to inspect statetree size (filecoin-project/lotus#9982)
    • Adds two commands, lotus-shed stat-actor and lotus-shed stat-obj that work with an offline lotus repo to report dag size stats.
  • feat: shed: encode address to bytes (filecoin-project/lotus#10105)
    • Adds a lotus-shed address encode for encoding a filecoin address to hex bytes.
  • feat: chain: export-range (filecoin-project/lotus#10145)
    • Adds a lotus chain export-range command that can create archival-grade ranged exports of the chain as quickly as possible.
  • feat: stmgr: cache migrated stateroots (filecoin-project/lotus#10282)
    • Cache network migration results to avoid running migrations twice.
  • feat: shed: Add a tool to read data from sectors (filecoin-project/lotus#10169)
    • Adds a lotus-shed sectors read command that extract data from sectors from a running lotus-miner deployment.
  • feat: cli: Refactor renew and remove extend (filecoin-project/lotus#9920)
    • Refactors the lotus-miner sectors extend command to have the functionality of lotus-miner sectors renew. The lotus-miner sectors renew command has been deprecated.
  • feat: shed: Add beneficiary commands (filecoin-project/lotus#10037)
    • Adds the beneficiary address command to lotus-shed. You can now use lotus-shed actor propose-change-beneficiary and lotus-shed actor confirm-change-beneficiary to change beneficiary addresses.

Improvements

Dependencies

Others

Contributors

Contributor Commits Lines ± Files Changed
Hannah Howard 2 +2909/-6026 84
Łukasz Magiera 42 +2967/-1848 95
Steven Allen 20 +1703/-1345 88
Alfonso de la Rocha 17 +823/-1808 86
Peter Rabbitson 9 +1957/-219 34
Geoff Stuart 12 +818/-848 29
hannahhoward 5 +507/-718 36
Hector Sanjuan 6 +443/-726 35
Kevin Li 1 +1124/-14 22
zenground0 30 +791/-269 88
frrist 1 +992/-16 13
Travis Person 4 +837/-53 24
Phi 20 +622/-254 34
Ian Davis 7 +35/-729 20
Aayush 10 +378/-177 40
Raúl Kripalani 15 +207/-138 19
Arsenii Petrovich 7 +248/-94 30
ZenGround0 5 +238/-39 15
Neel Virdy 1 +109/-107 58
ychiao 1 +135/-39 3
Jorropo 2 +87/-82 67
Marten Seemann 8 +69/-64 17
Rod Vagg 1 +55/-16 3
Masih H. Derkani 3 +39/-27 12
raulk 2 +30/-29 5
dependabot[bot] 4 +37/-17 8
beck 2 +38/-2 2
Jennifer Wang 4 +20/-19 19
Richard Guan 3 +28/-8 5
omahs 7 +14/-14 7
dirkmc 2 +19/-7 6
David Choi 2 +16/-5 2
Mike Greenberg 1 +18/-1 1
Adin Schmahmann 1 +19/-0 2
Phi-rjan 5 +12/-4 5
Dirk McCormick 2 +6/-6 3
Aayush Rajasekaran 2 +9/-3 2
Jiaying Wang 5 +6/-4 5
Anjor Kanekar 1 +5/-5 1
vyzo 1 +3/-3 2
0x5459 1 +1/-1 1
lotus - v1.22.1

Published by arajasek over 1 year ago

v1.22.1 / 2023-04-23

Important Notice

This is a MANDATORY hotfix release that fixes a consensus-critical bug that was in v1.22.0 -- the necessary fix is https://github.com/filecoin-project/ref-fvm/pull/1750 and it is integrated into lotus via https://github.com/filecoin-project/lotus/pull/10735.
You can NOT use 1.22.0 for the nv19 upgrade, you MUST be on 1.22.1 or higher.

About This Release

This is the stable release of Lotus v1.22.1 for the upcoming MANDATORY network upgrade at 2023-04-27T13:00:00Z, epoch 2809800. This release delivers the nv19 Lighting and nv20 Thunder network upgrade for mainnet.

Note that you must be on a go version higher than Go 1.18.8, but lower than Go v1.20.0. We would recommend Go 1.19.7.

The Lighting and Thunder upgrade introduces the following Filecoin Improvement Proposals (FIPs), delivered by builtin-actors v11 (see actors v11.0.0):

  • FIP 0060 - Thirty day market deal maintenance interval
  • FIP 0061 - WindowPoSt grindability fix
  • FIP 0062 - Fallback method handler for multisig actor

Expedited nv19 Lightning ⚡️ rollout

In light of the recent degraded chain quality on the mainnet an expedited nv19 upgrade has been proposed and accepted to roll out the market cron mitigation (FIP0060) that will improve block validation times, and with that the delay in block production that is causing a decrease in the chain quality currently.

With this expedited roll out we want to inform you of some key changes and important dates:

The Lighting and Thunder upgrade now implements the following Filecoin Improvement Proposals (FIPs), delivered by builtin-actors v11 (see actors v11.0.0):

  • FIP 0060 - Thirty day market deal maintenance interval
  • FIP 0061 - WindowPoSt grindability fix
  • FIP 0062 - Fallback method handler for multisig actor

v11 Builtin Actor Bundles

Make sure that your lotus actor bundle matches the v11 actors manifest by running after upgrading:

lotus state actor-cids --network-version 19
Network Version: 19
Actor Version: 11
Manifest CID: bafy2bzacecnhaiwcrpyjvzl4uv4q3jzoif26okl3m66q3cijp3dfwlcxwztwo

Actor             CID  
datacap           bafk2bzacebslykoyrb2hm7aacjngqgd5n2wmeii2goadrs5zaya3pvdf6pdnq
init              bafk2bzaceckwf3w6n2nw6eh77ktmsxqgsvshonvgnyk5q5syyngtetxvasfxg
reward            bafk2bzacebwjw2vxkobs7r2kwjdqqb42h2kucyuk6flbnyzw4odg5s4mogamo
cron              bafk2bzacebpewdvvgt6tk2o2u4rcovdgym67tadiis5usemlbejg7k3kt567o
ethaccount        bafk2bzaceclkmc4yidxc6lgcjpfypbde2eddnevcveo4j5kmh4ek6inqysz2k
evm               bafk2bzacediwh6etwzwmb5pivtclpdplewdjzphouwqpppce6opisjv2fjqfe
storagemarket     bafk2bzaceazu2j2zu4p24tr22btnqzkhzjvyjltlvsagaj6w3syevikeb5d7m
storagepower      bafk2bzaceaxgloxuzg35vu7l7tohdgaq2frsfp4ejmuo7tkoxjp5zqrze6sf4
system            bafk2bzaced7npe5mt5nh72jxr2igi2sofoa7gedt4w6kueeke7i3xxugqpjfm
account           bafk2bzacealnlr7st6lkwoh6wxpf2hnrlex5sknaopgmkr2tuhg7vmbfy45so
placeholder       bafk2bzacedfvut2myeleyq67fljcrw4kkmn5pb5dpyozovj7jpoez5irnc3ro
eam               bafk2bzaceaelwt4yfsfvsu3pa3miwalsvy3cfkcjvmt4sqoeopsppnrmj2mf2
multisig          bafk2bzaceafajceqwg5ybiz7xw6rxammuirkgtuv625gzaehsqfprm4bazjmk
paymentchannel    bafk2bzaceb4e6cnsnviegmqvsmoxzncruvhra54piq7bwiqfqevle6oob2gvo
storageminer      bafk2bzacec24okjqrp7c7rj3hbrs5ez5apvwah2ruka6haesgfngf37mhk6us
verifiedregistry  bafk2bzacedej3dnr62g2je2abmyjg3xqv4otvh6e26du5fcrhvw7zgcaaez3a

Changelog

Dependencies

github.com/filecoin-project/go-state-types (v0.11.0-rc1 -> v0.11.1):

lotus - v1.22.0-rc6

Published by ZenGround0 over 1 year ago

This is the sixth release candidate for the upcoming MANDATORY 1.22.0 release of Lotus. This release will deliver the nv19 Lighting and nv20 Thunder network upgrade.

Note that this release candidate sets the calibration upgrade epoch, and does NOT set the epoch at which mainnet will upgrade; that detail will be finalized in the 1.22.0 release. Note that go version must be 1.19 to build this release.

The Lighting and Thunder upgrade introduces the following Filecoin Improvement Proposals (FIPs), delivered by builtin-actors v11 (see actors v11.0.0):

  • FIP 0060 - Thirty day market deal maintenance interval
  • FIP 0061 - WindowPoSt grindability fix
  • FIP 0062 - Fallback method handler for multisig actor

Lighting and Thunder

As you may have noticed, that we are doing a two-stage incremental network upgrades in this release. This essentially means that there will be two network versions rolled out together -- nv19 and nv20.

The two stage roll out is required for FIP-0061 - which introduces a new proof that reduces the grindability of windowPoSt and furthur secures the network. At the first upgrade, the new proof type will start to be accepted by the protocol, while the second upgrade (nv20) marks the spot when the old proof type will no longer be accepted. This allows for a smooth rollover period during which both proof types are accepted. Lotus will start generating the new proof types immediately after the nv19 upgrade.

This is something we've safely done before. The second upgrade is something of a "ghost" upgrade -- no migration runs, and no code changes, except that clients will start reporting the new network version of nv20 to the FVM.

Expedited nv19 rollout

In light of the recent degraded chain quality on the mainnet an expedited nv19 upgrade has been proposed and accepted to roll out the market cron mitigation (FIP0060) that will improve block validation times, and with that the delay in block production that is causing a decrease in the chain quality currently.

With this expedited roll out we want to inform you of some key changes and important dates:

The Lighting and Thunder upgrade now implements the following Filecoin Improvement Proposals (FIPs), delivered by builtin-actors v11 (see actors v11.0.0-rc.2):

  • FIP 0060 - Thirty day market deal maintenance interval
  • FIP 0061 - WindowPoSt grindability fix
  • FIP 0062 - Fallback method handler for multisig actor

Calibration nv19 Lighting and nv20 Thunder Upgrade

This release candidate sets:

  • The calibration nv19 Lighting upgrade at epoch 489094, 2023-04-20T14:00:00Z
  • The calibration nv20 Thunder upgrade at epch 492214, 2023-04-21T16:00:00Z

v11 Builtin Actor Bundles

You may also check that your lotus actor bundle matches the v11 actors manifest by running

./lotus state  actor-cids --network-version 19                  Tue Apr 18 11:07:40 2023
Network Version: 19
Actor Version: 11
Manifest CID: bafy2bzaceajj76ms4q2ka3ckzj2iiifl7ewxqdpolshcmhml32sqwolrmsh6e

Actor             CID  
eam               bafk2bzacebq2zzzkcra7ahantiq7xlkgcrkoq2azmeqpwhoie5gxcxhurbdp2
multisig          bafk2bzaceavnestm7obwlj3moeewcnuomzigfzys4lt26s3l3psjeligf4hju
paymentchannel    bafk2bzaceb4vefsbgswrciqo26wnacvqwpicqjfyicwsdjmv4rbcc4zeae72g
storagepower      bafk2bzacebdoeoiayvshg4zaxygmpuituux3ccd4gxfl3ile7ckpsyy7kpnms
cron              bafk2bzacediqvwudrujh7ejowrcumrnara7k2fonr35vulnggpwwwh3o32pmu
datacap           bafk2bzaceb7jkx5kxwwbkw5ehz63sq2b6urouegn4cdczlziaw5ea44u67ock
account           bafk2bzacecsf7v7mm4osq7cqbygyudg5zkwqbpbkmi3iluakugzvbd2cuucfw
ethaccount        bafk2bzacebrtwz4nyl3nkgjfpvtqjrjixnbujkr72fwzf5bqhpqfo4hgbjqo2
init              bafk2bzacebwfs5wna2dculc4mpl4a25cjrmguoscbebeb5g5yft4ntfpku7cu
placeholder       bafk2bzacedfvut2myeleyq67fljcrw4kkmn5pb5dpyozovj7jpoez5irnc3ro
storagemarket     bafk2bzacechvhqru32tw7zii2raii3upjrkybiubepjtartyhzibnac57gxyo
storageminer      bafk2bzacedu6tzfpxwzqym3z2j252d57gqj3jduap7ijf5mczvhmsirs6wxfs
verifiedregistry  bafk2bzacecjshlgoyim3r2uvhtjthu5udxpq5fvjy4pmpi3sy4v6a4chjvb4k
evm               bafk2bzacecoxcwxbukue76b4rrqhnjywgzjdlhbbemma4kqdzalpkgtdpog7q
reward            bafk2bzacedyl4uerjbw3tu5mcgkn7ftvr3j3poy4uv5tuzyuvcdinjb75cz3w
system            bafk2bzaceatn5cj3jqmrug65ur26jhrzcgmishv5hjfswvsyxsieccchhwies

Changelog

  • feat: upgrade: expedite nv19 filecoin-project/lotus#10681
    • Update changelog build version (commit: 67d419e)
    • Update actors v11 (commit: 5df4f75)
    • Correct epoch to match specified date (commit: a28fcea)
    • Fast butterfly migration to validate migration (commit: 37a0dca)
    • Make docsgen (commit: daba4ff)
    • Update go-state-types (commit: 244ca0b)
    • Revert FIP0052 (commit: 68ed494)
    • Modify upgrade schedule and params (commit: fa0dfdf)
    • Update go-state-types (commit: 19ae05f)

Dependencies

github.com/filecoin-project/go-state-types (v0.11.0-rc1 -> v0.11.0-rc2):

lotus - v1.23.0-rc1

Published by ZenGround0 over 1 year ago

This is the first release candidate of the upcoming optional but highly recommended feature release of Lotus. It includes numerous improvements and enhancements for node operators, ETH RPC-providers and storage providers.

Note: this release is bumped from the v1.21.0-rcx plus all the changes from v1.22.0 that supports the upcoming nv19 upgrade, scheduled on Apr 27th. You can find more details about the upgrade here

☢️ Upgrade Warnings ☢️

Before upgrading to this feature release read carefully through these bullet points:

  • Starting from this release, the SplitStore feature is automatically activated on new nodes. However, for existing Lotus users, you need to explicitly configure SplitStore by uncommenting the EnableSplitstore option in your config.toml file. To enable SplitStore, set EnableSplitstore=true, and to disable it, set EnableSplitstore=false. It's important to note that your Lotus node will not start unless this configuration is properly set. Set it to false if you are running a full archival node!
  • This feature release requires a minimum Go version of v1.19.7 or higher to successfully build Lotus. Additionally, Go version v1.20 and higher is now also supported.
  • Storage Providers: The proofs libraries now have CUDA enabled by default, which requires you to install (CUDA)[https://lotus.filecoin.io/tutorials/lotus-miner/cuda/] if you haven't already done so. If you prefer to use OpenCL on your GPUs instead, you can use the FFI_USE_OPENCL=1 flag when building from source. On the other hand, if you want to disable GPUs altogether, you can use the FFI_NO_GPU=1 environment variable when building from source.
  • Storage Providers: The lotus-miner sectors extend command has been refactored to the functionality of lotus-miner sectors renew.
  • Exchanges/Node operators/RPC-providers:: Execution traces (returned from lotus state exec-trace, lotus state replay, etc.), has changed to account for changes introduced by the by the FVM. Please make sure to read the Execution trace format change section carefully, as these are interface breaking changes
  • Syncing issues: If you have been struggling with syncing issues in normal operations you can try to adjust the amount of threads used for more concurrent FMV execution through via the LOTUS_FVM_CONCURRENCY enviroment variable. It is set to 4 threads by default. Recommended formula for concurrency == YOUR_RAM/4 , but max is 128. If you are a Storage Provider and are pushing many messages within a short period of time, exporting LOTUS_SKIP_APPLY_TS_MESSAGE_CALL_WITH_GAS=1 will also help with keeping in sync.
  • Catching up from a Snapshot: Users have noticed that catching up sync from a snapshot is taking a lot longer these day. This is largely related to the built-in market actor consuming a lot of computational demand for block validation. A FIP for a short-term mitigation for this is currently in Last Call and will be included network version 19 upgrade if accepted. You can read the FIP here.

Highlights

Execution Trace Format Changes

Execution traces (returned from lotus state exec-trace, lotus state replay, etc.), has changed to account for changes introduced by the FVM. Specifically:

  • The Msg field no longer matches the Filecoin message format as many of the message fields didn't make sense in on-chain sub-calls. Instead, it now has the fields To, From, Value, Method, Params, and ParamsCodec where ParamsCodec is a new field indicating the IPLD codec of the parameters.
    • Importantly, the Msg.CID field has been removed. This field is still present in top-level invocation results, just not inside the execution trace itself.
  • The MsgRct field no longer includes a GasUsed field and now has a ReturnCodec field to indicating the IPLD codec of the return value.
  • The Error and Duration fields have been removed as these are not set by the FVM. The top-level message "invocation result" retains the Error and Duration fields, they've only been removed from the trace itself.
  • Gas Charges no longer include "virtual" gas fields (those starting with v...) or source location information (loc) as neither field is set by the FVM.

A note on "codecs": FVM parameters and return values are IPLD blocks where the "codec" specifies the data encoding. The codec will generally be one of:

  • 0x51, 0x71 - CBOR or DagCBOR. You should generally treat these as equivalent.
  • 0x55 - Raw bytes.
  • 0x00 - Nothing. If the codec is 0x00, the parameter and/or return value should be empty and should be treated as "void" (not specified).
{
  "Msg": {
    "Version": 0,
    "To": "f01234",
    "From": "f04321",
    "Nonce": 1,
    "Value": "0",
    "GasLimit": 0,
    "GasFeeCap": "1234",
    "GasPremium": "1234",
    "Method": 42,
    "Params": "<base64-data-or-null>",
    "CID": {
        "/": "bafyxyz....."
    },
  },
  "MsgRct": {
    "ExitCode": 0,
    "Return": "<base64-data-or-null>",
    "GasUsed": 12345,
  },
  "Error": "",
  "Duration": 568191845,
  "GasCharges": [
    {
      "Name": "OnMethodInvocation",
      "loc": null,
      "tg": 23856,
      "cg": 23856,
      "sg": 0,
      "vtg": 0,
      "vcg": 0,
      "vsg": 0,
      "tt": 0
    },
    {
      "Name": "wasm_exec",
      "loc": null,
      "tg": 1764,
      "cg": 1764,
      "sg": 0,
      "vtg": 0,
      "vcg": 0,
      "vsg": 0,
      "tt": 0
    },
    {
      "Name": "OnSyscall",
      "loc": null,
      "tg": 14000,
      "cg": 14000,
      "sg": 0,
      "vtg": 0,
      "vcg": 0,
      "vsg": 0,
      "tt": 0
    },
  ],
  "Subcalls": [
    {
      "Msg": { },
      "MsgRct": { },
      "Error": "",
      "Duration": 1235,
      "GasCharges": [],
      "Subcalls": [],
    },
  ]
}

{
  "Msg": {
    "To": "f01234",
    "From": "f04321",
    "Value": "0",
    "Method": 42,
    "Params": "<base64-data-or-null>",
    "ParamsCodec": 81
  },
  "MsgRct": {
    "ExitCode": 0,
    "Return": "<base64-data-or-null>",
    "ReturnCodec": 81
  },
  "GasCharges": [
    {
      "Name": "OnMethodInvocation",
      "loc": null,
      "tg": 23856,
      "cg": 23856,
      "tt": 0
    },
    {
      "Name": "wasm_exec",
      "loc": null,
      "tg": 1764,
      "cg": 1764,
      "sg": 0,
      "tt": 0
    },
    {
      "Name": "OnSyscall",
      "loc": null,
      "tg": 14000,
      "cg": 14000,
      "sg": 0,
      "tt": 0
    },
  ],
  "Subcalls": [
    {
      "Msg": { },
      "MsgRct": { },
      "GasCharges": [],
      "Subcalls": [],
    },
  ]
}

SplitStore

This feature release introduces numerous improvements and fixes to tackle SplitStore related issues that has been reported. With this feature release SplitStore is automatically activated by default on new nodes. However, for existing Lotus users, you need to explicitly configure SplitStore by uncommenting the EnableSplitstore option in your config.toml file. To enable SplitStore, set EnableSplitstore=true, and to disable it, set EnableSplitstore=false. It's important to note that your Lotus node will not start unless this configuration is properly set. Set it to false if you are running a full archival node!

SplitStore also has some new configuration settings that you can set in your config.toml file:

  • HotstoreMaxSpaceTarget suggests the max allowed space (in bytes) the hotstore can take.
  • HotstoreMaxSpaceThreshold a moving GC will be triggered when total moving size exceeds this threshold (in bytes).
  • HotstoreMaxSpaceSafetyBuffer a safety buffer to prevent moving GC from an overflowing disk.

The SplitStore also has two new commands:

  • lotus chain prune hot is a much less resource-intensive GC and is best suited for situations where you don't have the spare disk space for a full GC.
  • lotus chain prune hot-moving will run a full moving garbage collection of the hotstore. This commands create a new hotstore before deleting the old one so you need working room in the hotstore directory. The current size of a fully GC'd hotstore is around 295 GiB so you need to make sure you have at least that available.

You can read more about the new SplitStore commands in the documentation.

RPC API improvements

This feature release includes all the RPC API improvements made in the Lotus v1.20.x patch releases. It includes an updated FFI that sets the FVM parallelism to 4 by default.

Node operators with higher memory specs can experiment with setting LOTUS_FVM_CONCURRENCY to higher values, up to 48, to allow for more concurrent FVM execution.

Experimental scheduler assigners

In this release there are four new expirmental scheduler assigners:

  • The experiment-spread-qcount - similar to the spread assigner but also takes into account task counts which are in running/preparing/queued states.
  • The experiment-spread-tasks - similar to the spread assigner, but counts running tasks on a per-task-type basis
  • The experiment-spread-tasks-qcount - similar to the spread assigner, but also takes into account task counts which are in running/preparing/queued states, as well as counting running tasks on a per-task-type basis. Check the results for this assigner on (storage-only lotus-workers here).
  • The experiment-random - In each schedule loop the assinger figures a set of all workers which can handle the task and then picks a random one. Check the results for this assigner on (storage-only lotus-workers here).

Graceful shutdown of lotus-workers
We have cleaned up some commands in the lotus-worker to make it less confusing how to gracefully shutting down a lotus-worker while there are incoming sealing tasks in the pipeline. To shut down a lotus-worker gracefully:

  1. lotus-worker tasks disable --all and wait for the worker to finish processing its current tasks.
  2. lotus-worker stop to detach it and do maintenance/upgrades.

CLI speedups

The lotus-miner sector list is now running in parallel - which should speed up the process from anywhere between 2x-10x+. You can tune it additionally with the check-parallelism option in the command. The Lotus-Miner info command also has a large speed improvement, as calls to the lotus legacy market has been removed.

New features

  • feat: splitstore: Pause compaction when out of sync (filecoin-project/lotus/#10641)
    • Pause the SplitStore compaction if the node is out of sync. Resumes the compation when its back in sync.
  • feat: splitstore: limit moving gc threads (#10621) (filecoin-project/lotus/#10621)
    • Makes moving gc less likely to cause node falling out of sync.
  • feat: splitstore: Update config default value (#10605) (filecoin-project/lotus/#10605)
    • Sets Splitstore HotStoreMaxSpaceTarget config to 650GB as default
  • feat: splitstore: Splitstore enabled by default (#10429) (filecoin-project/lotus#10429)
    • Enables SplitStore by default on new Lotus nodes. Existing Lotus users need to explicitly configure
  • feat: splitstore: Configure max space used by hotstore and GC makes best effort to respect (filecoin-project/lotus#10391)
    • Adds three new configs for setting the maximum allowed space the hotstore can take.
  • feat: splitstore: Badger GC of hotstore command (filecoin-project/lotus#10387)
    • Adds a lotus chain prune hot command, to run the garbage collection of the hotstore in a user driven way.
  • feat: sched: Assigner experiments (filecoin-project/lotus#10356)
    • Introduces experimental scheduler assigners that works better for setups that uses storage-only lotus-workers.
  • fix: wdpost: disabled post worker handling (filecoin-project/lotus#10394)
    • Improved scheduling logic for Proof-of-SpaceTime workers.
  • feat: cli: list claims and remove expired claims (filecoin-project/lotus#9875)
    • Adds a command to list claims made by a provider lotus filplus list-claims. And lotus filplus remove-expired-claims to remove expired claims.
  • feat: cli: make sectors list much faster (filecoin-project/lotus#10202)
    • Makes lotus-miner sector list checks run in parallel.
  • feat: cli: Add an EVM command to fetch a contract's bytecode (filecoin-project/lotus#10443)
    • Adds an lotus evm bytecode command to fetch a contract's bytecode.
  • feat: mempool: Reduce minimum replace fee from 1.25x to 1.1x (#10416) (filecoin-project/lotus#10416)
    • Reduces replacement message fee logic to help include update message replacements from developers using Ethereum tools like MetaMask.
  • feat: update renew-sectors with FIP-0045 logic (filecoin-project/lotus#10328)
    • Updates the lotus-miner sectors extend with FIP-0045 logic to include the ability to drop claims and set the maximum number of messages contained in a message.
  • feat: IPC: Abstract common consensus functions and consensus interface (filecoin-project/lotus#9481)
    • Add eudico's consensus interface to Lotus and implement EC behind that interface. This abstraction is the stepping-stone for Mir's integration.
  • fix: worker: add all tasks flag (filecoin-project/lotus#10232)
    • Adds an all flag for the lotus-worker tasks enable/disable cmds.
  • feat:shed:add cid to cbor serialization command (filecoin-project/lotus#10032)
    • Adds two lotus-shed commands, lotus-shed cid bytes and lotus-shed cid cbor to serialize cid to cbor and cid to bytes.
  • feat: add toolshed commands to inspect statetree size (filecoin-project/lotus#9982)
    • Adds two commands, lotus-shed stat-actor and lotus-shed stat-obj that work with an offline lotus repo to report dag size stats.
  • feat: shed: encode address to bytes (filecoin-project/lotus#10105)
    • Adds a lotus-shed address encode for encoding a filecoin address to hex bytes.
  • feat: chain: export-range (filecoin-project/lotus#10145)
    • Adds a lotus chain export-range command that can create archival-grade ranged exports of the chain as quickly as possible.
  • feat: stmgr: cache migrated stateroots (filecoin-project/lotus#10282)
    • Cache network migration results to avoid running migrations twice.
  • feat: shed: Add a tool to read data from sectors (filecoin-project/lotus#10169)
    • Adds a lotus-shed sectors read command that extract data from sectors from a running lotus-miner deployment.
  • feat: cli: Refactor renew and remove extend (filecoin-project/lotus#9920)
    • Refactors the lotus-miner sectors extend command to have the functionality of lotus-miner sectors renew. The lotus-miner sectors renew command has been deprecated.
  • feat: shed: Add beneficiary commands (filecoin-project/lotus#10037)
    • Adds the beneficiary address command to lotus-shed. You can now use lotus-shed actor propose-change-beneficiary and lotus-shed actor confirm-change-beneficiary to change beneficiary addresses.

Improvements

Dependencies

Others

Contributors

Contributor Commits Lines ± Files Changed
Hannah Howard 2 +2909/-6026 84
Łukasz Magiera 42 +2967/-1848 95
Steven Allen 20 +1703/-1345 88
Alfonso de la Rocha 17 +823/-1808 86
Peter Rabbitson 9 +1957/-219 34
Geoff Stuart 12 +818/-848 29
hannahhoward 5 +507/-718 36
Hector Sanjuan 6 +443/-726 35
Kevin Li 1 +1124/-14 22
zenground0 30 +791/-269 88
frrist 1 +992/-16 13
Travis Person 4 +837/-53 24
Phi 20 +622/-254 34
Ian Davis 7 +35/-729 20
Aayush 10 +378/-177 40
Raúl Kripalani 15 +207/-138 19
Arsenii Petrovich 7 +248/-94 30
ZenGround0 5 +238/-39 15
Neel Virdy 1 +109/-107 58
ychiao 1 +135/-39 3
Jorropo 2 +87/-82 67
Marten Seemann 8 +69/-64 17
Rod Vagg 1 +55/-16 3
Masih H. Derkani 3 +39/-27 12
raulk 2 +30/-29 5
dependabot[bot] 4 +37/-17 8
beck 2 +38/-2 2
Jennifer Wang 4 +20/-19 19
Richard Guan 3 +28/-8 5
omahs 7 +14/-14 7
dirkmc 2 +19/-7 6
David Choi 2 +16/-5 2
Mike Greenberg 1 +18/-1 1
Adin Schmahmann 1 +19/-0 2
Phi-rjan 5 +12/-4 5
Dirk McCormick 2 +6/-6 3
Aayush Rajasekaran 2 +9/-3 2
Jiaying Wang 5 +6/-4 5
Anjor Kanekar 1 +5/-5 1
vyzo 1 +3/-3 2
0x5459 1 +1/-1 1
lotus - v1.22.0-rc5

Published by ZenGround0 over 1 year ago

This is the fifth release candidate for the upcoming MANDATORY 1.22.0 release of Lotus. This release will deliver the nv19 Lighting and nv20 Thunder network upgrade.

Note that this release candidate sets the calibration upgrade epoch, and does NOT set the epoch at which mainnet will upgrade; that detail will be finalized in the 1.22.0 release. Note that go version must be 1.19 to build this release.

The Lighting and Thunder upgrade introduces the following Filecoin Improvement Proposals (FIPs), delivered by builtin-actors v11 (see actors v11.0.0):

  • FIP 0060 - Thirty day market deal maintenance interval
  • FIP 0061 - WindowPoSt grindability fix
  • FIP 0062 - Fallback method handler for multisig actor

Lighting and Thunder

As you may have noticed, that we are doing a two-stage incremental network upgrades in this release. This essentially means that there will be two network versions rolled out together -- nv19 and nv20.

The two stage roll out is required for FIP-0061 - which introduces a new proof that reduces the grindability of windowPoSt and furthur secures the network. At the first upgrade, the new proof type will start to be accepted by the protocol, while the second upgrade (nv20) marks the spot when the old proof type will no longer be accepted. This allows for a smooth rollover period during which both proof types are accepted. Lotus will start generating the new proof types immediately after the nv19 upgrade.

This is something we've safely done before. The second upgrade is something of a "ghost" upgrade -- no migration runs, and no code changes, except that clients will start reporting the new network version of nv20 to the FVM.

Expedited nv19 rollout

In light of the recent degraded chain quality on the mainnet an expedited nv19 upgrade has been proposed and accepted to roll out the market cron mitigation (FIP0060) that will improve block validation times, and with that the delay in block production that is causing a decrease in the chain quality currently.

With this expedited roll out we want to inform you of some key changes and important dates:

The Lighting and Thunder upgrade now implements the following Filecoin Improvement Proposals (FIPs), delivered by builtin-actors v11 (see actors v11.0.0-rc.2):

  • FIP 0060 - Thirty day market deal maintenance interval
  • FIP 0061 - WindowPoSt grindability fix
  • FIP 0062 - Fallback method handler for multisig actor

Calibration nv19 Lighting and nv20 Thunder Upgrade

This release candidate sets:

  • The calibration nv19 Lighting upgrade at epoch 489094, 2023-04-20T14:00:00Z
  • The calibration nv20 Thunder upgrade at epch 492214, 2023-04-21T16:00:00Z

v11 Builtin Actor Bundles

You may also check that your lotus actor bundle matches the v11 actors manifest by running

./lotus state  actor-cids --network-version 19                  Tue Apr 18 11:07:40 2023
Network Version: 19
Actor Version: 11
Manifest CID: bafy2bzaceajj76ms4q2ka3ckzj2iiifl7ewxqdpolshcmhml32sqwolrmsh6e

Actor             CID  
eam               bafk2bzacebq2zzzkcra7ahantiq7xlkgcrkoq2azmeqpwhoie5gxcxhurbdp2
multisig          bafk2bzaceavnestm7obwlj3moeewcnuomzigfzys4lt26s3l3psjeligf4hju
paymentchannel    bafk2bzaceb4vefsbgswrciqo26wnacvqwpicqjfyicwsdjmv4rbcc4zeae72g
storagepower      bafk2bzacebdoeoiayvshg4zaxygmpuituux3ccd4gxfl3ile7ckpsyy7kpnms
cron              bafk2bzacediqvwudrujh7ejowrcumrnara7k2fonr35vulnggpwwwh3o32pmu
datacap           bafk2bzaceb7jkx5kxwwbkw5ehz63sq2b6urouegn4cdczlziaw5ea44u67ock
account           bafk2bzacecsf7v7mm4osq7cqbygyudg5zkwqbpbkmi3iluakugzvbd2cuucfw
ethaccount        bafk2bzacebrtwz4nyl3nkgjfpvtqjrjixnbujkr72fwzf5bqhpqfo4hgbjqo2
init              bafk2bzacebwfs5wna2dculc4mpl4a25cjrmguoscbebeb5g5yft4ntfpku7cu
placeholder       bafk2bzacedfvut2myeleyq67fljcrw4kkmn5pb5dpyozovj7jpoez5irnc3ro
storagemarket     bafk2bzacechvhqru32tw7zii2raii3upjrkybiubepjtartyhzibnac57gxyo
storageminer      bafk2bzacedu6tzfpxwzqym3z2j252d57gqj3jduap7ijf5mczvhmsirs6wxfs
verifiedregistry  bafk2bzacecjshlgoyim3r2uvhtjthu5udxpq5fvjy4pmpi3sy4v6a4chjvb4k
evm               bafk2bzacecoxcwxbukue76b4rrqhnjywgzjdlhbbemma4kqdzalpkgtdpog7q
reward            bafk2bzacedyl4uerjbw3tu5mcgkn7ftvr3j3poy4uv5tuzyuvcdinjb75cz3w
system            bafk2bzaceatn5cj3jqmrug65ur26jhrzcgmishv5hjfswvsyxsieccchhwies

Changelog

  • feat: upgrade: expedite nv19 filecoin-project/lotus#10681
    • Update changelog build version (commit: 67d419e)
    • Update actors v11 (commit: 5df4f75)
    • Correct epoch to match specified date (commit: a28fcea)
    • Fast butterfly migration to validate migration (commit: 37a0dca)
    • Make docsgen (commit: daba4ff)
    • Update go-state-types (commit: 244ca0b)
    • Revert FIP0052 (commit: 68ed494)
    • Modify upgrade schedule and params (commit: fa0dfdf)
    • Update go-state-types (commit: 19ae05f)

Dependencies

github.com/filecoin-project/go-state-types (v0.11.0-rc1 -> v0.11.0-rc2):

lotus - v1.22.0-rc4

Published by filecoin-helper over 1 year ago

This is the fourth release candidate for the upcoming MANDATORY 1.22.0 release of Lotus. This release will deliver the nv19 Lighting and nv20 Thunder network upgrade.

Note that this release candidate sets the calibration upgrade epoch, and does NOT set the epoch at which mainnet will upgrade; that detail will be finalized in the 1.22.0 release.

The Lighting and Thunder upgrade introduces the following Filecoin Improvement Proposals (FIPs), delivered by builtin-actors v11 (see actors v11.0.0):

  • FIP 0060 - Thirty day market deal maintenance interval
  • FIP 0061 - WindowPoSt grindability fix
  • FIP 0062 - Fallback method handler for multisig actor

Lighting and Thunder

As you may have noticed, that we are doing a two-stage incremental network upgrades in this release. This essentially means that there will be two network versions rolled out together -- nv19 and nv20.

The two stage roll out is required for FIP-0061 - which introduces a new proof that reduces the grindability of windowPoSt and furthur secures the network. At the first upgrade, the new proof type will start to be accepted by the protocol, while the second upgrade (nv20) marks the spot when the old proof type will no longer be accepted. This allows for a smooth rollover period during which both proof types are accepted. Lotus will start generating the new proof types immediately after the nv19 upgrade.

This is something we've safely done before. The second upgrade is something of a "ghost" upgrade -- no migration runs, and no code changes, except that clients will start reporting the new network version of nv20 to the FVM.

Expedited nv19 rollout

In light of the recent degraded chain quality on the mainnet an expedited nv19 upgrade has been proposed and accepted to roll out the market cron mitigation (FIP0060) that will improve block validation times, and with that the delay in block production that is causing a decrease in the chain quality currently.

With this expedited roll out we want to inform you of some key changes and important dates:

The Lighting and Thunder upgrade now implements the following Filecoin Improvement Proposals (FIPs), delivered by builtin-actors v11 (see actors v11.0.0-rc.2):

  • FIP 0060 - Thirty day market deal maintenance interval
  • FIP 0061 - WindowPoSt grindability fix
  • FIP 0062 - Fallback method handler for multisig actor

Calibration nv19 Lighting and nv20 Thunder Upgrade

This release candidate sets:

  • The calibration nv19 Lighting upgrade at epoch 489094, 2023-04-20T14:00:00Z
  • The calibration nv20 Thunder upgrade at epch 492214, 2023-04-21T16:00:00Z

v11 Builtin Actor Bundles

You may also check that your lotus actor bundle matches the v11 actors manifest by running

./lotus state  actor-cids --network-version 19                  Tue Apr 18 11:07:40 2023
Network Version: 19
Actor Version: 11
Manifest CID: bafy2bzaceajj76ms4q2ka3ckzj2iiifl7ewxqdpolshcmhml32sqwolrmsh6e

Actor             CID  
eam               bafk2bzacebq2zzzkcra7ahantiq7xlkgcrkoq2azmeqpwhoie5gxcxhurbdp2
multisig          bafk2bzaceavnestm7obwlj3moeewcnuomzigfzys4lt26s3l3psjeligf4hju
paymentchannel    bafk2bzaceb4vefsbgswrciqo26wnacvqwpicqjfyicwsdjmv4rbcc4zeae72g
storagepower      bafk2bzacebdoeoiayvshg4zaxygmpuituux3ccd4gxfl3ile7ckpsyy7kpnms
cron              bafk2bzacediqvwudrujh7ejowrcumrnara7k2fonr35vulnggpwwwh3o32pmu
datacap           bafk2bzaceb7jkx5kxwwbkw5ehz63sq2b6urouegn4cdczlziaw5ea44u67ock
account           bafk2bzacecsf7v7mm4osq7cqbygyudg5zkwqbpbkmi3iluakugzvbd2cuucfw
ethaccount        bafk2bzacebrtwz4nyl3nkgjfpvtqjrjixnbujkr72fwzf5bqhpqfo4hgbjqo2
init              bafk2bzacebwfs5wna2dculc4mpl4a25cjrmguoscbebeb5g5yft4ntfpku7cu
placeholder       bafk2bzacedfvut2myeleyq67fljcrw4kkmn5pb5dpyozovj7jpoez5irnc3ro
storagemarket     bafk2bzacechvhqru32tw7zii2raii3upjrkybiubepjtartyhzibnac57gxyo
storageminer      bafk2bzacedu6tzfpxwzqym3z2j252d57gqj3jduap7ijf5mczvhmsirs6wxfs
verifiedregistry  bafk2bzacecjshlgoyim3r2uvhtjthu5udxpq5fvjy4pmpi3sy4v6a4chjvb4k
evm               bafk2bzacecoxcwxbukue76b4rrqhnjywgzjdlhbbemma4kqdzalpkgtdpog7q
reward            bafk2bzacedyl4uerjbw3tu5mcgkn7ftvr3j3poy4uv5tuzyuvcdinjb75cz3w
system            bafk2bzaceatn5cj3jqmrug65ur26jhrzcgmishv5hjfswvsyxsieccchhwies

Changelog

  • 67d419e16 Update changelog build version
  • 5df4f75dc Update actors v11
  • a28fceaa5 Correct epoch to match specified date
  • 37a0dca11 fast butterfly migration to validate migration
  • daba4ff5f docsgen
  • 244ca0b5f Update gst
  • 68ed494a6 Revert fip 0052
  • fa0dfdfd9 Modify upgrade schedule and params
  • 19ae05f3b Update go-state-types
lotus - v1.22.0-rc3

Published by jennijuju over 1 year ago

This is the third release candidate for the upcoming MANDATORY 1.22.0 release of Lotus. This release will deliver the nv19 Lighting and nv20 Thunder network upgrade.

Note that this release candidate sets the calibration upgrade epoch, and does NOT set the epoch at which mainnet will upgrade; that detail will be finalized in the 1.22.0 release.

The Lighting and Thunder upgrade introduces the following Filecoin Improvement Proposals (FIPs), delivered by builtin-actors v11 (see actors v11.0.0-rc.1):

  • FIP 0060 - Thirty day market deal maintenance interval
  • FIP 0061 - WindowPoSt grindability fix
  • FIP 0062 - Fallback method handler for multisig actor
  • FIP 0052 - Deals and sectors can be created and extended in 3.5 year intervals (+2 years from current params)
  • Activation bug fix - internal refactor of sector info fields fixing several outstanding bugs

Lighting and Thunder

As you may have noticed, that we are doing a two-stage incremental network upgrades in this release. This essentially means that there will be two network versions rolled out together -- nv19 and nv20.
The two stage roll out is required for FIP-0061 - which introduces a new proof that reduces the grindability of windowPoSt and furthur secures the network. At the first upgrade, the new proof type will start to be accepted by the protocol, while the second upgrade (nv20) marks the spot when the old proof type will no longer be accepted. This allows for a smooth rollover period during which both proof types are accepted. Lotus will start generating the new proof types immediately after the nv19 upgrade.
This is something we've safely done before. The second upgrade is something of a "ghost" upgrade -- no migration runs, and no code changes, except that clients will start reporting the new network version of nv20 to the FVM.

Calibration nv19 Lighting and nv20 Thunder Upgrade

This release candidate sets the calibration-net nv19 Lighting upgrade at epoch 489394, 2023-04-20T16:30:00Z and nv20 Thunder upgrade will be triggered automatically 11520 epoch later. The bundle the network will be using is v10.0.0 actors
(located at build/actors/v11.tar.zst) upon/post migration, manifest CID bafy2bzacedyne7vbddp2inj64ubztcvkmfkdnahwo353sltkqtsyzckioneuu.

lotus - v1.22.0-rc2

Published by ZenGround0 over 1 year ago

Release Candidate nv19 and nv20 upgrades
diff from rc1: small devnet fixes and set calibration epoch

nv19
FIP 0060 - Thirty day market deal maintenance interval
FIP 0061 - WindowPoSt grindability gix
FIP 0062 - Fallback method handler for multisig actor
FIP 0052 - Deals and sectors can be created and extended in 3.5 year intervals (+2 years from current params)
Activation bug fix - internal refactor of sector info fields fixing several outstanding bugs

nv20
Final part of FIP 0061 deprecating grindable window post proofs

lotus - v1.22.0-rc1

Published by ZenGround0 over 1 year ago

Release Candidate nv19 and nv20 upgrades

nv19
FIP 0060 - Thirty day market deal maintenance interval
FIP 0061 - WindowPoSt grindability gix
FIP 0062 - Fallback method handler for multisig actor
FIP 0052 - Deals and sectors can be created and extended in 3.5 year intervals (+2 years from current params)
Activation bug fix - internal refactor of sector info fields fixing several outstanding bugs

nv20
Final part of FIP 0061 deprecating grindable window post proofs

lotus - v1.21.0-rc3

Published by rjan90 over 1 year ago

This is an optional but highly recommended feature release of Lotus. It includes numerous improvements and enhancements for node operators, ETH RPC-providers and storage providers.

☢️ Upgrade Warnings ☢️

Before upgrading to this feature release read carefully through these bullet points:

  • Starting from this release, the SplitStore feature is automatically activated on new nodes. However, for existing Lotus users, you need to explicitly configure SplitStore by uncommenting the EnableSplitstore option in your config.toml file. To enable SplitStore, set EnableSplitstore=true, and to disable it, set EnableSplitstore=false. It's important to note that your Lotus node will not start unless this configuration is properly set. Set it to false if you are running a full archival node!
  • This feature release requires a minimum Go version of v1.19.7 or higher to successfully build Lotus. Additionally, Go version v1.20 and higher is now also supported.
  • Storage Providers: The proofs libraries now have CUDA enabled by default, which requires you to install CUDA if you haven't already done so. If you prefer to use OpenCL on your GPUs instead, you can use the FFI_USE_OPENCL=1 flag when building from source. On the other hand, if you want to disable GPUs altogether, you can use the FFI_NO_GPU=1 environment variable when building from source.
  • Storage Providers: The lotus-miner sectors extend command has been refactored to the functionality of lotus-miner sectors renew.
  • Exchanges/Node operators/RPC-providers:: Execution traces (returned from lotus state exec-trace, lotus state replay, etc.), has changed to account for changes introduced by the by the FVM. Please make sure to read the Execution trace format change section carefully, as these are interface breaking changes
  • Syncing issues: If you have been struggling with syncing issues in normal operations you can try to adjust the amount of threads used for more concurrent FMV execution through via the LOTUS_FVM_CONCURRENCY enviroment variable. It is set to 4 threads by default. Recommended formula for concurrency == YOUR_RAM/4 , but max is 48. If you are a Storage Provider and are pushing many messages within a short period of time, exporting LOTUS_SKIP_APPLY_TS_MESSAGE_CALL_WITH_GAS=1 will also help with keeping in sync.
  • Catching up from a Snapshot: Users have noticed that catching up sync from a snapshot is taking a lot longer these day. This is largely related to the built-in market actor consuming a lot of computational demand for block validation. A FIP for a short-term mitigation for this is currently in Last Call and will be included network version 19 upgrade if accepted. You can read the FIP here.

⭐️ Highlights ⭐️

SplitStore

This feature release introduces numerous improvements and fixes to tackle SplitStore related issues that has been reported. With this feature release SplitStore is automatically activated by default on new nodes. However, for existing Lotus users, you need to explicitly configure SplitStore by uncommenting the EnableSplitstore option in your config.toml file. To enable SplitStore, set EnableSplitstore=true, and to disable it, set EnableSplitstore=false. It's important to note that your Lotus node will not start unless this configuration is properly set. Set it to false if you are running a full archival node!

SplitStore also has some new configuration settings that you can set in your config.toml file:

  • HotstoreMaxSpaceTarget suggests the max allowed space (in bytes) the hotstore can take.
  • HotstoreMaxSpaceThreshold a moving GC will be triggered when total moving size exceeds this threshold (in bytes).
  • HotstoreMaxSpaceSafetyBuffer a safety buffer to prevent moving GC from an overflowing disk.

The SplitStore also has two new commands:

  • lotus chain prune hot is a much less resource-intensive GC and is best suited for situations where you don't have the spare disk space for a full GC.
  • lotus chain prune hot-moving will run a full moving garbage collection of the hotstore. This commands create a new hotstore before deleting the old one so you need working room in the hotstore directory. The current size of a fully GC'd hotstore is around 295 GiB so you need to make sure you have at least that available.

You can read more about the new SplitStore commands in the documentation.

RPC API improvements

This feature release includes all the RPC API improvements made in the Lotus v1.20.x patch releases. It includes an updated FFI that sets the FVM parallelism to 4 by default.

Node operators with higher memory specs can experiment with setting LOTUS_FVM_CONCURRENCY to higher values, up to 48, to allow for more concurrent FVM execution.

Experimental scheduler assigners

In this release there are four new expirmental scheduler assigners:

  • The experiment-spread-qcount - similar to the spread assigner but also takes into account task counts which are in running/preparing/queued states.
  • The experiment-spread-tasks - similar to the spread assigner, but counts running tasks on a per-task-type basis
  • The experiment-spread-tasks-qcount - similar to the spread assigner, but also takes into account task counts which are in running/preparing/queued states, as well as counting running tasks on a per-task-type basis. Check the results for this assigner on (storage-only lotus-workers here).
  • The experiment-random - In each schedule loop the assinger figures a set of all workers which can handle the task and then picks a random one. Check the results for this assigner on (storage-only lotus-workers here).

Graceful shutdown of lotus-workers
We have cleaned up some commands in the lotus-worker to make it less confusing how to gracefully shutting down a lotus-worker while there are incoming sealing tasks in the pipeline. To shut down a lotus-worker gracefully:

  1. lotus-worker tasks disable --all and wait for the worker to finish processing its current tasks.
  2. lotus-worker stop to detach it and do maintenance/upgrades.

CLI speedups

The lotus-miner sector list is now running in parallel - which should speed up the process from anywhere between 2x-10x+. You can tune it additionally with the check-parallelism option in the command. The Lotus-Miner info command also has a large speed improvement, as calls to the lotus legacy market has been removed.

Execution Trace Format Changes

Execution traces (returned from lotus state exec-trace, lotus state replay, etc.), has changed to account for changes introduced by the FVM. Specifically:

  • The Msg field no longer matches the Filecoin message format as many of the message fields didn't make sense in on-chain sub-calls. Instead, it now has the fields To, From, Value, Method, Params, and ParamsCodec where ParamsCodec is a new field indicating the IPLD codec of the parameters.
    • Importantly, the Msg.CID field has been removed. This field is still present in top-level invocation results, just not inside the execution trace itself.
  • The MsgRct field no longer includes a GasUsed field and now has a ReturnCodec field to indicating the IPLD codec of the return value.
  • The Error and Duration fields have been removed as these are not set by the FVM. The top-level message "invocation result" retains the Error and Duration fields, they've only been removed from the trace itself.
  • Gas Charges no longer include "virtual" gas fields (those starting with v...) or source location information (loc) as neither field is set by the FVM.

A note on "codecs": FVM parameters and return values are IPLD blocks where the "codec" specifies the data encoding. The codec will generally be one of:

  • 0x51, 0x71 - CBOR or DagCBOR. You should generally treat these as equivalent.
  • 0x55 - Raw bytes.
  • 0x00 - Nothing. If the codec is 0x00, the parameter and/or return value should be empty and should be treated as "void" (not specified).
{
  "Msg": {
    "Version": 0,
    "To": "f01234",
    "From": "f04321",
    "Nonce": 1,
    "Value": "0",
    "GasLimit": 0,
    "GasFeeCap": "1234",
    "GasPremium": "1234",
    "Method": 42,
    "Params": "<base64-data-or-null>",
    "CID": {
        "/": "bafyxyz....."
    },
  },
  "MsgRct": {
    "ExitCode": 0,
    "Return": "<base64-data-or-null>",
    "GasUsed": 12345,
  },
  "Error": "",
  "Duration": 568191845,
  "GasCharges": [
    {
      "Name": "OnMethodInvocation",
      "loc": null,
      "tg": 23856,
      "cg": 23856,
      "sg": 0,
      "vtg": 0,
      "vcg": 0,
      "vsg": 0,
      "tt": 0
    },
    {
      "Name": "wasm_exec",
      "loc": null,
      "tg": 1764,
      "cg": 1764,
      "sg": 0,
      "vtg": 0,
      "vcg": 0,
      "vsg": 0,
      "tt": 0
    },
    {
      "Name": "OnSyscall",
      "loc": null,
      "tg": 14000,
      "cg": 14000,
      "sg": 0,
      "vtg": 0,
      "vcg": 0,
      "vsg": 0,
      "tt": 0
    },
  ],
  "Subcalls": [
    {
      "Msg": { },
      "MsgRct": { },
      "Error": "",
      "Duration": 1235,
      "GasCharges": [],
      "Subcalls": [],
    },
  ]
}

{
  "Msg": {
    "To": "f01234",
    "From": "f04321",
    "Value": "0",
    "Method": 42,
    "Params": "<base64-data-or-null>",
    "ParamsCodec": 81
  },
  "MsgRct": {
    "ExitCode": 0,
    "Return": "<base64-data-or-null>",
    "ReturnCodec": 81
  },
  "GasCharges": [
    {
      "Name": "OnMethodInvocation",
      "loc": null,
      "tg": 23856,
      "cg": 23856,
      "tt": 0
    },
    {
      "Name": "wasm_exec",
      "loc": null,
      "tg": 1764,
      "cg": 1764,
      "sg": 0,
      "tt": 0
    },
    {
      "Name": "OnSyscall",
      "loc": null,
      "tg": 14000,
      "cg": 14000,
      "sg": 0,
      "tt": 0
    },
  ],
  "Subcalls": [
    {
      "Msg": { },
      "MsgRct": { },
      "GasCharges": [],
      "Subcalls": [],
    },
  ]
}

New features

  • feat: splitstore: Pause compaction when out of sync (filecoin-project/lotus/#10641)
    • Pause the SplitStore compaction if the node is out of sync. Resumes the compation when its back in sync.
  • feat: splitstore: limit moving gc threads (#10621) (filecoin-project/lotus/#10621)
    • Makes moving gc less likely to cause node falling out of sync.
  • feat: splitstore: Update config default value (#10605) (filecoin-project/lotus/#10605)
    • Sets Splitstore HotStoreMaxSpaceTarget config to 650GB as default
  • feat: splitstore: Splitstore enabled by default (#10429) (filecoin-project/lotus#10429)
    • Enables SplitStore by default on new Lotus nodes. Existing Lotus users need to explicitly configure
  • feat: splitstore: Configure max space used by hotstore and GC makes best effort to respect (filecoin-project/lotus#10391)
    • Adds three new configs for setting the maximum allowed space the hotstore can take.
  • feat: splitstore: Badger GC of hotstore command (filecoin-project/lotus#10387)
    • Adds a lotus chain prune hot command, to run the garbage collection of the hotstore in a user driven way.
  • feat: sched: Assigner experiments (filecoin-project/lotus#10356)
    • Introduces experimental scheduler assigners that works better for setups that uses storage-only lotus-workers.
  • fix: wdpost: disabled post worker handling (filecoin-project/lotus#10394)
    • Improved scheduling logic for Proof-of-SpaceTime workers.
  • feat: cli: list claims and remove expired claims (filecoin-project/lotus#9875)
    • Adds a command to list claims made by a provider lotus filplus list-claims. And lotus filplus remove-expired-claims to remove expired claims.
  • feat: cli: make sectors list much faster (filecoin-project/lotus#10202)
    • Makes lotus-miner sector list checks run in parallel.
  • feat: cli: Add an EVM command to fetch a contract's bytecode (filecoin-project/lotus#10443)
    • Adds an lotus evm bytecode command to fetch a contract's bytecode.
  • feat: mempool: Reduce minimum replace fee from 1.25x to 1.1x (#10416) (filecoin-project/lotus#10416)
    • Reduces replacement message fee logic to help include update message replacements from developers using Ethereum tools like MetaMask.
  • feat: update renew-sectors with FIP-0045 logic (filecoin-project/lotus#10328)
    • Updates the lotus-miner sectors extend with FIP-0045 logic to include the ability to drop claims and set the maximum number of messages contained in a message.
  • feat: IPC: Abstract common consensus functions and consensus interface (filecoin-project/lotus#9481)
    • Add eudico's consensus interface to Lotus and implement EC behind that interface. This abstraction is the stepping-stone for Mir's integration.
  • fix: worker: add all tasks flag (filecoin-project/lotus#10232)
    • Adds an all flag for the lotus-worker tasks enable/disable cmds.
  • feat:shed:add cid to cbor serialization command (filecoin-project/lotus#10032)
    • Adds two lotus-shed commands, lotus-shed cid bytes and lotus-shed cid cbor to serialize cid to cbor and cid to bytes.
  • feat: add toolshed commands to inspect statetree size (filecoin-project/lotus#9982)
    • Adds two commands, lotus-shed stat-actor and lotus-shed stat-obj that work with an offline lotus repo to report dag size stats.
  • feat: shed: encode address to bytes (filecoin-project/lotus#10105)
    • Adds a lotus-shed address encode for encoding a filecoin address to hex bytes.
  • feat: chain: export-range (filecoin-project/lotus#10145)
    • Adds a lotus chain export-range command that can create archival-grade ranged exports of the chain as quickly as possible.
  • feat: stmgr: cache migrated stateroots (filecoin-project/lotus#10282)
    • Cache network migration results to avoid running migrations twice.
  • feat: shed: Add a tool to read data from sectors (filecoin-project/lotus#10169)
    • Adds a lotus-shed sectors read command that extract data from sectors from a running lotus-miner deployment.
  • feat: cli: Refactor renew and remove extend (filecoin-project/lotus#9920)
    • Refactors the lotus-miner sectors extend command to have the functionality of lotus-miner sectors renew. The lotus-miner sectors renew command has been deprecated.
  • feat: shed: Add beneficiary commands (filecoin-project/lotus#10037)
    • Adds the beneficiary address command to lotus-shed. You can now use lotus-shed actor propose-change-beneficiary and lotus-shed actor confirm-change-beneficiary to change beneficiary addresses.

Improvements

Dependencies

Others

Contributors

Contributor Commits Lines ± Files Changed
Hannah Howard 2 +2909/-6026 84
Łukasz Magiera 42 +2967/-1848 95
Steven Allen 20 +1703/-1345 88
Alfonso de la Rocha 17 +823/-1808 86
Peter Rabbitson 9 +1957/-219 34
Geoff Stuart 12 +818/-848 29
hannahhoward 5 +507/-718 36
Hector Sanjuan 6 +443/-726 35
Kevin Li 1 +1124/-14 22
zenground0 30 +791/-269 88
frrist 1 +992/-16 13
Travis Person 4 +837/-53 24
Phi 20 +622/-254 34
Ian Davis 7 +35/-729 20
Aayush 10 +378/-177 40
Raúl Kripalani 15 +207/-138 19
Arsenii Petrovich 7 +248/-94 30
ZenGround0 5 +238/-39 15
Neel Virdy 1 +109/-107 58
ychiao 1 +135/-39 3
Jorropo 2 +87/-82 67
Marten Seemann 8 +69/-64 17
Rod Vagg 1 +55/-16 3
Masih H. Derkani 3 +39/-27 12
raulk 2 +30/-29 5
dependabot[bot] 4 +37/-17 8
beck 2 +38/-2 2
Jennifer Wang 4 +20/-19 19
Richard Guan 3 +28/-8 5
omahs 7 +14/-14 7
dirkmc 2 +19/-7 6
David Choi 2 +16/-5 2
Mike Greenberg 1 +18/-1 1
Adin Schmahmann 1 +19/-0 2
Phi-rjan 5 +12/-4 5
Dirk McCormick 2 +6/-6 3
Aayush Rajasekaran 2 +9/-3 2
Jiaying Wang 5 +6/-4 5
Anjor Kanekar 1 +5/-5 1
vyzo 1 +3/-3 2
0x5459 1 +1/-1 1
lotus - v1.21.0-rc2

Published by rjan90 over 1 year ago

This is an optional but highly recommended feature release of Lotus. It includes numerous improvements and enhancements for node operators, ETH RPC-providers and storage providers.

☢️ Upgrade Warnings ☢️

Before upgrading to this feature release read carefully through these bullet points:

  • Starting from this release, the SplitStore feature is automatically activated on new nodes. However, for existing Lotus users, you need to explicitly configure SplitStore by uncommenting the EnableSplitstore option in your config.toml file. To enable SplitStore, set EnableSplitstore=true, and to disable it, set EnableSplitstore=false. It's important to note that your Lotus node will not start unless this configuration is properly set. Set it to false if you are running a full archival node!
  • This feature release requires a minimum Go version of v1.19.7 or higher to successfully build Lotus. Additionally, Go version v1.20 and higher is now also supported.
  • Storage Providers: The proofs libraries now have CUDA enabled by default, which requires you to install (CUDA)[https://lotus.filecoin.io/tutorials/lotus-miner/cuda/] if you haven't already done so. If you prefer to use OpenCL on your GPUs instead, you can use the FFI_USE_OPENCL=1 flag when building from source. On the other hand, if you want to disable GPUs altogether, you can use the FFI_NO_GPU=1 environment variable when building from source.
  • Storage Providers: The lotus-miner sectors extend command has been refactored to the functionality of lotus-miner sectors renew. The issue where extions did not work has been fixed in this release candidate.
  • Exchanges/Node operators/RPC-providers:: Execution traces (returned from lotus state exec-trace, lotus state replay, etc.), has changed to account for changes introduced by the by the FVM. Please make sure to read the Execution trace format change section carefully, as these are interface breaking changes
  • Syncing issues: If you have been struggling with syncing issues in normal operations you can try to adjust the amount of threads used for more concurrent FMV execution through via the LOTUS_FVM_CONCURRENCY enviroment variable. It is set to 4 threads by default. Recommended formula for concurrency == YOUR_RAM/4 , but max is 48. If you are a Storage Provider and are pushing many messages within a short period of time, exporting LOTUS_SKIP_APPLY_TS_MESSAGE_CALL_WITH_GAS=1 will also help with keeping in sync.
  • Catching up from a Snapshot: Users have noticed that catching up sync from a snapshot is taking a lot longer these day. This is largely related to the built-in market actor consuming a lot of computational demand for block validation. A FIP for a short-term mitigation for this is currently in Last Call and will be included network version 19 upgrade if accepted. You can read the FIP here.

Highlights

SplitStore

This feature release introduces numerous improvements and fixes to tackle SplitStore related issues that has been reported. With this feature release SplitStore is automatically activated by default on new nodes. However, for existing Lotus users, you need to explicitly configure SplitStore by uncommenting the EnableSplitstore option in your config.toml file. To enable SplitStore, set EnableSplitstore=true, and to disable it, set EnableSplitstore=false. It's important to note that your Lotus node will not start unless this configuration is properly set. Set it to false if you are running a full archival node!

SplitStore also has some new configuration settings that you can set in your config.toml file:

  • HotstoreMaxSpaceTarget suggests the max allowed space (in bytes) the hotstore can take.
  • HotstoreMaxSpaceThreshold a moving GC will be triggered when total moving size exceeds this threshold (in bytes).
  • HotstoreMaxSpaceSafetyBuffer a safety buffer to prevent moving GC from an overflowing disk.

The SplitStore also has two new commands:

  • lotus chain prune hot is a much less resource-intensive GC and is best suited for situations where you don't have the spare disk space for a full GC.
  • lotus chain prune hot-moving will run a full moving garbage collection of the hotstore. This commands create a new hotstore before deleting the old one so you need working room in the hotstore directory. The current size of a fully GC'd hotstore is around 295 GiB so you need to make sure you have at least that available.

You can read more about the new SplitStore commands in the documentation.

RPC API improvements

This feature release includes all the RPC API improvements made in the Lotus v1.20.x patch releases. It includes an updated FFI that sets the FVM parallelism to 4 by default.

Node operators with higher memory specs can experiment with setting LOTUS_FVM_CONCURRENCY to higher values, up to 48, to allow for more concurrent FVM execution.

Experimental scheduler assigners

In this release there are four new expirmental scheduler assigners:

  • The experiment-spread-qcount - similar to the spread assigner but also takes nto account task counts which are in running/preparing/queued states.
  • The experiment-spread-tasks - similar to the spread assigner, but counts running tasks on a per-task-type basis
  • The experiment-spread-tasks-qcount - similar to the spread assigner, but also takes into account task counts which are in running/preparing/queued states, as well as counting running tasks on a per-task-type basis. Check the results for this assigner on (storage-only lotus-workers here).
  • The experiment-random - In each schedule loop the assinger figures a set of all workers which can handle the task and then picks a random one. Check the results for this assigner on (storage-only lotus-workers here).

Graceful shutdown of lotus-workers
We have cleaned up some commands in the lotus-worker to make it less confusing how to gracefully shutting down a lotus-worker while there are incoming sealing tasks in the pipeline. To shut down a lotus-worker gracefully:

  1. lotus-worker tasks disable --all and wait for the worker to finish processing its current tasks.
  2. lotus-worker stop to detach it and do maintenance/upgrades.

CLI speedups

The lotus-Miner sector list is now running in parallel - which should speed up the process from anywhere between 2x-10x+. You can tune it additionally with the check-parallelism option in the command. The Lotus-Miner info command also has a large speed improvement, as calls to the lotus legacy market has been removed.

New features

  • feat: splitstore: Splitstore enabled by default (#10429) (filecoin-project/lotus#10429)
    • Enables SplitStore by default on new Lotus nodes. Existing Lotus users need to explicitly configure
  • feat: splitstore: Configure max space used by hotstore and GC makes best effort to respect (filecoin-project/lotus#10391)
    • Adds three new configs for setting the maximum allowed space the hotstore can take.
  • feat: splitstore: Badger GC of hotstore command (filecoin-project/lotus#10387)
    • Adds a lotus chain prune hot command, to run the garbage collection of the hotstore in a user driven way.
  • feat: sched: Assigner experiments (filecoin-project/lotus#10356)
    • Introduces experimental scheduler assigners that works better for setups that uses storage-only lotus-workers.
  • fix: wdpost: disabled post worker handling (filecoin-project/lotus#10394)
    • Improved scheduling logic for Proof-of-SpaceTime workers.
  • feat: cli: list claims and remove expired claims (filecoin-project/lotus#9875)
    • Adds a command to list claims made by a provider lotus filplus list-claims. And lotus filplus remove-expired-claims to remove expired claims.
  • feat: cli: make sectors list much faster (filecoin-project/lotus#10202)
    • Makes lotus-miner sector list checks run in parallel.
  • feat: cli: Add an EVM command to fetch a contract's bytecode (filecoin-project/lotus#10443)
    • Adds an lotus evm bytecode command to fetch a contract's bytecode.
  • feat: mempool: Reduce minimum replace fee from 1.25x to 1.1x (#10416) (filecoin-project/lotus#10416)
    • Reduces replacement message fee logic to help include update message replacements from developers using Ethereum tools like MetaMask.
  • feat: update renew-sectors with FIP-0045 logic (filecoin-project/lotus#10328)
    • Updates the lotus-miner sectors extend with FIP-0045 logic to include the ability to drop claims and set the maximum number of messages contained in a message.
  • feat: IPC: Abstract common consensus functions and consensus interface (filecoin-project/lotus#9481)
    • Add eudico's consensus interface to Lotus and implement EC behind that interface. This abstraction is the stepping-stone for Mir's integration.
  • fix: worker: add all tasks flag (filecoin-project/lotus#10232)
    • Adds an all flag for the lotus-worker tasks enable/disable cmds.
  • feat:shed:add cid to cbor serialization command (filecoin-project/lotus#10032)
    • Adds two lotus-shed commands, lotus-shed cid bytes and lotus-shed cid cbor to serialize cid to cbor and cid to bytes.
  • feat: add toolshed commands to inspect statetree size (filecoin-project/lotus#9982)
    • Adds two commands, lotus-shed stat-actor and lotus-shed stat-obj that work with an offline lotus repo to report dag size stats.
  • feat: shed: encode address to bytes (filecoin-project/lotus#10105)
    • Adds a lotus-shed address encode for encoding a filecoin address to hex bytes.
  • feat: chain: export-range (filecoin-project/lotus#10145)
    • Adds a lotus chain export-range command that can create archival-grade ranged exports of the chain as quickly as possible.
  • feat: stmgr: cache migrated stateroots (filecoin-project/lotus#10282)
    • Cache network migration results to avoid running migrations twice.
  • feat: shed: Add a tool to read data from sectors (filecoin-project/lotus#10169)
    • Adds a lotus-shed sectors read command that extract data from sectors from a running lotus-miner deployment.
  • feat: cli: Refactor renew and remove extend (filecoin-project/lotus#9920)
    • Refactors the lotus-miner sectors extend command to have the functionality of lotus-miner sectors renew. The lotus-miner sectors renew command has been deprecated.
  • feat: shed: Add beneficiary commands (filecoin-project/lotus#10037)
    • Adds the beneficiary address command to lotus-shed. You can now use lotus-shed actor propose-change-beneficiary and lotus-shed actor confirm-change-beneficiary to change beneficiary addresses.

Improvements

Dependencies

Others

Execution Trace Format Changes

Execution traces (returned from lotus state exec-trace, lotus state replay, etc.), has changed to account for changes introduced by the FVM. Specifically:

  • The Msg field no longer matches the Filecoin message format as many of the message fields didn't make sense in on-chain sub-calls. Instead, it now has the fields To, From, Value, Method, Params, and ParamsCodec where ParamsCodec is a new field indicating the IPLD codec of the parameters.
    • Importantly, the Msg.CID field has been removed. This field is still present in top-level invocation results, just not inside the execution trace itself.
  • The MsgRct field no longer includes a GasUsed field and now has a ReturnCodec field to indicating the IPLD codec of the return value.
  • The Error and Duration fields have been removed as these are not set by the FVM. The top-level message "invocation result" retains the Error and Duration fields, they've only been removed from the trace itself.
  • Gas Charges no longer include "virtual" gas fields (those starting with v...) or source location information (loc) as neither field is set by the FVM.

A note on "codecs": FVM parameters and return values are IPLD blocks where the "codec" specifies the data encoding. The codec will generally be one of:

  • 0x51, 0x71 - CBOR or DagCBOR. You should generally treat these as equivalent.
  • 0x55 - Raw bytes.
  • 0x00 - Nothing. If the codec is 0x00, the parameter and/or return value should be empty and should be treated as "void" (not specified).
{
  "Msg": {
    "Version": 0,
    "To": "f01234",
    "From": "f04321",
    "Nonce": 1,
    "Value": "0",
    "GasLimit": 0,
    "GasFeeCap": "1234",
    "GasPremium": "1234",
    "Method": 42,
    "Params": "<base64-data-or-null>",
    "CID": {
        "/": "bafyxyz....."
    },
  },
  "MsgRct": {
    "ExitCode": 0,
    "Return": "<base64-data-or-null>",
    "GasUsed": 12345,
  },
  "Error": "",
  "Duration": 568191845,
  "GasCharges": [
    {
      "Name": "OnMethodInvocation",
      "loc": null,
      "tg": 23856,
      "cg": 23856,
      "sg": 0,
      "vtg": 0,
      "vcg": 0,
      "vsg": 0,
      "tt": 0
    },
    {
      "Name": "wasm_exec",
      "loc": null,
      "tg": 1764,
      "cg": 1764,
      "sg": 0,
      "vtg": 0,
      "vcg": 0,
      "vsg": 0,
      "tt": 0
    },
    {
      "Name": "OnSyscall",
      "loc": null,
      "tg": 14000,
      "cg": 14000,
      "sg": 0,
      "vtg": 0,
      "vcg": 0,
      "vsg": 0,
      "tt": 0
    },
  ],
  "Subcalls": [
    {
      "Msg": { },
      "MsgRct": { },
      "Error": "",
      "Duration": 1235,
      "GasCharges": [],
      "Subcalls": [],
    },
  ]
}

{
  "Msg": {
    "To": "f01234",
    "From": "f04321",
    "Value": "0",
    "Method": 42,
    "Params": "<base64-data-or-null>",
    "ParamsCodec": 81
  },
  "MsgRct": {
    "ExitCode": 0,
    "Return": "<base64-data-or-null>",
    "ReturnCodec": 81
  },
  "GasCharges": [
    {
      "Name": "OnMethodInvocation",
      "loc": null,
      "tg": 23856,
      "cg": 23856,
      "tt": 0
    },
    {
      "Name": "wasm_exec",
      "loc": null,
      "tg": 1764,
      "cg": 1764,
      "sg": 0,
      "tt": 0
    },
    {
      "Name": "OnSyscall",
      "loc": null,
      "tg": 14000,
      "cg": 14000,
      "sg": 0,
      "tt": 0
    },
  ],
  "Subcalls": [
    {
      "Msg": { },
      "MsgRct": { },
      "GasCharges": [],
      "Subcalls": [],
    },
  ]
}

Contributors

Contributor Commits Lines ± Files Changed
Hannah Howard 2 +2909/-6026 84
Łukasz Magiera 42 +2967/-1848 95
Steven Allen 20 +1703/-1345 88
Alfonso de la Rocha 17 +823/-1808 86
Peter Rabbitson 9 +1957/-219 34
Geoff Stuart 12 +818/-848 29
hannahhoward 5 +507/-718 36
Hector Sanjuan 6 +443/-726 35
Kevin Li 1 +1124/-14 22
zenground0 30 +791/-269 88
frrist 1 +992/-16 13
Travis Person 4 +837/-53 24
Phi 20 +622/-254 34
Ian Davis 7 +35/-729 20
Aayush 10 +378/-177 40
Raúl Kripalani 15 +207/-138 19
Arsenii Petrovich 7 +248/-94 30
ZenGround0 5 +238/-39 15
Neel Virdy 1 +109/-107 58
ychiao 1 +135/-39 3
Jorropo 2 +87/-82 67
Marten Seemann 8 +69/-64 17
Rod Vagg 1 +55/-16 3
Masih H. Derkani 3 +39/-27 12
raulk 2 +30/-29 5
dependabot[bot] 4 +37/-17 8
beck 2 +38/-2 2
Jennifer Wang 4 +20/-19 19
Richard Guan 3 +28/-8 5
omahs 7 +14/-14 7
dirkmc 2 +19/-7 6
David Choi 2 +16/-5 2
Mike Greenberg 1 +18/-1 1
Adin Schmahmann 1 +19/-0 2
Phi-rjan 5 +12/-4 5
Dirk McCormick 2 +6/-6 3
Aayush Rajasekaran 2 +9/-3 2
Jiaying Wang 5 +6/-4 5
Anjor Kanekar 1 +5/-5 1
vyzo 1 +3/-3 2
0x5459 1 +1/-1 1
Package Rankings
Top 0.79% on Proxy.golang.org
Related Projects
peerbit

P2P database framework with encryption, sharding and search

25 Apr 2022 184
estuary

A custom IPFS/Filecoin node that makes it easy to pin IPFS content and make Filecoin deals.

03 Mar 2021 239
rainbow

A specialized IPFS HTTP gateway

20 Oct 2021 55
kubo

An IPFS implementation in Go

26 Jun 2014 15,752
ipfs-desktop

An unobtrusive and user-friendly desktop application for IPFS on Windows, Mac and Linux.

30 May 2015 5,789
ipfs-docs

📚IPFS documentation platform

23 Oct 2019 286
ipfs-webui

A frontend for an IPFS Kubo node.

26 Sep 2014 1,513
nft.storage

**Notice: Uploads Decommissioned - Existing Data is Safe!** NFT.Storage Classic (classic.nft...

09 Mar 2021 619
valist

Web3-native software distribution. Publish and install executables, Docker images, WebAssembly, a...

14 Jul 2020 117
myel-ethonline

Content retrieval payment manager

03 Oct 2020 6
boxo

A set of reference libraries for building IPFS applications and implementations in Go.

28 Nov 2022 170
ipfs-gui

Creating standards and patterns for IPFS that are simple, accessible, reusable, and beautiful

23 Nov 2017 104
filehive

Filehive is a Filecoin-backed marketplace for datasets.

10 Nov 2020 15
distributions

Legacy dist.ipfs.tech website and artifact build tools

05 Aug 2015 48
good-karma-kit

😇 A Docker Compose bundle to run on servers with spare CPU, RAM, disk, and bandwidth to help the...

12 Apr 2021 302