lighthouse

🧑‍🔧 Release Candidate: Not for Production Use 🧑‍🔧

This is a release candidate. It is not production-ready and not recommended for mainnet use.

Whilst we do not recommend this release for critical and/or mainnet use, we would appreciate help testing this release on testnets.

For more information on release candidates, see: https://lighthouse-book.sigmaprime.io/advanced-release-candidates.html

Summary

This release candidate introduces support for the Ropsten Beacon Chain, among other changes.

The Ropsten Beacon Chain will launch on Monday, May 30th, 2022 3:00:00 PM +UTC. The long-standing Ropsten testnet is expected to undergo "the merge" in early June 2022. Any user that wishes to participate in the Ropsten merge must use this release candidate (or any following release).

Ropsten

Use the --network ropsten flag to join the Ropsten testnet.

Breaking Changes

❗ Database Schema Upgrade ❗

This release contains a backwards-incompatible database schema migration for any network that has undergone "the merge" (see #3157). At the time of writing, this includes Kiln and Kintsugi but excludes Prater and Mainnet. For clarity:

• Prater, Mainnet: users who upgrade to v2.3.0-rc.0 are able to downgrade to v2.2.x and v2.1.x releases.
• Kiln, Kinstugi: users who upgrade to v2.3.0-rc.0 are not able to downgrade to any prior release

All Changes

• v2.3.0-rc.0 (#3218)
• Avoid parallel fork choice runs during sync (#3217)
• Document database migrations (#3203)
• Decrease proposer boost to 40% (#3201)
• Call per_slot_task from a blocking thread (v2) (#3199)
• Remove build status badge from README (#3195)
• Fix Rust 1.61 clippy lints (#3192)
• Add Ropsten configuration (#3184)
• Run fork choice before block proposal (#3168)
• Log out response status when we hit PayloadIdUnavailable (#3190)
• Tiny improvement: PeerManager and maximum discovery query (#3182)
• Add client authentication to Web3Signer validators (#3170)
• Avoid unnecessary slashing protection when publishing blocks (#3188)
• Fix http header accept parsing problem (#3185)
• Remove DB migrations for legacy database schemas (#3181)
• Prevent attestation to future blocks from early attester cache (#3183)
• Allow TaskExecutor to be used in async tests (#3178)
• [refactor] Refactor Option/Result combinators (#3180)
• Remove lcli block packing analysis (#3179)
• Separate execution payloads in the DB (#3157)
• Exclude EE build dirs from Docker context (#3174)
• Add remotekey API support (#3162)
• Fix: no version info in homebrew package (#3167)
• Don't panic in forkchoiceUpdated handler (#3165)
• Fix Execution Engine integration tests (#3163)
• Change the url of the blog post (#3161)
• Poll shutdown timeout in rpc handler (#3153)
• log upgrades + prevent dialing of disconnecting peers (#3148)
• Disallow attesting to optimistic head (#3140)
• keep failed finalized chains to avoid retries (#3142)

Binaries

See pre-built binaries documentation.

The binaries are signed with Sigma Prime's PGP key: 15E66D941F697E28F49381F426416DC3F30674B0

Summary

This release fixes a bug in Lighthouse v2.2.0 that prevented some nodes from completing sync.

The majority of nodes running v2.2.0 will not have been affected by the bug, but nodes attempting to sync from genesis would get stuck at slot 0 forever, and nodes performing checkpoint sync would fail to download blocks from before the Altair hard fork (Oct 2021). The bug occurred as a result of changes made to RPC limits in preparation for the Merge, and has been fixed in PR https://github.com/sigp/lighthouse/pull/3146. Please see that pull request for more detail.

Users upgrading from v2.1 should also read the release notes for v2.2.0 to understand a new protocol feature called proposer boost which has been enabled in v2.2: https://github.com/sigp/lighthouse/releases/tag/v2.2.0

Backwards Compatibility

This release is fully backwards-compatible with the previous releases v2.2.0 and v2.1.x.

Downgrading from v2.2.1 to v2.1.x is also supported without any manual intervention.

Update Priority

This update is a Medium Priority update for users of Lighthouse v2.2.0, and a Low Priority update for v2.1.x.

See Update Priorities for more information about this table.

All Changes

• v2.2.1 (#3149)
• Don't log crits for missing EE before Bellatrix (#3150)
• Fix rpc limits version 2 (#3146)
• Transition Block Proposer Preparation (#3088)
• Fix SPRP default value in cli (#3145)
• Ensure VALID response from fcU updates protoarray (#3126)
• Add tests for importing blocks on invalid parents (#3123)

Binaries

See pre-built binaries documentation.

The binaries are signed with Sigma Prime's PGP key: 15E66D941F697E28F49381F426416DC3F30674B0

Summary

This release includes a minor protocol upgrade named proposer boost, which is designed to mitigate several attacks against fork choice.

It is recommended that all users upgrade over the course of the next few weeks.

Proposer Boost

Background

Proposer boost mitigates attacks involving the delayed publishing of blocks and attestations. By creating disagreement amongst honest nodes, an attacker could frustrate consensus and cause re-orgs. For more detail on the attacks mitigated, please see Three Attacks on Proof-of-Stake Ethereum by researchers from the Ethereum Foundation and Stanford University.

Mitigation

Proposer boost mitigates the attacks by granting additional power to the proposer of each block, equal to 70% of the weight of the attesters in a single slot. This 70% "boost" is only granted to timely blocks, which prevents the delayed-publishing attack while also incentivizing on-time block publication in general. For more information please see the accompanying pull request to the beacon chain specification: https://github.com/ethereum/consensus-specs/pull/2730.

Upgrade Strategy

By virtue of being a protocol change, proposer boost needs to be rolled out by a large fraction of the nodes on the network in order to be effective. This Lighthouse release is coordinated with forthcoming releases from the other consensus clients.

Although it's best for all nodes on the network to adopt proposer boost within a short time frame, we estimate that the impact of upgrades spaced over the course of a few weeks will be minimal. For more detailed analysis please see https://github.com/sigp/lighthouse/pull/3131.

In addition to theoretical analysis and unit testing, proposer boost has also undergone extensive testing on the Prater testnet, where it has been enabled for several months.

Other New Features

• The beacon node and validator client HTTP APIs can now be hosted on IPv6 addresses (https://github.com/sigp/lighthouse/pull/3104).
• The concurrency of request handling during sync was improved (https://github.com/sigp/lighthouse/pull/3036).
• A command for manually inspecting the beacon node's database was added (https://github.com/sigp/lighthouse/pull/3129).
• Lots of progress was made towards The Merge, including:
  • Groundwork for MEV (https://github.com/sigp/lighthouse/pull/3062)
  • Bugfixes (https://github.com/sigp/lighthouse/pull/3133, https://github.com/sigp/lighthouse/pull/3132, https://github.com/sigp/lighthouse/pull/3125, https://github.com/sigp/lighthouse/pull/3071).

Backwards Compatibility

This release is fully backwards-compatible with the previous release.

Downgrading from v2.2.0 to v2.1.x is also supported without any manual intervention.

Update Priority

This table provides priorities for which classes of users should update particular components.

See Update Priorities for more information about this table.

All Changes

• v2.2.0 (#3139)
• Update mdbook runner to Ubuntu 20.04 (#3138)
• Update and consolidate dependencies (#3136)
• Fix merge rpc length limits (#3133)
• Conserve disk space by raising default SPRP (#3137)
• Enable proposer boost on mainnet and GBC (#3131)
• Strip newline from jwt secrets (#3132)
• Add lighthouse db command (#3129)
• Kiln mev boost (#3062)
• json rpc id to value (#3110)
• Increase timeouts for EEs (#3125)
• Update engine state in broadcast (#3071)
• Improve slashing import log (#3122)
• Fix typos in docs (#3121)
• Target geth master in integration testing (#3120)
• Optionally skip RANDAO verification during block production (#3116)
• Add merge objects to lcli parse-ssz subcommand (#3119)
• feat: Update ASCII art (#3113)
• Support IPv6 in BN and VC HTTP APIs (#3104)
• Add Nethermind integration tests (#3100)
• Remove sync await points (#3036)
• Add Proposer Cache Pruning & POS Activated Banner (#3109)
• Fix ganache windows CI attempt 2 (#3107)
• Fix ganache in windows CI (#3105)
• Correct typos book (#3099)
• Updates to tests and local testnet for Ganache 7 (#3056)

Binaries

See pre-built binaries documentation.

The binaries are signed with Sigma Prime's PGP key: 15E66D941F697E28F49381F426416DC3F30674B0

Summary

This is a security-focussed release to address a denial-of-service vulnerability in OpenSSL (CVE-2022-0778).

Lighthouse does not rely on OpenSSL for any of its P2P functionality, but does make use of OpenSSL TLS when making HTTPS connections. We recommend that users upgrade to this release from v2.1.4 if they are using HTTPS URLs in their configuration and are concerned about the denial-of-service risk, see below for details.

OpenSSL Vulnerability Detail

If configured, Lighthouse will use HTTPS to connect to URLs passed to --eth1-endpoints, --checkpoint-sync-url or --beacon-nodes (from the validator client). These URLs typically correspond to servers trusted by the user, but still pose a denial-of-service risk if the connection is intercepted by an intermediary (e.g. an ISP or public hotspot router). We believe that attacks are unlikely to occur in practice, and that their impact would be relatively minor — lighthouse would hang, but there would be no slashing risk nor risk of signing keys being compromised.

For more information please see:

• OpenSSL security advisory: https://www.openssl.org/news/secadv/20220315.txt
• Lighthouse fix: https://github.com/sigp/lighthouse/pull/3095

Backwards Compatibility

Other than the OpenSSL fix, this release is functionally equivalent to the previous release, and is 100% backward compatible.

Update Priority

This table provides priorities for which classes of users should update particular components.

See Update Priorities for more information about this table.

All Changes

• v2.1.5 (#3096)
• Update openssl for CVE-2022-0778 (#3095)
• Clarify proposers message is about current epoch (#3084)
• Add minimum supported Rust version (#3082)

Binaries

See pre-built binaries documentation.

The binaries are signed with Sigma Prime's PGP key: 15E66D941F697E28F49381F426416DC3F30674B0

Summary

This release contains optimisations to P2P networking to improve validator performance. It also contains new functionality for The Merge and out-of-the-box support for the Kiln testnet via the --network kiln flag.

This release will increase the default peer count from 50 to 80. Users who don't specify a custom --target-peers value should see an increased peer count after updating. This is normal and expected.

Other notable features include:

• Ignore non-routable IPs in discovery (#3023)
  • See the "Breaking Changes" section for more details.
• Removal of unnecessary error/warning logs related to sync committees (#2999, #3086)
• Support for requesting proposer duties for the next epoch via the HTTP API (#2963)
• Addition of an API to compute block packing efficiency (#2879)
• Improved performance when a VC encounters timeout errors for a BN (#3022)
• Addition of --validator-dir alias, to aid usability (#3034)
• Fixes to missing quotes on integer values on the API (#3074, #3080)

Increased Peer Count

Ensuring peak performance on the P2P network frequently involves trade-offs between bandwidth usage and ensuring there is a diverse and useful set of peers. The trade-off space varies over time as the network increases in size and as different implementations evolve.

As described in this comment, increasing the default peer count from 50 to 80 has shown to provide significant benefits in peer management whilst making a less significant impact on network bandwidth.

The upside of this change is that we expect less "InsufficientPeers" errors, therefore reducing penalties caused by missed/late attestations or sync messages. The downside is that we expect users to notice a ~50% increase in bytes transmitted on the network and a ~15% increase in bytes recieved (mileage may vary).

Users who prefer minimal bandwidth are free to use --target-peers 50 and maintain the status-quo. That being said, we expect most users will be happy with the new default value of 80.

Breaking Changes

Discovery and Non-Routable IPs

In #3032, Lighthouse now ignores non-globally-routable IP addresses during discovery. This prevents some cloud providers (e.g., Hetzner) from banning Lighthouse under the (false) premise that it is maliciously scanning the private network.

We only expect this to impact developers running local testnets and enterprise users with sophisticated internal P2P routing. This change can be reverted by adding the --enable-private-discovery flag to the BN.

Kintsugi

The Kintsugi testnet is not supported by this release. Users should prefer Kiln.

VC and BN Update Procedure

The VC and BN between this release and the previous are compatible. You may update them independently.

Update Priority

This table provides priorities for which classes of users should update particular components.

See Update Priorities for more information about this table.

All Changes

• v2.1.4 (#3076)
• Ignore attestations to finalized blocks (don't reject) (#3052)
• Lower debug! logs to trace! (#3053)
• Rename random to prev_randao (#3040)
• Custom RPC request management for sync (#3029)
• Update libp2p (#3039)
• Filter non global ips from discovery (#3023)
• Rename Eth1/Eth2 in documents (#3021)
• Increase default target-peer count to 80 (#3005)
• Enforce Optimistic Sync Conditions & CLI Tests (v2) (#3050)
• Cleanup small issues (#3027)
• Retrospective invalidation of exec. payloads for opt. sync (#2837)
• Update to Rust 1.59 and 2021 edition (#3038)
• Ensure logfile respects the validators-dir CLI flag (#3003)
• Add aliases for validator-dir flags (#3034)
• Avoid hogging the fallback status lock in the VC (#3022)
• Increase Bors timeout, refine target-branch-check (#3035)
• Add API to compute block packing efficiency data (#2879)
• Allow proposer duties request for the next epoch (#2963)
• Fix sync committee polling for 0 validators (#2999)
• Improved peer management (#2993)
• Use SmallVec in Bitfield (#3025)
• Engine API v1.0.0.alpha.6 + interop tests (#3024)
• Update to consensus-specs v1.1.9 (#3016)
• Update block reward API docs (#3013)

Binaries

See pre-built binaries documentation.

The binaries are signed with Sigma Prime's PGP key: 15E66D941F697E28F49381F426416DC3F30674B0

Summary

This release addresses a bug in rust-libp2p which has been causing the Lighthouse networking stack to perform poorly since the v2.1.0 release.

This bug-fix significantly improves peer stability and is recommended for any user who is seeing an erratic peer-count and/or missing attestations due to InsufficientPeers.

Fix to rust-libp2p

Since the v2.1.0 release, users have been reporting issues with peer stability manifesting as missed attestations.

After months of developer hours investigating this issue and making peer stability improvements via other methods, we are relieved to have identified an obscure but impactful bug in rust-libp2p.

Whilst user feedback will be the ultimate indicator of success, we are confident that this release will represent a major improvement in peer stability.

Lighthouse relies on a regular "heartbeat" from rust-libp2p in order to perform many critical tasks. An unfortunate regression in rust-libp2p caused the heartbeat to become irregular and stretch from a 700ms interval to several minutes. We have raised this issue with the rust-libp2p developers who have been receptive and are working on a fix. To provide relief to our users, Lighthouse is using a Sigma Prime fork of rust-libp2p with a hot-fix applied.

Breaking Changes

The Pyrmont network is no longer natively supported by Lighthouse (#2543). This means that the --network pyrmont flag will no longer work.

Since the Pyrmont testnet is abandoned and in a state of disarray, we do not expect this to be of concern to our users. The Prater testnet has superseded the Pyrmont testnet.

Users can still opt to use the --testnet-dir flag to manually sync the Prymont network, if desired.

VC and BN Update Procedure

The VC and BN between this release and the previous are compatible. You may update them independently.

Update Priority

This table provides priorities for which classes of users should update particular components.

See Update Priorities for more information about this table.

All Changes

• v2.1.3 (#3017)
• Remove Pyrmont testnet (#2543)
• libp2p upgrade + gossipsub interval fix (#3012)
• Add "update priority" (#2988)
• Allow per validator fee recipient via flag or file in validator client (similar to graffiti / graffiti-file) (#2924)
• Bump crossbeam-utils to fix cargo-audit CI failure (#3004)
• Fix Docker run -p for both TCP and UDP (#2998)
• Gossip cache timeout adjustments (#2997)
• Correct a dial race condition (#2992)
• upgrade libp2p (#2933)
• Retry gossipsub messages when insufficient peers (#2964)

Binaries

See pre-built binaries documentation.

The binaries are signed with Sigma Prime's PGP key: 15E66D941F697E28F49381F426416DC3F30674B0

Summary

This release significantly improves peer stability and is recommended for any user who is seeing an erratic peer-count and/or missing attestations due to InsufficientPeers.

In addition to peer stability improvements, this release also contains:

• Eth2 Key Manager API support for the validator client (#2736)
• Docker Hub images with CPU-optimized "modern" BLST (#2966)
• Support for --network gnosis (#2931)

Peer Scoring Stability

There appears to be a significant portion of peers which are transmitting late messages on the p2p gossip network. The cause is yet to be determined and it's clear that more time will be required to investigate. The late messages are not likely to consume a significant amount of system or network resources, so it seems beneficial to network stability to temporarily tolerate peers who exhibit this behaviour (see (#2976). Once the cause(s) of the late messages is/are determined, we will revisit the peer-scoring parameters.

Optimized Docker Images

If you are running Lighthouse under Docker on a modern CPU then you can probably upgrade to the latest-modern tag for a 20-30% speed-up on signature verification.

If you are running the Lighthouse image manually, use latest-modern or v2.1.2-modern in your docker commands, e.g.

docker run sigp/lighthouse:latest-modern

If you are using lighthouse-docker then you can upgrade by setting LIGHTHOUSE_VERSION=latest-modern in your .env file.

To check whether your CPU is compatible with the optimized image, see Portability in the book. For more information on all the available images, see Available Docker Images.

VC and BN Update Procedure

The VC and BN between this release and the previous are compatible. You may update them independently.

Update Priority

This table provides priorities for which classes of users should update particular components.

User Classes and Priorities

Staking users are those who use lighthouse bn and lighthouse vc to stake on the Beacon Chain. Non-staking users are those who run a lighthouse bn for non-staking purposes (e.g., data analysis or applications).

High priority updates should be completed as soon as possible (e.g., hours or days). Medium priority updates should be completed at the next convenience (e.g., days or a week). Low priority updates should be completed in the next routine update cycle (e.g., two weeks).

All Changes

• v2.1.2 (#2980)
• Add checks to prevent fwding old messages (#2978)
• Add strict penalties const bool (#2976)
• Fix small typo in error log (#2975)
• local testnet: Fix an error on startup (#2973)
• Run setup.sh foreground in order to avoid timing issues (#2970)
• Optimized Docker images (#2966)
• Reduce gossip history (#2969)
• Parse uint256 as decimal string (#2957)
• Implement standard keystore API (#2736)
• Native support for Gnosis Beacon Chain network (#2931)
• Avoid looking up pre-finalization blocks (#2909)
• Add API to compute discrete validator attestation performance (#2874)
• Outaded flag in lighthouse book (#2965)
• Implement API for block rewards (#2628)
• Add flag to disable confirmation when performing voluntary exits (#2955)
• Set mmap threshold to 128KB in malloc utils (#2937)
• Code quality improvents to the network service (#2932)
• Document why we hash downloaded blocks for both sync algs (#2927)
• Fix errors from local testnet scripts on MacOS (#2919)

Binaries

See pre-built binaries documentation.

The binaries are signed with Sigma Prime's PGP key: 15E66D941F697E28F49381F426416DC3F30674B0

Summary

This update addresses a bug that can cause missed attestations accompanied by a Could not publish message error: InsufficientPeers log. We recommend all users update to this release, at their convenience. Even users who are not affected by the bug immediately will likely experience it to some degree, eventually.

The bug fix is relevant to the BN and users are not strictly required to update the VC as well. However, we still recommend updating the VC to this version.

Notable changes include:

• Fix for increased frequency of missed attestations accompanied by a Could not publish message error: InsufficientPeers log (#2948).
• Compatibility between Lighthouse VCs and Nimbus beacon nodes (#2940, #2943, #2952).
• Fix to HTTP query string parsing (#2908).

Known Issues

The validator client compatibility changes inadvertently introduced a bug that results in a panic when a Lighthouse v2.1.1 validator client (VC) connects to a Lighthouse beacon node (BN) of version v2.0.1 or earlier. If you update your beacon node(s) and validator client(s) to v2.1.1 then your setup will be unaffected, the panic will only occur if a beacon node is kept back on v2.0.1 or earlier. We have confirmed that Infura beacon nodes are also unaffected. For more information please see https://github.com/sigp/lighthouse/pull/2957.

Breaking Changes

There are no breaking changes between v2.1.0 and v2.1.1.

Users upgrading from a version prior to v2.1.0 should be sure to read the v2.1.0 release notes and understand the breaking changes involved.

Update Priority

This table provides priorities for which classes of users should update particular components.

User Classes and Priorities

Staking users are those who use lighthouse bn and lighthouse vc to stake on the Beacon Chain. Non-staking users are those who run a lighthouse bn for non-staking purposes (e.g., data analysis or applications).

High priority updates should be completed as soon as possible (e.g., hours or days). Medium priority updates should be completed at the next convenience (e.g., days or a week). Low priority updates should be completed in the next routine update cycle (e.g., two weeks).

All Changes

• 5f628a71d v2.1.1 (#2951)
• 69288f616 VC: don't warn if BN config doesn't match exactly (#2952)
• b9b3ea70d Fix metric name for monitoring (#2950)
• ca29b580a Increase target subnet peers (#2948)
• f35a33716 Quote validator indices when posting duties (#2943)
• 799aedd63 Add default config options for transition constants (#2940)
• f0f327af0 Removed all disable_forks (#2925)
• fc7a1a7dc Allow disconnected states to introduce new peers without warning (#2922)
• a8ae9c841 Add linkcheck workflow (#2918)
• 0116c8d46 Change type of extra fields in ConfigAndPreset (#2913)
• d06f87486 Support duplicate keys in HTTP API query strings (#2908)

Binaries

See pre-built binaries documentation.

The binaries are signed with Sigma Prime's PGP key: 15E66D941F697E28F49381F426416DC3F30674B0

Summary

This update provides significant optimisations and bug-fixes for mainnet users.
We recommend all users update to this release, at their convenience. Users
need to update both their Beacon Node (BN) and Validator Client (VC) when
updating.

Notable changes include:

• Adds a mechanism to improve attestation performance during heavy disk I/O (#2872).
• Improves cache performance during skip slots and late-blocks (#2849, #2832).
• Fixes cases where peers are excessively penalized and disconnected, resulting in
  sharp drops in peer counts (#2903, #2894).
• Switches the slasher database to MDBX, resulting in a smaller disk footprint (#2776).
• Changes default behaviour when using a relative --datadir (#2682, #2846).
• Introduces support for running merge testnets (e.g. Kintsugi) (#2768).

Users should expect this release candidate to provide more stability, reduced resource consumption
and better validator performance.

Known Issues

Increased Memory Footprint

During testing, some nodes exhibited a temporarily increased beacon node memory
footprint of about 150% (~3GB) after upgrading from v2.0.1 to v2.1.0. The memory usage returns to normal after the first restart.

This increased footprint occurs during a recent database migration. The
migration causes memory fragmentation which increases the overall memory
footprint.

Users who observe an increased memory footprint can choose to restart their
beacon node to get the "normal" footprint again. Users who don't mind the
increased footprint can leave their node running with no harm.

Since the migration only happens once (i.e. the first time running v2.1.0 or
later) and the memory usage is within reasonable bounds, the developers have
opted to leave the migration as-is.

Teku/Infura Compatibility

Connecting a Lighthouse validator client to a Teku (or Infura) beacon node produces a warning:

WARN Beacon node config does not match exactly, advice: check that the BN is updated and configured for any upcoming forks

This is due to some minor differences in how the two clients serve data on the /config/spec endpoint when new hard forks are defined. The warning is cosmetic (if annoying), and the two clients will continue to function in its presence.

Breaking Changes

The VC from this release (v2.1.0) is not compatible with Lighthouse BNs from prior versions.
Users MUST update both Lighthouse BNs and VCs to v2.1.0.

New dependencies

Lighthouse now requires libclang when building from source. For details on the packages to install for different platforms please see the updated Building from Source section of the book.

❗ Database Schema Upgrade ❗

Some changes have been made to Lighthouse's database schema to support "the merge", proposer
boosting, and other optimisations. The database schema upgrade will be applied automatically upon
upgrading. Once a beacon node's database has been upgraded by Lighthouse v2.1.0 it will no longer be
compatible with any v2.0.x release, meaning that if you want to revert to a previous version of
Lighthouse you will have to re-sync. We anticipate that this will not be necessary, but Checkpoint
Sync is a good way to re-sync quickly if required.

Slasher Database Changed from LMDB to MDBX

The following section is relevant only to users running a slasher.

The slasher's underlying database has been upgraded to MDBX in
order to address disk usage issues with LMDB (https://github.com/sigp/lighthouse/issues/2538).
Combined with more efficient attestation storage, this should result in around 5-10x less disk space
being used over a long period.

The upgrade to MDBX will happen automatically and destructively upon running v2.1.0. Any previous
LMDB database will be deleted and replaced with a fresh MDBX database. We made the decision to
delete the database rather than copy it, in order to reduce complexity and to avoid running out of
space while copying hundreds of gigabytes of data. Although some LMDB slasher databases had
ballooned to 800GB+ they still only contained data from the last 18 days (4096 epochs) and will be
rebuilt to an equivalent smaller database after running the new release for 18 days.

If you are concerned about everyone upgrading at the same time you could choose to wait a random
number of days between 1 and 18 before upgrading. However we suspect that this is not necessary due
to the presence of Prysm slashers on the network, and the natural staggering of upgrades amongst
users.

Relative Paths in Data Directory

In this release, a bug with parsing the --datadir flag resulted in relative paths being relative
to the users home directory, rather than to the present working directory. This has been fixed in
#2682 and #2846.

Backwards compatibility is maintained by opening the path relative to the home directory, if it
exists. If that path does not exist, the path is assumed to be relative to the present working
directory.

Update Priority

This table provides priorities for which classes of users should update particular components.

Users Classes and Priorities

Staking users are those who use lighthouse bn and lighthouse vc to stake on the Beacon Chain. Non-staking users are those who run a lighthouse bn for non-staking purposes (e.g., data analysis or applications).

High priority updates should be completed as soon as possible (e.g., hours or days). Medium priority updates should be completed at the next convenience (e.g., days or a week). Low priority updates should be completed in the next routine update cycle (e.g., two weeks).

All Changes

• v2.1.0 (#2928)
• Make /config/spec backwards compat for VC (#2934)
• Update to spec v1.1.8 (#2893)
• Use "release candidate" in book (#2920)
• v2.1.0-rc.1
• v2.1.0-rc.0
• Remove grandparents from snapshot cache (#2917)
• Lazy hashing for SignedBeaconBlock in sync (#2916)
• PeerDB Status unknown bug fix (#2907)
• Network performance tuning (#2608)
• Remove gitter from readme (#2914)
• Fix broken links in book (#2912)
• Rust 1.58 lints (#2906)
• Remove penalty for attesting to unknown head (#2903)
• Antithesis docker workflow (#2877)
• Add peer score adjustment msgs (#2901)
• Avoid penalizing peers for delays during processing (#2894)
• Skip serializing proposer boost if null (#2899)
• Avoid peer penalties on internal errors for batch block import (#2898)
• Update dependencies including sha2 (#2896)
• Add early attester cache (#2872)
• Document Homebrew package (#2885)
• Adjusting ARCHIVE_URL (#2892)
• Add tests for flags enable-enr-auto-update and disable-packet-filter (#2887)
• Fix off-by-one in block packing lcli (#2878)
• Allow value for beacon_node fee-recipient argument (#2884)
• Use ? debug formatting for block roots in beacon_chain.rs (#2890)
• Update to superstruct v0.4.1 (#2886)
• Fix assert in slashing protection import (#2881)
• Update rust version in lcli Dockerfile (#2876)
• Only import blocks with valid execution payloads (#2869)
• Additional networking metrics (#2549)
• Allow to set validator password via reimport (#2868)
• Optimise slasher DB layout and switch to MDBX (#2776)
• Add configurable block replayer (#2863)
• Unban peers at the swarm level when purged (#2855)
• Minor Edit on Port Forward Reference (#2867)
• Downgrade AttestationStateIsFinalized error to debug (#2866)
• Enable mallinfo2 behind feature flag (#2864)
• Update docker images to Ubuntu latest (#2862)
• do not count dialing peers in the connection limit (#2856)
• Update OpenSSL (#2865)
• Optimise balances cache in case of skipped slots (#2849)
• v1.1.6 Fork Choice changes (#2822)
• Merge devnet 3 (#2859)
• Update rusqlite from yanked version (#2861)
• Remove wrong duplicated comment (#2751)
• Optimise snapshot cache for late blocks (#2832)
• fix cache miss justified balances calculation (#2852)
• 1.57.0 lints (#2850)
• Restrict network limits based on merge fork epoch (#2839)
• Remove duplicate slot_clock method (#2842)
• Support legacy data directories (#2846)
• Kintsugi Diva comments (#2836)
• Cleanup Comments & Fix get_pow_block_hash_at_ttd() (#2835)
• Kintsugi review comments (#2831)
• Bump crate versions (#2829)
• Removed PowBlock struct that never got used (#2813)
• Kintsugi on_merge_block tests (#2811)
• Increase network limits (#2796)
• Ensure difficulty/hash/epoch overrides change the ChainSpec (#2798)
• Implement engine API v1.0.0-alpha.4 (#2810)
• Skip memory intensive engine test (#2809)
• Fixes after rebasing Kintsugi onto unstable (#2799)
• Remove old uses of testnet
• Fix Uint256 deserialization (#2786)
• 1.1.5 merge spec tests (#2781)
• Fix arbitrary check kintsugi (#2777)
• Move merge-f2f docker to kintsugi (#2774)
• Kintsugi rebase patches (#2769)
• Accept TTD override as decimal (#2676)
• v1.1.1 spec updates (#2684)
• [Merge] Optimistic Sync: Stage 1 (#2686)
• Fix db paths when datadir is relative (#2682)
• [Merge] Optimistic EL verification (#2683)
• [Merge] Block validator duties when EL is not ready (#2672)
• Disable notifier logging from dummy eth1 backend (#2680)
• Update lcli pubkey replace command (#2677)
• Fixed bugs for m3 readiness (#2669)
• Misc changes for merge testnets (#2667)
• Add BeaconChainHarness tests for The Merge (#2661)
• Disable autotests for beacon_chain (#2658)
• Add merge fork_epoch and fork_version to Config (#2663)
• add automated docker build for merge-f2f branch (#2654)
• [Merge] Add execution API test vectors from Geth (#2651)
• [Merge] Add serde impls for Transactions type (#2649)
• [Merge] Implement execution_layer (#2635)
• Finished Gossip Block Validation Conditions (#2640)
• Fork boundary fix (#2646)
• Store execution block hash in fork choice (#2643)
• Update test vectors to v1.1.0 (#2642)
• Handle merge fork in web3signer (#2631)
• Update merge consensus to v1.1.0-beta.5 (#2630)
• Fix clippy lints on merge-f2f (#2626)
• Fix consensus, SSZ, tree hash & run merge EF tests (#2622)
• Initial merge changes
• Add background file logging (#2762)
• Status'd Peer Not Found (#2761)
• Correctly update range status when outdated chains are removed (#2827)
• Allow additional subnet peers (#2823)
• Revert peer DB changes from #2724 (#2828)
• Use published ssz/tree_hash (#2825)
• Ensure consistent log formatting (#2819)
• Inform dialing via the behaviour (#2814)
• Move peer db writes to eth2 libp2p (#2724)
• Sync wrong dbg assertion (#2821)
• Update Lighthouse Dependencies (#2818)
• Fix decoding max length (#2816)
• Investigate and correct RPC Response Timeouts (#2804)
• Fix simulator issues (#2802)
• Add a section on using infura as the checkpoint sync provider (#2797)
• Add fork choice EF tests (#2737)
• Add regression tests for boot_node (#2749)
• Move the peer manager to be a behaviour (#2773)
• De-duplicate attestations in the slasher (#2767)
• Add minimum supported version for checkpoint sync in book (#2779)
• Add cargo vendor test (#2076)
• Peer manager cfg (#2766)
• Do not compute metrics in the network service if the cli flag is not set (#2765)
• Ensure dependent root consistency in head events (#2753)
• Fix linting error on Windows (#2759)
• Check proposer index during block production (#2740)
• Add op pool metrics for attestations (#2758)
• Update prometheus to v0.13.0 (#2757)
• Relax late sync committee penalty (#2752)
• Upgrade to latest libp2p (#2605)
• Fixed Gossip Topics on Fork Boundary (#2619)
• Prevent double import of blocks (#2647)
• Add API version headers and map_fork_name! (#2745)
• Fix purge-db edge case (#2747)
• Add note about java dependency (#2746)
• Add metrics for individual async tasks (#2735)
• Fix test warnings on Rust 1.56.0 (#2743)
• Add a waker to the RPC handler (#2721)
• Update next_fork_subscriptions correctly (#2688)
• Update mainnet altair types test (#2738)
• remove double backfill sync state (#2733)
• Resolve Rust 1.56 lints and warnings (#2728)
• Tiny fix: wrong log level (#2720)
• Add flag to disable lock timeouts (#2714)
• Rename eth2_libp2p to lighthouse_network (#2702)
• Export slashing protection per validator (#2674)
• Ignore cargo audit advisory (#2730)

Binaries

See pre-built binaries documentation.

The binaries are signed with Sigma Prime's PGP key: 15E66D941F697E28F49381F426416DC3F30674B0

🧑‍🔧 Release Candidate: Not for Production Use 🧑‍🔧

This is a release candidate. It is not production-ready and not recommended for mainnet use.

Whilst we do not recommend this release for critical and/or mainnet use, we would appreciate help testing this release on testnets.

For more information on release candidates, see: https://lighthouse-book.sigmaprime.io/advanced-release-candidates.html

Summary

Users need to update both their Beacon Node (BN) and Validator Client (VC) when updating.

Notable changes include:

• Adds a mechanism to improve attestation performance during heavy disk I/O (#2872).
• Improves cache performance during skip slots and late-blocks (#2849, #2832).
• Fixes cases where peers are excessively penalized and disconnected, resulting in
  sharp drops in peer counts (#2903, #2894).
• Switches the slasher database to MDBX, resulting in a smaller disk footprint (#2776).
• Changes default behaviour when using a relative --datadir (#2682, #2846).
• Introduces support for running merge testnets (e.g. Kintsugi) (#2768).

Users should expect this release candidate to provide more stability, reduced resource consumption
and better validator performance.

Note: the v2.1.0-rc.0 version was not released due to CI failures. Use v2.1.0-rc.1 instead.

Breaking Changes

The VC from this release (v2.1.0-rc.1) is not compatible with Lighthouse BNs from prior versions.
Users MUST update both Lighthouse BNs and VCs to v2.1.0.

❗ Database Schema Upgrade ❗

Some changes have been made to Lighthouse's database schema to support "the merge", proposer
boosting, and other optimisations. The database schema upgrade will be applied automatically upon
upgrading. Once a beacon node's database has been upgraded by Lighthouse v2.1.0 it will no longer be
compatible with any v2.0.x release, meaning that if you want to revert to a previous version of
Lighthouse you will have to re-sync. We anticipate that this will not be necessary, but Checkpoint
Sync is a good way to re-sync quickly if required.

Beacon Node API Changes

The GET eth/v1/config/spec endpoint has been modified to support new constants introduced in
Bellatrix (a.k.a. "the merge"). Since v2.1.0-rc.1, VCs will expect some Bellatrix constants to be
present in the spec endpoint. Using a v2.1.0-rc.1 VC with an earlier version BN will produce
errors logs similar to "missing field `TERMINAL_TOTAL_DIFFICULTY`".

Slasher Database Changed from LMDB to MDBX

The following section is relevant only to users running a slasher.

The slasher's underlying database has been upgraded to MDBX in
order to address disk usage issues with LMDB (https://github.com/sigp/lighthouse/issues/2538).
Combined with more efficient attestation storage, this should result in around 5-10x less disk space
being used over a long period.

The upgrade to MDBX will happen automatically and destructively upon running v2.1.0. Any previous
LMDB database will be deleted and replaced with a fresh MDBX database. We made the decision to
delete the database rather than copy it, in order to reduce complexity and to avoid running out of
space while copying hundreds of gigabytes of data. Although some LMDB slasher databases had
ballooned to 800GB+ they still only contained data from the last 18 days (4096 epochs) and will be
rebuilt to an equivalent smaller database after running the new release for 18 days.

If you are concerned about everyone upgrading at the same time you could choose to wait a random
number of days between 1 and 18 before upgrading. However we suspect that this is not necessary due
to the presence of Prysm slashers on the network, and the natural staggering of upgrades amongst
users.

Relative Paths in Data Directory

In this release, a bug with parsing the --datadir flag resulted in relative paths being relative
to the users home directory, rather than to the present working directory. This has been fixed in
#2682 and #2846.

Backwards compatibility is maintained by opening the path relative to the home directory, if it
exists. If that path does not exist, the path is assumed to be relative to the present working
directory.

All Changes

• v2.1.0-rc.1
• v2.1.0-rc.0
• Remove grandparents from snapshot cache (#2917)
• Lazy hashing for SignedBeaconBlock in sync (#2916)
• PeerDB Status unknown bug fix (#2907)
• Network performance tuning (#2608)
• Remove gitter from readme (#2914)
• Fix broken links in book (#2912)
• Rust 1.58 lints (#2906)
• Remove penalty for attesting to unknown head (#2903)
• Antithesis docker workflow (#2877)
• Add peer score adjustment msgs (#2901)
• Avoid penalizing peers for delays during processing (#2894)
• Skip serializing proposer boost if null (#2899)
• Avoid peer penalties on internal errors for batch block import (#2898)
• Update dependencies including sha2 (#2896)
• Add early attester cache (#2872)
• Document Homebrew package (#2885)
• Adjusting ARCHIVE_URL (#2892)
• Add tests for flags enable-enr-auto-update and disable-packet-filter (#2887)
• Fix off-by-one in block packing lcli (#2878)
• Allow value for beacon_node fee-recipient argument (#2884)
• Use ? debug formatting for block roots in beacon_chain.rs (#2890)
• Update to superstruct v0.4.1 (#2886)
• Fix assert in slashing protection import (#2881)
• Update rust version in lcli Dockerfile (#2876)
• Only import blocks with valid execution payloads (#2869)
• Additional networking metrics (#2549)
• Allow to set validator password via reimport (#2868)
• Optimise slasher DB layout and switch to MDBX (#2776)
• Add configurable block replayer (#2863)
• Unban peers at the swarm level when purged (#2855)
• Minor Edit on Port Forward Reference (#2867)
• Downgrade AttestationStateIsFinalized error to debug (#2866)
• Enable mallinfo2 behind feature flag (#2864)
• Update docker images to Ubuntu latest (#2862)
• do not count dialing peers in the connection limit (#2856)
• Update OpenSSL (#2865)
• Optimise balances cache in case of skipped slots (#2849)
• v1.1.6 Fork Choice changes (#2822)
• Merge devnet 3 (#2859)
• Update rusqlite from yanked version (#2861)
• Remove wrong duplicated comment (#2751)
• Optimise snapshot cache for late blocks (#2832)
• fix cache miss justified balances calculation (#2852)
• 1.57.0 lints (#2850)
• Restrict network limits based on merge fork epoch (#2839)
• Remove duplicate slot_clock method (#2842)
• Support legacy data directories (#2846)
• Kintsugi Diva comments (#2836)
• Cleanup Comments & Fix get_pow_block_hash_at_ttd() (#2835)
• Kintsugi review comments (#2831)
• Bump crate versions (#2829)
• Removed PowBlock struct that never got used (#2813)
• Kintsugi on_merge_block tests (#2811)
• Increase network limits (#2796)
• Ensure difficulty/hash/epoch overrides change the ChainSpec (#2798)
• Implement engine API v1.0.0-alpha.4 (#2810)
• Skip memory intensive engine test (#2809)
• Fixes after rebasing Kintsugi onto unstable (#2799)
• Remove old uses of testnet
• Fix Uint256 deserialization (#2786)
• 1.1.5 merge spec tests (#2781)
• Fix arbitrary check kintsugi (#2777)
• Move merge-f2f docker to kintsugi (#2774)
• Kintsugi rebase patches (#2769)
• Accept TTD override as decimal (#2676)
• v1.1.1 spec updates (#2684)
• [Merge] Optimistic Sync: Stage 1 (#2686)
• Fix db paths when datadir is relative (#2682)
• [Merge] Optimistic EL verification (#2683)
• [Merge] Block validator duties when EL is not ready (#2672)
• Disable notifier logging from dummy eth1 backend (#2680)
• Update lcli pubkey replace command (#2677)
• Fixed bugs for m3 readiness (#2669)
• Misc changes for merge testnets (#2667)
• Add BeaconChainHarness tests for The Merge (#2661)
• Disable autotests for beacon_chain (#2658)
• Add merge fork_epoch and fork_version to Config (#2663)
• add automated docker build for merge-f2f branch (#2654)
• [Merge] Add execution API test vectors from Geth (#2651)
• [Merge] Add serde impls for Transactions type (#2649)
• [Merge] Implement execution_layer (#2635)
• Finished Gossip Block Validation Conditions (#2640)
• Fork boundary fix (#2646)
• Store execution block hash in fork choice (#2643)
• Update test vectors to v1.1.0 (#2642)
• Handle merge fork in web3signer (#2631)
• Update merge consensus to v1.1.0-beta.5 (#2630)
• Fix clippy lints on merge-f2f (#2626)
• Fix consensus, SSZ, tree hash & run merge EF tests (#2622)
• Initial merge changes
• Add background file logging (#2762)
• Status'd Peer Not Found (#2761)
• Correctly update range status when outdated chains are removed (#2827)
• Allow additional subnet peers (#2823)
• Revert peer DB changes from #2724 (#2828)
• Use published ssz/tree_hash (#2825)
• Ensure consistent log formatting (#2819)
• Inform dialing via the behaviour (#2814)
• Move peer db writes to eth2 libp2p (#2724)
• Sync wrong dbg assertion (#2821)
• Update Lighthouse Dependencies (#2818)
• Fix decoding max length (#2816)
• Investigate and correct RPC Response Timeouts (#2804)
• Fix simulator issues (#2802)
• Add a section on using infura as the checkpoint sync provider (#2797)
• Add fork choice EF tests (#2737)
• Add regression tests for boot_node (#2749)
• Move the peer manager to be a behaviour (#2773)
• De-duplicate attestations in the slasher (#2767)
• Add minimum supported version for checkpoint sync in book (#2779)
• Add cargo vendor test (#2076)
• Peer manager cfg (#2766)
• Do not compute metrics in the network service if the cli flag is not set (#2765)
• Ensure dependent root consistency in head events (#2753)
• Fix linting error on Windows (#2759)
• Check proposer index during block production (#2740)
• Add op pool metrics for attestations (#2758)
• Update prometheus to v0.13.0 (#2757)
• Relax late sync committee penalty (#2752)
• Upgrade to latest libp2p (#2605)
• Fixed Gossip Topics on Fork Boundary (#2619)
• Prevent double import of blocks (#2647)
• Add API version headers and map_fork_name! (#2745)
• Fix purge-db edge case (#2747)
• Add note about java dependency (#2746)
• Add metrics for individual async tasks (#2735)
• Fix test warnings on Rust 1.56.0 (#2743)
• Add a waker to the RPC handler (#2721)
• Update next_fork_subscriptions correctly (#2688)
• Update mainnet altair types test (#2738)
• remove double backfill sync state (#2733)
• Resolve Rust 1.56 lints and warnings (#2728)
• Tiny fix: wrong log level (#2720)
• Add flag to disable lock timeouts (#2714)
• Rename eth2_libp2p to lighthouse_network (#2702)
• Export slashing protection per validator (#2674)
• Ignore cargo audit advisory (#2730)

Binaries

See pre-built binaries documentation.

The binaries are signed with Sigma Prime's PGP key: 15E66D941F697E28F49381F426416DC3F30674B0

Summary

This release contains a fix for a runtime panic in peer discovery that can cause Lighthouse to shutdown unexpectedly (#2711). It is recommended for all users.

Any user that is running a version prior to v2.0.0 must update before the Altair upgrade on the 27th of October. Nodes running v2.0.0 are fully Altair-ready. However, nodes running any release prior to v2.0.1 will not be patched against the panic fixed in #2711.

Other notable features in this release include:

• Fixes to the standard API (#2712)
• More resilient slashing protection imports (#2598)
• A reduction in validator monitor logging verbosity (#2606)

We recommend updating both the Beacon Node (BN) and Validator Client (VC), however there are no known incompatibilities between v2.0.0 and v2.0.1. Users upgrading from a release prior to v2.0.0 must update both the BN and VC together.

Users upgrading from a release prior to v2.0.0 are advised to read the v2.0.0 Release Notes as they contain important information about breaking changes.

Update Priority

This table provides priorities for which classes of users should update particular components.

Users Classes and Priorities

Staking users are those who use lighthouse bn and lighthouse vc to stake on the Beacon Chain. Non-staking users are those who run a lighthouse bn for non-staking purposes (e.g., data analysis or applications).

High priority updates should be completed as soon as possible (e.g., hours or days). Medium priority updates should be completed at the next convenience (e.g., days or a week). Low priority updates should be completed in the next routine update cycle (e.g., two weeks).

All Changes

• Release v2.0.1 (#2726)
• Correct peer connection transition logic (#2725)
• Add Altair tests to op pool (#2723)
• Update to consensus-spec-tests v1.1.3 (#2722)
• Reduce lock contention in backfill sync (#2716)
• Change "too many resources" to "insufficient resources" in eth2_libp2p (#2713)
• Allow custom certificates when connecting to BN (#2703)
• Update discovery (#2711)
• Quoted altair fields (2.0) (#2712)
• Get arbitrary check passing (2.0) (#2710)
• Add BeaconChainHarness::builder (#2707)
• Make slashing protection import more resilient (#2598)
• Reduce validator monitor logging verbosity (#2606)
• Add TLS capability to the beacon node HTTP API (#2668)
• Refactor Peerdb and PeerManager (#2660)
• Fix cargo audit warns for nix, psutil, time (#2699)
• Avoid negative values in malloc_utils metrics (#2692)
• Update broken api links (#2665)
• Don't log errors on resubscription of gossip topics (#2613)
• Add test_logger as feature to logging (#2586)
• Fix name collision in script (#2678)

Binaries

See pre-built binaries documentation.

The binaries are signed with Sigma Prime's PGP key: 15E66D941F697E28F49381F426416DC3F30674B0

Summary

Lighthouse v2.0.0 is a mandatory upgrade release for all mainnet users. It contains the activation schedule for the beacon chain's first hard-fork: Altair, occurring in around 3 weeks on the 27th of October.

In addition, Lighthouse v2.0.0 includes several new features and optimisations which are described below.

Altair on Mainnet

The v2.0.0 series sets the fork epoch for the Altair upgrade on mainnet, which is scheduled for epoch 74240, i.e. Oct 27, 2021, 10:56:23am UTC. All mainnet users must upgrade to v2.0.0 before this date. Any validators that do not upgrade will cease to follow the chain, will miss 100% of their assigned attestations and blocks, and will be penalised for inactivity.

You must update both the beacon node and validator client together, so that both are aware of the fork.

Checkpoint Sync

Lighthouse beacon nodes can now be synced in a matter of minutes using the new Checkpoint Sync feature. Checkpoint sync accelerates sync by downloading a finalized checkpoint state from an existing beacon node via HTTP, and syncing forward from that point. Checkpoint sync is a first-class Lighthouse feature, and can be used for both staking and archive nodes.

See the docs in the Lighthouse book for more details: https://lighthouse-book.sigmaprime.io/checkpoint-sync.html

⚠️ Database Schema Upgrade ⚠️

The checkpoint sync feature required a small change to Lighthouse's database schema, which will be applied automatically upon upgrading. Once a beacon node's database has been upgraded by Lighthouse v2.0.0 it will no longer be compatible with any v1.x release, meaning that if you want to revert to a previous version of Lighthouse you will have to re-sync from genesis.

Batch BLS Verification for Attestations

An optimisation providing a substantial speed-up to attestation verification has been merged in PR #2399 and is enabled by default. We have observed significant reductions in CPU usage and attestation processing times on both Prater and mainnet.

The optimisation works by grouping the signatures of up to 64 attestations for verification in bulk, which is faster than verifying each signature individually. The batching is opportunistic and will take effect only when 2 or more attestations are queued. In the case of invalid signatures it degrades gracefully to verifying signatures individually, as before. You can read more about this optimisation in the implementation pull request: #2399.

Web3Signer Support

Lighthouse's validator client has been extended with support for the Web3Signer remote signer from Consensys. Using a remote signer allows the validator client to defer responsibility for signing to an external service, which can be administered and audited independently. This is mostly of interest to validators operating at scale, and we are looking forward to seeing the integrations that will be built on top of this feature in the coming weeks and months.

Documentation for using Web3Signer with Lighthouse can be found here: https://lighthouse-book.sigmaprime.io/validator-web3signer.html

Peering and Discovery Improvements

Several improvements have been made to Lighthouse's networking stack, including an upgrade to a more efficient version of discv5 (#2577), and changes to make Lighthouse's peering more robust and resilient (#2530, #2604, #2650).

Update Priority

This table provides priorities for which classes of users should update particular components.

Users Classes and Priorities

Staking users are those who use lighthouse bn and lighthouse vc to stake on the Beacon Chain. Non-staking users are those who run a lighthouse bn for non-staking purposes (e.g., data analysis or applications).

High priority updates should be completed as soon as possible (e.g., hours or days). Medium priority updates should be completed at the next convenience (e.g., days or a week). Low priority updates should be completed in the next routine update cycle (e.g., two weeks).

All Changes

The full list of changes since v1.5.2 is:

• Release v2.0.0 (#2673)
• Fix I/O atomicity issues with checkpoint sync (#2671)
• Fix chain_id value in config/deposit_contract RPC method (#2659)
• Create a local testnet (#2614)
• Release v2.0.0-rc.0 (#2634)
• Consistent tracking of disconnected peers (#2650)
• Remove unused deps (#2592)
• Add BlockTimesCache to allow additional block delay metrics (#2546)
• Improve valmon inclusion delay calculation (#2618)
• Add interop metrics (#2645)
• Refine VC->BN config check (#2636)
• Activate Altair on mainnet at epoch 74240 (#2632)
• Add contribution and proof event (#2527)
• Provide lcli tooling for attestation packing analysis (#2480)
• Update zeroize_derive (#2625)
• Implement SSZ union type (#2579)
• Update spec tests to v1.1.0-beta.4 (#2548)
• Correct bug in sync (#2615)
• Batch BLS verification for attestations (#2399)
• Implement checkpoint sync (#2244)
• Increase connection limits and allow priority connections (#2604)
• Add a note about doppelganger protection interoperability to the docs (#2607)
• Improved handling of IP Banning (#2530)
• Subscribe to altair gossip topics 2 slots before fork (#2532)
• Update mainnet bootnodes (#2594)
• Experimental discovery (#2577)
• Web3Signer support for VC (#2522)
• Shutdown gracefully on panic (#2596)
• Reduce network debug noise (#2593)
• Update sloggers to v2.0.2 (#2588)

Binaries

See pre-built binaries documentation.

The binaries are signed with Sigma Prime's PGP key: 15E66D941F697E28F49381F426416DC3F30674B0

⚠️ Pre-Release: Not for Production Use ⚠️

This is a pre-release. It is not production-ready and not recommended for mainnet use.

Whilst we do not recommend this release for critical and/or mainnet use, we would appreciate help testing this release on testnets.

For more information on pre-releases, see: https://lighthouse-book.sigmaprime.io/advanced-pre-releases.html

Summary

This is the first pre-release for Lighthouse v2.0.0, which includes several new major features. Please read the following sections closely, as this release includes a database schema migration that prevents downgrading after upgrading.

Altair on Mainnet

The v2.0.0 series sets the fork epoch for the Altair upgrade on mainnet, which is scheduled for epoch 74240, i.e. Oct 27, 2021, 10:56:23am UTC. All mainnet users will have to upgrade to the final v2.0.0 release before this date, but should not upgrade to this pre-release right now. The final v2.0.0 release is scheduled for the week after this pre-release, allowing at least 3 weeks for all stakers to upgrade.

Checkpoint Sync

Lighthouse beacon nodes can now be synced in a matter of minutes using the new Checkpoint Sync feature. Checkpoint sync accelerates sync by downloading a finalized checkpoint state from an existing beacon node via HTTP, and syncing forward from that point. Checkpoint sync is a first-class Lighthouse feature, and can be used for both staking and archive nodes.

See the docs in the Lighthouse book for more details: https://lighthouse-book.sigmaprime.io/checkpoint-sync.html

⚠️ Database Schema Upgrade ⚠️

The checkpoint sync feature required a small change to Lighthouse's database schema, which will be applied automatically upon upgrading. Once a beacon node's database has been upgraded by Lighthouse v2.0.0 it will no longer be compatible with any v1.x release, meaning that if you want to revert to a previous version of Lighthouse you will have to re-sync.

Batch BLS Verification for Attestations

An optimisation providing a substantial speed-up to attestation verification has been merged in PR #2399 and is enabled by default. We have observed significant reductions in CPU usage and attestation processing times on both Prater and mainnet.

The optimisation works by grouping the signatures of up to 64 attestations for verification in bulk, which is faster than verifying each signature individually. The batching is opportunistic and will take effect only when 2 or more attestations are queued. In the case of invalid signatures it degrades gracefully to verifying signatures individually, as before. You can read more about this optimisation in the implementation pull request: #2399.

Web3Signer Support

Lighthouse's validator client has been extended with support for the Web3Signer remote signer from Consensys. Using a remote signer allows the validator client to defer responsibility for signing to an external service, which can be administered and audited independently. This is mostly of interest to validators operating at scale, and we are looking forward to seeing the integrations that will be built on top of this feature in the coming weeks and months.

Documentation for using Web3Signer with Lighthouse can be found here: https://lighthouse-book.sigmaprime.io/validator-web3signer.html

Peering and Discovery Improvements

Several improvements have been made to Lighthouse's networking stack, including an upgrade to a more efficient version of discv5 (#2577), and changes to make Lighthouse's peering more robust and resilient (#2530, #2604, #2650).

All Changes

The full list of changes, including some not covered above is below:

• Release v2.0.0-rc.0 (#2634)
• Consistent tracking of disconnected peers (#2650)
• Remove unused deps (#2592)
• Add BlockTimesCache to allow additional block delay metrics (#2546)
• Improve valmon inclusion delay calculation (#2618)
• Add interop metrics (#2645)
• Refine VC->BN config check (#2636)
• Activate Altair on mainnet at epoch 74240 (#2632)
• Add contribution and proof event (#2527)
• Provide lcli tooling for attestation packing analysis (#2480)
• Update zeroize_derive (#2625)
• Implement SSZ union type (#2579)
• Update spec tests to v1.1.0-beta.4 (#2548)
• Correct bug in sync (#2615)
• Batch BLS verification for attestations (#2399)
• Implement checkpoint sync (#2244)
• Increase connection limits and allow priority connections (#2604)
• Add a note about doppelganger protection interoperability to the docs (#2607)
• Improved handling of IP Banning (#2530)
• Subscribe to altair gossip topics 2 slots before fork (#2532)
• Update mainnet bootnodes (#2594)
• Experimental discovery (#2577)
• Web3Signer support for VC (#2522)
• Shutdown gracefully on panic (#2596)
• Reduce network debug noise (#2593)
• Update sloggers to v2.0.2 (#2588)

Binaries

See pre-built binaries documentation.

The binaries are signed with Sigma Prime's PGP key: 15E66D941F697E28F49381F426416DC3F30674B0

Summary

This release contains useful optimizations for mainnet and testnet users. It also contains improvements to logging, tightened file permissions and usability improvements.

Mainnet users will benefit from #2547, which reduces the time it takes to set a block as the canonical head. This reduces the likelihood of late-to-arrive blocks causing missed head votes for attesters. This should see a marginal increase in validator rewards.

Prater and Prymont users will benefit from #2560 which greatly reduces the load during block processing. This will help overloaded nodes run more smoothly. Any Prater or Pyrmont user running v1.5.1 or earlier should update at their next convenience. This optimization was included in v1.5.2-rc.0, so there is no pressing need to update testnet nodes running that release-candidate.

Cross-compatibility between Lighthouse and Teku has been improved by #2587. This will benefit Infura users across mainnet and testnets.

Breaking Changes

The breaking changes listed here are introduced between v1.5.1 and this this release. No breaking changes have been introduced since v1.5.2-rc.0.

Tighter File Permissions

The log file generated with the --logfile flag will now have tighter file permissions (600). Previously, all users had read permissions. Now, read permissions are only granted to the user running the lighthouse binary. Since the log file is created anew on each restart, these permissions will take affect immediately after updating.

The API token files generated by the lighthouse vc will also be created with the same restricted file permissions (600). If the API token files already exist, no changes will be made to their permissions.

Restricting access to log files and API tokens is a security best-practice.

Remote Signer

The lighthouse remote_signer command has been removed. This feature was incomplete and has been archived. At present, there are no plans to reinstate it. Once #2522 is complete, Lighthouse will support remote signing via Web3Signer.

Update Priority

This table provides priorities for which classes of users should update particular components.

Users Classes and Priorities

Staking users are those who use lighthouse bn and lighthouse vc to stake on the Beacon Chain. Non-staking users are those who run a lighthouse bn for non-staking purposes (e.g., data analysis or applications).

High priority updates should be completed as soon as possible (e.g., hours or days). Medium priority updates should be completed at the next convenience (e.g., days or a week). Low priority updates should be completed in the next routine update cycle (e.g., two weeks).

All Changes

Changes are since the v1.5.1 release.

• f9bba92db v1.5.2 (#2595)
• e4ed42a9d Fix nightly bump num bigint (#2591)
• f7dd24c99 Add quoted u64/u64_vec to SyncCommitteeSubscription (#2589)
• 46cd67d51 Case insensitive match for ForkName (#2587)
• 8d2650fff Make local-testnet setup more discoverable (#2572)
• 3b0f3247c Fix typo of vars.env (#2574)
• 1eefd6d41 Add how users should report security vulnerabilities for this repository (#2562)
• ddbd4e696 v1.5.2-rc.0 (#2565)
• 9c785a9b3 Optimize process_attestation with active balance cache (#2560)
• f4aa1d8ae Archive remote_signer code (#2559)
• d9910f96c Remove SIGPIPE handler (#2558)
• ac274221c Run altair specific checks with the simulator (#2556)
• 6f18f9589 Update file permissions (#2499)
• 50321c667 Updates to make crates publishable (#2472)
• 5a3bcd290 Validator monitor support for sync committees (#2476)
• 44fa54004 Persist to DB after setting canonical head (#2547)
• 1031f79ac Improve ergonomics of adding a new network config (#2489)
• b4dd98b3c Shutdown after sync (#2519)
• 10945e061 Revert bad blocks on missed fork (#2529)
• 6b65b6f3b removed verify_exit_time_independent_only function (#2509)
• bc14d1d73 Add more unix signal handlers (#2486)
• 99737c551 Improve eth1 fallback logging (#2490)
• beab306e0 Fix log output for INFO Found no doppelganger (#2551)

Binaries

See pre-built binaries documentation.

The binaries are signed with Sigma Prime's PGP key: 15E66D941F697E28F49381F426416DC3F30674B0

⚠️ Pre-Release: Not for Production Use ⚠️

This is a pre-release. It is not production-ready and not recommended for mainnet use.

Whilst we do not recommend this release for critical and/or mainnet use, we do recommend that users upgrade their Prater nodes to this version.

For more information on pre-releases, see: https://lighthouse-book.sigmaprime.io/advanced-pre-releases.html

Summary

The motivation for this release candidate is to apply an optimization to Prater nodes (#2560). Without this optimization, a significant increase in block import and API response times will be observed for all blocks since the Altair fork on Prymont (2020/09/02). Whilst this does not represent a consensus bug, it may cause increased CPU load and decreased performance for attesters and API consumers.

Updating to this release (or later) will be required for all Prater users. However, since Prater is a testnet, we do not consider this an urgent priority. We kindly request that users update their Prater nodes at their next convenience.

There is no action required for mainnet users. Do not use this release candidate on mainnet.

Breaking Changes

Tighter File Permissions

The log file generated with the --logfile flag will now have tighter file permissions (600). Previously, all users had read permissions. Now, read permissions are only granted to the user running the lighthouse binary. Since the log file is created anew on each restart, these permissions will take affect immediately after updating.

The API token files generated by the lighthouse vc will also be created with the same restricted file permissions (600). If the API token files already exist, no changes will be made to their permissions.

Restricting access to log files and API tokens is a security best-practice.

Remote Signer

The lighthouse remote_signer command has been removed. This feature was incomplete and has been archived. At present, there are no plans to reinstate it. Once #2522 is complete, Lighthouse will support remote signing via Web3Signer.

All Changes

• ddbd4e696 v1.5.2-rc.0 (#2565)
• 9c785a9b3 Optimize process_attestation with active balance cache (#2560)
• f4aa1d8ae Archive remote_signer code (#2559)
• d9910f96c Remove SIGPIPE handler (#2558)
• ac274221c Run altair specific checks with the simulator (#2556)
• 6f18f9589 Update file permissions (#2499)
• 50321c667 Updates to make crates publishable (#2472)
• 5a3bcd290 Validator monitor support for sync committees (#2476)
• 44fa54004 Persist to DB after setting canonical head (#2547)
• 1031f79ac Improve ergonomics of adding a new network config (#2489)
• b4dd98b3c Shutdown after sync (#2519)
• 10945e061 Revert bad blocks on missed fork (#2529)
• 6b65b6f3b removed verify_exit_time_independent_only function (#2509)
• bc14d1d73 Add more unix signal handlers (#2486)
• 99737c551 Improve eth1 fallback logging (#2490)
• beab306e0 Fix log output for INFO Found no doppelganger (#2551)

Binaries

See pre-built binaries documentation.

The binaries are signed with Sigma Prime's PGP key: 15E66D941F697E28F49381F426416DC3F30674B0

Summary

This update to Lighthouse contains the following notable changes:

• Support for the Altair upgrade on the Prater testnet (scheduled for the 2nd of September, 2021).
• A fix to a recent issue where a drop in peer count resulted in sub-optimal attestation performance.
• The fork schedule API has been updated for Altair (see #2524)

Prater Altair Upgrade

All Prater users must upgrade their VC and BN prior to 02/09/2021, 12:24:00 pm (UTC). Nodes that do not upgrade to v1.5.1 or later before that time will be unable to follow the head and will require a re-sync in order to rejoin the network.

Prater users must upgrade both the VC and BN to v1.5.1 or later, otherwise the VC will log warnings about "incompatibility" with the BN.

Peer drop resulting in sub-optimal attestation performance

Users have reported occurrences of a combination of "Prior attestation known" debug logs, drops in peer counts and sub-optimal attestation performance.

Our investigations have found instances of old attestations (32-33 slots old) circulating on the network. Whilst we do not believe that these messages should be on the network (i.e., they are a useless burden), Lighthouse was being overzealous in banning peers who were sending these messages.

Thanks to changes introduced in #2540 and #2542, Lighthouse will not ban peers for this behaviour.

The Lighthouse team will continue to investigate why these old messages are still circulating on the network.

Known Issues

• Unable to restart lighthouse if altair fork is missed (#2526)

Note: There are no known issues affecting mainnet.

Update Priority

This table provides priorities for which classes of users should update particular components.

Note: this update is "high priority" for all Prater users due to the upcoming Altair fork,

Users Classes and Priorities

Staking users are those who use lighthouse bn and lighthouse vc to stake on the Beacon Chain. Non-staking users are those who run a lighthouse bn for non-staking purposes (e.g., data analysis or applications).

High priority updates should be completed as soon as possible (e.g., hours or days). Medium priority updates should be completed at the next convenience (e.g., days or a week). Low priority updates should be completed in the next routine update cycle (e.g., two weeks).

All Changes

• b0ac3464c v1.5.1 (#2544)
• 440542572 Expand gossip duplicate cache time (#2542)
• 9fb94fbeb Add Altair fork epoch for Prater (#2537)
• 3fdad38eb Remove penality for duplicate attestation from same validator (#2540)
• 09545fe66 Increase maximum gossipsub subscriptions (#2531)
• d3b4cbed5 Packet filter cli option (#2523)
• aca49fc45 Security update for openssl (#2539)
• 5b8436e33 Fork schedule api (#2525)
• bc0599246 Windows binaries (#2492)
• 02fd54bea Refactor discovery queries (#2518)

Binaries

See pre-built binaries documentation.

The binaries are signed with Sigma Prime's PGP key: 15E66D941F697E28F49381F426416DC3F30674B0

Summary

This is a substantial update to Lighthouse which contains many new features, performance improvements and fixes. Importantly, due to added support for the Altair hard-fork, this release is not backwards-compatible with v1.4.0 or earlier. You can safely upgrade to this release without re-syncing the chain, but will need to re-sync to downgrade (see Breaking Changes below).

With this caveat clearly stated, the Lighthouse team is proud to present this release which contains:

• A refined and improved networking stack.
• Doppelganger protection (disabled by default).
• The reorg event has been added to the HTTP API.
• Caching of attestations for unknown blocks, to improve validator rewards (#2319)
• Hardware-optimized hashing (#2426)
• Faster and more reliable attesting for validators (#2478)
• Support for compiling under the latest version of Rust (#2483)
• Support for Altair on the Pyrmont testnet
• Countless bug/annoyance fixes.

Breaking Changes

• This update involves a change to the Beacon Node database schema which is not backwards compatible. After upgrading, reverting to v1.4.0 or earlier will require users to delete the existing beacon node database and re-sync. This does not affect the slasher or validator client. Re-syncing is only required on a downgrade (reversion), there is no need to re-sync if upgrading to this release from an earlier version – the database migration will happen automatically.
• Changes have been made to the beacon node API which will result in previous versions of the Lighthouse validator client logging warnings about "incompatibility". The solution is simple; update and restart both the beacon node and validator client.
• Some Prometheus metrics and the /lighthouse/validator_inclusion endpoints have been modified to be Altair-compatible. See #2416 for detail.

Known Issues

The validator client is incompatible with Teku beacon nodes on Prater due to a difference in how Prater configuration is represented. If you are running a Lighthouse validator client with a Teku beacon node on Prater (including via Infura) we recommend remaining on v1.4.0 until the incompatibility is resolved. Mainnet and Pyrmont are unaffected. More information here.

There are a couple of Altair-related issues that do not affect mainnet, and will be fixed in the next release. Importantly, if you run a Pyrmont node and didn't update to a v1.5.0 release candidate before the Altair fork, then issue #2526 means that upgrading now will be ineffective. We plan to have this fixed in the next release.

• Unable to restart lighthouse if altair fork is missed (#2526)
• Fork schedule API is not updated for Altair (#2524)

Update Priority

This table provides priorities for which classes of users should update particular components.

Users Classes and Priorities

Staking users are those who use lighthouse bn and lighthouse vc to stake on the Beacon Chain. Non-staking users are those who run a lighthouse bn for non-staking purposes (e.g., data analysis or applications).

High priority updates should be completed as soon as possible (e.g., hours or days). Medium priority updates should be completed at the next convenience (e.g., days or a week). Low priority updates should be completed in the next routine update cycle (e.g., two weeks).

All Changes

• 90d5ab156 v1.5.0 (#2535)
• f2a8c6229 Metrics and DEBG log for late gossip blocks (#2533)
• 12fe72bd3 Always require auth header in VC (#2517)
• c7379836a v1.5.0-rc.1 (#2516)
• bc6a848de Quote integers in sync committee contributions (#2515)
• c0a2f501d Upgrade dependencies (#2513)
• d17350c0f Lower penalty for past/future slot errors (#2510)
• 4c4ebfbaa v1.5.0 rc.0 (#2506)
• 4af6fcfaf Bump libp2p to address inconsistency in mesh peer tracking (#2493)
• 33ff51a09 Add Altair fork schedule for Pyrmont (#2501)
• 54f92cc26 Use THC for state.inactivity_scores (#2504)
• 7b46c4bb7 Add docs for Pre-Releases (#2505)
• ceda27371 Ensure doppelganger detects attestations in blocks (#2495)
• ff85b0524 Add docs for doppelganger protection (#2496)
• 71ab16e40 Register vals with doppelganger earlier (#2494)
• 17a2c778e Altair validator client and HTTP API (#2404)
• 350b6f19d Print only unique doppelgangers (#2500)
• fc8969302 Update mdbook version in github actions (#2497)
• e8c0d1f19 Altair networking (#2300)
• 6a620a31d Fix starting-epoch check in doppelganger (#2491)
• 187425cdc Bump discv5 to v0.1.0-beta.9 (#2479)
• c5786a882 Doppelganger detection (#2230)
• 834ee98bc Fix flag in redundancy docs (#2482)
• 303deb996 Rust 1.54.0 lints (#2483)
• 8efd9fc32 Add AttesterCache for attestation production (#2478)
• 1d4f90e2e Bump tests to v1.1.0-beta.2 (#2481)
• 923486f34 Use bulk verification for sync_aggregate signature (#2415)
• 6e3ca48cb Cache participating indices for Altair epoch processing (#2416)
• f5bdca09f Update to spec v1.1.0-beta.1 (#2460)
• 84e6d7195 Tree hash caching and optimisations for Altair (#2459)
• 74aa99c40 Document BN API security considerations (#2470)
• 63923eaa2 Bump discv5 to v0.1.0-beta.8 (#2471)
• 17b6d7ce8 Add http-address flag to VC (#2467)
• bcf8ba631 Add lcli Dockerfile and auto-build to CI (#2469)
• 9a8320bea Merge pull request #2389 from sigp/network-1.5
• 08fedbfcb (origin/pr/2389) Libp2p Connection Limit (#2455)
• 6818a9417 Discovery update (#2458)
• 381befbf8 Ensure disconnecting peers are added to the peerdb (#2451)
• 059d9ec1b Gossipsub scoring improvements (#2391)
• c62810b40 Update to Libp2p to 39.1 (#2448)
• 3c0d3227a Global Network Behaviour Refactor (#2442)
• 64226321b Relax requirement for enr fork digest predicate (#2433)
• c1d2e35c9 Bleeding edge discovery (#2435)
• f4bc9db16 Change the window mode of yamux (#2390)
• 6fb48b45f Discovery patch (#2382)
• 4aa06c955 Network upgrades (#2345)
• b0f5c4c77 Clarify eth1 error message (#2461)
• a3a7f39b0 [Altair] Sync committee pools (#2321)
• 8fa6e463c Update direct libsecp256k1 dependencies (#2456)
• fc4c61147 Remove msg about longer sync with remote eth1 nodes (#2453)
• 304fb05e4 Maintain attestations that reference unknown blocks (#2319)
• 9656ffee7 Metrics for sync aggregate fullness (#2439)
• 27aec1962 Add more detail to "Prior attestation known" log (#2447)
• 20fce117f Start a release checklist (#2270)
• 2c691af95 Use hardware acceleration for SHA256 (#2426)
• a7b7134ab Return more detail when invalid data is found in the DB during startup (#2445)
• 371c216ac Use read_recursive locks in database (#2417)
• b3c7e59a5 Adjust beacon node timeouts for validator client HTTP requests (#2352)
• b4689e20c Altair consensus changes and refactors (#2279)
• 89361573d Update testnet-validator.md (#2432)
• 78e5c0c15 Capture a missed VC error (#2436)
• 406e3921d Use forwards iterator for state root lookups (#2422)
• 73d002ef9 Update outdated dependencies (#2425)
• 206486006 Add Lighthouse version and commit hash to Prometheus metrics (#2427)
• 379664a64 Improve compilation error on 32-bit (#2424)
• 9461ac2d5 Run apt-get upgrade to get latest security updates (#2418)
• 2357a4186 Fix typo in advanced networking docs (#2412)
• 6583ce325 Minify slashing protection interchange data (#2380)
• b84ff9f79 rust 1.53.0 updates (#2411)
• 3dc1eb5eb Ignore inactive validators in validator monitor (#2396)
• 98ab00cc5 Handle Geth pre-EIP-155 block sync error condition (#2304)
• b1657a60e Reorg events (#2090)
• 3261eff0b split outbound and inbound codecs encoded types (#2410)
• a526145b4 Fix remote signer test (#2400)
• dffe31c31 Add an account command to enable/disable validators (#2386)

Binaries

See pre-built binaries documentation.

The binaries are signed with Sigma Prime's PGP key: 15E66D941F697E28F49381F426416DC3F30674B0

⚠️ Pre-Release: Not for Production Use ⚠️

This is a pre-release. It is not production-ready and not recommended for mainnet use.

Whilst we don't recommend this release for critical and/or mainnet use, we do recommend that users upgrade their Pyrmont and Prater nodes to this version. We assume that nothing critical is occurring on these testnets.

For more information on pre-releases, see: https://lighthouse-book.sigmaprime.io/advanced-pre-releases.html

Summary

Compared to v1.5.0.-rc.0, this pre-release contains three fixes:

1. A fix to the API to add quotes around a 64-bit integer. (#2515)
2. Dependency upgrades to remove a "yanked" warning whilst compiling. (#2513)
3. A change to our network peer scoring to reduce the impact of early sync committee messages. (#2510)

The most notable of these changes is (3), it addresses an issue on an Altair devnet where we saw a significant number of peer disconnections due to early sync committee messages. Although these messages shouldn't occur in theory, they are occurring in practice. As such, we recommend that all Pyrmont users should upgrade to this release before the Altair fork at 2021-08-19 12:00:07Z.

Users upgrading from v1.4.0 or earlier read the release notes in v1.5.0.-rc.0 to understand the breaking changes introduced there.

All Changes

• c7379836a v1.5.0-rc.1 (#2516)
• bc6a848de Quote integers in sync committee contributions (#2515)
• c0a2f501d Upgrade dependencies (#2513)
• d17350c0f Lower penalty for past/future slot errors (#2510)

Binaries

See pre-built binaries documentation.

The binaries are signed with Sigma Prime's PGP key: 15E66D941F697E28F49381F426416DC3F30674B0

⚠️ Pre-Release: Not for Production Use ⚠️

This is a pre-release. It is not production-ready and not recommended for mainnet use.

Whilst we don't recommend this release for critical and/or mainnet use, we do recommend that users upgrade their Pyrmont and Prater nodes to this version. We assume that nothing critical is occurring on these testnets.

For more information on pre-releases, see: https://lighthouse-book.sigmaprime.io/advanced-pre-releases.html

Pyrmont Altair Upgrade

This update is mandatory for all nodes on the Pyrmont testnet. At 2021-08-19 12:00:07Z, Pyrmont will undergo an irregular state transition to upgrade to the Altair specification (i.e., there will be a hard fork).

Users running Lighthouse v1.4.0 or earlier will fail to undergo this state-transition and will be left on the wrong side of history.

The Eth2 developers (including the Lighthouse team) are upgrading Pyrmont in anticipation for the same upgrade on mainnet. Failing to upgrade your Pyrmont nodes will impede their efforts to deliver a smooth upgrade on mainnet. Please upgrade your Pyrmont nodes!

To be clear, mainnet users should not implement this release-candidate version. Prater users may upgrade, but are not required to.

Summary

This is a substantial update to Lighthouse with two important caveats:

• This update involves a change to the Beacon Node database schema which is presently not backwards compatible. After upgrading the Beacon Node (BN), users will need to delete the existing BN database and re-sync in order to revert to a previous version. This does not affect the slasher or validator client. Re-syncing is only required on a downgrade, there is no need to re-sync if upgrading to this release from an earlier version --- the database migration will happen automatically.
• Pyrmont users must upgrade their BN and Validator Client (VC) at the same time. Due to the upcoming Altair upgrade, pre-v1.5.0-rc.0 VCs and BNs are incompatible. This only applies to Pyrmont, other networks are not affected.

Failing to upgrade both the BN and VC on Pyrmont will result in the VC declaring the BN to be "incompatible". The solution is simple; update and restart both processes.

With the caveats clearly stated, the Lighthouse team is proud to present this release which contains:

• Full support for the upcoming Altair fork on the Pyrmont testnet at 2021-08-19 12:00:07Z.
• A refined and improved networking stack.
• Doppelganger protection (disabled by default).
• The reorg event has been added to the HTTP API.
• Countless bug/annoyance fixes.
• Caching of attestations for unknown blocks, to improve validator rewards (#2319)
• Hardware-optimized hashing (#2426)
• Faster and more reliable attesting for validators (#2478)
• Support for compiling under the latest version of Rust (#2483)

Breaking Changes

• The database schema used for v1.5.0-rc.0 nodes (including freshly-synced nodes and those upgraded from earlier versions) is not compatible with earlier versions of Lighthouse. This was mentioned earlier.
• Exclusively for Pyrmont nodes, the VC and BN must both be running v1.5.0-rc.0. Earlier versions are not compatible with v1.5.0-rc.0.
• Some Prometheus metrics and the /lighthouse/validator_inclusion endpoints have been modified to be Altair-compatible. See #2416 for detail.

All Changes

• 4c4ebfbaa v1.5.0 rc.0 (#2506)
• 4af6fcfaf Bump libp2p to address inconsistency in mesh peer tracking (#2493)
• 33ff51a09 Add Altair fork schedule for Pyrmont (#2501)
• 54f92cc26 Use THC for state.inactivity_scores (#2504)
• 7b46c4bb7 Add docs for Pre-Releases (#2505)
• ceda27371 Ensure doppelganger detects attestations in blocks (#2495)
• ff85b0524 Add docs for doppelganger protection (#2496)
• 71ab16e40 Register vals with doppelganger earlier (#2494)
• 17a2c778e Altair validator client and HTTP API (#2404)
• 350b6f19d Print only unique doppelgangers (#2500)
• fc8969302 Update mdbook version in github actions (#2497)
• e8c0d1f19 Altair networking (#2300)
• 6a620a31d Fix starting-epoch check in doppelganger (#2491)
• 187425cdc Bump discv5 to v0.1.0-beta.9 (#2479)
• c5786a882 Doppelganger detection (#2230)
• 834ee98bc Fix flag in redundancy docs (#2482)
• 303deb996 Rust 1.54.0 lints (#2483)
• 8efd9fc32 Add AttesterCache for attestation production (#2478)
• 1d4f90e2e Bump tests to v1.1.0-beta.2 (#2481)
• 923486f34 Use bulk verification for sync_aggregate signature (#2415)
• 6e3ca48cb Cache participating indices for Altair epoch processing (#2416)
• f5bdca09f Update to spec v1.1.0-beta.1 (#2460)
• 84e6d7195 Tree hash caching and optimisations for Altair (#2459)
• 74aa99c40 Document BN API security considerations (#2470)
• 63923eaa2 Bump discv5 to v0.1.0-beta.8 (#2471)
• 17b6d7ce8 Add http-address flag to VC (#2467)
• bcf8ba631 Add lcli Dockerfile and auto-build to CI (#2469)
• 9a8320bea Merge pull request #2389 from sigp/network-1.5
• 08fedbfcb Libp2p Connection Limit (#2455)
• 6818a9417 Discovery update (#2458)
• 381befbf8 Ensure disconnecting peers are added to the peerdb (#2451)
• 059d9ec1b Gossipsub scoring improvements (#2391)
• c62810b40 Update to Libp2p to 39.1 (#2448)
• 3c0d3227a Global Network Behaviour Refactor (#2442)
• 64226321b Relax requirement for enr fork digest predicate (#2433)
• c1d2e35c9 Bleeding edge discovery (#2435)
• f4bc9db16 Change the window mode of yamux (#2390)
• 6fb48b45f Discovery patch (#2382)
• 4aa06c955 Network upgrades (#2345)
• b0f5c4c77 Clarify eth1 error message (#2461)
• a3a7f39b0 [Altair] Sync committee pools (#2321)
• 8fa6e463c Update direct libsecp256k1 dependencies (#2456)
• fc4c61147 Remove msg about longer sync with remote eth1 nodes (#2453)
• 304fb05e4 Maintain attestations that reference unknown blocks (#2319)
• 9656ffee7 Metrics for sync aggregate fullness (#2439)
• 27aec1962 Add more detail to "Prior attestation known" log (#2447)
• 20fce117f Start a release checklist (#2270)
• 2c691af95 Use hardware acceleration for SHA256 (#2426)
• a7b7134ab Return more detail when invalid data is found in the DB during startup (#2445)
• 371c216ac Use read_recursive locks in database (#2417)
• b3c7e59a5 Adjust beacon node timeouts for validator client HTTP requests (#2352)
• b4689e20c Altair consensus changes and refactors (#2279)
• 89361573d Update testnet-validator.md (#2432)
• 78e5c0c15 Capture a missed VC error (#2436)
• 406e3921d Use forwards iterator for state root lookups (#2422)
• 73d002ef9 Update outdated dependencies (#2425)
• 206486006 Add Lighthouse version and commit hash to Prometheus metrics (#2427)
• 379664a64 Improve compilation error on 32-bit (#2424)
• 9461ac2d5 Run apt-get upgrade to get latest security updates (#2418)
• 2357a4186 Fix typo in advanced networking docs (#2412)
• 6583ce325 Minify slashing protection interchange data (#2380)
• b84ff9f79 rust 1.53.0 updates (#2411)
• 3dc1eb5eb Ignore inactive validators in validator monitor (#2396)
• 98ab00cc5 Handle Geth pre-EIP-155 block sync error condition (#2304)
• b1657a60e Reorg events (#2090)
• 3261eff0b split outbound and inbound codecs encoded types (#2410)
• a526145b4 Fix remote signer test (#2400)
• dffe31c31 Add an account command to enable/disable validators (#2386)

Binaries

See pre-built binaries documentation.

The binaries are signed with Sigma Prime's PGP key: 15E66D941F697E28F49381F426416DC3F30674B0

Summary

This is a low-priority update. It primarily contains optimizations which reduce the impact of the Beacon Node on RAM, CPU, disk I/O and Eth1 nodes.

Whilst we suggest all users update to this release at their convenience, there is no pressing reason to do so unless they are concerned that the resource requirements of Lighthouse are outpacing their hardware.

Notable features in this release are:

• A reduction in memory footprint. On mainnet, this reduces memory usage from ~6 GB to ~1.5 GB. (#2299)
• Improvements to avoid disk and memory intensive operations. (#2289, #2376)
• An 80% reduction in Eth1 queries. (#2340)
• Beta Windows support, not presently recommended for production. (#2333)

As mentioned in the "Memory Allocator Tuning" section, some users running Linux without glibc may need to apply the --disable-malloc-tuning CLI flag to lighthouse bn (the Beacon Node) in order to avoid a panic at startup. We expect this scenario to be very rare and have not encountered it during internal testing or during external release candidate testing. We only recommend using this flag if you experience a failure as it will substantially increase memory usage. We only expect to see this issue on obscure Linux installations; MacOS, Ubuntu, Amazon Linux and other common operating systems are unaffected.

Breaking Changes

• #2272 changes the behaviour of the HTTP API when retrieving blocks/block-roots on a slot that was skipped. Previously, Lighthouse returned the value for the block at the previous slot. Now, to keep in-line with other clients, Lighthouse will return a 404.
• #2299 automatically configures the GNU memory allocator (see "Memory Allocator Tuning" section for more detail).
• #2251 adds additional fields to the lighthouse/health HTTP API endpoint.
• #2392 ensures that using the --eth1-endpoints flag enables syncing the Eth1 chain. This fixes a regression in a previous release.

Update Priority

This table provides priorities for which classes of users should update particular components.

Users Classes and Priorities

Staking users are those who use lighthouse bn and lighthouse vc to stake on the Beacon Chain. Non-staking users are those who run a lighthouse bn for non-staking purposes (e.g., data analysis or applications).

High priority updates should be completed as soon as possible (e.g., hours or days). Medium priority updates should be completed at the next convenience (e.g., days or a week). Low priority updates should be completed in the next routine update cycle (e.g., two weeks).

Notes

Memory Allocator Tuning

With #2299, Lighthouse now automatically configures the GNU Memory Allocator at startup to provide a more optimal memory footprint.

This alone reduces the memory consumption of Lighthouse by at least 50%. However, some Linux systems may have an incompatible memory allocator. This will cause Lighthouse to panic and exit at startup (only at startup, never after it has begun operation).

If users experience a panic and instant exit during startup, do not fear. Simply provide the --disable-malloc-tuning flag to completely disable this new feature.

For example:

lighthouse --disable-malloc-tuning bn --staking

If you do experience this error, please raise an issue. We are eager to ensure Lighthouse runs on all platforms by default.

We have never seen this panic during our testing or on the release candidate, but we are aware it is possible on some obscure platforms.

Changes to Announced Features

Since Lighthouse Update #36 announced a list of features and a timeline for v1.4.0, the development team has decided to split the features into two, separate releases:

• v1.4.0: to be released approximately one week earlier than scheduled, but only containing:
  • Beta Windows support, not presently recommended for production.
  • An 80% reduction in Eth1 queries. (Note: this does not reduce the time required to sync the Eth1 cache).
  • A reduction in memory footprint (on mainnet, this reduces memory usage from ~6 GB to ~1.5 GB).
• v1.5.0: to be release late June, containing:
  • Altair testnet support.
  • Doppelganger protection.
  • Support for remote signing in the validator client.

All Changes

• v1.4.0 (#2402)
• Modify Malloc Tuning (#2398)
• Only perform malloc tuning for beacon node (#2397)
• Make less logs for attn with unknown head (#2395)
• Fix options for --eth1-endpoints flag (#2392)
• v1.4.0-rc.0 (#2379)
• Revert "Network protocol upgrades (#2345)" (#2388)
• Add early check for RPC block relevancy (#2289)
• Return a specific error for frozen attn states (#2384)
• Return more detail in Eth1 HTTP errors (#2383)
• Use the forwards iterator more often (#2376)
• Minimum Outbound-Only Peers Requirement (#2356)
• Reduce outbound requests to eth1 endpoints (#2340)
• Correct issue with dialing peers (#2375)
• Network protocol upgrades (#2345)
• Tune GNU malloc (#2299)
• Monitoring service api (#2251)
• More stringent dialing (#2363)
• Add Windows to Bors config (#2358)
• Enable Compatibility with Windows (#2333)
• Write validator definitions atomically (#2338)
• Add NETWORK_ID variable (#2330)
• Updated to comply with new clippy formatting rules (#2336)
• Add testing for beacon node and validator client CLI flags (#2311)
• Add SensitiveUrl to redact user secrets from endpoints (#2326)
• cargo audit fix (#2331)
• Filter Disconnected Peers from Discv5 DHT (#2219)
• Fix typos in redundancy docs (#2320)
• 404's on API requests for slots that have been skipped or orphaned (#2272)
• Allow specifying alternative url for deposit_contract (#2295)
• Add a no-wait flag for voluntary exits (#2292)
• Fix readme typo (#2312)

Binaries

See pre-built binaries documentation.

The binaries are signed with Sigma Prime's PGP key: 15E66D941F697E28F49381F426416DC3F30674B0

Windows for Lighthouse is still in beta and binaries are not yet provided.