lighthouse

Summary

This is a low-priority release. It contains several minor bug fixes and performance improvements which we recommend rolling out on mainnet nodes at your convenience. This release also includes support for the Holešky testnet, with genesis on 15 September 2023.

Note that there is no v4.4.0 release due to a bug found late in the release cycle. The bug was fixed in PR #4687, which is included in this release.

The highlights of v4.4.1 are:

• [NEW] Introduction of the lighthouse validator-manager command for managing validators on a running validator client. Documentation can be found in a new chapter of the Lighthouse book: https://lighthouse-book.sigmaprime.io/validator-manager.html.
• Improved scheduling for HTTP requests (#4462). Lighthouse will now balance the load of HTTP requests against other important duties, and avoid becoming overwhelmed. We hope that this improves stability for nodes supporting a large number of validators or processing frequent HTTP queries.
• Disk space savings for new nodes through state pruning (#4610, #4663).
• Bugfix for backfill sync erroring upon completion (#4465).
• Support for the Holešky testnet via --network holesky (#4653).
• Support for the Chiado Gnosis testnet via --network chiado (#4530).

🐞 Known Issues 🐞

We have identified a minor bug in the code related to state pruning that can cause an error message to be logged for a limited time after upgrading:

ERRO Missing chunk in forwards iterator

It can be safely ignored, as it does not impact the node's ability to follow the chain or perform validator duties. However, API queries for historic blocks & states may fail until the issue resolves itself. This only affects nodes with an existing database. Nodes with lower --slots-per-restore-point will heal faster, and we expect the maximum time to heal to be around 8192 slots (~27 hours). The issue is being tracked in https://github.com/sigp/lighthouse/issues/4697.

⚠️ Breaking Changes ⚠️

Breaking Change: State pruning for new nodes

Newly synced nodes will no longer store historic states by default. This applies both to checkpoint sync and genesis sync. Node operators may opt-in to historic state storage by reconstructing states with the --reconstruct-historic-states flag.

Existing nodes will continue storing historic states newer than the checkpoint from which they synced. We are working on a command to prune historic states from existing nodes, which will likely be available in the next release.

The pruned status of a node can be determined using the /lighthouse/database/info API, which will show the value of state_upper_limit as 18446744073709551615. For more details see #4610, #4663.

Breaking Change: Inclusion delay in HTTP rewards API

Support for the phase0 hard fork was added to the attestation rewards API. This required the addition of a new field for the inclusion_delay, which has been standardised upstream in beacon-APIs: https://ethereum.github.io/beacon-APIs/#/Rewards/getAttestationsRewards.

See #4474, #4520 for details.

Breaking Change: HTTP status code for duplicate blocks

Lighthouse will now return an HTTP 202 ACCEPTED response when attempting to publish a block that is already known. This behaviour goes against the current beacon-APIs spec, but is likely better for the majority of API consumers. Previously, a 400 BAD REQUEST response could cause the caller (usually a validator client) to think that block publication had failed when it had not. We have begun a discussion to change the spec to allow the new error code.

The status code can also be set to a custom value using the flag --http-duplicate-block-status. See https://github.com/sigp/lighthouse/pull/4655 for details.

New HTTP APIs

The Lighthouse BN supports the following new HTTP APIs:

• SSZ variant of POST /eth/v{1,2}/beacon/blocks (#4479).
• SSZ variant of POST /eth/v{1,2}/beacon/blinded_blocks (#4504).
• POST /eth/v1/validator/liveness/{epoch} (#4343).
• GET /eth/v1/builder/states/{state_id}/expected_withdrawals (#4390).

Update Priority

This table provides priorities for which classes of users should update particular components.

See Update Priorities more information about this table.

The Beacon Node may be updated without also updating the Validator Client. However we recommend upgrading both components for consistency.

All Changes

• Release v4.4.1 (#4690)
• Remove double-locking deadlock from HTTP API (#4687)
• Release v4.4.0 (#4673)
• Pin foundry toolchain version to fix stuck CI jobs
• feat: add chiado (#4530)
• Fix bug in block root storage (#4663)
• Add Holesky (#4653)
• Increase web3signer_tests timeouts (#4662)
• Show --gui flag in help text (#4660)
• Do not log slot clock error prior to genesis (#4657)
• Send success code for duplicate blocks on HTTP (#4655)
• update dependencies (#4639)
• rust 1.72 lints (#4659)
• Use lockfile with cross and fix audit fail (#4656)
• Revive mplex (#4619)
• Implement expected withdrawals endpoint (#4390)
• Remove Antithesis docker build workflow (#4642)
• Fix the link of anvil in lighthouse book (#4641)
• Fix beacon-processor-max-workers (#4636)
• ForkChoice: remove an impossible case for parent_checkpoints (#4626)
• Remove checkpoint alignment requirements and enable historic state pruning (#4610)
• Fix Prefer Builder Flag (#4622)
• Update blst to 0.3.11 (#4624)
• add metrics::VALIDATOR_DUTIES_SYNC_HTTP_POST for post_validator_duties_sync (#4617)
• attester_duties: remove unnecessary case (#4614)
• Don't kill SSE stream if channel fills up (#4500)
• Fix disable backfill flag not working correctly (#4615)
• Improve HTTP API error messages + tweaks (#4595)
• ForkChoice: remove head_block_root field (#4590)
• opt maybe_update_best_child_and_descendant: remove an impossible case (#4583)
• Use the native concurrency property to cancel workflows (#4572)
• Fix bug of init_from_beacon_node (#4613)
• use account_manager::CMD instead of magic string (#4612)
• remove optional_eth2_network_config (#4611)
• minor optimize process_active_validator: avoid a call to state.get_validator (#4608)
• align editorconfig with rustfmt (#4600)
• Fix some typos (#4565)
• Improve transport connection errors (#4540)
• Use BeaconProcessor for API requests (#4462)
• Add validator-manager (#3502)
• Increase slashing protection test SQL timeout to 500ms (#4574)
• Support SSZ request body for POST /beacon/blinded_blocks endpoints (v1 & v2) (#4504)
• Update doppelganger note about sync committee contributions (#4425)
• Shift networking configuration (#4426)
• CommitteeCache.get_all_beacon_committees: set correct capacity to avoid realloc (#4557)
• CommitteeCache.initialized: fail early if possible (#4556)
• fix typo (#4555)
• upgrade to libp2p 0.52 (#4431)
• Deprecate exchangeTransitionConfiguration functionality (#4517)
• Update Lighthouse book FAQ (#4510)
• add ssz support in request body for /beacon/blocks endpoints (v1 & v2) (#4479)
• Development feature flag - Disable backfill (#4537)
• Add Eth Version Header (#4528)
• Fix incorrect ideal rewards calculation (#4520)
• Fix bug in lcli transition-blocks and improve pretty-ssz (#4513)
• Restore upstream arbitrary (#4372)
• Implement liveness BeaconAPI (#4343)
• Add self-hosted runners v2 (#4506)
• Phase 0 attestation rewards via Beacon API (#4474)
• Cleanup unreachable code in lcli::generate_bootnode_enr and some tests (#4485)
• Speed up CI by installing foundry with Github action (#4505)
• Rust 1.71 lints (#4503)
• CI fix: add retries to eth1 sim tests (#4501)
• Clean up local testnet files without prompting (#4498)
• Remove wget dependency (#4497)
• Update mev-rs and remove patches (#4496)
• Remove instances of required arguments with default values (#4489)
• Use unique arg names for eth1-sim (#4463)
• Fix HTTP state API bug and add --epochs-per-migration (#4236)
• bump proc-macro2 (#4464)
• Remove hidden re-exports to appease Rust 1.73 (#4495)
• Add state-root command and network support to lcli (#4492)
• Move the BeaconProcessor into a new crate (#4435)
• Bump default checkpoint sync timeout to 3 minutes (#4466)
• Correct checks for backfill completeness (#4465)

Binaries

See pre-built binaries documentation.

The binaries are signed with Sigma Prime's PGP key: 15E66D941F697E28F49381F426416DC3F30674B0

Summary

This release is a low-priority release for all users, except those using the Gnosis Chain.

For Gnosis users, this release includes the Gnosis Merge parameters. Gnosis users will need to update to this release before November 30th or they will fail to go through the merge transition and will be left following the wrong chain. Therefore, this release is high-priority for Gnosis users.

Ethereum mainnet and Ethereum testnet (e.g., Goerli, Sepolia) users may choose to update to this release, but it is low-priority.

Alongside the Gnosis changes, this release includes:

• ⚠️ Database schema upgrade to v13 ⚠️
• Support for a new and upcoming improvement to checkpoint sync (#2915)
• Addition of the --checkpoint-sync-url-timeout flag (#3710)
• Improvements to sync committee message signing with optimistic EEs (#3624)

Breaking Changes

⚠️ Breaking Change: Database Schema v13 ⚠️

To support new features the database schema has been upgraded to v13. The schema upgrade will be applied automatically upon upgrading and should not take more than a few seconds.

To downgrade, follow the instructions in the book for Database Migrations.

Breaking Change: Sync Commmittee Fallback Behaviour

The signing of sync committee messages by the validator client has been improved in a way that is incompatible with old versions of the Lighthouse beacon node. We do not expect many users to be affected because only versions prior to v3.0.0 are incompatible.

For details please see: https://github.com/sigp/lighthouse/pull/3624

Gnosis Merge

Gnosis chain is merging! 🎉

The expected timeline is:

• November 30 2022: Bellatrix hard fork activated on the consensus layer.
• December 5-11 2022: Terminal total difficulty reached, merge completed.

You must update both the Lighthouse beacon node and validator client before November 30. Failure to update will result in missed validator rewards and a corrupt beacon node database (requiring a re-sync).

You must also ensure you're running a compatible verison of Nethermind (v1.14.6 or newer) and that it is correctly connected to your Lighthouse beacon node. For further instructions please see:

• Merge Migration - Lighthouse Book
• Gnosis Chain Merge Guide

Deposit Snapshot Sync

We are pleased to announce the readiness of deposit snapshot sync, which enables nodes to quickly sync the deposit contract.

When setting up a new Lighthouse node using checkpoint sync, a snapshot of the deposit contract will be downloaded from the checkpoint sync provider. Initially the only supported provider is Lighthouse v3.3.0, although we expect checkpointz and the other clients soon will roll out support soon.

If the checkpoint sync provider does not support the feature you'll see a pair of warning logs:

WARN Remote BN does not support EIP-4881 fast deposit sync
WARN Failed to finalize deposit cache

These warnings are harmless, and Lighthouse will gracefully fallback to syncing the deposit contract cache from the execution node.

Depsosit snapshot support is the result of many months of work by @ethDreamer, who not only implemented the feature in Lighthouse but also wrote the spec for it, EIP-4811.

For further details on the implementation see: https://github.com/sigp/lighthouse/pull/2915

Update Priority

This table provides priorities for which classes of mainnet users should update particular components.

Note: this update is high-priority for Gnosis chain users.

The Beacon Node may be updated without the Validator Client, however we recommend updating both components for completeness. Gnosis users must update both the beacon node and the validator client.

See Update Priorities for more information about this table.

All Changes

• v3.3.0 (#3741)
• Lower deposit finalization error to warning (#3739)
• Schedule gnosis merge (#3729)
• Super small improvement: Remove unnecessary mut (#3736)
• Add metrics for subnet queries (#3721)
• Simplify GossipTopic -> String conversion (#3722)
• compile with beta compiler on CI (#3717)
• Health Endpoints for UI (#3668)
• CI gardening maintenance (#3706)
• Sync committee sign bn fallback (#3624)
• Add --light-client-server flag and state cache utils (#3714)
• add checkpoint-sync-url-timeout flag (#3710)
• Blinded block and RANDAO APIs (#3571)
• Register blocks in validator monitor (#3635)
• Added Merkle Proof Generation for Beacon State (#3674)
• Blocklookup data inconsistencies (#3677)
• Update stale sections of the book (#3671)
• Clarify error log when registering validators (#3650)
• Fix rust 1.65 lints (#3682)
• Deposit Cache Finalization & Fast WS Sync (#2915)
• Update discv5 (#3171)
• Book spelling and grammar corrections (#3659)
• Added lightclient server side containers (#3655)

Summary

This medium-priority release is a patch release for the recently released v3.2.0, fixing a performance regression.

Please see the release notes for v3.2.0 for a summary of included features and breaking changes:

https://github.com/sigp/lighthouse/releases/tag/v3.2.0

Update Priority

This table provides priorities for which classes of users should update particular components.

The Beacon Node may be updated without the Validator Client and vice versa, as long as both BN and VC are running a v3.x release.

Beacon nodes and validator clients on pre-v3.0.0 versions have been unable to follow the chain since September 6 2022 when the Bellatrix hard fork was activated.

See Update Priorities for more information about this table.

All Changes

• Release v3.2.1 (#3660)
• Revert "Optimise HTTP validator lookups" (#3658)

Binaries

See pre-built binaries documentation.

The binaries are signed with Sigma Prime's PGP key: 15E66D941F697E28F49381F426416DC3F30674B0

This minor release contains some important protection from excessive resource consumption in some uncommon cases (#2130). We recommend all users update both their BNs and VCs to this version.

The breaking changes in this release should be insignificant for most, see below for more information.

New Features

The most notable feature is the new "validator monitor" that allows a BN to provide additional logging and metrics for specific validators.

This provides the long-awaited "validator balance" metric and many, many others.

Additionally, you get additional logs about your validators activities (e.g. attestation inclusion in blocks).

See the validator monitor docs for more information.

Breaking Changes

Breaking changes are fairly minor in this release, but may have edge-case impacts for those running custom or non-Lighthouse validator clients.

• Disallow attestation production earlier than head (#2130)
  • Does not permit the creation of AttestationData via the API for slots earlier than the head. This is a fast patch to an immediate issue, in the future we plan to implement a more flexible solution with #2169.

All Changes

• Detailed validator monitoring (#2151)
• Fix bug from #2163 (#2165)
• Disallow attestation production earlier than head (#2130)
• Represent slots in secs instead of millisecs (#2163)
• Add lcli command to replace state pubkeys (#1999)
• Simplify enum -> str with strum (#2164)
• Modify proto array loop (#2154)
• Automate docker version tag (#2150)
• Update docs: Change --beacon-node to --beacon-nodes (#2145)
• Fix: end_slot returns incorrect value (#2138)
• Fix timing issue in obtaining the Fork (#2158)
• remove protected environment (#2135)
• Clippy 1.49.0 updates and dht persistence test fix (#2156)
• Fix race condition when creating LocalBeaconNode (#2137)
• Disconnect from peers quicker on internet issues (#2147)
• Use cross in linux x86 64 release flow (#2136)
• Add docs about redundancy (#2142)
• update smallvec (#2152)
• replace tempdir by tempfile (#2143)
• Reduce ping interval (#2132)
• Remove duplicate log in BN fallback (#2116)
• Ssz state api endpoint (#2111)
• test_decode_malicious_status_message improvements (#2104)
• Fix typo in ssz_snappy.rs comment (#2103)

Binaries

See pre-built binaries documentation.

The binaries are signed with Sigma Prime's PGP key: 15E66D941F697E28F49381F426416DC3F30674B0

We recommend all users upgrade to v1.0.4, particularly users who have previously experienced database corruption, or who are running a slasher. Please see below for details of breaking changes, including removal of the websocket server.

Breaking Changes

• Removed the legacy websocket server in favour of the standardised server-sent-events API (#1920).
• Disabled Medalla and other legacy testnet support by default, improving compilation time (#1989)

Summary

• Improved handling of I/O errors during block processing. We hope that this spells the end of corrupt databases and resyncs! (#2068)
• Fix a bug in the slasher that caused it to miss some blocks (#2047).
• Networking improvements that strengthen peering, and may improve attestation effectiveness (#2075, #2041).
• New --eth1-purge-cache flag that can be used to clean out the Eth1 cache without a resync. This should only be used if your node is unable to recover from an error like FailedToInsertDeposit(NonConsecutive {..}). The flag is supplied to the beacon node command like so: lighthouse bn --eth1-purge-cache (#2039).

All Changes

• Softer penalties for missing blocks (#2075)
• Multiarch docker GitHub actions (#2065)
• Revert fork choice if disk write fails (#2068)
• Improve compile time (#1989)
• Downgrades a valid log (#2057)
• fix default network handling (#2029)
• More metrics + RPC tweaks (#2041)
• update dependencies (#2032)
• Remove audit ignore ws server (#2051)
• Fix default values and --network flag in Voluntary exits book page (#2056)
• Pass failed gossip blocks to the slasher (#2047)
• Add a eth1-purge-cache cli option (#2039)
• Server sent events (#1920)
• Validators endpoint status code (#2040)
• Update blst to 0.3.2 (#2034)
• Fingerprint new client identify agent string (#2027)
• Pretty-print EIP-3076 tests (#1977)
• Normalize keystore passwords (#1972)
• Minor fixes (#2038)
• Fix new clippy lints (#2036)
• delete validator-dir path printing in subcommands (#2025)
• Add Content-Type to metrics server (#2019)
• Fix broken custom data directories link (#2000)

Docker Hub

NEW in this release: multi-arch Docker images. Thanks to amazing work by @realbigsean in #2065,
our Docker Hub images are now capable of running on x86_64 or ARM64! That includes
the Raspberry Pi 4 🎉

You can get the latest release via the :latest or :v1.0.4 tags:

• sigp/lighthouse:latest
• sigp/lighthouse:v1.0.4

The latest dev build is available via the :latest-unstable tag:

• sigp/lighthouse:latest-unstable

For the multi-arch build, please use :latest rather than :v1.0.4.

Binaries

See pre-built binaries documentation.

The binaries are signed with Sigma Prime's PGP key: 15E66D941F697E28F49381F426416DC3F30674B0

We recommend all users to upgrade to v1.0.3, especially if they're experiencing timeouts with their eth1 endpoint.

Please note

Users may see a new log message:

Nov 30 21:04:28.268 WARN Syncing eth1 block cache   est_blocks_remaining: initializing deposits, msg: sync can take longer when using remote eth1 nodes, service: slot_notifier

This log indicates that your beacon node is downloading blocks and deposits from your eth1 node. When the est_blocks_remaining is initializing_deposits, your node is downloading deposit logs. It may stay in this stage for several minutes. Once the deposits logs are finished downloading, the est_blocks_remaining value will start decreasing.

It is perfectly normal to see this log when starting a node for the first time or after being off for more than several minutes.

If this log continues appearing sporadically during operation, there may be an issue with your eth1 endpoint.

Summary

• Increasing timeouts and decreases payload size for log request from eth1 nodes.
• Reduces the impact of timeouts when they do occur.
• Logs information about eth1 syncing whilst users are waiting for genesis.
• Adds the lighthouse bn --private flag which zeros-out graffiti and does not publish client information in the identify protocol.

All Changes

• Add privacy option (#2016)
• Improve eth1 block sync (#2008)
• No string in slog (#2017)
• Update PGP key in README (#1986)
• Drop discovery log to trace (#2007)

DockerHub

• sigp/lighthouse:v1.0.3

Binaries

See pre-built binaries documentation.

The binaries are signed with Sigma Prime's PGP key: 15E66D941F697E28F49381F426416DC3F30674B0

We recommend all users to upgrade to v1.0.2, especially if they're intending to stake with a genesis validator.

This release updates core dependencies in Lighthouse and is highly encouraged for all nodes to run for mainnet.

Summary

• Updates the core executor to a tokio 0.3 runtime and modifies the structure of the task executor
• Updates libp2p (security fixes, spec incompatibility updates)
• Updates BLST to the latest official version of BLST
• UPnP handling improvements

All Changes

• Upgrade to tokio 0.3 and related core sub-dependencies (#1839)
• Update BLST to the latest version (#1979)
• Fix type in docs (#1990)

DockerHub

• sigp/lighthouse:v1.0.2

Binaries

See pre-built binaries documentation.

The binaries are signed with Sigma Prime's PGP key: 15E66D941F697E28F49381F426416DC3F30674B0

We recommend all users to upgrade to v1.0.1, especially if they're intending to stake with a genesis validator.

Since this release affects lockfiles, users should be especially cautious that there is no other lighthouse vc process running on the same host when upgrading to this release. Running two validator clients at the same time, with the same keys can lead to slashing.

Summary

• Changes to flock-based lockfiles which alleviates the need for users to run with --delete-lockfiles after an abrupt shutdown.
• Adds the --eth1-endpoints flags which alllows for redundant eth1 nodes.
• Adds the --metrics flag (and other related flags) to lighthouse vc, allowing for Prometheus metrics.
  • Example dashboard found here: https://github.com/sigp/lighthouse-metrics/blob/master/dashboards/ValidatorClient.json

All Changes

• Fallback nodes for eth1 access (#1918)
• Disable snappy in LevelDB to fix build issues (#1983)
• Log better error message (#1981)
• Use OS file locks in validator client (#1958)
• Sync metrics (#1975)
• Add metrics to VC (#1954)
• Fix #1964: remove mainnet warnings which no longer apply (#1970)
• Update ENR construction and mainnet bootnodes (#1968)
• FAQ/Doc updates (#1966)
• Update docs whenever unstable changes (#1969)
• Status race condition (#1967)

DockerHub

• sigp/lighthouse:v1.0.1

Binaries

See pre-built binaries documentation.

This release marks Lighthouse transitioning into a production/stable development phase.

We recommend all users to upgrade to v1.0.0, especially if they're intending to stake with a genesis validator.

Summary

• The mainnet genesis state has been included in the lighthouse binary.
• The --testnet flag has been replaced with --network
  • The default network has been changed from medalla to mainnet
• The slasher has been added (see the docs)
• Many other bug-fixes and improvements

All Changes

• Bump to v1.0.0, run cargo update
• Add mainnet genesis state (#1959)
• Remove experimental software warning (#1957)
• Update security details in README (#1956)
• Refine and test slashing protection semantics (#1885)
• Add Prysm and Teku boot nodes (#1953)
• Update validator guide for mainnet (#1951)
• check if the slashing protection database is locked before creating keys (#1949)
• Sync Bug fixes (#1950)
• Change --testnet flag to --network (#1751)
• Tweak slasher DB schema and pruning (#1948)
• Implement slasher (#1567)
• Improve UX whilst VC is waiting for genesis (#1915)
• Add flag to import all attestations (#1941)
• move sync state to the chains KV (#1940)
• Fix race condition in seen caches (#1937)
• Fix metrics http server error messages (#1946)
• Expand eth1 block cache, add more logs (#1938)
• Ensure deposit signatures do not use aggregate functions (#1935)
• Move some rpc processing to the beacon_processor (#1936)
• Add additional libp2p tests (#1867)
• Document system requirements (#1934)
• Add validation to kdf parameters (#1930)
• [Remote signer] Add signer consumer lib (#1763)
• Reject invalid utf-8 characters during encryption (#1928)
• Update pool/attestations and committees endpoints (#1899)
• Avoid string initialization in network metrics and replace by &str where possible (#1898)
• Add new flag to set blocks per eth1 query (#1931)
• Add lighthouse bootnodes (#1929)

DockerHub

• sigp/lighthouse:v1.0.0

Binaries

See pre-built binaries documentation.

This release is recommended for all users. Due to API updates, it's essential that you restart both the beacon node and validator client after updating.

Summary

• The --testnet pyrmont flag has been added, providing full support for the Prymont testnet.
• A memory leak has been fixed in #1923.
• Two PRs (#1922, #1921) reduce memory footprint by reducing allocations likely to cause fragmentation.
• Database compaction is run less frequently in #1916, reducing disk I/O and therefore prolonging disk life.

All Changes

• Address queue congestion in migrator (#1923)
• Refine compaction (#1916)
• Avoid some allocations in BlockSignatureVerifier (#1922)
• Avoid allocations on VariableList (#1921)
• router: drop requests from peers that have dc'd (#1919)
• Validate eth1 chain id (#1877)
• Added fn to count unicode characters (#1903)
• Performance improvement for db reads (#1909)
• Add link to Lighthouse mailing list (#1913)
• Check whistle-blower index (#1911)
• Add Pyrmont testnet (#1904)
• Update logs + do not downscore peers if WE time out (#1901)
• add slot validation to attestation_data endpoint (#1888)
• Misc Peer sync info adjustments (#1896)

DockerHub

• sigp/lighthouse:v0.3.5

Binaries

See pre-built binaries documentation.

This release provides many incremental improvements to networking, storage and the HTTP API as we prepare for mainnet.

It is recommended for all users running on the Medalla testnet. Due to API updates, it's essential that you restart both the beacon node and validator client after updating.

Summary

• The pruning of the database has been improved through the use of a compaction pass. If you've noticed your Medalla datadir grow beyond 80-90GB, this release will reduce it back down to a comfortable level around 10-15GB. No resync is required (#1871).
• We have merged Gossipsub scoring, which hardens the networking layer against malicious peers. This has been the result of many weeks of research, and will be essential for network stability once mainnet launches (#1668).
• We have updated the HTTP API to align more closely with the evolving standard. The revised standard allowed us to fix a regression in validator client performance with large numbers of validators (100+) (#1831, #1828).
• Lighthouse now includes a --testnet mainnet flag, for monitoring of the deposit contract before genesis. With your beacon node connected to a mainnet Eth1 node you can watch the deposits stream in 🎉 Although this is our first release with some mainnet compatibility, it is not intended for making mainnet deposits or running a validator on mainnet. We will ship a v1.0.0 release closer to genesis, as described in our v1.0.0 release schedule here: https://lighthouse.sigmaprime.io/update-32.html

All Changes

• handle peer state transitions on gossipsub score changes + refactoring (#1892)
• Peer endpoint updates (#1893)
• do subnet discoveries until we have MESH_N_LOW many peers (#1886)
• Update EF tests to 1.0.0 (#1875)
• Gossipsub scoring (#1668)
• Update tiny-bip39 dependency (#1887)
• Add toledo support (#1874)
• Update vc testnet script (#1859)
• Standard beacon api updates (#1831)
• Compact database on finalization (#1871)
• Improve validator key cache lock handling (#1837)
• Add --testnet mainnet and start HTTP server before genesis (#1862)
• Update slashing protection interchange to v5 (#1816)
• Seen addresses store port (#1841)
• Remove mention of OpenSSL from documentation (#1844)
• Changed http:// to https:// on some links (#1869)
• Update remote signer README (#1870)
• Update README.md (#1868)
• [Remote signer] Fold signer into Lighthouse repository (#1852)
• Update libp2p (#1865)
• Fix fn documentation
• Add warnings for deposits (#1858)
• Ignore RPC messages of disconnected peers and remove old peers based on disconnection time (#1854)
• Prevent errors for stream termination race (#1853)

DockerHub

• sigp/lighthouse:v0.3.4

Binaries

See pre-built binaries documentation.

This release includes numerous networking stability improvements and fixes, and new HTTP API features.

Critically, it fixes an issue with the included bootnode addresses that prevented nodes from finding peers when starting a beacon node from scratch on v0.3.2 (#1849).

It also adds a long-awaited command for submitting voluntary exits, documented here: https://lighthouse-book.sigmaprime.io/voluntary-exit.html

Finally, if you run on a VPS and had trouble with the optimized v0.3.2 build, it might be worth trying again, as we've fixed a false positive in our detection of CPU features (#1846).

Due to the API changes, it is essential to restart both the validator client and beacon node after upgrading.

Changes: Summary

• fix unbanning of peers (#1838)
• More sync edge cases + prettify range (#1834)
• Beacon state validator id filter (#1803)
• Only run http_api tests in release (#1827)
• Add cli option for voluntary exits (#1781)
• add quoted serialization util for FixedVector and VariableList (#1794)
• Tweak head syncing (#1845)
• Return eth1-related data via the API (#1797)
• Update to discv5 bootnodes (#1849)
• Downgrade ADX check to a warning (#1846)

DockerHub

• sigp/lighthouse:v0.3.3

Binaries

See pre-built binaries documentation.

We recommend that all users upgrade their software to this latest release as it contains important interoperability improvements.

Changes: Summary

• Update to spec v1.0.0-rc.0 (#1765)
• Inform peers of requests that exceed the maximum rate limit + log downgrade (#1830)
• Ensure eth1 deposit/chain IDs are used from YamlConfig (#1829)
• Address clippy lints, panic in ssz_derive on overflow (#1714)
• Update testnet configs, change on-disk format (#1799)

DockerHub

• sigp/lighthouse:v0.3.2

Binaries

See pre-built binaries documentation.

This release provides breaking changes in the networking layer and in the db schema.

This version upgrades the discovery mechanism of Lighthouse to Discovery v5.1. This means Lighthouse will not be able to discover peers with previous versions of Lighthouse. There has also been a DB schema update. Running v0.3.1 on an older database will cause Lighthouse to migrate the older db to the new version. Users will not be able to downgrade their db after this.

We encourage all Medalla users to upgrade to this version to participate in the new networking changes and to improve peer discovery.

Changes: Summary

• Improved RPC validation (#1117)
• Improved docs and various bug fixes (#1756, #1759, #1761, #1760, #1762, #1747, #1800)
• Update the gossipsub message-id to latest specification (#1752)
• Add gossipsub topic filters (#1767)
• Correct issues and improve sync logic (#1791, #1804, #1796)
• Allow the Validator Client to start without validators (#1779)
• Implement further node endpoints (#1778)
• Reduce memory usage (#1798)
• Optimise attester slashing (#1745)
• Update to discv5.1 (#1786)
• Update libp2p (#1819)

DockerHub

• sigp/lighthouse:v0.3.1

Binaries

Binaries are supplied for two platforms:

• x86_64-unknown-linux-gnu: AMD/Intel 64-bit processors (most desktops, laptops, servers)
• aarch64-unknown-linux-gnu: 64bit ARM processors (Raspberry Pi 4)

Additionally there is also a -portable suffix which indicates if the portable feature is used:

• Without portable: uses modern CPU instructions to provide the fastest signature verification times (may cause Illegal instruction error on older CPUs)
• With portable: approx. 20% slower, but should work on all modern 64-bit processors.

Note: we do not yet provide binaries for MacOS or Windows native.

This is a major release which contains breaking features. The breaking features require users to perform:

• A resync of the database, since databases from v0.2.x are incompatible v0.3.0
• Some data directory migration for existing setups (see below)

Data Directory Migration

The directory structure has changed, such that the ~/.lighthouse dir (AKA "datadir") now contains one sub-directory for each --testnet value. What was previously ~/.lighthouse/beacon is now ~/.lighthouse/medalla/beacon or ~/.lighthouse/altona/beacon.

If you have an existing ~/.lighthouse directory you will need to move it into a sub-directory that matches the testnet name.

Example

Assume there is an existing ~/.lighthouse directory for Medalla:

.lighthouse
  |- beacon
  |- validators
  |- secrets

The directory should be re-arranged as such:

.lighthouse
  |- medalla
      |- beacon
      |- validators
      |- secrets

Note: you will likely need to edit the validators/validator_definitions.yml file to reflect the additional directory nesting.

Changes: Summary

Breaking Features

• The previous HTTP API has been replaced with a partially-complete standard Eth2.0-API implementation. See which endpoints are implemented at https://github.com/sigp/lighthouse/issues/1434.
• Prometheus metrics are no longer served on the same port as the HTTP API (default 5052). Metrics must be enabled with the --metrics flag and their address/port/CORS settings are managed separately to the API.
• The ~/.lighthouse directory structure has changed (mentioned earlier in the release notes).
• Lockfiles on EIP-2335 keypairs are now strictly enforced. The --delete-lockfiles flag can be used to recover from a scenario where the validator client detects existing lockfiles from a previous crash. Use --delete-lockfiles with caution and read the CLI help first.
• The Metamask deposit functionality has been removed from the Lighthouse docs. We direct users to the Eth2 launch pad instead.
• The slashing protection database is now stricter by default. The validator client will refuse to start if it finds validator keys without a corresponding database. When migrating your datadir to the new layout you should ensure you move slashing_protection.sqlite to ~/.lighthouse/medalla/validators/slashing_protection.sqlite (or $datadir/validators/slashing_protection.sqlite). More information here: https://lighthouse-book.sigmaprime.io/slashing-protection.html#misplaced-slashing-database
• Use 24-word mnemonics by default, with options for other lengths.

Non-Breaking

• Add the --testnet zinken option, with full support for the upcoming testnet.
• The --wss-checkpoint flag has been added to protect against weak subjectivity attacks. Users are not expected to require this flag until after mainnet launch.
• There is now a mechanism for caching BLS keys obtained from EIP-2335 keystores, to prevent long wait times when starting the VC. This cache should be transparent to users in most cases.
• Add a RESTful HTTP API to the VC.
• Add UPnP networking support.
• Support for the slashing interchange format.
• Various security fixes.
• Various stability fixes.

Changes: Detail

• (origin/staging) Strict slashing protection by default (#1750)
• Bump version to v0.3.0 (#1743)
• Update docs for v0.3.0 (#1742)
• Validator dir creation (#1746)
• Add zinken testnet (#1741)
• Improve command help (#1740)
• Implement key cache to reduce keystore loading times for validator_client (#1695)
• Clean up obsolete TODOs (#1734)
• Update external deps (#1711)
• Upgrade discovery and restructure task execution (#1693)
• Improve error handling in network processing (#1654)
• From panic to crit (#1726)
• Document need for port 9000 to be open (fix #730) (#731)
• Increase content-id length (#1725)
• Sync fixes (#1716)
• Use Drop impl to send worker idle message (#1718)
• Update libp2p (#1728)
• Tidy some TODOs (#1721)
• Address a couple of TODOs (#1724)
• Add check for head/target consistency (#1702)
• Add UPnP support for Lighthouse (#1587)
• Wallet creation: Make mnemonic length configurable, default to 24 words. (#1697)
• Update tiny-bip39 dependency to one implementing zeroize (#1701)
• Weak subjectivity start from genesis (#1675)
• Implement VC API (#1657)
• Implement slashing protection interchange format (#1544)
• Add database schema versioning (#1688)
• Implement standard eth2.0 API (#1569)
• Directory restructure (#1532)
• Fix validator lockfiles (#1586)
• Update key derivation to latest EIP-2333 (#1633)
• Update Cargo.lock (#1735)
• Improve error handling in network processing (#1654)
• From panic to crit (#1726)
• Document need for port 9000 to be open (fix #730) (#731)
• Increase content-id length (#1725)
• Allow truncation of pubkey cache on creation (#1686)
• Remove macos tests (#1687)

Dockerhub

• sigp/lighthouse:v0.3.0

Binaries

Binaries are supplied for two platforms:

• x86_64-unknown-linux-gnu: AMD/Intel 64-bit processors (most desktops, laptops, servers)
• aarch64-unknown-linux-gnu: 64bit ARM processors (Raspberry Pi 4)

Additionally there is also a -portable suffix which indicates if the portable feature is used:

• Without portable: uses modern CPU instructions to provide the fastest signature verification times (may cause Illegal instruction error on older CPUs)
• With portable: approx. 20% slower, but should work on all modern 64-bit processors.

Note: we do not yet provide binaries for MacOS or Windows native.

This release updates Lighthouse for the upcoming Spadina testnet launch.

• Adds the Spadina genesis state and associated boot nodes.
• Updates discovery (improves NAT filtering in the DHT).
• Network bug fixes related to peer management.

Changes

• https://github.com/sigp/lighthouse/commit/1ef4f0ea12f76ccf775a41ff0012c2d14f1044d8 Updates gossipsub attestation verification (#1667)
• https://github.com/sigp/lighthouse/commit/7bf14908dc37415c4f3bb7e7a7bac20432566523 Adds Spadina genesis file and extended Spadina bootnodes (#1681)
• https://github.com/sigp/lighthouse/commit/6c1d7f55bfdb30bb8dbad623388a2d11305779ec Update Lighthouse Spadina bootnode (#1685)
• https://github.com/sigp/lighthouse/commit/ae287739659c5dd4101691cf674f6886fab5113b Network bug fixes (#1684)

Dockerhub

• sigp/lighthouse:v0.2.13

Binaries

Binaries are supplied for two platforms:

• x86_64-unknown-linux-gnu: AMD/Intel 64-bit processors (most desktops, laptops, servers)
• aarch64-unknown-linux-gnu: 64bit ARM processors (Raspberry Pi 4)

Additionally there is also a -portable suffix which indicates if the portable feature is used:

• Without portable: uses modern CPU instructions to provide the fastest signature verification times (may cause Illegal instruction error on older CPUs)
• With portable: approx. 20% slower, but should work on all modern 64-bit processors.

Note: we do not yet provide binaries for MacOS or Windows native.

This release contains new features and a fix to an error introduced in the previous release (v0.2.11).

• Adds support for the --testnet spadina flag.
• Reverts a change that caused "Illegal instruction" on ARM processors.
• Adds a --staking flag to the beacon node which enables the API and Eth1 syncing.
• Allow for interactive password entry on all account manager flows.

Changes

• 5688f21bb Spadina support (v2) (#1670)
• 258b28469 Update consensus code and tests to v0.12.3 (#1655)
• 3412a3ec5 Remove saturating arith from state_processing (#1644)
• 28b6d921c Remove banned peers from DHT and track IPs (#1656)
• 15638d144 Beacon node does not quit on eth1 errors (#1663)
• b8013b7b2 Super Silky Smooth Syncs, like a Sir (#1628)
• 80e52a026 Subscribe to core topics after sync (#1613)
• 80ecafaae Add --staking flag (#1641)
• b75df2950 minimize the number of places we are calling update_pubkey_cache (#1626)
• 1801dd1a3 Interactive account passwords (#1623)
• 62c8548ed Revert "Update BLST, add force-adx support (#1595)" (#1649)
• a97ec318c Subscribe to subnets an epoch in advance (#1600)
• 7aceff4d1 Add safe_sum and use it in state_processing (#1620)
• 4fca30639 Update BLST, add force-adx support (#1595)

Dockerhub

• sigp/lighthouse:v0.2.12

Binaries

Binaries are supplied for two platforms:

• x86_64-unknown-linux-gnu: AMD/Intel 64-bit processors (most desktops, laptops, servers)
• aarch64-unknown-linux-gnu: 64bit ARM processors (Raspberry Pi 4)

Additionally there is also a -portable suffix which indicates if the portable feature is used:

• Without portable: uses modern CPU instructions to provide the fastest signature verification times (may cause Illegal instruction error on older CPUs)
• With portable: approx. 20% slower, but should work on all modern 64-bit processors.

Note: we do not yet provide binaries for MacOS or Windows native.

Solves an issue which caused the beacon node to resume from the database using a canonical head that is no longer consistent with fork choice. The canonical head is now derived from fork choice when resuming from DB.

Also adds the new "trusted peers" feature (#1640).

Changes

• bd39cc8e2 Apply hotfix for inconsistent head (#1639)
• 14ff38539 Add trusted peers (#1640)

Binaries

Binaries are supplied for two platforms:

• x86_64-unknown-linux-gnu: AMD/Intel 64-bit processors (most desktops, laptops, servers)
• aarch64-unknown-linux-gnu: 64bit ARM processors (Raspberry Pi 4)

Additionally there is also a -portable suffix which indicates if the portable feature is used:

• Without portable: uses modern CPU instructions to provide the fastest signature verification times (may cause Illegal instruction error on older CPUs)
• With portable: approx. 20% slower, but should work on all modern 64-bit processors.

Solves an issue which caused some nodes to stall during sync.

Changes

• a17f74896 Fix bad assumption when checking finalized descendant (#1629)
  • Fixes a "batch failed" error during syncing.
• 49ab41459 Shift gossipsub validation (#1612)
• 2074beccd Gossipsub message id to shortened bytes (#1607)
• e5fc6bab4 Remove redundant decompression in process_deposit (#1610)
• c9596fcf0 Temporary Sync Work-Around (#1615)
• c6abc5611 Prevent large step-size parameters (#1583)
• 7f1b93690 ignore too early / too late attestations instead of penalizing them (#1608)
• 810de2f8b Static testnet configs (#1603)
  • Solves issues downloading github files experienced by users behind the GFW. Thanks @danielschonfeld.
• 052587688 Dial cached enr's before making subnet discovery query (#1376)
• d79366c50 Prevent printing binary in RPC errors (#1604)
• b19cf02d2 Penalise bad peer behaviour (#1602)
• dfe507715 Remove references to rust-docs (#1601)

Binaries

Binaries are supplied for two platforms:

• x86_64-unknown-linux-gnu: AMD/Intel 64-bit processors (most desktops, laptops, servers)
• aarch64-unknown-linux-gnu: 64bit ARM processors (Raspberry Pi 4)

Additionally there is also a -portable suffix which indicates if the portable feature is used:

• Without portable: uses modern CPU instructions to provide the fastest signature verification times (may cause Illegal instruction error on older CPUs)
• With portable: approx. 20% slower, but should work on all modern 64-bit processors.

This release contains new user facing functionality (key recovery) and fixes an issue which cause validators to produce attestations which would not be included on chain. We have observed significant improvement in rewards using this release.

Changes

• Mnemonic key recovery (#1579)
  • Allows users to recover validator keys from mnemonics, thanks to @realbigsean
• Update state before producing attestation (#1596)
  • Critical bug fix by @pawanjay176 which improves validator attestation inclusion.
• Don't quote slot and epoch, for now (#1597)
• Use simple logger builder pattern (#1594)
• conforming to the p2p specs, requiring error_messages to be bound (#1593)
• Update discv5 to v10 (#1592)
• Add serde_utils module with quoted u64 support (#1588)
• Revert "add a github action for build multi-arch docker images (#1574)" (#1591)
• Avoid Printing Binary String to Logs (#1576)
• add a github action for build multi-arch docker images (#1574)
• Allow to use the same password when importing multiple keystores (#1479) (#1510)
  • Improves UX for importing from EF launchpad, thanks to @adetante
• Minor documentation fixes (#1297)
• Extended Gossipsub metrics (#1577)
• Revert 1502 - Switching docker user to lighthouse (#1578)
• Creates a new lighthouse user and makes it the default user to be use… (#1502)
• Increase logging channel capacity (#1570)
• Eth1 network exit on wrong network id (#1563)
• Use Gossipsub 1.1 (#1516)
  • Major networking milestone, this protocol will be used for mainnet. Congrats to @AgeManning and @blacktemplar.
• fix change flag name end_after_checks to continue_after_checks (#1573)

Binaries

Binaries are supplied for two platforms:

• x86_64-unknown-linux-gnu: AMD/Intel 64-bit processors (most desktops, laptops, servers)
• aarch64-unknown-linux-gnu: 64bit ARM processors (Raspberry Pi 4)

Additionally there is also a -portable suffix which indicates if the portable feature is used:

• Without portable: uses modern CPU instructions to provide the fastest signature verification times (may cause Illegal instruction error on older CPUs)
• With portable: approx. 20% slower, but should work on all modern 64-bit processors.