The NFT Geyser plugin that powers metaplex APIs
AGPL-3.0 License
This repo houses a validator plugin that is a key part of the Metaplex's Digital Asset RPC API. It is responsible for getting information out of the Solana validator and sending it to a message bus in a binary format. While this component was built to serve the APIs, it was designed to allow any message bus tech to be used. That being said, it can be used for many use cases.
Solana 1.10.41 or greater is required. Your calling components must support V2 GeyserPlugin types
It is built on the following principles.
If you are building the Metaplex RPC API infrastructure please follow the instructions in Metaplex RPC API infrastructure.
If you are using this plugin for your bespoke use case then the build steps are below.
NOTE -> M1 macs may have issues. Linux is best.
cargo build
for debug or
cargo build --release
for a release build.
You will now have a libplerkle.so file in the target folder. This is the binary that you will pass into the validator using the following option.
--geyser-plugin-config plugin-config.json
The plugin config for plerkle must have this format, but you can put whatever keys you want
EXAMPLE PLEASE DONT CONSIDER THIS THE PERFECT CONFIG
{
"libpath": "/.../libplerkle.so",
"enable_metrics": false,
"env": "local",
"handle_startup": true, // set to false if you dont want initial account flush
"accounts_selector": {
"owners": [
"metaqbxxUerdq28cj1RbAWkYQm3ybzjb6a8bt518x1s"
]
},
"transaction_selector": {
"mentions": [
"BGUMAp9Gq7iTEuizy4pqaxsTyUCBK68MDfK752saRPUY"
]
}
}
This config file points to where your plugin library file is, and what programs it is listening to. This is the standard Solana geyser plugin config file that the validator reads.
There are some other bits of configuration needed. Environment Variables. The process running the validator must have access to environment variables. Those variables are as follows
RUST_LOG=warn
PLUGIN_MESSENGER_CONFIG='{ messenger_type="Redis", connection_config={ redis_connection_str="redis://redis" } }'
The PLUGIN_MESSENGER_CONFIG determines which compiled messenger to select and a specific configuration for the messenger.
Producer Configuration
Lower Scale Low network latency
PLUGIN_MESSENGER_CONFIG='{pipeline_size_bytes=1000000,local_buffer_max_window=10, messenger_type="Redis", connection_config={ redis_connection_str="redis://redis" } }'
High Scale Higher latency
PLUGIN_MESSENGER_CONFIG='{pipeline_size_bytes=50000000,local_buffer_max_window=500, messenger_type="Redis", connection_config={ redis_connection_str="redis://redis" } }'
Consumer Configuration
PLUGIN_MESSENGER_CONFIG='{batch_size=1000, message_wait_timeout=5, retries=5, consumer_id="random_string",messenger_type="Redis", connection_config={ redis_connection_str="redis://redis" } }'
PLUGIN_ACCOUNT_STREAM_SIZE=250000000
PLUGIN_SLOT_STREAM_SIZE=250000
PLUGIN_TRANSACTION_STREAM_SIZE=25000000
PLUGIN_BLOCK_STREAM_SIZE=250000
NOTE: in 1.4.0 we are not sending to slot status.
The plugin exposes the following statsd metrics
This repo contains a docker file that allows you to run and test the plerkle plugin using a test validator.
To test it you can build the container withdocker compose build
and run it with docker compose up
.
You will want to change the programs you are listening to in ./docker/runs.sh
. Once you spin up the validator send your transactions to the docker host as you would a normal RPC.
Any program .so files you add to the /so/ file upon running the docker compose system will be added to the local validator.
You need to name the so file what you want the public key to be:
metaqbxxUerdq28cj1RbAWkYQm3ybzjb6a8bt518x1s.so
This is because of this line in the docker-compose.yml
file.
- ./programs:/so/:ro
You can comment this out if you dont want it.
You can run ./docker/build.sh
to create a build of the geyser plugin and solana, then output the artifacts
to your local filesystem so you can run the validator.
Note that differences in image distro could cause incompatible GLibc versions.
NOTE WE DO NOT PUBLISH THE PLUGIN ANY MORE:
plerkle_messenger-https://crates.io/crates/plerkle_messenger plerkle_serialization-https://crates.io/crates/plerkle_serialization