chainquery

Chainquery parses and syncs the LBRY blockchain data into structured SQL

MIT License

Stars
2.3K
Committers
11

Bot releases are hidden (Show)

chainquery - v2.0.4

Published by tiger5226 almost 5 years ago

Changelog

19adcb3 Add Transactions to ChainSync job Fixed bug where raw column for transactions were not being saved.
94273d7 Add recovery from panic in chainsync.
344e9b9 Fix bug where we double deploy due to notification for the same commit.

chainquery - v2.0.3

Published by tiger5226 almost 5 years ago

Changelog

a057c86 Only log errors when its an app error. Otherwise just debug the output.

chainquery - v2.0.2

Published by tiger5226 almost 5 years ago

Changelog

fe59c26 Added timeout for sql api so there are no long running queries against the database. Sharing is caring.
cc380a1 Fix import cycle in test files

chainquery - v2.0.1

Published by tiger5226 almost 5 years ago

Changelog

8f07337 Added ChainSync job that synchronizes automatically. It will go block by block and make sure it is in sync with lbrycrd. This will add block data synchronization. There are two new config parameters for this job. See the default config file. Added two flags for debug and trace mode so it can be set as a flag without modifying the config file. This is useful for ide debugging.
79b3014 Fix Linting errors.

chainquery - v2.0.0

Published by tiger5226 almost 5 years ago

Changelog

002d389 Add command to process specific block only.
73fa89a Add comment to ProcessBlock
13dbdba Add index to input table to make certificate sync Adjust query to take advantage of the index
7bb1d96 FIX gofmt & golint violations
077a4c2 FIX issue where its possible that claims can be created without a name.
8080f57 FIX update spent claims bug where nothing gets updated FIX bug where claim_id was never getting set on the output when processed ADD run command to run individual chainquery jobs instead of the whole application ADD new columns to track against the latest utxo applied to a claim CHANGE name of transaction value sync function name for alignment CHANGE owner of claims to be the first output that creates it instead of the latest output that updates it CHANGE claim trie sync to consider the latest utxo for whether its spent
4adad85 Fix certificate sync to use the latest transaction_hash so it doesn't fail on updated claims.
bebf9bd If creating a tag fails, don't fail processing. Log and continue.
d2a063f Increase blocks generated to 800 so it includes seqwit tx at block 600 Add waiting for blocks in e2e test ( processing )
38035d6 Merge pull request #134 from lbryio/claim_ownership
217114f Put defer into its own function.
76c6d97 Update e2e test to use docker build with deprecated RPCs
cf6fff4 fix flag reset

chainquery - v1.9.0

Published by tiger5226 almost 5 years ago

Changelog

1822c16 ADD ability to stop process during transaction syncing instead of after for a faster shutdown.
8cb6cb4 ADD configurable profiling mechanism for environments. FIX all panics in the datastore.
a4ab4a2 Add block height debug logging for every block Fix concurrency, to continue with channel queue when error is hit and return the first error when done.
00c5282 Add data folder to git ignore since its used to hold lbrycrd data
c478f66 Add goimports to travis for build.sh
f5a65f7 Add handling of non standard scripts that are not claim scripts
3f5530d Add segwit support
e06eeb2 Add special handling of genesis block since v17 doesn't allow using getrawtransaction on the genesis block
d271ca1 Add support for nulldata output script type
f9eae67 Add support for nulldata output script type Fix concurrency bug Remove processing of genesis block (v0.17.2.1) Add error tracing Update e2e testing
b0712f8 Add trace to errors during e2e test Add docker-compose or build to be local Add
1dee1b5 Additional panic removals
557d991 Adjustments for go mods and fix for client.
260f408 FIX bug hanging the syncing process.
1ad1962 FIX nometadata argument removal
ff10443 Fix bug where sdhash is greater than 120 characters.
ff6a7c0 Fix panic by returning error so the app doesnt crash.
f0f89b5 Fix panic by returning error so the app doesnt crash.
96421bb Fixed bindata for travis build
da1309f Fixed travis build
b47c48d Include MySQL version number
325508b Merge branch 'column_size_fix'
8e7edd2 Merge branch 'master' of https://github.com/lbryio/chainquery
fbc8e5f Merge branch 'master' of https://github.com/lbryio/chainquery
c7d79a8 Merge branch 'master' of https://github.com/lbryio/chainquery
f2c191a Merge pull request #123 from AlessandroSpallina/master
664a65d Merge pull request #125 from lbryio/fix_lbrycrd_client
b471665 Merge pull request #126 from lbryio/fix_testnet
2214d9c Merge pull request #129 from StrikerRUS/patch-1
ce68eb2 Merge pull request #131 from lbryio/cleanup
efa6bca Merge pull request #133 from lbryio/align-lbrycrd
183decd Remove panic from outpoint.go for missing address
2a0645e Resolve error from unhandled get call for datastore.
847aa29 UPDATE e2e test to use lbrycrd v0.17.3.1 UPDATE readme to have contributors run the e2e test
d27f6f4 UPDATE remove bindata.go from repo since its auto generated
77010f7 Update api generation
63a756a Update btcd libraries to avoid hardcoded chain parameters Update bindata library
2b74dfd Update build binary to be executable.
120ef4d Update daemon to only run jobs when the chain is caught up.
ab5ff48 Update daemon to only run jobs when the chain is caught up.
0e14593 Update test.sh to include additional packages for testing
7da0287 Update test.sh to include additional packages for testing
ae6dfcf Update travis build to not change go.mod
c2ba24d Update travis to remove go gets that should only be in the lint.sh
e17012b align structures to lbrycrd
8131544 bump year in license
2ea85b3 fix columns
c9cd89a fix transaction hashes column
e747264 fix wrong cut and paste in cmd description
282a4a5 fixed bug where it gets stuck at height zero
c86a416 init gomod
1b95b3e return "ERROR" as bidstate when there is no associated output.
f2389e2 return "ERROR" as bidstate when there is no associated output.
e2cc63b try updating go mod

chainquery - v1.8.1

Published by tiger5226 over 5 years ago

Changelog

b4d0821 fix tags formatting
927654d tags hotfix

chainquery - v1.8.0

Published by tiger5226 over 5 years ago

Changelog

424cd86 Added auto-update script template for users to auto-update their chainquery instance based on the master branch.
c8fb7c7 Always sync claim regardless of spend status since mempool continuously processes transactions which could be an update.
d7ea2dc Merge branch 'master' of https://github.com/lbryio/chainquery
f40d944 Merge pull request #112 from lbryio/new_metadata
14953bd Merge pull request #113 from lbryio/build-lbry-io
fcf5034 Merge pull request #114 from lbryio/test_deploy
1738733 Merge pull request #115 from lbryio/fix_claim_updates
f799439 Merge pull request #116 from lbryio/fix_sqlboiler
e24b787 Update sqlboiler model files based on latest version.
44237fe added build file with commit hash in it for AWS bucket
b5ee777 added chainquery.txt file to the build artifacts
b1dab7c added new metadata columns,upgraded lbryschema.go
84996c5 added support to publish linux build to build.lbry.io
46b8f2b move into two providers to get two sets. upgraded go
46a7832 remove all upserts as its not supported in sqlboiler for mysql. Added Tag and ClaimTag to datastore operations to handle upsert more easily.
d6c2bd0 remove artifact of travis-artifacts
c99f0d3 update lbryschema.go revision due to bug in sdk that allowed a fee to exist without an address in the metadata. This removes address validation from the decoding process.
0dfd359 updated master for latest of sqlboiler
d24ca8b upsert instead of inserting on relation set. If its already there processing will get stuck.

chainquery - v1.7.2

Published by tiger5226 over 5 years ago

Changelog

7e5851b add persist folder for e2e testing to gitignore
b5e9db6 update value passed into sign function due to https://github.com/lbryio/lbry/commit/90bef98bc3e814f412943a6ad2be9cc4a974ae41#diff-43d431f99630ca3e6208e2671256abd0
a9e234d updated e2e testing you use auto build docker images from lbry-docker
68e839c use latest version of lbryschema.go to fix proto serialization issue.
2af62ec user master branch

chainquery - v1.7.1

Published by tiger5226 over 5 years ago

Changelog

979dc3a removed testing logic around block heights

chainquery - v1.7.0

Published by tiger5226 over 5 years ago

Changelog

6f703ed -Migrated chainquery from lbry.io to lbry.com
7abff9e -fixed missing setting of json value. -fixed place where we were setting cert processed to false incorrectly.
5a6c728 Added e2e tests for proto3 support.
2068b46 Merge pull request #109 from ykris45/patch-1
94a8ea8 Merge pull request #110 from lbryio/proto3_support
c1de50e Merge pull request #111 from lbryio/proto3_support
364fc4a Update README.md
680d76d fix linting error
aad1cc6 fixed bindata
1a1c0b7 upgrade to latest version of lbry.go, nlopes, and pkg/errors
4c024f2 upgrade to latest version of lbryschema.go

chainquery - v1.6.3

Published by tiger5226 over 5 years ago

Changelog

84f875b -Added block chain name as a configurable parameter for Chainquery.
e70daa5 -Added check for 0 blockheight before setting claim height.
40a2125 -Fixed bug where timestamps for changes to an output would not be found when checking for spent outputs.
007bb78 -fixed bug where script parsing for nonstandard transaction addresses were hardcoded to the mainnet.
7e36602 -fixed init of job status bug
a940c04 -fixed json bug when starting from scratch.

chainquery - v1.6.2

Published by tiger5226 over 5 years ago

Changelog

f5bcc9a -Added new api for syncing a specific claimname.
037ca9d -Fixed bug where if we process a claim name and then a claim update but a claimtrie sync happens in between, it was getting marked spent which prevent future updates from getting "other than spent" bid states.
a9444be -Fixed bug with mempool cleanup where it was deleting tx's before they blocks that removed them from the mempool were processed. Changed it to only consider removing mempool transactions that are older than 1 hr ( Stale ).

chainquery - v1.6.1

Published by tiger5226 over 5 years ago

Changelog

085c293 -Fixed bug where claims were decoding from a hardcoded blockchain name instead of the global configurable variable.
6551528 -set the address list after the fact on non-standard transactions since lbrycrd does not populate it.
683eda3 -turn off debugging of claimtrie sync.

chainquery - v1.6.0

Published by tiger5226 over 5 years ago

Changelog

ec6d65e -Added claims to e2e test.
714109a -Added documentation to important variable.
18a4154 -Added flushing of mempool capability if a tx is no longer in the mempool. This solves the issue where a doublespend never gets put in a block and gets invalidated from the mempool and removed.
b7ddf9a -Added logic to sync claims that have passed their valid at height since the last claim trie sync. This fixes the takeover bug where the state never gets updated. -Added JSON state column to job status to store state specifics for each job.
7d95cc9 -Added maxfailures as a live configurable parameter.
53b86aa -add no metadata back.
d127196 -corrected lint naming issue.
379aab3 -reduced code complexity of mempool job function
b82bc8b Merge pull request #98 from lbryio/takeovers
de3ae25 Merge pull request #99 from lbryio/double_spend

chainquery - v1.5.1

Published by tiger5226 over 5 years ago

Changelog

0825f85 incremented data version to reprocess all claims.
70992ca made processing all claims a background process for upgrade.

chainquery - v1.5.0

Published by tiger5226 over 5 years ago

Changelog

430e212 - Upgraded SQLBoiler to 3.2.0 - Added License,LicenseURL, and Preview Metadata columns - Fixed bug where we were not setting NSFW column - Fixed bug where we were not reset thing fields during a claim update - Added data migration to reprocess all claims
843d671 -moved commands to cobra library -added e2e test functionality and script
6c310a5 Added claim to e2e testing with regtest Adjusted e2e to use master branch of lbry-docker
87896ab Change default config
5f50099 Merge branch 'master' into default-changes
b47c885 Merge pull request #71 from lbryio/default-changes
0cf2ff4 Merge pull request #82 from lbryio/tooling
360a0e5 Merge pull request #84 from lbryio/fix_value_sync
4374efe Merge pull request #87 from lbryio/metadata_columns
eaebb5f Merge pull request #92 from lbryio/metadata_columns
913b973 fixed column ordering for ci
7f05c89 fixed linting.
e3efd37 fixed startup bug that causes error at block 0
111736c update suggestion
82bab18 value sync was diffing credits/debits which are the fees. We just want the value of the transaction.

chainquery - v1.4.0

Published by tiger5226 almost 6 years ago

Changelog

3a1e8cd -Added missing upsert -Fixed bug where setting SendReceive for TA would panic instead of returning error. -Modified/Added triggers to simplify them for performance.
8151ce3 -Added regtest for chainquery startup.
81e34f6 -Added usage of stopper debugging. -Added address.balance & transaction.value fields -Added triggers in migration to calculate those fields on specific changes. -Added migration to calculate the fields for prior data. -Updated lock file for SQLBoiler V3 and lbry.go update for stopper debugging.
5b6a4c1 -Added validation api to validate specific blocks or a range of blocks. -Added processing api to re-process specific blocks or a range of blocks. -Added validation job to run daily to make sure the chain data is up to date. -Removed initial run of all jobs. -Added authorization for apis. -Added live configuration for keys.
9d68190 -Fixed bug where we were expiring claims based on the old rules of the blockchain, if they were created before the hardfork. If they were to expire AFTER the hardfork, their expiration should have been extended.
e0252cb -Fixed bug with the api query where row closing was not deferred.
9b898fe -fixed bug where routine required to end for shutdown was stuck sleeping. This should just be in the background unrelated to shutdown.
1b62e0b -fixed datastore issue where we were still using the cache.
5e737bf -fixed missing rows closure.
ee0ae99 Added address balance, transaction value sync apis and cron jobs.
1c990fa Fixed potential for data corruption. If block processing error'd out, the partially processed block was not rolled back. In order to roll a block back an overhaul of the concurrent transaction processing was needed so that an error could be returned and handled. The stopper pattern was implemented for all go routines at each level ( concurrent transaction, input, output processing ). Advanced debug logging was added as well to make investigations easier. Lastly, the grabbing of transaction details from LBRYcrd were serialized to allow for an algorithm that determines in which sequence transactions are assigned to available workers to be processed. Over a set of blocks, the loss of performance was minimal going from 4 min 42 sec to 4 min 44 sec. This algorithm can be improved significantly in the future.
3874efa Merge pull request #65 from lbryio/fix_data_corruption
ba5a9f9 Merge pull request #76 from lbryio/triggers
0a5545e Merge pull request #81 from lbryio/regtest
b921ceb Remove migration of data as it will take too long in production.
2e9aca2 Update trigger migration to show the credit value of the whole transaction.
3d9c529 Upgrade chainquery to sqlboiler V3.
800b475 addressed review comments and linting
c18648c fixed review items to make the code more readable and fewer parameters. Added syncManager.
3908b6d fixed review items, cleaned up getblockinfo, and removed excessive parameters.
27c4390 test
0a3016c test

chainquery - v1.3.2

Published by tiger5226 almost 6 years ago

Changelog

085cf96 -Moved claimtrie sync to be 100% in the background that can be killed for graceful shutdown. -Added closing of specific channels for graceful shutdown.
e18485e -fixed send on closed channel issue

chainquery - v1.3.1

Published by tiger5226 almost 6 years ago

Changelog

35c4bc2 incremented to golang version 1.11.2
f445c60 not running when returned
7ee66ce return from iteration if we cannot get the block height due to error from lbrycrd.