Notes Hive Public API node performance

in LeoFinance2 years ago (edited)

Abstract

image.png


As part of the Hive - Peerplays Sidechain implementation, we are now attempting to connect the sidechain listener to the Hive Public API endpoints. Previously we have been using fullfledged Hive TESTNET and Peerplays TESNET and the cross chain transfer from Hive TESTNET to Peerplays TESTNET was possible in around ~10 seconds. We have been expecting the same implementation to complete cross chain transfer of HIVE/HBD in less than 30 seconds. We have been streaming the Hive blockchain and looking for transactions in account_history_api.get_transaction every second. But with the Hive mainnet, the API nodes adds latency which results the cross chain swaps to slow down to around 80-100 seconds.

Next steps are to attempt to listen for only the LIB (ie the irreversible blocks) every 3 seconds instead of looking for transactions every second. Further optimizations in the required RPC calls can also provide better performance. (Results will be announced).

In general it looks like solutions like HAF, ie moving the account history to proven and battle tested general purposed databases like PostgreSQL can provide much needed response times for financial applications.

Current Challenges

  • block_api.get_block call is slow (details to follow)

  • account_history_api.get_transaction Often fails on the Hive Mainnet's public full nodes

  • some of the nodes seems to have compiled with SKIP_BY_TX_ID=ON & thus fails to return full data.

Hive - Peerplays Sidechain listener logic

The following logic is used for listening the transactions in the Hive blockchain in real time. Upon meeting the criteria, deposit or withdraw of HIVE/HBD assets are invoked and under goes consensus process by the Sidechain Operator Nodes. Note that the implementation is in C++ and for now we are using a slower curl based implementation that supports HTTPS as part of the Peerplays chain's Hive listener. Initially we have been using HTTP and thus that also might be adding a small amount of delay.

  • call, once, database_api.get_version to get chain_id
  • call, once, condenser_api.get_config to verify are we running on testnet or mainnet
  • loop start
  • call, each second, database_api.get_dynamic_global_properties to monitor for new blocks (ie, we are looking into head_block_number field)
  • if** new block is available (ie, head_block_number is increased), we call block_api.get_block, to get the block
  • we are iterrating through block transactions, in order to find any transaction with transfer_operation of interest
  • we call account_history_api.get_transaction, for each transaction id in a block (these calls fail on mainnet)
840114ms th_a       rpc_client.cpp:171            send_post_request    ] ### Request URL:    https://api.hive.blog:443      840114ms th_a       rpc_client.cpp:172            send_post_request    ] ### Request:        { "jsonrpc": "2.0", "id": 5, "method": "account_history_api.get_transaction", "params": { "id": "bbea1d35998433e534e8b57f0945c32d1bbd3bd0", "include_reversible": "true" } }      840114ms th_a       rpc_client.cpp:174            send_post_request    ] ### Response:       {"jsonrpc":"2.0","error":{"code":-32003,"message":"Assert Exception:false: Unknown Transaction bbea1d35998433e534e8b57f0945c32d1bbd3bd0","data":{"code":10,"name":"assert_exception","message":"Assert Exception","stack":[{"context":{"level":"error","file":"account_history_api.cpp","line":250,"method":"get_transaction","hostname":"","timestamp":"2021-10-12T13:14:00"},"format":"false: Unknown Transaction ${t}","data":{"t":"bbea1d35998433e534e8b57f0945c32d1bbd3bd0"}}]}},"id":5}
  • we parse the transaction, looking for transfer_operation with son-account as a receiver
  • if we find it, we create appropriate deposit object
  • SONs are calling account_history_api_get_transaction, again during deposit object verification, to confirm that the deposit transaction exists, and that data in the transaction matches data in the deposit object (sender, receiver, asset symbol, amount)
  • for withdrawals we use network_broadcast_api.broadcast_transaction to bradcast payout transaction, and we monitor if the transaction reached irreeversible block, by calling database_api.get_dynamic_global_properties and looking into field last_irreversible_block_num. If transaction reached irreversible block, withdrawal is confirmed.

Issue logs

  • account_history_api.get_transaction
840114ms th_a       rpc_client.cpp:171            send_post_request    ] ### Request URL:    https://api.hive.blog:443      840114ms th_a       rpc_client.cpp:172            send_post_request    ] ### Request:        { "jsonrpc": "2.0", "id": 5, "method": "account_history_api.get_transaction", "params": { "id": "bbea1d35998433e534e8b57f0945c32d1bbd3bd0", "include_reversible": "true" } }      840114ms th_a       rpc_client.cpp:174            send_post_request    ] ### Response:       {"jsonrpc":"2.0","error":{"code":-32003,"message":"Assert Exception:false: Unknown Transaction bbea1d35998433e534e8b57f0945c32d1bbd3bd0","data":{"code":10,"name":"assert_exception","message":"Assert Exception","stack":[{"context":{"level":"error","file":"account_history_api.cpp","line":250,"method":"get_transaction","hostname":"","timestamp":"2021-10-12T13:14:00"},"format":"false: Unknown Transaction ${t}","data":{"t":"bbea1d35998433e534e8b57f0945c32d1bbd3bd0"}}]}},"id":5}
  • Nodes with SKIP_BY_TX_ID=ON

    curl -s --data '{"jsonrpc":"2.0","id":5,"method":"account_history_api.get_transaction","params":{"id":"bbea1d35998433e534e8b57f0945c32d1bbd3bd0","include_reversible":"true"}}' https://hive-api.3speak.tv/

    Response:

{"jsonrpc":"2.0","error":{"code":-32003,"message":"Assert Exception:false: This node's operator has disabled operation indexing by transaction_id","data":{"code":10,"name":"assert_exception","message":"Assert Exception","stack":[{"context":{"level":"error","file":"account_history_api.cpp","line":226,"method":"get_transaction","hostname":"","timestamp":"2021-10-13T12:05:21"},"format":"false: This node's operator has disabled operation indexing by transaction_id","data":{}}]}},"id":5}

Transaction times


Note that these tests are obviosly not done using multiple locations.

The commands used to test are here https://gitlab.com/PBSA/peerplays/-/issues/191#note_713042607

Sort:  

Bunch of notes from me:

  • Some API nodes (like mine) might have rate limiting enabled
  • There's no point in using SKIP_BY_TX_ID=ON these days
  • There's no need using such frequent get_dynamic_properties calls, blocks won't come faster than once per 3 seconds
  • You can use transaction_status_api to track status of transactions, it should be way cheaper in matter of computer resources
  • You could use local instance of a simple consensus node with transaction_status_api where you could run frequent calls against, while still using public node with account history for other calls.

Thanks @gtg for your points.

Some API nodes (like mine) might have rate limiting enabled

That explains why there is a variation in the response times.

There's no point in using SKIP_BY_TX_ID=ON these days

Yes - We may need to create a post and inform everyone that the RocksDB based plugin works just fine irrespective either options.

I got confused by this one big time. At first ended up concluding the disparity from the TESTNET and the mainnet is this option. Only handful of nodes have this option using at this point.

The last 3 points ---> Yes, we are trying to use different calls and optimizing the workflow and it looks like the performance of the listener is much better now.

As for the local node, these nodes already runs a graphene based chain and we are trying to reduce the over all expense to run the sidechain nodes. This is one of the reasons to go with the public nodes.

Further it looks like HAF may make things even better and we can ignore the account history altogether.

This post has been manually curated by @bhattg from Indiaunited community. Join us on our Discord Server.

Do you know that you can earn a passive income by delegating to @indiaunited. We share 80 % of the curation rewards with the delegators.

Here are some handy links for delegations: 100HP, 250HP, 500HP, 1000HP.

Read our latest announcement post to get more information.

image.png

Please contribute to the community by upvoting this comment and posts made by @indiaunited.

You are working for the good of the platform. Thank you!

feeling good that some Indian working to development of this platform..... really appreciating your work... keep it up 👍

thank you!

Can this system also be used for cross-chain swaps between Hive and other platforms? I would be interested in discussing this with you if you have time and want to reach out.

The tech is part of the core Peerplays blockchain and currently supports Bitcoin and Hive blockchains (Note Hive Support is in the final QA phase). Ethereum support is in progress. You can natively use the tech to swap HIVE to say BTC or HBD to BTC in couple of months. HIVE/HBD to ETH will be few months down the line. Ethereum support will also open up other pairs like HIVE to an ERC20 token. The swap itself will be done on the DEX and cross chain transfer will be using the decentralized, elected nodes (SONs).

Once the UI for the DEX is available, I think you or anyone can use the swap functionality out of the box. Once the UI is ready in a month or so, I can reach out.

This is super great to hear! Looking forwards to doing such swaps with platforms that have integrated the SPK Network using this tech

For the @spknetwork tokens the swaps will be available as soon the tokens are created. The following will work of the box:

  • Trading SPK network tokens with HIVE, HBD or BTC
  • Swapping of HIVE or HBD or BTC to a tradable SPK network token : This will essentially math the order book to the best available order and do the settlement.

Eventually the swap screen should like the following

image.png