Below are a few highlights of the Hive-related programming issues worked on by the BlockTrades team since my last report.
Hived: core blockchain node software
We’re planning to release 1.27.7 sometime in July (probably near the end of July). As part of this release, we’re also setting what we expect will be the final “trigger date” for hardfork 28: September 24, 2024 at 14:00 UTC (to allow a little time after the upcoming HiveFest in Croatia).
Most of the changes for 1.27.7 are already done, but we’re giving about a month for our production testing. Both hardfork-related changes and some other changes associated with this release can be found in links below (and/or discussed in subtopics below):
- https://gitlab.syncad.com/hive/hive/-/milestones/6#tab-issues
- https://gitlab.syncad.com/hive/hive/-/milestones/7#tab-issues (unstarted issues described here won’t be part of the release)
Split block logs (enables “lite” hived nodes for services)
We are in the middle of implementing the last changes associated with the split block logs functionality that I discussed in my previous report and we expect to complete that work this week. Then we plan to devote a week to production testing of the new various new features associated with block log splitting.
We added another new feature based on the new split block log technology: hived now opens block log files in read-only mode and then switches to read/write mode when the node begins syncing. This change allows old block log parts to be shared across multiple instances of hived, further reducing storage space required to support multiple hived nodes capable of supplying sync blocks.
Other hived changes
- Cleanup of signature generation code: now hived uses the
bip0052
format everywhere instead of thefc::legacy
code. Old legacy signatures are still verified correctly. - Slight redefinition of virtual operations holding assets: where possible they were changed to use specialized versions of
tiny_asset
to limit the runtime C++ object size and more importantly, the size of their binary serialized form (this reduces HAF database size).
- Fix for bug in P2P layer associated with block compression changes that caused some hived nodes to crash (not sure why this bug took so long to appear, however).
BLS signatures research
We did some initial research on BLS signatures, the report is here: https://gitlab.syncad.com/hive/hive/-/wikis/Research-of-BLS-signatures
We plan to first implement support for BLS in the new Wax API library, where it will initially be useful for 2nd layer apps, then assuming we find no problems, add support for BLS in the 1st layer (in HF 29, which is scheduled for a relatively quick release after HF 28).
Beekeeper (cryptographic key management software)
Current we are working to finish up the python wrapper for BeeKeeper, which is needed to support Wax’s transaction builder for Python.
HAF API node
Added support for optional Prometheus and Grafana monitoring.
HAF (Hive Application Framework)
We’ve just about completed the redesign of the “app main loop” for HAF to make it more difficult for app developers to make an error when designing their app’s processing logic. That work is here:
https://gitlab.syncad.com/hive/haf/-/merge_requests/504
Other changes to HAF
- We’ve implemented a new approach for HAF application schemas that allows the same application to be deployed multiple times (to different schemas and contexts): https://gitlab.syncad.com/hive/haf/-/merge_requests/484
- Storage space optimizations (reduces current storage requirements by around 300GB):
- Removed hive_rowid indexes for irreversible applications (i.e. hivemind)
- Elimination of block_num, op_type_id columns from hive.operations and hive.account_operations tables. They are replaced with function calls decoding this data from operation_id
- Re-using new virtual operation serialization to limit vops storage size in the database (this feature was discussed in the hived section above)
- Elimination of operation timestamp column from operations table and replacing it with referenced block creation time.
2nd layer “Lite accounts” that are transportable across the Hive ecosystem
As mentioned in my last report, we are creating a new HAF app to manage the creation and maintenance of “Lite” accounts that can be used by any 2nd layer app to sign 2nd layer transactions. The specification for this also includes documentation for how we plan to support 2nd layer transactions. https://gitlab.syncad.com/hive/haf/-/issues/214.
Most recently we worked out how we will handle the namespaces for lite accounts (using decentralized ids):
The default namespace of la
(for light account) can be used by any hive account. So any hive account will be able to create a lite account called did:la:null
for example, as there are no special requirements to claim any account name in the la namespace. Names in this namespace are strictly first-come, first-served.
We’ll also have a special namespace for hive accounts called hive
so that all hive accounts have one “verified” associated light account. For example, the light account called did:hive:justme
can only be edited using the 1st layer keys of the Hive account called justme
. So hive accounts don’t need to worry about someone grabbing their unique 2nd layer lite account in the hive namespace.
2nd layer accounts of this type are useful if you want to associate additional 2nd layer specific keys with your Hive account so that everyone knows who you are. For example, you might want to add a “battle” key to play a 2nd layer game.
We also have a plan for how to create namespaces for other blockchains and also non-blockchain apps such as google/facebook, etc, but I’ll go into that more in a later report.
HAF Block Explorer and Balance Tracker APIs
- Adjusted for new HAF version
- Support for account authority data
- Support account reputation info in API calls
- Optimizations of get_account API call performance (time reduced from 1s to 50ms)
- Auto-disable block/comment search APIs when optional indexes are not built
- OpenAPI rewrite support: This is part of work I described in my previous post to auto-generate some useful standards-conforming REST calls via rewrite rules AND generate interactive swagger pages for HAF apps (HAF block explorer is our “test case” for the new technology). Most recently we worked out how to best handle REST calls that take array parameters.
Wax (New multi-language Hive API library)
We’re preparing the first official release of the Wax npm package at the end of this month (2 weeks).
Latest improvements to Wax
- Implemented TS operation builder and properties serializer for witness_set_properties operation
- Implemented witness_set_properties text formatter to make it readable for humans
- Implemented API healthchecker component
- Package size reduced from 5.4MB to 2.3 MB (compressed size)
- New methods for key utility functions (suggest_brain_key, generate private key from password)
Clive: command-line and TUI wallet for Hive
- Published release: https://gitlab.syncad.com/hive/clive/-/releases/v1.27.5.11
- New features in progress (planned for June 19th release): https://gitlab.syncad.com/hive/clive/-/merge_requests/399
UI changes cover account alarm management. CLI changes include bugfixes to recurrent transfer and custom_json support.
Imagehoster
- Fix for URL handling code that could crash imagehoster
- Updated docker images to use newer version of node.js
- Deployed version now supports SVG images
Some of our work in progress (or planned for near future)
- Creation of HAF-based Lite Accounts application (implementation in progress). This also includes support for 2nd layer transactions which should be useful for many types of HAF apps.
- Developing spec for 2nd layer smart contract processing engine (most of docs are here: https://gitlab.syncad.com/hive/smarc/-/tree/abw_smarc_design/doc?ref_type=heads )
- Many hived improvements: https://gitlab.syncad.com/hive/hive/-/issues/675
- Official release of wax and workerbee npm package
- Finish new OpenAPI documentation of existing REST APIs (balance tracker and HAFAH are next)
- Create a release candidate for Denser (replacement for Condenser)
- Continue adding new commands to Clive
- Release new reputation tracker app
- More hivemind performance improvements (continued replacement of Python code). Replacement of python web server with PostgREST server should be completed in time for July release.
- Integrate reputation_tracker as a sub-app inside hivemind. This should improve space optimization (now hivemind must collect all votes to recalculate reputation) and sync speed. We also hope to have this in July release.
While there was no reason not to do it, only the trivial part of the task was done so it took very little time, and it also serves as a protection against passing wrong assets to virtual operations, the impact was extremely disappointing despite hundreds of millions of vops in database. Therefore even though we could still continue with more complex cases, we most likely won't because there would be no gain. It also puts into question somewhat similar but separate task of merging vops related to comment payouts - at the very least we'll have to estimate the impact before making any changes.
Since the vast majority of the operations are custom_json, it's not surprising the impact on the size of the operations table was small.
So what is our final goal that we want to achieve ? And how far we are from that ? It's good to see we are working, but its equally important to figure out, how it aligns to the goal.
We have several goals for our current work:
I smell the confidence, thank you so much for the detailed response. I understand the challenge, but who dares wins and 2025 will be ours. Kudos to you and the team.
Thanks for the work @blocktrades !
Congratulations @blocktrades! Your post has been a top performer on the Hive blockchain and you have been rewarded with this rare badge
You can view your badges on your board and compare yourself to others in the Ranking
If you no longer want to receive notifications, reply to this comment with the word
STOP
Check out our last posts:
Keep up the great work guys 👍
👍
this is always good to know, some good work getting done.
!LOL
!ALIVE
lolztoken.com
But not as fast as his brother Sudden Lee.
Credit: manuvert
@blocktrades, I sent you an $LOLZ on behalf of cryptoyzzy
(8/10)
Delegate Hive Tokens to Farm $LOLZ and earn 110% Rewards. Learn more.
😱 I need to catch up, Soooo many changes!
It is exciting to know that we already have a date for the next HardFork. I see each of these updates shaping the future of the Hive blockchain. 🚀
Hola @blocktrades , es muy importante el trabajo que estás realizando, felicitaciones. Leyendo tu publicación, me di cuenta que mucho me falta por aprender de este mundo blockchain y su lenguaje técnico, gracias por los apoyos que he recibido de tu parte, me han ayudado mucho.
This projects are really good, important and helpful to all hive users and also to the Platform. 👍👏