๐Ÿ Meet WorkerBee: The Easy Way to Build Smart Blockchain Bots

in HiveDevs โ€ข 8 days ago

๐Ÿ Meet WorkerBee: The Easy Way to Build Smart Blockchain Bots

Welcome to WorkerBee โ€” the library that makes building powerful bots operate on the Hive blockchain as easy as programming a web page! </>

We already announced this library in several of our articles in the past: here and at initial rough presentation of our tools. For one and a half years our developers (mostly using this library - our hope is to see your use cases too) have proven that the initial version works well (it was used as a heart of optimistic UI implementation in Denser).

Moreover, we have a few ideas how to simplify the use and provide you with "ready to go solutions" for very common usecases without losing any ability to process completely custom one.

Our answer to your common needs are predefined processing primitives, but allowing mixing them according to your needs as well as still providing ability to use custom bot trigger conditions.

So whether you want to track new posts, watch account changes or react to blockchain events, WorkerBee takes care of the hard stuff so you can focus on what โœจ you want your app to do, not the gritty details of how ๐Ÿง .

ai-bee.png

๐Ÿ‘‰ Why Use WorkerBee?

WorkerBee is designed for every programmer, not only for blockchain developers! Here is what makes it special:

  • No More Endless Loops! ๐Ÿ”„
    Just say what you want to listen to - WorkerBee does all the waiting and event handling behind the scenes.

  • No Blockchain Headaches! ๐Ÿงฉ
    Forget complicated APIs and coding tricks. Write logic much like you would on regular web apps (think: โ€œWhen a new post appears by Alice, send me a ping!โ€).

  • Keep Your Sandbox Clean ๐Ÿงผ
    WorkerBee shields your code from the messy details of blockchain data, so your app stays flexible and easy-to-maintain.

  • One Interface, Many Sources ๐Ÿ—ƒ๏ธ
    Switch from live blockchain data to a historical database or new data source (e.g. SQL) - all without changing your appโ€™s logic!

  • Easy to Expand ๐Ÿ“ˆ
    Start simple but add new events, rules, or channels as your needs grow.

  • Fully typed โ„น๏ธ
    Library APIs have well defined types and functional interfaces with special support for IDE IntelliSense

wb-fully-typed.gif

๐Ÿ•น๏ธ What Can You Build with WorkerBee?

  • ๐Ÿ’ฌ Automatic reward bots for blog comments,
  • ๐Ÿ“… Custom notifications for changes in your favorite accounts,
  • ๐Ÿ“Š Powerful monitoring dashboards,
  • ๐Ÿ“ข Alert/response bots for blockchain events,
  • ๐ŸŒŽ Web2 social media integrations,
  • ๐ŸŽฎ On-chain Games,
  • ...and much more ๐Ÿ˜Ž!

๐Ÿงฑ Bot building blocks

To provide a flexible way of creating the triggers tied to some blockchain events, the library offers a set of functional primitives being specialized to the features offered by Hive blockchain. For example, if you want to watch new posts written by your friend, the WorkerBee allows you to use onPosts(author) condition which completely isolates you from low level blockchain operations and also from the knowledge how to gather them.

Such building blocks (called also as condition primitives) can be joined into more complex expressions by using or/and logical operators which allow you to build custom "logical expressions" matching your data processing criteria.

Involving given condition primitive, automatically passes gathered data (e.g. post permlinks) to your callback function.

We also allowed forcing the library to additionally collect and pass to your callback some different data collected at runtime - to use this feature you should call special methods called providers which on demand collect, transform and provide data to your callback function when it should be called after satisfied evaluation of defined condition.

For example this code

   workerbee.observe.onBlock().provideAccounts("account1", "account2").subscribe({
     next: (data) => {
       console.log("Account 1 data:", data.accounts["account1"]);
       console.log("Account 2 data:", data.accounts["account2"]);
     }
   });

allows gathering data specific to given accounts and passing them to your callback. What is important - you do not need to know anything about any API needed to do it.

To make it work, the library internals are built by using a set of objects having different responsibilities:

  • collectors - their role is to handle a communication layer between blockchain and workerbee library. Current implementation is based on RPC API calls, but its abstract design allows switching them in the future for example to SQL based one.
  • filters - such a class of objects is responsible for gathering data from collectors and evaluating the condition specific to given filter class
  • data providers - their function is to gather data from collectors, transform them into official WorkerBee interface to be finally passed to your callback.

The last important thing is the finalization of the observer object previously defined from the set of condition and provider primitives. It happens once you call subscribe method.

Further chapters contain library architecture description in great technical detail, including data flow performed during each processing cycle.

๐Ÿ› ๏ธ Data Flow at a Glance

WorkerBee consists of multiple layers of data analysis:

Blockchain Data โ›“ โ†’ Collectors ๐Ÿ›’ โ†’ Data Evaluation Context ๐Ÿง  โ†’ Filters ๐Ÿ”Ž โ†’ Providers ๐Ÿšš โ†’ Your Observer Callbacks ๐Ÿคณ

  1. Blockchain Data โ›“ - The library operates on the blockchain data on an abstract level - it can fetch data from multiple sources, such as JSON-RPC (hived Node) API, REST API, HAF SQL, etc. It is worth mentioning that at the moment all the features offered by WorkerBee need to work only consensus node.

  2. Collectors ๐Ÿ›’ are strictly connected with Classifiers which enforce an expected collector result. In other words the Classifier identifies collector implementation which can provide a given class of data (and separate collector implementations can be provided for requested classifier by using a Factory object pattern). This way an user can fetch data from any supported source without changing anything in its code. A set of needed classifiers (and chosen collectors) is determined by requested Filters and Data-Providers described below. The collector data acquisition is executed only on demand to avoid wasting a computational power until their execution is needed.

  3. The Data Evaluation Context (DEC) ๐Ÿง  is the brain: It manages dependency injection, caches, shared store, cycle timing and orchestrates everything behind the scenes. And due to Smart Cache we can get the data we want, only when it is actually needed by the framework and limit the API calls when they can be reused. The DEC is central hub of communication layer and allows sharing cached data between filters & collectors which want to consume them and collectors being responsible for gathering them.

  4. Filters ๐Ÿ”Ž run all in parallel with predefined conditions. Upon listener creation, a user can define advanced AND/OR logic between filters, extending their functionality and creating complex expressions. There is implicit OR association - using two subsequent filter building blocks involves OR condition. For example:

     workerbee.observe.onAccountsBalanceChange(false, "username").onAccountsMetadataChange("username").subscribe({
       next: () => {
         console.log("Someone is messing with @username account !!!");
       }
     });
    

    The WorkerBee filter implementation supports Short Circuit Evaluation scheme so it does not waste time on waiting for all the filters.

  5. Providers ๐Ÿšš evaluate only when at least the filter associated to the whole observer object passes. Providers also run in parallel, transforming the collected data to public WorkerBee object model and passing it to the user callback.

  6. Your observer callback ๐Ÿคณ runs with data normalized by providers. This function is also async, which means the main loop does not wait for it in order to execute next iteration. The result is that you can instantly react to blockchain changes and broadcast transactions without worrying about loosing other live data ๐Ÿ‘€

DEC-cycle-chart.png

๐ŸŒ„ Real-Life Scenario: Blog Post & Account Tracking

Let us say your bot needs to monitor blog posts and comments by Alice, and manabar load of Bob's in order to vote for Alice's content

To achieve this we need to use the following filters & providers:

  • Filters
    • PostFilter: triggers when bot detects new posts authored by Alice,
    • CommentFilter: fires when Alice made some new comments,
    • AccountFullManabarFilter: will match when Bob's account manabar load exceeds specified percent
  • Providers
    • PostProvider: retrieves data specific to all detected Alice's posts,
    • CommentProvider: gathers data of all new Alice's comments,
    • ManabarProvider: receives Bob's manabar value

Further part of this article shows the code doing it (please do not be disappointed by the small amount and simplicity of the attached code - it was our intention to allow simple coding of complex things as described above - we hope you will appreciate it :-) ).

gantt.png

๐Ÿ”ƒ A Typical Notification Cycle

As we have previously promised, WorkerBee eliminates a need to write its own mainloop to trigger the whole bot machinery - the library performs it in its own specific way - what has technical and performance related explanation. This process depends on current evaluation phase. Let say we have to process thousands of past blocks to get posts created for the last week - it means that your bot should consume data as fast as it can ๐Ÿš€, because there is nothing to wait for - all requested blocks already exist (and thanks to OBI they are usually written in a rock ๐Ÿชจ ). Things seem to be different when a bot enters live sync (reaches blockchain head). Without special progamming techniques, it can start naive API spamming process for periodic (usually too frequent) lookup for new data. The workerbee library (as its natural equivalent: ๐Ÿ ) is smart: since it knows query context, it can create massive queries (and data processing) for past data and periodic (time triggered - 2s based) for live sync. Another big helper in this process is abstract layer specific to Collectors which sometimes have different implementation for past and live phases. Well done object oriented design always brings good benefits :-)

So let us analyze the role of all actors involved into work during single notification cycle (let say once per block):

  • Data Evaluation Context: DEC ๐Ÿง  initializes and injects collectors required by filters and providers (including their dependencies)
  • Concurrently ๐Ÿ”Ž:
    • PostFilter requests OperationCollector via DEC and fetches data from the blockchain. OperationCollector needs BlockCollector, which needs DGPOCollector (dynamic global object properties collector). It finds Alice's post and matches.
    • CommentFilter requests OperationCollector, and since PostFilter already requested it, it only waits for the fullfilled cache in DEC and does not call the API again. It searches for Alice's comments, but is interrupted by PostFilter match, which finished its own work earlier and due to short circuit evaluation scheme cancelled other evaluations.
    • AccountFullManabarFilter requests RcAccountCollector. It fetches RC data of Bob's, but it is also cancelled by PostFilter match.
  • Concurrently ๐Ÿšš:
    • PostProvider: Retrieves all new Alice's posts from the cache.
    • CommentProvider: Retrieves all new Alice's comments from the cache.
    • ManabarProvider: Retrieves Bob's manabar load from the cache
  • End Result: Your observer receives {post, comment, rc data} โ€“ ready to act! ๐Ÿคณ

We will publish soon the explanation of the differences between collectors (and why we need to have so many of them) :-)

๐Ÿš€ WorkerBee in Action: Stablitity, Efficiency and Performance

Stability

As mentioned before, our library introduces its own typed data layer. It means that your bots start to be independent on direct blockchain API calls (which are changing over time). Data provided based on such calls are wrapped by WorkerBee library to provide call results in normalized WorkerBee format. Moreover, some of them need to use special API call techniques to hide errors e.g. specific to too many requests. So taking care of network call errors is also moved from your code into common implementation of the library communication layer.

Performance and efficiency

To get desirable performance results, out library introduces a few optimization aspects. Most important ones cover concurrency (asynchronuous execution) and caching specific to single notification cycle. Because of encapsulation, all such techniques are hidden from a client code. Below you can see greater details specific to information gathered during testing and benchmarking of the library.

WorkerBeeโ€™s Built-In Caching Delivers ๐Ÿ›ต

  • At least 50% fewer API calls in real-world multi-filter scenarios!
  • Consistent Data: Every component works from a single state snapshot per cycle.
  • Smooth Scaling: From average of 6 to 3 API calls per cycle in real workloads.
  • Linear Complexity: Dependency chains do not explode resource usage.

Cache Invalidation Strategy ๐ŸŽฏ

  • Per-Cycle: Cache is completely cleared at the start of each notification cycle
  • Atomic: All data represents a consistent blockchain state snapshot
  • Dependency-Safe: Collector dependencies always see the same cached data
  • Memory Efficient: No long-term memory accumulation from caching

DEC Caching Performance Example Benchmark ๐Ÿ’ซ

WorkerBee without DEC Cache ๐Ÿ˜ญWorkerBee with DEC Cache ๐Ÿ˜ŽNaive implementation ๐Ÿง™โ€โ™‚๏ธ
API Calls per cycle *6310
Network Latency500ms250ms830ms
Throughputโ€“+2ร—-0.7x
Node LoadHighLowVery High
Intelligence+5+420-9001
Stamina+1+42-9001

* - A Number of calls calculated based on the scenario of operation filter, 3 accounts filter and operation & account provider

๐Ÿงฐ Core Patterns & Architecture

WorkerBee leverages proven design patterns for clarity, flexibility, and speed:

  • ๐Ÿ‘€ Observer Pattern: Event-driven, just like front-end UIs.
  • ๐Ÿšธ Dependency Injection: Clean orchestration and loose coupling between data collectors and business logic.
  • โ™Ÿ๏ธ Strategy Pattern: Factories swap data sources (RPC, SQL, etc.) on demand.
  • ๐Ÿ„ Fluent API: Build logic by chaining calls, ร  la jQuery or modern Node.js libraries.
  • ๐Ÿ”› Type-Safe Interfaces: Compile-time safety and great DX.
  • โ™ป๏ธ Fine-Grained Caching: Every component shares the most recent data snapshot for max performance.

๐Ÿ“ฆ High-Level APIs (Think Query-Language Simplicity)

The WorkerBee interface aims to be as powerful as SQL, but as friendly as jQuery. You build what you want to see:

bot.observe
  .onPosts("alice")
  .onAccountsFullManabar("gtg")
  .subscribe({
    next({ posts }) {
        for(const { permlink } of posts.alice) {
            console.log(`Alice created post "alice/${permlink}"`);
            console.log("Voting as gtg as it has full manabar...");
        }
    },
    error: console.error
  });

filters-graph.png

๐Ÿšง On-going work

We are currently working on improving the past data collection and adding new filters & providers. ๐ŸŽจ

Thanks to that, you will be able to react to even more events on blockchain! โš›๏ธ

We would love to hear your feedback so we can make our software even better than it already is! ๐Ÿ˜Ž

๐Ÿ”œ Future of WorkerBee

  • ๐Ÿ›ข๏ธ SQL Integration
  • ๐Ÿ›Œ REST API support
  • ๐Ÿ Python implementation of WorkerBee
  • ๐Ÿค– Implementing new bots
  • ๐Ÿ‘จ๐Ÿปโ€๐Ÿ’ป Spreading the use of WorkerBee across all automation applications!

โœ… WorkerBee is already in production!

Optimistic UI feature in Denser allows visualizing and tracking the state of blockchain transactions specific to UI actions like votes, comments:

like-like-denser.gif

You guessed it right - It's WorkerBee-powered! โšก - WorkerBee awaits the confirmation of the vote operation in the blockchain and triggers UI updates. All such features have been achieved just by calling to overloaded broadcast method which additionally does such tracking - so again we have oneline code implementation !!.

You can try it yourself - please go to Denser blog with revolutionary HiveSense AI Search! ๐Ÿ“š

Besides, we also created multiple small bot applications integrating Hive platform with other services like RSS, some chats... We will describe them in more details in our next post soon.

โšก Ready to Try WorkerBee?

Whether you build blockchain bots, analysis tools or want a better event architecture โ€“ start simple, scale up, and never worry about main-loop spaghetti or wasted API calls again.

WorkerBee: All the power of blockchain events - none of the hassle.

๐Ÿ“ Getting Started

Please check out the WorkerBee Documentation ๐Ÿ“ก!

WorkerBee is your friendly helper for blockchain bots โ€” fast, efficient, and super easy to use.
Let it do the buzzing while you focus on building something amazing! ๐Ÿง‘๐Ÿปโ€๐Ÿ’ป

Happy coding and see you soon again !!!

Sort: ย 

WorkerBee looks slick! Building bots on Hive just got easier ๐Ÿš€ Great resources linked too! ๐Ÿ

How do you find and set this up?

You can check out the documentation and install the package from the npm ๐Ÿ˜ƒ

  1. Install Node.js
  2. In an empty directory, create new npm project: npm init -y es6
  3. Install WorkerBee library from the npm: npm i @hiveio/workerbee
  4. Create index.js file with the example content:
    import WorkerBee from "@hiveio/workerbee";
    
    const bot = new WorkerBee();
    
    await bot.start();
    
    console.log("๐Ÿš€ Bot started! Waiting for new blocks...");
    
    for await (const { id, number } of bot)
      console.log(`๐ŸŽ‰ Got block #${number} with ID: ${id}`);
    
  5. Run your script: node index.js

Excellent information. Regards.

Wow! WorkerBee is a true game changer for Hive developers! So many amazing features โ€“ post tracking, account monitoring, and event-based automation โ€“ all made so easy, with so little coding. It feels like blockchain is about to become as easy as regular web development!โ€

Brother, its typed API and fluent chaining syntax is heart-winning! Smart Caching has done wonders โ€“ it seems like blockchain dev has become fun and easy now! Now many ideas are coming to mind โ€“ I am planning to make something amazing soon!

Man, reading this has given me some great ideas! I am thinking of making a bot that will alert me whenever my favorite author posts something new โ€“ I have never seen anything like this on Hive before!โ€

This sounds fascinating. @borniet, take a look at this, oh Botlord. ๐Ÿ˜ ๐Ÿ™ ๐Ÿ’š โœจ ๐Ÿค™

This is definitely interesting stuff!!! Will look into it deeper ;-)
!HOPE
!PIZZA
!INDEED
!ALIVE
!BBH
!STRIDE
!WEIRD

Oh, good, I'm glad that you saw it! Cool, I figure some aspects of it may be useful. ๐Ÿ˜ ๐Ÿ™ ๐Ÿ’š โœจ ๐Ÿค™

!HOPE
!INDEED
!MMB
!STRIDE

Yeah, definitely! Even if only to see how they did it and learn from it ;-)

!HOPE
!PIZZA
!INDEED
!ALIVE
!BBH
!STRIDE
!WEIRD

Exactly, that's what I was figuring. ๐Ÿ˜ ๐Ÿ™ ๐Ÿ’š โœจ ๐Ÿค™

!HOPE
!INDEED
!WEIRD
!ZOMBIE

Really you provide best bots in Hive Blockchain.
These bots make more interesting Hive Blockchain and make good impact.
Thanks for this good job ๐Ÿ‘

This looks good enough to try. Will give it a go when I start a new script or am modifying one of my current ones enough that I think adding this will provide benefits.

Excelente para miel