How to setup 3Speak Video Encoder node?

in Threespeak2 years ago (edited)

▶️ Watch on 3Speak

🙏 Namaste Hive Community members

With this video, I'll illustrate how to set up the encoder node on your system.

FAQs

First of all what is encoder node?

So, whenever someone uploads a video to 3speak, it has to be processed. You might be wondering what processing?
Imagine you uploaded a video from your iPhone 12 Pro with highest resolution and video size is up-to 1 GB.
Do you want your users with low data bandwith to suffer? Of course you won't.

So, encoder nodes will encode the video into different sizes like 480, 720, 1020 etc so that it can be viewable with limited data badnwith.
Video encoding is high processing task.

Now imagine, there are many users uploading video, 3speak can not limit itself to just 2-3 computers for encoding.
It may result in-to long queue and users may have to wait for video to get published.
and if 3speak does that, it becomes a centralised service provider which 3speak wants to avoid.
It has to be community driven.

This is where encoder-node-operator comes into picture.
Community powers the community.
They share the load, they take the job of video-encoding and guess what they also get reward of 1% as a beneficiary.

What kind of system do I need?

Anything which has processing power more than intel i7.

Do I need graphic card?

Not at all.

What kind of internet connection do I need?

You need a wired connection. Wifi would work but you'll get lesser jobs because of fluctuating internet connection.

How much internet speed is required?

Anything above 40 mbps

How much RAM is required?

Anything above 8 GB

What operating system is required?

You can use any operating system - macOS, ubuntu, windows.

What softwares are required?

You need nodejs 16, and ffmpeg installed on your system.

Would VPS work?

I wouldn't recommend. I paid for heavy-weight contabo server for over a year & it does not encode a single video in a day.
But if you want to try, Give it a try. I wouldn't stop you :)

Steps to setup encoder node

Setup PM2 (if you already have PM2, skip this)

  1. Run command npm install pm2 --global

Setup IPFS (if you already have IPFS, skip this)

  1. Open https://dist.ipfs.io/#go-ipfs
  2. Download binary (based on your OS)
  3. Run install.sh command which is inside downloaded zip file.
  4. Go to your work directory e.g. ~/Users/sagar
  5. Run ipfs init - this you would run once in life time.
  6. Run pm2 start "ipfs daemon" --name ipfs - again once in life time.
    Next time onwards, you just need to say pm2 restart ipfs

Setup FFMPEG

Please google yourself - "how to install ffmpeg"

Setup video encoder node

  1. Open terminal & navigate where you want to setup encoder-node
    e.g. ~/Users/sagar/projects
  2. Run git clone https://github.com/spknetwork/video-encoder.git
  3. cd video-encoder
  4. npm install --force
  5. npm run build
  6. pm2 start "npm -- run start" --name video --time --max-memory-restart 6144M -f - once in life time. Next time onwards, you just need to say pm2 restart video

NOTE: Make sure to run pm2 save so that if system restarts, you can bring it back to life.

What to change in ~/.spk-encoder/config file?

Update the gateway_client => gateway_url value. as follows.

  "gateway_client": {
    "gateway_url": "https://encoder-gateway.infra.3speak.tv",
    "queue_max_length": 1,
    "queue_concurrency": 1,
    "async_uploads": false
  },
  "remote_gateway": {
    "api": "https://encoder-gateway.infra.3speak.tv",
    "enabled": true
  },

Update the node name, hive name

  "node": {
    "name": "Sagar M1 macBookPro",
    "privateKey": "Sxxxxxxxxxxxxxxxxxxxxxxx=",
    "cryptoAccounts": {
      "hive": "sagarkothari88"
    },
    "publicKey": "did:key:zxxxxxxxxxxxxxxxxxxxxxxx"
  },

CRITICAL WARNING:

  1. DO NOT TOUCH PRIVATE KEY OR PUBLIC KEYs.
  2. Those are system generated & it shouldn't be touched.
  3. DO NOT PASTE any of your HIVE keys here.

What to do If I restart my system?

pm2 resurrect

More questions?

Ask me in the comments section or ping me on 3speak-discord=>encoder-nodes.

Like the work I do?

  • 👍 Please up-vote my content.
  • 📥 Please follow my hive-blog. @sagarkothari88

Vote me as your hive-witness

If you are on DTube

If you are on Blurt

To appreciate my contribution, you can vote me as your Blurt Witness.

Jai Shree Krishna.
See you next time.

Sort:  

Please also add settings to transcode to 240p and 144p, as not all users have high bandwidth data lines to watch videos in high resolution !
Many thanks.
Regards, Stefan.

Legit why can't this just be build into the 3speak app download and just toggle a button to have my computer work as an encoder like Theta edge node does. Super simple and tons of people can then get involved.

We're working on it.

In the config from git version is missing this
"remote_gateway":{ "api": "https://encoder-gateway.infra.3speak.tv", "enabled": true },

I'll leave it here for someone to copy and paste if necessary...

I Always Get this error.


jobInfo {"data":{"queueJob":{"reason":"RANK_REQUIREMENT","job":null}}} { node_id: 'did:key:z6MkvXLtWnXFXZRHLyBqCmxtoLntHthVsRf3NbnnYXULd1v1' }


It would be nice to know what the criteria is for that message, since I have a server with Core i7 - 32 Gb RAM - 1 TB NVMe - 2 TB SSD -
Internet 1Gbit

@zuun.net - congratulation. Your node is up & running. I've talked about this in the video.
I've 5 different systems & out of 5, my 3 system gets those message.

"RANK Requirement" indicates that your node doesn't have RANK at all to get new jobs.
When all nodes having good rank are busy, a node with no-rank will be picked & will be measured for it's performance. If performance is good, RANK will be updated after the your node finishes the video-encoding.

Now that you have a better rank (than NO RANK), your node will be preferred node for next video encoding.

I hope you got the idea.

what makes your node have a higher rank

At the moment, there are very limited beta users
and due to that, there are very little videos to encode for encoder-nodes
So, don't panic if your system doesn't get jobs for few days.

Very informative thank you.

Are rewards up and running or is this still in testing phase?

!PIZZA

Rewards are up and running.

PIZZA!

PIZZA Holders sent $PIZZA tips in this post's comments:
@revisesociology(1/10) tipped @sagarkothari88 (x1)

Join us in Discord!

Congratulations @sagarkothari88! You have completed the following achievement on the Hive blockchain and have been rewarded with new badge(s):

You received more than 5000 upvotes.
Your next target is to reach 6000 upvotes.

You can view your badges on your board and compare yourself to others in the Ranking
If you no longer want to receive notifications, reply to this comment with the word STOP

Check out the last post from @hivebuzz:

Hive Power Up Day - July 1st 2022
NFT for peace - Thank you for your continuous support
Support the HiveBuzz project. Vote for our proposal!

This system properties can only be met by few people. Also the mbs network here in my country is somewhat rare, as a matter of fact I don't think I have witnessed 40mbs from our network providers before

This would have been much easier if its encrypted in 3speak from onset.

However, you're a Geek 🤓...you've really shared values. Thanks Champ!

Ubuntu is not an operating system. It is a Linux distribution. And even not a recommendable.

🙏 Prabhu 🙏 (Master in my language)

Thank you so much Sir.I learnt a lot from your post

You're welcome. I am glad that it helped you learn more. I'm committed to do even more contribution.

Thank you for this. With these, some of us will learn basic coding skills from the blockchain. It is nice to have you sir @theycallmedan

Can I monitor it working in real time?

Very nice Thank you for this .With these some of us we learn basic coding skills from the block chain coding is very important you're really shared value
Congratulations....
You have completed the following achievement on the HIVE blockchain......