Hello Steemians, we would like to open up public discussion of another minor addition to HF21: removing the limit to the number of custom JSON operations that can be included in a block by a single account.
For App Developers
Many Steem developers have informed us that this limit is adversely impacting the user experience in their applications. We want to do everything we can to make sure that Steem is the best blockchain protocol for powering web applications with amazing user experiences, which is why we believe it is worth discussing this change.
We have attempted to keep private discussions of this change to a minimum and a final decision has not been made. We would like all discussions on this topic to occur in the comments section of this post, or in follow up posts where appropriate. This way Steem users can gain as much insight into the change, and the reasons for supporting or opposing it, as possible.
Why Limit Custom JSON ops?
When this limit was created Steem only had bandwidth to rate-limit transactions. The big cost of custom JSON is an undo state for the plugins that process custom JSON. Because the Resource Credit is more sophisticated than the original bandwidth system, it more accurately captures that cost, and is therefore much more effective at rate limiting excessive use of custom JSON ops. This makes the limit antiquated.
DOS?
The DOS attack that led to the introduction of this limit did not impact the reliability of the Steem Blockchain. All witnesses continued to produce blocks without interruption, but it did result in an outage on Steemit.com. There is still the opportunity for the operation to be sent in a burst which could result in a short hiccup for nodes processing custom JSON.
Increase to 5
After weighing the pros and cons of such a change, we would recommend increasing the limit to 5 custom JSON ops per block. This will greatly increase flexibility for DAPP developers while continuing to ensure the stability of said DAPPs. This would enable us to re-evaluate the change prior to the SMT hardfork and decide to either relax the limit further, remove it entirely, or keep it where it is based on usage by DAPP developers.
Feedback
We have done our best to ensure that private discussions of this change were kept to a minimum, and that no final decisions on this change were made. We invite those who support this change, and those who do not, to voice their opinions in the comments to this post. The goal is to give Steem users one place where they can come to learn about this change, understand the various sides, and voice their support or lack thereof.
The Steemit Team
You have our support for more custom json!
You have our support for more custom json!
You have our support for more custom json!
You have our support for more custom json!
You have our support for more custom json!
You have our support for more custom json!
Sincerely Team Booster
I would like to remove this limit as much as possible. We have been having to built big work arounds because of this limit.
Make app developers have proper amount of SP but limits per block make everything clunky and complex.
How much do you estimate 3speak would need to run smoothly? 10? 20?
If a limit of 5 is not enough for 3speak, how will this ever be enough for future DAPPs that we can't even dream about yet?
As a workaround you could create a number of accounts and split RC over them to create this.
This change is super important to steem-engine and steemmonsters so that user actions never conflict and stop something from happening. It gets more important as new apps and automation are added.
We also would be greatly benefited as well!
Also making the necesary node/jussi changes to allow more than 2000 characters would be amazing as well that way we would decrease number of json transactions on a daily basis.
What happens if USER A with 20 token X sends 20 to USER B and USER C in the same block? Who gets prioritized?
The transaction that gets included in the block earlier by the witness producing the block.
Really interesting question. My guess is that asset transfers would be limited to one per block. Maybe there are a sub-type of transactions that are easier to bundle. E.g. if you and I play a game that ends in the transfer of an asset, then maybe all of our moves within the game can be bundled into a single block, while the asset transfer operation is included in its own block.
Any chance STEEMIT will make it so that only 1 of each unique coustom_json id per block?
In the end it depends on the software hosted by the witness to order the transactions. Whichever it deems first will be the one to be executed. In terms of custom json token transfers, both will be included in the block but the API of the server will only accept the first tx in the block.
Any chance STEEMIT will make it so that only 1 of each unique coustom_json id per block?
If there is a quick way to limit it again in case of emergency, I am okay with removing the limit completely.
These are all great ideas, no just scrap that silly EIP. Everything else you are doing is good for the price of steem, that is likely going to hurt it, it likely already has.
I approve this change, because as you mentioned, it will increase flexibility for DAPP developers.
I fully agree with the limit increase, the flexibility/speed it will give apps is crucial. The onus on adjusting to the change is on the app devs.
There's also no need for it to be a witness parameter. If there's a reason to adjust it upwards or downwards in the future, that adjustment is unlikely to be critical enough to alone warrant a HF and will be part of the next logical HF regardless.
I don't object at all to this change, but I wanted to caution those using custom_json's for derived consensus data, that this may change how you interpret ordering.
You can no longer do async/parallel application of transactions within a block as the account would not be guaranteed unique.
In other words, right now one can shortcut ordering as blocks are clearly totally ordered. With this change, one needs to pay attention to the order of transactions within a block to ensure deterministic application across observers.
Good point but I'm pretty sure this limit was implemented in 'non-enforced soft fork' and can't be fully relied upon for any security purpose anyway. If anyone is doing that they are currently vulnerable.
Are there serious downsides to having it be a witness-configured limit instead of a hard-coded limit?
Letting witnesses configure it gives us flexibility so that if we want to adjust it later, we don't need a hardfork or replay.
A possible downside is that a witness decides to ignore the other witnesses and ends up bricking accounts. But I think that's a risk with almost any op, not just custom_json. A witness doing that has to modify steemd and explicitly attack the community, which is grounds for unapproving such a witness.
You can't unapprove a backup.
In this case, the account being bricked has to also participate in the attack upon itself by trying to get too many ops included in blocks.
Not really. It is perfectly okay to broadcast multiple transactions at one time, I believe? They'll just trickle in one per block unless some witness puts multiples in a block. But I'm not really sure how this ends up bricking the account though. It seems different than the overuse of RCs that we saw earlier, since that affected the RC balance. There is no balance here.
It is currently implemented entirely via soft consensus (plugins). The biggest downside currently would be developer time to reimplement the system to use witness voted limits.
I think having generic "witness-configured parameter" logic added to the Steem blockchain would really be a plus. Once done, you will not have to reinvent the wheel.
I see many use case, on top of the custom_json limit, that would benefit being implemented as a witness-configured parameter.
Allowing more than 1 custom_json and rasing the size limit will be a great improvement for all steem bases apps. I support these changes.
Posted using Partiko Android
This is a great opportunity to improve the user experience of NextColony.
We're supporting this change.
This would a great improvement for all DApp developers. The current system is great compared to other blockchains, but this would help to lift the user experience closer to the level on non blockchain apps.
I wonder, what else will be added to HF21?
Is there a plan?
Are you making this up on the go?
Do you have a release manager?
Not that any testing is needed, I guess?
HF AZ-5
We make these determinations on a case-by-case basis, provide our recommendations, and solicit feedback from the community as well as the Witnesses. If the community wants something, and their Witnesses want something, we will do our best to oblige them especially if the changes are minor, low risk, and will not delay the hardfork.
I'm in favor of this change, seems low risk, and beneficial to dapps.
That's great news, I think steemit should focus on making the chain more developer-friendly, we've seen what community devs can achieve and it's important for attracting new business and giving dapps room to grow
Looks like a good change. A big win for the user experience with almost zero bad impact on the blockchain. I support this change as a backup witness.
I prefer the change to 5.
We fully agree with this change and will install a fork containinig it.
I‘m in favor of increasing the limit to 5. I don’t think this needs to be a witness parameter right now. Maybe in the future, but for now there are bigger fish to fry.
Just do this and SPS, then seriously consider cancelling the other changes.
Posted using Partiko Android
There's a lot of changes ... which ones. The change from 15 minutes to 1 minute?
They're coming out with a lot of small but solid updates each day.
2e12 reward curve concerns me the most. The rest are simple on their own but combined effects are confusing.
Posted using Partiko Android
Love this one! Good job communicating each day btw.
It'd be good to remind people about the timing of the proposed Hard Fork... keep it in their minds when this may happen and give them a reason to interact sooner than later.
I support the idea of gradual change. Stability is one of the top priorities.
This new steem is much better than the old steem.
Ya'll have managed to find a second change that we can agree on.
That's a start.
If it makes nextcolony load faster, even better.
Nice, an intelligent exchange of ideas and opinions under a Steemitblog. Gotta love that. No idea what you are talking about exactly, but I don't mind. Much better than the usual hate/sarcasm mix.
Posted using Partiko iOS
Great right?! So I can explain a little bit if you're interested. Custom JSON ops are essentially a way of storing code on the blockchain so that developers can make apps more decentralized and transparent but at very low cost (free even). This enables something we refer to as "Soft Consensus" which I like to describe as "trust through validation" rather than "trust through decentralized computation." The idea behind protocols like EOS and Ethereum is that in order for users to trust apps, the computations powering those apps must be performed by the blockchain itself. Steem's approach is different. We say that, yes, there are SOME computations users want done by the blockchain like those relating to token transfers, token vests, voting actions, the storing of speech (i.e. content), and a few others. But that's it. Developers can't ask Steem to "calculate Pi to 1,000,000 decimal points."
Bounding the amount of computational work Steem can be asked to do enables the efficiency which allows for many of the special features of Steem like free transactions. This creates a unique opportunity for developers, in that they can decentralize the mechanics of their app without incurring the costs that come from having that code processed by the blockchain. Granted, this is something that you can do on practically any blockchain. The problem is that this would be extremely expensive to do with any other blockchain, therefore if you were going to store your code on Ethereum you might as well have it computed by Ethereum as well.
Soft Consensus only really makes sense on Steem because of the speed and fee-less nature of it. By hosting the code to your app on Steem you accomplish two things: 1. you enable your code to be audited by developers who can then validate that you are executing the on-chain code faithfully, and 2. you open your code to 3rd party developers. For example, because the market for SteemMonster exists in Soft Consensus, as far as Steem is concerned, there is no such thing as SteemMonsters, just a bunch of meaningless custom JSON ops. But inside those custom JSON ops is all the code powering their market, which 3rd parties can use to build their own apps. So now anyone can build their own market for SteemMonsters cards, which is how PeakMonsters.com exists.
It is my thesis that of all the code needed to power a given app, 99% of that code does not need to be decentralized at all. Of the code that need to be decentralized (i.e. on a blockchain), 99% of that code (the 1%) only needs to be in soft consensus where it can be audited. Only the remainder (1% of 1%) needs to be on-chain and executed on-chain. I believe that what we try to do with Steem is to look at that .01% of code that is super critical for decentralized applications, and add those to Steem. This enables Steem to perform those Smart Contracts exceptionally well, at an unrivaled low cost.
Hope that helps!
I respect your right to feel that way, however, the key value add of Bitcoin and all blockchains is asset possession. Your argument for soft consensus is actually the same argument Peter Schiff makes for why he thinks Bitcoin has no value.
Schiff argues that gold is better than bitcoins in every way. This is because to him the soft consensus of validation that he owns the gold is enough.
This is in contrast to the "Not Your Keys, Not Your Bitcoin" argument. The argument for Bitcoin's value is that possession is crucial. Stocks, bonds, and commodities like gold and silver are not ever in your possesion even when you have a note that verifies that it belongs to you. Corporations still have many ways to get out of giving you the asset and/or the value of that asset.
The whole case being made by Bitcoiners and Etherians is that if you do not have full possession of your asset it isn't really yours. This becomes evidentally so in international business dealing wherein your ability to punish overseas businesses for misbehavior below the $20,000 range becomes quite difficult and rarely worthwhile.
Soft consensus is merely evidence of right to something. The problem with that is at the highest levels we see that evidence to a right is not enough. When Germany wanted to repatriate their gold the US Reserve bank indicated that it would take 7 years. That's ridiculous, if it was there it should not have taken so long. Clearly, possession is essential at all levels.
Now, if you need validation for a fairly non-valuable item then it is fine. No body cares about a receipt for a donut, but its good for it to be censorship-resistant in case someone claims that you walked out without paying. Soft consensus is great for accounting purposes, but for storing valuables like NFTs its not such a great thing.
Here's a perfect example of how validation is not enough. When MagicDice did their exit-scam, there was a mountain of evidence that MagicDice owed its players tokens/dividends. The Steem blockchain has transaction details full of validation of those players rights. Still, they got jack all.
Why is validation of ownership rights on the Steem blockchain not enough for the victims of MagicDice? Because if its not your keys, its not your bitcoin... Possession is everything.
This change is amazing! That's exactly what I needed. :)
Bring it on and make STEEM ready for the rise of Dapps.
Yes! We urgently need this change.
Would be beneficial for nextcolony to allow more game commands in one block.
Effectively limitations reduce discussions and there are discussions that deserve a good debate because of the level of interest they have.
One of the questions they ask me is: why are the discussions so limited?
Efectivamente las limitaciones disminuyen las discusiones y hay discusiones que bien merecen un buen debate por el nivel de interés que poseen.
Una de las preguntas que me hago es: ¿por qué son tan limitadas las discusiones?
Yes, please. This is an essential changed. However, five operations per block seems a little small (although, 5 is better than nothing is 10 isn't agreeable). I would preferably prefer to see this increased to ten. Specifically, this would benefit Steem Engine greatly as well as the airdrop bot that I have created which gets around this limitation.
Furthermore, can we please increase the limit of 2000 characters for a custom JSON operation for Jussi backed nodes? I would ideally like to see this increased to 10000 characters to make life a little easier from a development perspective.
I'm for it.