We're working on mitigating it. Stay tuned.
There are lots of rumors and misinformation floating around. Don't believe everything you read.
(In the meantime, check out busy.org or chainbb.com. :)
We're working on mitigating it. Stay tuned.
There are lots of rumors and misinformation floating around. Don't believe everything you read.
(In the meantime, check out busy.org or chainbb.com. :)
Some questions:
Seeing 5XX errors on a global site like this really hurts confidence. Twitter has their fail whale. Github has the angry pink unicorn. Can we get something for steemit as well? A static page communicating that you are aware of an outage goes a long way.
Thanks for listening. I hope you and your team are able to navigate through this quickly and put things in place to ensure it doesn't happen again in the future.
Edit: More thoughts here.
I 100% agree to status pages! Steemit is lacking many things that can be expected from an entry level decent webservice. STINC seems not to care about the community that makes them big! But I think in cryptospace anything goes, spamming scamming and enormous egocentric behaviours and not understanding how to deal with communities, consumers etc including the lack of understanding how to actually make a good social network with a monetisation model for those who need to create, run, manage and operate it.
Well said!
Spot on, Luke.
As the popularity and visibility of the platform grows, we also need to put a little more effort into the "public face" of Steemit. We can't honestly expect the world to take us seriously enough that Steemit becomes a "household name" if we come across as a Made in Bob's Garage Production.
Yes, I think we need our own "Fail Mascot" here. We have tons of talented graphic designers here... maybe even turn it into a community contest/challenge.
Not super impressed with the twitter response... a little too "home made" and not very professional.
Luke nailed it. All 4 points were spot on.
Whoever is in control of the Twitter account needs to be removed immediately.
"Luke nailed it. All 4 points were spot on."
AGREED!
And as for @sneak's quote...
"We're working on mitigating it. Stay tuned."
Stay "TUNED" where exactly? Twitter?
Man...
I only use SteemIt because it's the only Blockchain out there that does what it does.
But based on it's "Team," it is also THE WORST.
That twitter exchange is horrible... Stuff like that puts steemit in a terrible light. I also would like to know who is running the account.
I just put some thoughts down in a post. We need our own fail mascot. I'm trying to stay positive about this stuff, but it's difficult when some basics aren't in place like a status page or a static "We're working on it!" page.
A little bit tragic, at least for me personally, that I made such a confident post on why we needed Steem and used a picture of the Reddit failure one...
Brovo, Sir!
Some basic, basic steps need to be taken.
What is the worst case scenario if we NEVER do that?
Seriously, though - this is a rare occurrence. If we do none of that, then what is the delta between doing all of that? There are some major security considerations involved in doing that that Twitter and GitHub don’t have to contend with.
We are different than other companies, and will likely do a lot of things differently than people are used to. Some will be better, some will be worse. In this case, though, I ask you to consider the alternative. It’s confusing for the subset of active users for the subset of time we are down. What is the harm done?
That Twitter thing was a straight fuck up, though.
Worst case? The company and the site will not be taken seriously by professional investors and brands who might otherwise integrate and risk their brand reputation by being associated with this project. I know that's an extreme case, but please hear me out.
This, I think, gets at the core of concern I've been hearing from the community over the past year+ I've been here. 10 hours of down time for a brand is serious harm done. Any and all downtime that isn't well-communicated and explained is harm done. Most professional companies fully and completely understand this. If Steemit, inc does not, that's really concerning. People that may have been supporters of the platform may never come back because of that failed first impression. It seems more shady if the site returns a default browser error than if the site has a status page and explains a professional team of developers know about the issue and are working on it. If people can't review a history of previous downtime on a status page, they can't evaluate if the site is legit or a scam during those outages. Too many people already think anything cryptocurrency related is a scam and impressions like this don't help improve that perception.
I have to respectfully disagree. Being out this long due to a DDoS attack, yes, that's very rare. Seeing a 5XX response on steemit.com? Unfortunately not very rare. Over the past year, it has happened many, many times to me and others. IMO, it's well past time to have a status page and a professional 5XX response page. For each hard-fork that I can remember, the site experienced some issues. IMO, it would be much better to display a status page instead of a broken site.
GitHub deals with PCI and HIPPA compliant source code for companies processing billions and billions of dollars worth of transactions. They have very serious security considerations. Same for Twitter. Can you imagine the brand fallout (or even global fallout) if the Twitter account of the president was hacked into?
I think I understand your perspective, but I hope you're open to hearing an outside perspective as well. What you're saying sounds elitist to me. Arguing Steemit has more advanced security concerns than other sites and therefore can't have a global CDN or a professional status page doesn't make sense to me. You have vendors for your web servers, your DNS, your image hosting, etc, etc. As I said before, if you don't trust your vendors then you need new vendors. If you do trust them but a status page, professional 5xx landing page, and clear communication are not priorities, then just state that instead of bringing up security concerns that, to me, don't make much sense.
I'm open to being completely wrong here and not fully understanding the unique challenges you face with this site, but so far, what I'm arguing for here seems pretty obvious to me.
I know I'm being tough, but I really am on your side. I've always been a big supporter, and I regularly get flak about it in the chat rooms. I really want Steemit, Inc to succeed. Unfortunately, too many people use the term "STINK" instead. IMO, being humble about weaknesses and open to criticism and improvement suggestions (and implementing them) will go a long way towards improving community relations.
Thanks for responding. I love that I can openly (and hopefully respectfully) voice my concerns and be heard directly by you and your team. I look forward to hanging out at Steemfest2 and meeting you all in person so we can tell war stories of major site outages I've experienced as well.
Tossing around PCI and HIPAA (not HIPPA lol) without understanding the specific security requirements of steemit.com in this instance just tells me “I don’t know what I’m talking about”.
That’s not elitist, it’s just you not understanding the specific risks to this site.
I’m happy to take some time at steemfest to explain in depth to you why what you’re proposing is a bad idea.
I think that’s vastly overblown, and I think you’re making it up to win an argument. Any downtime, splash page or no, harms the brand. I asked for the delta.
Sorry for misspelling HIPAA, thanks for pointing that out. I have some slight dyslexia, so it's unfortunately common for me to mix letters up like that.
I think the community would benefit from understanding more about the specific security concerns of this website. If I, a ten year veteran of my own software as a service company which deals directly with security, am ignorant of it then most others are as well. If our frustration is based on ignorance, please help remove that frustration through education. I'd really appreciate reading a post by you or the steemitdev account so I can better understand what makes Steemit so different.
Is this something you or your team will put together? Communication is key, and I keep hearing from the community how people want more of it.
I don't understand what you mean about a delta. You asked for the worst case scenario. I tried to come up with one like you asked. Then in a separate paragraph mentioned a delta. What do you mean by delta? Do you mean what's the difference in harm between going down without a status page, site down page, or clear communication about the outage, what caused it, and how it was resolved compared to having none of that like we do now? IMO, it's quite large. People are left with the impression this site is not professional and not ready for mainstream adoption or integration.
It seems the site is down right now again. I'm glad to see a Tweet about it, but without a status page or a static site down page, how is anyone supposed to know your team is working on this and taking steps to ensure it won't happen again?
protection for everyone !!!!
Bound to happen as STEEM ruffles feathers.
our myanmar steemit cummunity is in silence at this time , sir
::: ^ :::
our myanmar steemit cummunity is in silence at this time , sir
::: ^ :::
hi, i am new in steemit.com. I need your help. Brother can you please give me some STEEM POWER, I really need this, take care, bye
Steemit should have backup servers and a proper DDOS protection, and also add a NEWS button/section on the website so people know what the hell is going on. How many users are going to see this post and know there was a DDOS?
Yeah it was down for a few hours, it saved my voting power for a while haha
And... we're back!
Peer to peer networking shows its strength again.
yeah ! that is right
::: ^ :::
yeah ! that is right
::: ^ :::
While everyone is making valid points, I would just like to point out that many of us commenting here need to get over themselves and realize that calling for someone to lose their position becuase some assholes decided to try and ruin steemit for a while, is a bit overboard and that outrage is miplaced. People make mistakes, so what. You want someone to get in trouble for being unprepared to deal with an angry twitter mob?! Get over yourselves.
With respect.
Poprocks Out!
IMHO next time Steemit should redirect to an interesting Fail page like the famous Twitter Fail Whale... at least there's something to look at while the system is under maintanence...
Hackers are bitches !!!
That's a bummer. Hope you can combat it. At least we have alternative ways to access the system. I'm using Busy
For me, when Steemit is down, Busy is down too. When Steemit comes up, Busy is working too...
They are separate sites, so shouldn't be linked. I'm using Busy right now even though Steemit is down
nice.. Steemit's IT teams sould launch an investigation into those issues...
RESPECT! :)
OMG! Everything is alright right now. Thx you so much
Thank you for keeping us updated on the status. I am glad the site is back up again and the attack stopped.
Heeey @sneak. ¿Why suggest using third party apps and not even insinuate to try and start using steemit.com Twin Brother Guts once and for all? }:)
Why should I trust you with my keys ?
For browsing your service is nice, thanks.. but for account management, I don't know.
¿Aren't you using your posting key already to sign in on the others third party apps like eSteem, Busy or Chainbb right now?
I suspect you need to explore and actually READ a little bit more out there to gain actual knowledge mate. :)
I use them with the apps made by people that I know and trust.
Are you saying that the only key that I could use with your service is my posting key, that no-one can use another key and lets say use their wallet through your service ?
People that you know and trust. ¿Huh?
Ok, ¿How well you Know and would you Trust on.. eermm.. Hmm.. Ah yes! for example; @steemitdev & @steemitblog?
¿Are you willing to explore and read a little bit more now? :)
ps. And btw, it is not my service hehehe
@por500bolos, you ware right.
Ref: @steemitdev/help-us-test-new-performance-optimizations-for-steemit-com
I guess they didn't want to point the attackers to that site too.
You are absolutely right in your appreciation my dear @zinovi. They indeed cleverly didn't want to point the attackers to that site too. :)
So now, I am really happy that you have finally captured my cryptic message. After having had the will to explore & read beyond the usual just a little bit more, and then, you own now that more accurate knowledge and wisdom. Hehehe
Cheers!! }:)
what's this?
Awesome thanks for the updates. Cheers ;)
Still not up my side but i know things will come back ok...thanks for the update @sneak
@good-karma's eSteem app works fine on mobile devices, and https://busy.org works okay for computers during the issue.
it would be interesting to know who's the people behind it. most of the full-nodes are down as well (at least unresponsive). thanks for the update @sneak.
I take it that this is a good sign since Steem is now in the crosshairs. Comes with the added fame. I hope you guys are able to mitigate it without any problems!
Thank you for the update...
Is this why I haven't been able to get anywhere near the entire Steemiit website since last night?
I thought my ip had been blocked.
Link to my previous comment before I just tell you briefly here that only Busy works for me still and barely so.
I'm not propagandizing or putting on the tinfoil hat. All I'm saying is that no matter how we look at it, this downtime for so many Steem apps does show that there is an issue.
Well said @lukestokes.
If we like to run steemit as a professional service,
we have to take care of it like a professional service
So let us act like pros and find a way!
Thanks for this post. I am sure you are super busy and finding solutions!
Just curious, how's the fix?
Another DDOS? Seeing a generic 504 white page again. Feeling sad. Frustrated. Discouraged. Comments and posts have been just completely disappearing lately. No JS console errors. Just... spins and spins. Eventually says
Loading...
and the comment is gone. Had it happen on a root post last night also. Is your team using Steemit.com enough to see these issues also? Is there any logging to indicate how often it's happening system-wide?I've defended the steemit.com team for over a year now, just telling people to be patient and encouraging them that it will get worked out. Was I wrong? I'm looking forward to Steemfest2 where we can discuss this stuff in more detail in person.