Yeah, I know. You don't visit ENWorld.
Still, in the world of RPGs, ENWorld and RPG.net are the two big boys.
ENWorld is currently offline and has been for... no idea (see how often I peek?)
I'm not sure if it's three months of everything lost or just forum posts, but the loss of three months of forum posts is a HUGE hit.
It looks like the backup was corrupted, thus the huge loss.
Of course, this reminded me to back up The Tavern, which I rarely do. A hair over 90mb.
Might I suggest we all take this as a shared lesson and back up our stuff.
In Blogger, go to "Settings" - "Other Settings" - "Import & back up" - click "Back up Content" - it may take a minute or two - or five.
This isn't the first time it happened. Well, hackers took it down the last time, but the effect was the same: ENWorld down. Considering the bother that caused a few years ago, I'm amazed Morrus doesn't have better backups. I don't know if he can Kickstarter his way out this time.
ReplyDeleteand the KS in question was in violation of Kickstarter's own rules, which they willingly broke and pushed the ENWorld KS.
Deletehopefully those on the PBP games have their own backups.
stuff like this frustrates your base and you tend to lose readership, at least in the short term
He's at the standard step 2.
DeleteThe first time your database gets junked for whatever reason (hardware failure, someone making a silly mistake and deleting it, evil hackers, etc.) you realize people weren't joking about making backups.
Next time it goes belly up you can almost be sure that the backups won't work. Either they are corrupted themselves or they weren't backing up everything or they were silently failing. Then you realize people weren't joking when they said to test the backups.
Next time the backups will destroyed along with the actual data and he'll realize that people weren't joking when they said the backups need to be not just remote from the actual data but not writable from there except for the actual taking of the backup.
Everyone learns these all the hard way...
I believe they use vBulletin and MySQL. I have to wonder how big a full dump gets, and how fragmented the files get. It sounds like it takes time just to reindex the tables (which have to be done regularly).
DeleteThey also literally defrauded at least some of their backers on that KS. (I was one of them.) ENWorld is a scummy organization and I feel no sympathy for the travails they create for themselves through the combination of their unethical and incompetent behavior.
DeleteAs a professional database administrator, I feel the need to not only recommend REGULAR backups that meet your criteria for data loss (say... 24 hours worth?) but also you need to TEST them, that means taking a secondary site, and restoring said backups at least once a quarter just to make sure they're not corrupted. Case in Point: ENWorld didn't test their backups.
ReplyDeleteA backup isn't a backup until it's restored. ;)
DeleteVery existential. ;)
DeleteI agree with the testing of backups with a restore function, it's something I've done a lot in my career.
DeleteHowever, there are two factors involved with testing a full restore: cost and complexity. A staging environment will probably cost more, especially if you're talking about a second virtual or physical machine, and the cost also depends on just how much data there is. I've seen and experienced conditions where there is no practical way to test a backup because there is no staging environment available, because the cost is prohibitive (regardless of how important it might seem, some places can't afford or won't pay for it).
For instance, Tenkar could easily restore his blog to another server because it's mostly a blog and some comments, and I don't think he gets anywhere near ENWorld's traffic. ENWorld is a huge beast with a long history--it's mostly a message board system in existence since Eric Noah started watching 3e's development pattern. It might cost a lot of time and effort to setup a staging environment and truly test it, which could be prohibitive to do on a regular basis.
There's a lot of challenges to ENWorld. It's a popular site but I doubt it brings in much revenue, keeping things maintained well would probably take somebody with experience in coding and databases, and that costs money--or one could volunteer but I doubt they could dedicate enough time to it that it would need. I don't know ENWorld's technology stack but it probably is an off-the-shelf web forum software with customization, and forums tend to be slowly dying and there's probably some technical debt building up there. Unlike a general content site which could be rebuilt from the ground up, it's somewhat more difficult to convert a lot of forum threads, user accounts, and maintain legacy redirects so you don't lose SEO visibility.
This comment has been removed by the author.
DeleteConspiracy theories anyone?
ReplyDeleteThis comment has been removed by the author.
DeleteActually I do hit EnWorld pretty regularly. The front page news stories keep me in touch with what's going on in the industry. It was actually pretty useful.
ReplyDeleteShudder...that is something I wouldn't even wish on Whitman. Damn...I really feel for them.
ReplyDeleteNot sure why I'm having trouble posting this, but let me try when it's not a reply...(to AJ Fritz)
ReplyDeleteI agree with the testing of backups with a restore function, it's something I've done a lot in my career.
However, there are two factors involved with testing a full restore: cost and complexity. A staging environment will probably cost more, especially if you're talking about a second virtual or physical machine, and the cost also depends on just how much data there is. I've seen and experienced conditions where there is no practical way to test a backup because there is no staging enviornment available, because the cost is prohibitive (regardless of how important it might seem, some places can't afford or won't pay for it).
For instance, Tenkar could easilly restore his blog to another server because it's mostly a blog and some comments, and I don't think he gets anywhere near ENWorld's traffic. ENWorld is a huge beast with a long history--it's mostly a message board system in existence since Eric Noah started watching 3e's development pattern. It might cost a lot of time and effort to setup a staging environment and truly test it, which could be prohibitive to do on a regular basis.
There's a lot of challenges to ENWorld. It's a popular site but I doubt it brings in much revenue, keeping things maintained well would probably take somebody with experience in coding and databases, and that costs money--or one could volunteer but I doubt they could dedicate enough time to it that it would need. I don't know ENWorld's technology stack but it probably is an off-the-shelf web forum software with customizations, and forums tend to be slowly dying. Unlike a general content site which could be rebuilt from the ground up, it's somewhat more difficult to convert a lot of forum threads, user accounts, and maintain legacy redirects so you don't lose SEO visibilty.
Hopefully they can get back on track. I think that web site is a hard one to handle.
Hey, I keep trying to post a comment on the ENWorld stuff and it's disappearing after about 5 minutes...makes me wonder if the ENWorld situation could be contagious.
ReplyDeleteYou were snagged as spam. No idea what criteria Google uses to flag as such.
ReplyDeleteI saw your comments in email. Didn't know they were spammed. Working from phone so hopefully this worked.
And no, I don't censor.
I didn't think you were censoring, I was more concerned that there was a glitch with your blog--it'd be ironic if there was.
DeleteIf it was a spam flag, I suspect the flag came from three capital letters that started with S in the third paragraph.
Took your advice and backed up my own blog. All 521kb of it :D
ReplyDelete