mysql – welcome to somewhereville (.com)

July 6, 2024

Goodbye, web.com. Hello, greengeeks.com

With hopes that this doesn't come back and bite me when something big breaks, a quick post to note that somewhereville.com is, after 23 years on interland.com (which I selected to go along with the land/ville theme way back when) and then web.com, now being hosted on greengeeks.com.

I can't speak to the actual numbers, but a number of us folk sitting on "legacy" services on web.com and checking their website umteen times a day had a very long mid-May when a poorly-planned php 7.4 to 8.3 upgrade brought websites offline for nearly two weeks (kicking and screaming into the future, as it were).

It went from horrible…

To bad…

To stuff I've seen plenty of times thanks to bad Jetpack updating…

To the point of at least seeing the site…

With that problem resolved, a historical restriction of 300 MB to MySQL databases for legacy accounts resulted in my being frozen out of my WordPress Dashboard for yet another week.

This locked-out issue had to be self-diagnosed because there were no red flags on the web.com support side (despite several asks), but a simple flipping on of WP-DEBUG in wp-config.php:

// Enable WP_DEBUG mode
define( 'WP_DEBUG', true );

// Enable Debug logging to the /wp-content/debug.log file
define( 'WP_DEBUG_LOG', true );

… produced the following:

WordPress database error: [INSERT, UPDATE command denied to user 'adminaccount'@'IP.IP.IP.IP' for table 'wp_wfConfig']
INSERT INTO wp_wfConfig (name, val, autoload) values ('lastPermissionsTemplateCheck', '1716472207', 'yes') ON DUPLICATE KEY UPDATE val = '1716472207', autoload = 'yes'

WordPress database error: [INSERT, UPDATE command denied to user 'adminaccount'@'IP.IP.IP.IP' for table 'wp_wfConfig']
INSERT INTO wp_wfConfig (name, val, autoload) values ('serverDNS', '1716472207;407;209.17.116.160', 'yes') ON DUPLICATE KEY UPDATE val = '1716472207;407;209.17.116.160', autoload = 'yes'

WordPress database error: [INSERT, UPDATE command denied to user 'adminaccount'@'IP.IP.IP.IP' for table 'wp_options']
INSERT INTO `wp_options` (`option_name`, `option_value`, `autoload`) VALUES ('_transient_doing_cron', '1716472208.0914690494537353515625', 'yes') ON DUPLICATE KEY UPDATE `option_name` = VALUES(`option_name`), `option_value` = VALUES(`option_value`), `autoload` = VALUES(`autoload`)

Warning: Cannot modify header information - headers already sent by (output started at /data/50/5/86/11/5086011/user/6116905/htdocs/swv/wp-includes/class-wpdb.php:1850) in /data/50/5/86/11/5086011/user/6116905/htdocs/swv/wp-includes/pluggable.php on line 1435

Warning: Cannot modify header information - headers already sent by (output started at /data/50/5/86/11/5086011/user/6116905/htdocs/swv/wp-includes/class-wpdb.php:1850) in /data/50/5/86/11/5086011/user/6116905/htdocs/swv/wp-includes/pluggable.php on line 1438

Which indicated an inability to write to the database. And how does a personal blog fill 300 MB of text with less than 200 posts over 23 years? And why was the sum total of my database under 100 MB when checking inside of phpMyAdmin?

Those are great questions. I asked those questions, and also asked why cnyo.org was sitting at 222 MB when my database backup file peaked at 8 MB.

The answer, from five different call/chat attempts, was MyTime Support – a pay service that offered the opportunity of getting reimbursed if the problem was legitimately web.com's, but which required forking over $59.99 for the privilege of a one-time call for an answer that any reasonable support service should have provided ("Thank you for letting me know. The My Time Support price for database issues is $59.99. Shall I add it for you?").

And that was the last plastic straw.

I looked at three hosting companies – dreamhost.com, nixihost.com, and greengeeks.com based on a bunch of frantic "We gotta get outta this place, if it's the last thing we ever do" searches. This was when I also learned about the animosity some folks have for the parent company EIG (? See wikipedia. Then take a look at reddit).

If you want to know, the primary selling point for the move to GreenGeeks was in the email. The catch-all email, to be exact. One of the perks of running your own domain name is you can have an email address for everything you want, meaning everything I sign up for can have its own email address associated with it (then let email client filters do the busy work for you. Either by selling or hacking, you'd be amazed where these custom email addresses end up over the decades). If you don't have a catch-all, and have been doing this for two decades, you risk having to remake N^M unique email addresses to get access to them all again. This might have forced staying with web.com (because providers don't much like the spam risk associated with this) and that took dreamhost and nixihost out of the running (according to official word from one instance of each of their chat supports. Your mileage may have varied).

Migration was almost flawless – on mine and GreenGeek's sides. They currently offer a free migration service, but it would have taken longer to find all the login information than just run UpDraftPlus (just the free version!) to backup my site on web.com and then restore it on greengeeks.com.

To make life easy, I decided to create a new WordPress site on GreenGeeks with their one-click installer and just restore my old copy with UpDraftPlus. I had an issue with the Default theme during the initial install, so picked a different theme from the one-click install list. When I initially created the WordPress site for my other moved legacy website cnyo.org and ran the restore from the Dashboard, my first visit ended as below:

Not ideal. And not ideal in 6 browsers, 3 machines, with and without a VPN turned on. A possible solution, as proposed by support, was a server-side cache clearing, as they would have cached the created site upon creation but not the migrated site upon initial migration (so I could get full access to the Dashboard, could go to most any other link on the site because the initial installation didn't have any links to play with, but would always land on the front page of the one-click install version until their cache was cleared out).

A clean-out on their side and I was good to go (which would have happened anyway, but not over the 15 minutes it took to diagnose and resolve).

Same problem with somewhereville.com – except the site loaded completely fine on the browser I was migrating in but not any other (?). 15 minutes and two quick chats and problem resolved. On the other hand, email hand-off from web.com to greengeeks.com was flawless and completed over the course of minutes (I didn't miss that one vital email that changed everything for me. Or whatever.).

According to GreenGeeks practices, I'm 300% green with GreenGeeks. I've got two kids now, so I have to think about these things. Also, the site and email are a hell of a lot faster than they were.

Having been burned a bit by the reduction in quality support over the last five or so years, and having played with the migration of cnyo.org hosting and email first (domain name transfer to another company and hosting to GreenGeeks) to see what the process would look like, I have learned a few important lessons about web.com in the process that I provide below if you hadn't had the pleasure (not unreasonable items for the time being):

do a chat first to get an official record of what you want to know and don't forget to copy + paste if problems/situations come up you're not happy with.
at least for web.com, where the support doesn't seem to keep a record of the times you ask questions unless it gets to the point of a ticket, ask three or four times on chat and by phone if you don't like the answer you're getting. Despite the training, some people will dig a little deeper than others and I had gotten out of three previous issues that they tried to throw at MyTime Support.
make sure your account is closed completely if/when you jump ship. I got an invoice for a forwarding service I thought was cancelled when they said "everything is closed" and had to call again to get my $2.34 back (and even more-so officially close the account).
Seven-year itch – I have the following from my chat when I called to close the account, but when I called to verify, the person on the other end knew of no such thing. Watch out for differing policies over two different mediums from the same company: "Unfortunately, due to tax reasons, we are required by law to keep your account open for 7 years from the last time logged in after all products were deleted. By keeping the account open, if there is ever an audit, we can provide you all necessary invoices and documents from the account. I can assure you moving forward, there should be no unauthorized charges as I already made sure that there are no active products and credit cards save in your account."
Domain name transfer is slow (5 days), but the updating of nameservers and propagation is fast (for me, 5 minutes to be getting email and website).
UpDraftPlus made the whole process very easy – I would have moved providers sooner if I'd known this. Once you create the brand new WordPress site, (1) the Restore overwrites the whole damned thing and (2) your wp-config.php doesn't need you if their one-click installer works right, so you don't need to know anything about your database or login credentials. If your provider offers a WordPress install option, jumping ship might be even easier than that.

December 12, 2013July 21, 2017

"From Kurdistan With Love" or Some Things To Do Before And/Or After Your WordPress Site Gets Hacked

"Hopefully, because he's busy." – Commissioner Gordon, The Dark Knight

On the plus side, www.somewhereville.com received its first update in just over 5 months. On the minus side, the new post was less than useful in many ways. I received a timely email from Dr. Obi Griffith of the Washington University in St. Louis Division of Oncology noting that my entire site was differently-down (thanks to the hijacking of my Sanger (And Illumina 1.3+ (And Solexa)) Phred Score (Q) ASCII Glyph Base Error Conversion Tables page that he linked to on a biostars site thread – so my thanks to Obi for catching something I likely would have gone weeks without noticing!).

The snapshot below shows the state of swv as of 3 December 2014. On the bright side (minus a friendly conspiracy to get someone else in trouble), I can say with some certainty that Serwan performed the content-ectomy (twitter: @S3RW4N, current email (although I suspect it won't last long): serwan_007 – at cymbal – hotmail.com, on the Facebook, etc. All sites subject to change as people try to track him/her down post-attack (he/she's been prolific if nothing else)).

Exhibit A. Flag is waving in the actual version.

Several problems. To begin, it's a gaudy hack, complete with rolling text and techno music. Second, the Television New Zealand (TVNZ) news service thought this hack to be significant enough to warrant actual coverage on their website when a similar file-swap on a WordPress (or WordPress-esque) site brought down the Health and Sports Fitness Club in Sandringham (syracuse.com didn't give me the time of day). I commend this Kurdish hacker group for their ratings. Third, the manner in which files were replaced in the blog (specifically meaning the index.php file) blocked every other post on the site from being accessed, so every link anyone had posted to a page anywhere else on the Internets was made useless.

That said, I appreciate that Serwan generally performs fairly benign attacks on websites. File replacements were clearly identified from a simple date sorting, the important MySQL database content wasn't touched, and Serwan even went as far as to set up a second Admin account so that I could quickly retake control of the site.

So, in light of the plight of the Kurdish people, I left the hacked version up for a few hours as I pondered what to do, which I discuss below.

My Spotty Procedure For Recovery:

What follows is a list of obvious and less obvious things to consider when recovering your WordPress blog from a hack. There are plenty of websites that show how to protect your site in the first place, then others that explain how to revive it (provided you do your own due diligence and back your site up regularly enough). What's below is not complete, but you can rest assured that google is your friend in such matters, so keep your keywords targeted and see what comes up.

General Considerations:

1. Don't use your blog. My last post at the time dated back to June 25th, during which time I've made several full backups (and kept WordPress up-to-date, the last time being 7 November 2013) of my entire site. In this respect, I was well set up to quickly recover from a hacking incident.

2. Keep a copy of your current running version of WordPress handy for file replacements. In my case, index.php was written over. All I had to do to recover was uncompress my WordPress 3.7.1 download, upload index.php to my server, and the site was back and running.

3. Have you backed up lately? This phrase has been in the .sig of my emails for many, many years. If your entire life is lived in the Googleverse (email, images, documents, etc.), then you're fine until the Earth's magnetic poles shift and wipe all the hard drives out (just kidding. I think). If you're a computational scientist and have TBs of data, it's up to you to make sure you have access to it all again. Same applies to WordPress. I've a biweekly alarm that tells me to back up several websites and I've an encrypted .txt file with all of the login info and steps needed to perform this backup. You should absolutely be doing the same if you're not.

4. Set up an additional Administrator. In my case, my admin account was hacked to change the associated user email address to Serwan's email. Obviously, attempting to log in, change the password, or what have you simply sent little pings of your futile attempts to the hacker. Having that second admin account will allow you to reroute your login efforts (and if they're both hacked into, there's still a way around. Will get to below).

5. Make a real password. At the risk of de-securing my sites by providing personal info, my typical password looks something like this:

d@!25fj014or&ydoSDfu

20 characters long, upper and lower, numbers, and non-alphanumeric characters. If you care about your site security, stay the hell away from the dictionary.

6. Dry-run your SHTF moment. Are you a survivalist? Can you identify edible berries by sight, build a lean-to, or stitch an open wound? Or are you the Marty Stouffer of the camping section at Target? If you've never had to work your way back from a complete disaster, you likely won't know how to do it either quickly, efficiently, or securely.

Ergo, do another WordPress installation in a sub-folder of your main installation, create a new database, make your site pretty, perform a full backup of your database and uploaded media, then break it, either by deleting core files or corrupting your database (deleting a table would do the trick). If you can put the site back together again (the uploading of the database back onto your server likely being the worst part of the whole process), you're likely in good shape for the real deal.

7. Harden WordPress. The good people at WordPress even tell you how to (although, admittedly, I thought I did all of this, so maybe there's something being missed that will go into a future iteration of this page).

8. Get rid of "admin." Several of the sites discussing WordPress hacks report that having this default account (or account default'ed) is a top-5 problem when trying to keep people out of your site. So get rid of it. Easily. Set up a new account, give it administrative privileges, then delete the admin account, which will ask you to attribute the current admin posts to another admin account.

9. Delete deactivated plugins if you're not going to use them. Plugins are developed by people. People often have lives that keep them from timely updates of security exploits. If you're using a plugin, that's one thing. If a deactivated plugin languishes in your plugins folder, never gets updated, and some hacker writes something specifically to exploit a security flaw in that old, poorly maintained plugin, that's all on you. So don't risk your pocket knife being a projectile as you walk into the MRI room and get rid of the knife before it comes a problem.

10. I know nothing about it yet, but am giving Wordfence a whirl presently.

11. Hey, check your blog every once in a while to make sure it's still you and not Serwan.

For The Specific Attack (From Easy To Harder):

1. FTP in and check file dates. The offending .php files (index.php and a hello.php containing the techno) were both dated 3 December 2013. Everything else was, at its newest, 7 November 2013 (from my last WordPress update). This made finding the hacked and previously not-present files easy. A cluster of important files with identically modification times and dates is an easy giveaway.

2. FTP in and check ALL the file dates. One never knows when something else is going to be placed into a themes folder, plugin folder, etc., to keep track of site access (that's why I delete all deactivated plugins). So, sort by date and scour the whole site for modifications and new files.

3. If you make it into your site, go right to your User Settings, change the email address, then change your password.

4. Check out something like Sucuri SiteCheck. Hopefully, this search will complement your initial search as well as test against known threats. I ran a Sucuri on a similarly-hacked site (in this case, indoorstinkbugtrap.com) and received the following notification of defacement (so the check worked).

securi.net results for fellow victim indoorstinkbugtrap.com.

5. If you can't make it in the front door, crawl through the plumbing. You can change your admin account from within MySQL using, for instance, phpMyAdmin (check your hosting provider for details if this is new information to you). In the case of phpMyAdmin, you can modify the admin account in six easy steps.

1. Log in to phpMyAdmin

2. Click on the Structure Button in wp_users (red circle)

3. Click on Browse (told you this was easy)

4. Click the edit button for your administrative account (red circle)

5. Change the email address back to your email and delete the current password.

6. Save and go back to our WordPress site, then request a new password.

And, While We're At It:

Serwan's twitter image currently features a white hat (the Gandalf-ian sign of a good guy/gal hacker) and a long list of sites that have been defaced with otherwise useless, feral medadata promoting Kurdish Hackers for google to get confused by. A search for somewhereville.com in google left the following bad taste in its results page for a week after:

Hacked By Serwan. Allah Is Greatest. Long Live Kurdistan. Thanks To All Kurdish Hackers. Follow @S3RW4N FB.com/Mr.S995

If I may be so bold (and I've told Serwan the same), the Kurdish people had a long history of getting steamrolled by an oppressive regime that, regretfully, first-world countries didn't put enough into stopping or acknowledging until the tanks rolled South into Kuwait. If you're gong to label yourself an ethical hacker, fine. Mangle the front-end of someone's WordPress site. That said, you could be educating others on the Kurdish people by including a few links into your hack. I live in America, where certain news services use "Muslim" and "Islam" in headlines purely for shock value when they want to appeal to an audience so narrow-minded that their hearing is susceptible to the Casimir Effect. I recommend adding the wikipedia article on Kurdistan and the Al-Anfal Campaign to future hacks (and I'm sure Serwan could find more) to provide a little substance to your efforts unless, of course, your goal is just to be a stupid-ass script-kiddie hacker.

If you're gonna hack, at least try to be productive. Meantime, this was a valuable lesson for myself on what to do to try to keep WordPress from falling into the same limbo during a time when I might not have had an hour to fix it.