January 2014

Wow! That was some Third Birthday Party I had. I think someone must have spiked my drinks because I was on and off line for some time. Not dead but in some sort of coma. Never again. What has been going on?
– Survival UK (30-Jan-2014)

Sadly January has not been a good month for me. Several things have happened on a personal basis, nothing major but just so much trivial stuff it is unbelievable. Death by 1,000 cuts. The site issues being the one that was most visible to others and I would like to explain what happened to the sites.

The tale of woe starts with a few calls and emails from users stating that the forum is timing out or displaying server errors. On investigation it was really slow and as there had been no changes to the site I called the ISP support desk. This was 1&1 internet btw. They reported that my sites were taking up too many process resources and that the latest version of WordPress was very resource intensive. This came as a bit of a surprise as I was on an unlimited package so I stated this. Seems my package has been obsolete for some time and the recommended I moved to the latest super dooper Unlimited Plus, new servers with more memory etc. Only an extra £3 or so. So I said I would think about it.

I then went back to the sites and reduced the number of plugins etc. to reduce the load. I spoke to others who were hosted by me and asked them to do the same and discovered that one had upgraded to the latest version of WordPress. I suspect now that this was the straw that made the difference and we started hitting the limits.

The next day everything was suddenly worse. Every site using PHP, which includes the main site (WordPress) and the forum (MyBB) refused to work. Even simple PHP I’d written myself on other sites just stopped working. This wasn’t just SurvivalUK but every site I hosted. After a while I discovered that every PHP file in my disk space had been edited and a line inserted that basically killed the code. Even now I’m not sure what it does as it is very long and complicated and I have not had time to analyse it yet.

I managed to edit the main files for the forum, reinstalling the main software and editing the line out of every PHP file, a lot, in each plugin. All to no avail. The sites were still failing with timeouts and server errors. So I read the instructions for upgrading the account to the new sooper dooper package and then hit the key. They said it would take 6 hours on their website. Despite several calls it look three days. Three days of slow websites with timeouts and server errors. After three days I was on the super dooper package and to my surprise it was no better than it was before, timeouts and server errors.

So I called support yet again and during the discussion they mentioned that throttling was on on my account. He didn’t know why but he turned it off. The plot thickens. I asked about the code changes again and again met with a denial that it was anything to do with them. 15 domains, backup files that couldn’t be accessed via a domain all modified and nobody knew how. It was as if a script had edited every PHP file and added in a single line at the top but nobody had access to do that except me and, err… its on the tip of my tongue, Oh. them.

I reinstalled every site without any improvement. I was still told there were too many processes and it was recommended that I move up to the next package in line. A server of my own. My sooper dooper unlimited plus was again limited. For six WordPress sites, two forums, MyBB and PHPBB, as well as several custom built PHP scripts which had been working fine just over a week ago I was being forced to upgrade to my own server. I’ve come to the conclusion now that as my sites grew other site owners had moved from my shared server to the new sooper dooper packages or other ISPs and I didn’t have a lot of real competition for resources. On the new package I was competing with people like me and the server couldn’t handle it. So much for sooper dooper and umlimited Plus, what is the minus like? So, yet again, I needed to fix it and quickly. I was getting earache from a lot of people.

I’ve had enough of 1&1. They are great when everything is fine, uptime is good, but hit any issues and their policies suck. I had all my sites taken down a while ago because of one post on a site. No negotiation take the site down or we will. The site admin didn’t and every site in my package simply died. Every one. Bunch of PC whiners. So time for a new ISP.

I then had a look around and decided that I was going to be more careful in the future. I’ll have to domains at one location, with the sites at others, perhaps diverse sites. So I looked for some recommended hosting and decided to move SurvivalUK forums to 5Quid hosting, which I document everywhere as Squid, while I decided what I was going to do with the rest of the sites. They recommended DomainMonster for Domain hosting so after a look around at other offerings I went there. This gives me the option of changing ISPs within hours just by pointing my domains at any new ISP hosting platform.

I installed the forums on 5Quid hosting no issues with the aid of their technical support team. The service they give is fantastic. I couldn’t recommend them more. Within hours the forum was back up and running and I sent out a message to all. I just needed to work on the main site now. The only issue I had though is that the forum was OK space wise but not processor wise the main site was way too big for their package space wise. I could see issues with resources on the forums which I needed more so I opted for putting the main site on another host to spread the load and risk.

I looked around and chose GoDaddy as the site for the Main site with its associated storage and being a major ISP bandwidth was not going to be an issue for all the downloads. I signed up and installed the main site on to GoDaddys servers. Soon, the main site was in but not working properly. Whoo Hoo. But when I come to install the sub domains download, etc. I found that they would only go in a sub directories of the main site, this meant that every other site, subdomain and therefore all files could be visible under the SurvivalUK.net domain for those that knew how to look. I had to fix that.

This is where I made my big mistake. Unknown to myself to repoint the domain elsewhere on GoDaddy meant I had been allocated new name servers during the process. I had therefore disrupted the DNS to all the SurvivalUK sites and not just the main site but the forum and my emails were dead as well and I didn’t know. I knew the main site was dead as I was working on it but the forum was fine as far as I knew.

So I fiddled with the main site and it took me two days to realise it was a DNS issue and called support at GoDaddy at which point I discovered that I had lost contact with the forum and the emails as the DNS was not resolving. As a DNS entry can take up to 48 Hrs to be propagated around the world I updated the DNS with what I had noted were my GoDaddy name servers, unfortunately the incorrect ones and while I waited I decided to move the forum on to GoDaddy as well. After a further two days of down time it still had not fixed the issue so I called support again where we discovered that the nameservers I was using were the wrong ones. I reset them and six hours later the forum was up and running again and the main site was in the directory I wanted and visible but not working. I had to reinstall the software to get it back up. Both sites are now up and running on GoDaddy.

Whilst I was waiting the few days for the DNS to resolve I moved the rest of the sites on to 5Quid hosting. They seem to be doing well and started transferring the domains to DomainMonster.

So current status.

1. All sites up and running except a WordPress site which I will fix in the next few days by moving to 5Quid.
2. Most domains transferred to a Domain service except SurvivalUK.net and three others which I will start transferring Sunday evening. Not expecting any disruptions but I will not put money on it.
3. All Emails for SurvivalUK will need to be transferred. I’ll be doing that Sunday. So if you have a SurvivalUK email anyone sending you email will get a message to retry tomorrow. Sorry about that. You can have a full refund if you wish.
4. Once everything is off 1&1 I’ll terminate that contract. Good riddance.

So, as I said, January has not been good for me. Although nothing serious by itself just lots of trivial stuff that is overloading me. Now the sites look like they are stable I would guess that it should get back to normal.

That is if it isn’t a January issue but a 2014 issue. Mmmm. Fingers crossed.