trouble.net.au blogs

Folks,

Trouble has just undergone its biggest change in a few years: It is now CURRENT because I upgraded it to Ubuntu 8.04, Hardy Heron yesterday. Yay!

For those who use it, the biggest change is that we’re now running Apache 2.2, which thoroughly screwed my authentication setup for a few hours yesterday. There was also a glitch during the upgrade where Trouble forgot who its users were for about an hour in the middle of the day. This is important because anyone with a high frequency mailing-list subscription on Trouble may have been kicked due to ‘no such user’ bounce messages generated during this window. If so, my sincere apologies: I am usually much better at avoiding this kind of thing.

As always, O user-base, let me know if anything isn’t to your liking.

After extensive research, I have come to some conclusions about the entry-level hosting market:

1. If I don’t mind being in the USA, it still doesn’t get cheaper than ServerPronto, for equivalent resources, not even with VPS.
2. If I insist on local (Australian) hosting for latency reasons, I would have to go with a VPS, and probably pay more, especially for traffic allowance.
3. The pain of moving to a new OS instance, essentially rebuilding from scratch, is too horrible to contemplate right now.

So Trouble is staying put for the time being.

In case anyone is interested, here’s a list of Australian entry-level VPS places, compiled with the help of the SAGE-AU technical mailing list:

• netlogistics.com.au
• labyrinthdata.net.au
• gplhost.com
• xenion.com.au
• web24.com.au
• rimuhosting.com

I am, however, probably going to upgrade, once Hardy Heron is out, since it’s the next LTS release, and as such it has an in-place automated upgrade path from 6.06, which is getting a bit long in the tooth. This will be preceded by plenty of careful backing up and provision of lots of nice rollback mechanisms, so there will probably be more news here before any such upgrade goes ahead.

Later-the-same-day-edit: E points out that crucial.com.au are currently having a special which brings their rates down roughly into line with the competition!

Trouble is moving.

It may not be tomorrow, nor even the next day, but Trouble will move.

You may have noticed that Trouble vanished on Wednesday and reappeared late last night.  This is because ServerPronto decided to cancel my account with them without warning or any reason that they could explain. This has now been remedied, but not explained.

So, we’re moving, probably to entic.net.

Folks, heads-up: The email filter-maker in squirrelmail is broken, probably has been since the disk-crash. I will fix this when time allows, and post the fact here, but it’s a big job, so it won’t happen today.

If you require ultra-urgent fixes, like because you’re being spam-bombed by something, call me.

You may have noticed that Trouble was down all last night. This was because it started getting hard IO errors on its one and only disk, and somehow autonomously decided to make it’s own /var area read-only. It took a hard reset to get it back on its feet this morning.

In reaction to this, I have installed the smartmontools package and set it up to frantically mail me if anything goes even slightly wrong with the disk. Will keep you posted, if the posting forum remains available.

In unrelated but reassuring news, the OSSEC author Daniel Cid has posted a straightforward exaplanation of the spurious rootkit detection issue: netstat won’t list a socket if it is allocated but never used, however attempts to re-allocate the socket will still get ‘in use’ messages. This will very likely happen a lot if you have something moderately hefty stomping on your available ports like, say, the LDAP which underlies trouble. Hmmm. There may be a fix, we hope. I have asked for one, anyway.

Thanks to a burst of furious wheel-reinvention and a lot of horse-trading, we now have a nice simple daily off-site backup being generated. The backups are encrypted and traded with another ServerPronto subscriber who now in turn backs up their system onto Trouble.

At some stage in the not-too-distant future, I will get around to publishing the thoroughly over-engineered ‘hedge’ script that’s handling the backups, since it seems to fulfill a useful purpose. I’ll post a pointer here when I do.

Also, wanting to welcome two new admins, and acknowledge one long-standing one. Wanting to, but I’m not going to use anyone’s real names or even usernames here. You know who you are.

Long aens have passed since I last updated this blog, mostly because very little has happened. My paranoia about having been cracked persists, but so does the $70 USD fee for re-imaging which serverpronto would charge me if I wanted to be really, really sure.
Many tools and much mailing-list reading have lead me to the conclusion that I was probably just imagining things. Certainly there has been no mysterious pressure on my disk space, no inexplicable system anomalies, and no sign of this IP on any spam, exploit or DOS blacklists.

Like Descartes, I must resign myself to the knowledge that if evil demons are in fact deceiving me about the state of things, they at least seem to be doing an adequate job of it.

At least now we have the rudiments of a backup system, and a lot more security than before. It is, I suppose, astonishing enough that Trouble is running at all, given how little time I have to work on it these days.

The morals of this story:

1. Never run an out-of-date copy of chkrootkit.
2. Never do security analysis at midnight after a long and difficult day.
3. Don’t assume that just because some asshole has managed to foo-up your insecure web app, that they’ve kitted your entire machine. Apache can save you from some things.

We now return you to your regularly scheduled paranoia. And I scurry off to do some hardening exercises. Yurk.

It looks suspiciously like someone has just cracked Trouble and put a root kit on it.

That’s really anti-social guys, and very very annoying. What did I do to you?

If you want to use Trouble for something, for christ’s sake, just grow up and ask me like an adult. I will almost certainly say ‘yes’, especially if you tell me how you got in.

For all the legitimate users of Trouble who come in the front way, please be advised that Trouble may be going down again for a few more days.

Merry Christmas.

As of this week, the Trouble blogs are now also on released, supported code. Yay!

I am immensely enjoying using a real blogging engine, even if no-one else sems to care.

Still outstanding: You may notice that there is no longer a ‘register’ link on the front page. This is deliberate. For the time being I have decided that there is no particular cause to allow automated registration, expecially in light of the fact that last time I did this, some asshole used my site as a base for Phishing. If you want an account, ask. I will not hesitate to give you one if I know you, even vaguely, and I will probably give you one even if I don’t. Since I have largely abandoned virtual addresses, a Trouble account will now come with a whole buttload of nifty features, whether you intend to use them or not: secure shell access, arbitrary amounts of disk space, wild and crazy web scripting power, and the ever present factor that if you want a feature, you can ask me for it, and my curiousity will probably get the better of me.