The school's weather station webpage seems to have stuffed it sometime around Thanksgiving. Today somebody finally noticed and alerted the support staff.

My boss asks "where's the documentation?".

Right.  There is none. This system has been in place for ten years or more and fails occasionally.  When that happens we go in and fix it.

Start with the webpage that actually displays the data. It's pulling the data from a file that is supposed to be automagically updated. Except we don't believe in magic. The file didn't get updated. Now to find out why.

Since this is a scheduled event, cron has to be involved. Let's have a look at the crontab file. Hmmm. It's pulling the changes from another file that is supposed to be automagically updated. That one hasn't been changing either. What changes that file? It isn't cron. Or is it? That file is symlinked to a file on another computer. Let's go have a look at the other computer. Ah, I see. There's a crontab running there which generates the contents of the update file from a data file via a collection of python scripts. Let's have a look at those. 

As I suspected, they are pulling data from yet another file that is automagically updated. Right. It hasn't changed since November either. What changes this file? Time to scan the logs. Nothing.

OK, it's time to start from the other direction. The weather station is connected to a PC in the corner of a lab. Let's have a look there. It's hung and totally unresponsive. OK, maybe that's the problem. I reboot it. Then go back to the webpage. Nope. Nothing has changed.

OK, somehow the data has to get from the weather station computer to the other computer where the python scripts can munge it. Let's have a look at the logs.

The logs say everything is fine, but it isn't fine. Nothing. It's not happening. Well this is interesting. I check connectivity and network connections. They're OK. We've got an IP addess and pings work just fine. A closer look reveals that there's a Windows task scheduler which occasionally FTP's the weather files across the net to the second Unix box. The logs don't show any errors. Hmmm. The files aren't being FTP'd though. They aren't making it. Then I see a notice at the bottom of the screen. Updates were applied some time since the computer was last powered on - six months ago. OK, what updates? Windows firewall. Right. So I have a look, and sure enough the computer's FTP connection has been firewalled because of an automatic update. The FTP's are silently failing - and indicating success. This is pure evil. After several minutes I'm able to get in with an administrator account that can fix the firewall and do so. 

Then have another look. Still nothing happening. What could be the problem now? Ah, on reboot FTP is automatically disabled on the weather station software - again without any warnings. The logs again say everything is working and files are being transferred. More evil. What's the use of having log files if they lie to you? I turn on the FTP. Bingo - now the files get through. Now back to the second computer to manually process the files and dump them into the directory where the third computer can pick them up. Then back to the third computer to manually update the processed files.

Yay! It works.

Back to the documentation. How would somebody document stuff like this? There's just too much that can go wrong. I could use up a tree or two writing it all down. This is why we've got systems folks. 

 

 

Meanwhile, Russia shows off their new military uniform.

Interesting to see what this does to their next military campaign. Opposing forces will be lining up outside the Kremlin to surrender - especially if the Russians promise to torture their captives.    

This morning on talk radio they were discussing the power grid selloff - and one of the callers mentioned seeing on a website the 'California perspective', wherein it was claimed that prices would almost certainly rise. Hmmm. Wonder if that was my website...

WASHINGTON (AP) -- A large U.S. spy satellite has lost power and propulsion and could hit the Earth in late February or March, government officials said Saturday.

The satellite, which no longer can be controlled, could contain hazardous materials, and it is unknown where on the planet it might come down, they said.

----

Hmmm. Hazardous materials. Highly unlikely. What would be likely is if this claim were being made to keep 'visitors' away from the crash site until any secrets can be secured. Wonder if it's one of the Keyholes? That would certainly be a prize to any foreign government. It'll be a race to see who gets there first in any case.

I didn't used to care about these things - living largely outside the normal orbit trajectories, but that may not be true anymore. Skylab rained down over Western Australia, and I'm certain the U.S. government would classify anything outside Sydney as 'sparsely populated'.    

Yesterday I was walking on the path between buildings and a lady walking nearby stopped me.

"Watch out for the spider!"

Yikes. Yes, I do want to watch out. Where is it?

"Look right there, you can see the web..."

Yes. I see a web. Thanks for letting me know. Little did I know that I was now hopelessly caught in her web.

"It's one of those Asian spiders ... jibber jabber ... they were responsible for the burning of 7 million witches in the middle ages ... jibber jabber ... of course the Catholic church knew what was really going on ... jibber jabber ... Carl Sagan warned about what would happen if the DNA sequence was leaked ... jibber jabber ... Now the aliens have our DNA and they're using it to ... jibber jabber ... Prime Minister Rudd  met secretly with the church and they're ... jibber jabber ... You're an American, aren't you? Bush has had secret meetings with the aliens ... jibber jabber ..."

Finally I could take it no more and politely told the lady that it was fascinating listening to her talk non-stop, but I really had work to be doing.

"Oh, I understand. I hope you learned something and it sunk into your sub-concious... jibber jabber ..."

No. I've really got to go. Yes, I learned something. Bush has secret meetings with aliens who stole our DNA from the catholic church and it was all because of those darn spiders and the witches.

Thanks.

"Don't mention it."

How can I not mention it? This is all fascinating. It explains a whole lot of things.   

 

A bit of drama out on the F6 (Highway 1) this morning. A car smacked a cyclist near Kembla Grange. The authorities closed the freeway just about 1km north of me. This turned out to be a rather unfortunate place to be. In order to get off the freeway, the entire freeway behind me had to be emptied out, so that myself and the cars ahead could turn around and get back to the nearest off ramp. This took about an hour.

Once turned around, everything was diverted to surface streets to go around the accident. This of course choked them up as well.

But the real drama was just beginning to unfold. I was moving in a corridor of traffic along the Princes at 2km/hour (it certainly beats 0km), when it became apparent that my bladder couldn't hold out much longer. But there were no turnoffs, no place to go, because doing so would involve getting back into the slowly moving chain. The only option was to hold and grit teeth.

Finally was able to pull off in Figtree (another hour later) and found a Wooly's. I stepped up to the customer service desk shifting quickly from one leg to the other and clenching my teeth. The girl was busy finding a discount coupon for the older fellow ahead of me. After several minutes of this I finally went over to the nearest checkstand, interrupting the transaction in progress and proclaimed that there was going to be a rather embarrassing moment ahead if I didn't find a toilet very quickly.

Luckily she was sympathetic and fumbled around for a key and pointed me towards the toilets.

Just barely made it.

Finally arrived at work almost 2.5 hours late, to confront a panic in the server room. What a day!

I felt a bit nostalgic about Bayshore freeway traffic; though this same situation probably wouldn't have unfolded there. In the states, it is common practice in situations of death on the road to take a few pictures and roll the body off to the side. It is much more important to keep the freeway open than it is to investigate the accident. Sure, one or two people's lives have been severely disrupted, but contrast that with shutting down an entire regional economy for a few hours. Yes, I'm being a bit overly dramatic now... but it's not too far from the truth. 

 

Sir Edmund Hillary, who was the first person to set foot on the top of Mt. Everest, died. He is arguably the most famous kiwi that ever lived and I believe put the phrase 'Because it's there' into the social vocabulary. It's hard to imagine that only 50+ years ago Mt. Everest had never been conquered by anybody. One of the last parts of the earth to be touched by humans. Nowadays it seems every aspiring mountaineer has left candy bar wrappers (and sometimes their bodies) on the trail to the top. 

So would you buy a used car from this woman?

 

The FBI is re-opening the DB Cooper case to try and figure out if anybody is willing to talk after 30+ years of mystery. 

For those too young to remember, a man known as Dan Cooper hijacked a plane in the Pacific Northwest (U.S.A.) in 1971 and demanded $200,000 cash (a large fortune in those days) and a parachute or the plane would be blown up. The ransom was paid, the plane flew towards Mexico and Mr. Cooper bailed out - possibly somewhere near the Oregon/Washington border.

He (and the money) vanished without a trace, although a kid found a few thousand in tattered $20 bills in the woods several years later that presumably came from the ransom loot. Whether he lived or died after the jump has long been the subject of speculation. 

Mr. Cooper - if you're out there,  hope you're doing well; and a toast - "You took the money and ran". Cheers mate.