ABS’s night of disaster as servers crash and millions fail to complete the census

ABS Census home page. ABS

The Australian census has proved a disaster tonight as the Australian Bureau of Statistics (ABS) threw up its hands and declared that its online service would not be available on the night that the census was due to be completed and people should try again in the morning.

After struggling to convince the Australian public that its most private data was safe in their hands, the crash of its online service couldn’t have come at a worse time.

Again, to be fair to the ABS, the fault is shared with IBM who won the AU $9.6 m contract to host the census this year.

The ABS had tested the application at 150 percent of the expected load and paid Revolution IT AU $470,000 to carry out these tests. This lead to the claim that the website would be able to handle 1 million form submissions every hour. This sounds impressive but it actually equates to only 277 a second. When nearly 11 million people are accessing the site at roughly the same time, this would actually be a very small number.

Another issue with load testing is that it is hard to replicate the exact nature of how millions of users will actually try and access the site. Load testing usually just ramps up the number of accesses to the site evenly whereas the demand from real users is unlikely to be this even.

What the ABS didn’t count on is the fact that slow response times and failures cause people to constantly refresh or resubmit their forms and this just makes matters worse.

It seems inconceivable that a company like IBM would underestimate the expected load on their servers so badly. One can only assume that there is more to the story than is apparent at this time. IBM is still tarnished from its association with the AU $1.2 billion failed Queensland Health payroll project.

For the ABS however, they have not only damaged their own reputation and their ability to convince anyone to take seriously any of their technical claims, but they have brought into question the ability of any government agency to be able to run technology projects of this scale. This is specifically relevant given the recent discussions about running elections online.

“#CensusFail” was top of the hashtags trending on Twitter this evening although the majority of tweets were people venting their frustration rather than shedding any light on why the system failed.

From a technical perspective, it is not clear whether the failure was simply a result of the load on the services or actual bugs in the underlying system. The fact that the servers are now offline and won’t be restored until tomorrow suggests that the issues were more fundamental than just being overloaded.

It is also possible that given the publicity involved, the sites may have been subjected to some specific effort to crash them, including the possibility that they were targeted by hackers running a denial of service attack.

In any event, the ABS has proved that its ability to communicate with the Australian public in an effective way has been as deficient as its technology. There is also a clear question as to whether the entire census will need to be rerun as it has already been severely compromised.

Would anyone actually still trust the results that come from the census as it has been run so far?

Update: An ABS spokesperson has confirmed that the ABS census site was subjected to a malicious cyber-attack and that this was the reason the site was eventually taken down. It is not clear what capabilities the ABS would have put in place to deal with this type of attack. There are specific devices, software and services that can be used to defend against “distributed denial of service” (DDOS) attacks but whether the ABS was using any of this type of capability is unknown.

The difficulty the ABS will now face is that the short amount of time they have left to run the census will not give them much scope to get servers restarted, explain what they want the public to do and defend against future or ongoing attacks. It is possible that they will simply have to abandon the census at this time and try again with better infrastructure.

A possible motivation for the attack could be hacktivists targeting the ABS census as a result caused by the controversy around requiring names and addresses.