Thursday, August 18, 2016

PM&C sets a new benchmark for public engagement in Open Government Partnership membership process

This morning the Department of Prime Minister and Cabinet (PM&C), through its ogpau.govspace.gov.au site, put out a call for stakeholders to express their interest in joining an Interim Working Group to help co-draft Australia’s National Action Plan (NAP) for the Open Government Partnership (OGP).

The approach that PM&C confirmed this morning is a very innovative and progressive one. I believe it is a model for government/civil society engagement in Australia that other agencies should pay close attention to.

PM&C proposed forming an Interim Working Group to decide which actions to put to the government for final sign-off and inclusion in the NAP.

The Group is expected to have up to 12 members, comprised of equal representation between government officials and civil society stakeholders. It will be co-chaired by a senior government official and a civil society representative.

Anyone can submit an expression of interest to join the Group, with expressions to outline relevant experience and expertise related to supporting transparency, accountability and open government.

It is extremely rare for government agencies in Australian to agree to 'share power' with external stakeholders in this manner during a decision-making process. The usual approach is to invite feedback from outside, but make decisions inside agencies, usually in a 'black box' manner.

The collaborative approach outlined by the OGP team in PM&C is a far more transparent and engaging one. It shows respect by granting near equal standing to external stakeholders and, through sharing decision-making responsibility, is more likely to result in shared ownership and ongoing commitment to implementing the decisions made.

The Interim Working Group announcement is the first public step progressing Australia's membership of the OGP  since the federal election. It follows a multi-stage consultation process which has included:

  • initial stakeholder information sessions run in Brisbane, Sydney, Melbourne and Canberra in December 2015 (Disclosure - I ran them on behalf of DPM&C, the presentation slides are here and a video of one of the sessions here).
  • a dual consultation process in February/March 2016 involving both an external wiki collecting action ideas for Australia's NAP  (over 200 collated ideas here) and an internal consultation with government agencies to identify actions they could commit to.
  • a Canberra-based co-creation workshop in April 2016 involving roughly 60 attendees from civil societies, agencies and individual stakeholders which aimed to aggregate and filter the collated ideas into 10-15 actions for the NAP (outcomes here). My report on the workshop, which I attended is here.
I'm very optimistic about this process, as the Department of Prime Minister and Cabinet has demonstrated significant engagement and commitment to the outcome and a willingness to listen to and involve external stakeholders throughout the decision-making process.

I hope other agencies keep a close eye on this process and the outcomes and consider where a similar approach might help them achieve public goals in a more effective and sustainable way.

Read full post...

Friday, August 12, 2016

What follows #CensusFail

I think it is now safe to say that, technically at least, #CensusFail has peaked, with the ABS and IBM successfully restoring most access for the Census 2016 site.

While there are still scattered reports of failures, not recognizing JavaScript is turned on, issues in some browsers and variable levels of access for people with VPNs, by and large the site is limping home.

Increasingly it appears that there was no large denial of service attack on the ABS, just a cascading series of issues which made the census service vulnerable to demand peaks, with perhaps a small attack being sufficient to drive it over the edge.

The repercussions and fallout for the incident will occur for a long time. Several official reviews are already in motion, all of IBM's advertising in Australia remains offline, and the ABS has not changed its engagement and communications course in any perceivable respect.

The ABS is likely to be feeling the initial impacts of the next demand spike - not of census traffic, but of Freedom of Information requests, with journalists, privacy advocates, IT experts and others all interested in understanding what decisions were made, where and by whom.

Hopefully the ABS will scale its capability effectively, unlike the Census experience, or take the high road and proactively release information, including server logs (anonymised of course) that allow external parties to understand the progression of events and clarify what occurred and the good work the ABS did to protect the data of Australians (their key commitment) throughout the incident.

The real risk now is that politicians and ABS management will try to switch back to business as usual too quickly, answering to official enquiries about the incident but refusing to answer to their real owners - Australian citizens. There's a tendency in most organisations to spring back into normal operations to quickly after a crisis, forgetting that the collective external memory is often longer than insiders expect.

The consequences of #CensusFail are likely to have ripples affecting every major government IT project, significantly reducing political and public trust in digital initiatives by many federal agencies, as well as impacting on state and local government initiatives.

In many other digital projects politicians and citizens will ask for additional safeguards to protect against a potential #CensusFail, no matter how unlikely it may be. This will add cost and time to these projects, pushing up IT expenditures at a time when budgets are being cut, causing agencies to delay and defer the more expensive or ambitious projects and attempt to keep limping along on existing infrastructure for just a year or two more.

In extreme cases this may increase risk, with already old systems pushed beyond their commercial lifespans, in broader cases it will harm innovation and cause governments to fall further behind their peers elsewhere in the world.

This seems a bleak picture, but I don't blame the ABS for this. It is a consequence of the lack of political IT expertise we continue to see across many Australian governments and of the risk-averse cultures that continue to flourish across governments despite the increasing downside risk of this stance.

It takes a long time to turn a big ship, and in this instance the ship is Australia. We are not educating our children or adults adequately to master a digital world and we do not have the level of IT knowledge or capabilities we need as yet to sustain a first-world infrastructure.

This flows through our corporations, our public service and our political leadership and it cannot be solved by the import or outsourcing of expertise.

I wish the ABS well in rebuilding their reputation following #CensusFail. In five years when the next Census is held the organisation will still be dealing with the fallout from 2016.

However the real failure and fallout in 2021 will be much greater if Australians have not invested in building our collective digital expertise to the levels we require to continue to grow our national wealth and economy, to sustain our standards of living and maintain our place as a first world democracy.

Read full post...

Wednesday, August 10, 2016

#CensusFail - What the ABS did well, what they didn't & what other agencies should learn from it - PLUS: Who attacked the Census?

I feel sorry for the ABS guys this morning - they've just seen three years of planning and effort effectively blow up in their faces, and I expect many staff are now scrambling to address the issues from overnight in order to move forward.

I know how dedicated and hard working they are, and how committed they've always been to providing a statistically accurate picture of Australia to support good government policy and services.

However I also feel it is appropriate to look at what's happened with the Census 2016 process - the good and the bad - and provide some context and thoughts for others across government on what they can learn from the ABS's experiences.

It is right after Census night, and much remains unclear - I expect a more complete picture of events to emerge over the following days and weeks. However there's still much that can be observed, critiqued and analysed from the events thus far.

Let's look at the facts to start.

This was the ABS's 17th Australian Census and the organisation has a highly enviable international reputation for its capability to effectively collect, securely store and usefully distribute Australian statistics. It is one of the best organisations of its kind globally and has been highly trusted and respected by successive governments and the Australian public for how it has conducted itself over the last 105 years.

Despite this, successive governments have progressively cut the ABS's funding, with the ABS forced to consider making the Census 10 yearly, rather than 5 yearly, due to budget cuts. In fact if not for a $250 million 5-year grant from the Coalition government in 2015 to upgrade ageing (30yr old) computer systems, it is questionable whether the ABS would be positioned to maintain Census timing and reliability.

The 2016 Census was the first in Australian history to be predominantly online. The ABS has had an online capability since the 2006 Census and was moving digital in degrees. For 2016 the ABS estimated it would cost at least $110 million extra to hold the census predominantly in paper, so the move to digital first was both sensible and pragmatic given the ABS's reduced funding.

As a result, the ABS aimed to have at least 65% of Australian households (call it about 6.5 million) complete the Census online.

Around the same time the ABS (not the government) also made a decision to retain names for up to four years (up from 18 months in 2011 and 2006) after the Census and create linking keys which would be retained indefinitely. The purpose was to allow the ABS to connect Census data with other datasets to uncover deeper statistical trends to better inform policy.

The ABS did conduct a form of public engagement over this decision to retain name data and linking keys, however this was quite limited - I wasn't aware when it was held and apparently it only received three public responses indicating concern.

The same proposal was discussed in 2006 and 2011 and rejected due to privacy concerns. I'm not sure how the ABS felt the situation or environment had changed to make this proposal acceptable.

Over the last four months privacy advocates, senior ex-staff of the ABS and non-government politicians have become increasingly vocal over this privacy change - with the discussion coalescing online at #CensusFail over the last two weeks (just prior to 'Census night').

The ABS's response to this has been to largely repeat (in various ways) 'you can trust us', and avoid publicly engaging with the issue or its underlying causes in detail.

Partly as a reaction to not feeling heard or engaged, the voices of opposition have gotten louder and louder, reaching the media and prompting at least seven Federal politicians to publicly announce they would not be providing their names to the ABS in the Census. Alongside this, IT specialists have tested the edges of the ABS's Census systems and highlighted several apparent vulnerabilities - which the ABS has also not engaged with in a public way beyond 'trust us'.

At the same time ABS phone lines for Census were widely reported to be congested - with the ABS in its site saying from 8 August that people should delay calling until the 10th (after Census night on the 9th).

It's worth noting that alongside this slowly rising opposition, the ABS was quite outspoken about how people who did not complete the Census could be charged $180 per day indefinitely until completed and those who provided false information could be charged $1,800 - although they were careful to wrap this stick in a little cotton wool, stating that there were only about 100 fines issued in 2011.

The ABS's Census campaign, similar to past Censuses, focused on 'Census night' - where Australians could take a 'pause' to respond to the Census and inform future policy for Australia. They tried to create a party atmosphere, with the Census being an engaging family experience that could be undertaken on Census night.

In fact people were able to complete the Census up to two weeks earlier (and 2 million households did), as well as complete the Census up to six weeks afterwards without any prospect of a penalty. While some people understood this, the ABS's communications campaign focused very much on that one Census night on 9th August - which was an approach likely to concentrate demand for the online Census system into a 5pm to 10pm period on that one night.

On Census night the Prime Minister and many others commented publicly via social channels on how easy the system was to use and complete - until around 7:15pm when the first issues began being reported online, with failures to submit completed Censuses and the Census site being slow and unresponsive.

From 7:30pm the level of complaints had escalated enormously, and shortly thereafter the ABS reports it took the site offline due to a series of at least four denial of service attacks on the site, one of which exploited a vulnerability in a third party service.

The Australian Signals Directorate (the government agency that other government agencies calls in when foreign interests or organised criminals digitally attack - and manage much of Australia's cyberwarfare capability) said that the attack was malicious and foreign-based.

However the ABS's Twitter account continued to cheerily post about completing the Census, TV and radio ads continued, and the ABS didn't announce the site was down temporarily until 8:38pm - almost an hour later.

The ABS then didn't announce the service would not be restored until 10:59pm - over three hours later.

This morning we learnt officially about the attack and the ABS's response. The ABS shut down the site to protect the data they had already collected, protecting the privacy of the 2 million plus Census forms successfully submitted earlier in the night.

The ABS's chief statistician, David Kalisch, told ABS radio this morning (10 August) that, "after the fourth attack, which took place just after 7.30pm [on Tuesday AEST], the ABS took the precaution of closing down the system to ensure the integrity of the data."

As reported in the Sydney Morning Herald, Mr Kalisch He also described the events that caused the issue: the system's geo-blocking protection was not working effectively, a hardware router failed, and a monitoring system "threw up queries we needed to investigate".

We've also learnt that the Privacy Commissioner is now investigating the matter.

The Minister responsible has said that the "Census [was] not attacked or hacked" - though this is somewhat of a half-truth. A Denial of Service attack is an attack, and can be used to attempt to 'brute force' access underlying data. The ABS successfully defended the site's integrity and prevented data loss (hacking), but it was definitely an attack.

Now there is currently some doubt over whether there was a Denial of Service attack, however for the purposes of this post, I'll take the ABS's word for it.

More details will emerge over the days and weeks to come, but let's look at what went wrong and why.

What went wrong and why

The issues raised prior to Census night may have explicitly focused on privacy and security, but they were really about engagement. Simply speaking, many people in the community felt that they had not been sufficiently engaged about the ABS's decision to change how long it kept personally identifiable data or how it would be linked to other datasets.

The ABS's consultation approach - which I missed entirely - appears to have failed to engage the group who were most likely to be concerned about privacy considerations, and the agency's attitude after privacy concerns were raised was too dismissive and high-handed.

This isn't simply my view - I've seen the same basic concerns raised time and time again about a level of ABS engagement arrogance and refusal to go more deeply into a conversation than 'trust us' and 'those concerned are crackpots'.

So the first thing the ABS did wrong was have an insufficiently long or engaging process of socialising the Census changes and reassuring key voices by demonstrating a very interactive process of addressing potential issues and concerns.

This perhaps speaks to the ABS biting off too much in this Census - both going to a digital-first model and introducing changes which many people felt increased the privacy risks. If the ABS had focused on one of these changes this Census (digital-first), and had then introduced the data retention changes next Census, they would have had a much easier job of it.

The concerns around privacy escalated as people found potential security holes in the Census system - such as this plaintext issue - which the ABS essentially ignored and dismissed.

Whether or not this, and other issues, were merely perceptional or were real issues which the ABS then addressed, acknowledging the concerns and addressing them in a mature and engaging manner would have gone some way to address the concerns and satisfy those raising them. Instead all the community received was government motherhood statements amounting to 'trust us'.

In a time when trust in government is low and people are regularly bombarded with media stories about data breaches, 'trust us' has no meaningful use as a government message. Instead the ABS needed to have a highly flexibly and collaborative approach to communication - inviting privacy advocates to special sessions with Census officials who could explain the technical nuances of what was being done, and policy officials who could explain the benefits of the approach.

With this style of engagement the ABS could have transformed the privacy community into advocates for the Census, rather than opponents, and greatly limited the pre-Census jitters which has resulted in a high profile loss of faith in the ABS and in the government.

A side-effect of the ABS's failure to engage was the creation and growth of the #CensusFail hashtag on Twitter - which then became a lightening rod for the subsequent issues the Census experienced on the night of 9 August.

With the ABS focusing attention through its media campaign on 'Census night' it was inevitable that most people would attempt to complete their Census forms online in a very narrow window - between 5pm and 10pm on August 9th. This was even though it was possible to complete the Census earlier or later.

With the Census being a national event and the high profile of 'Census night' due to both the ABS's positive #MyCensus campaign and the negative #CensusFail campaign, it was highly probably that someone would see the narrow high-volume Census night window as an opportunity to embarrass the Australian Government, make a privacy point or attempt to steal a massive honeypot of data that could have been mined for decades for commercial and political gain.

I'm sure the ABS foresaw this. They'd done extensive loadtesting and everything I'm sure they could to secure the system. However they were also partially responsible for creating the window in which there would be a peak load that could be exploited.

As such, the highly probably happened. Someone (or someones), somewhere in the world attacked the Census process with a series of four brute force Denial of Service attacks (again I'm taking the ABS's word that there were active attacks).

These attacks aim to flood servers with so much traffic that they give up secure information or fall over and stop working.

We know from the Australian Signals Directorate that the attacks were launched primarily from offshore. I'm unsure if the ABS had designed its system to separate known foreign and domestic traffic, which could have helped mitigate part of these attacks - but there are ways for attackers to mask their locations, so this is not a certain way to counter them.

We know that the first three attacks caused little damage other than minor delays and hiccups, but the fourth found a vulnerability in a third-party service and the ABS pulled the Census site down.

This was absolutely the right technical approach to take, but again the ABS made a series of unforced errors in its engagement.

Firstly, in the days before the Census the ABS declared that its system was unsinkable - "It won't crash". Technically this was true - it didn't. However the impact for users was the same as a crash, the system went offline.

The Prime Minister similarly said that ABS Census privacy "was absolute". This, to anyone involved in privacy and security is simply untrue - and the ABS's actions on Census night created a perception that the Prime Minister was lying or poorly informed by the ABS.

Both the actions above created a situation where the ABS over-promised and under-delivered. Rather than making people trust the ABS and Census approach, it has created more fear and uncertainty and made the ABS look, in the words of New Matilda, "like a deer caught in the headlights".

It would have been preferable if the ABS had made it clear that they had taken the advice of security and privacy specialists and taken all the actions within in their power to protect the Census process.

Rather than declaring "It won't crash", they should have said "whatever happens we will safeguard your data to the best of our ability", and highlighted that people could complete the Census over a period of time at their leisure, rather than having to pile in on 'Census night'.

In other words, agencies should underpromise and overdeliver. The ABS, like other government departments, already had a high level of trust and faith from Australians. Making undeliverable claims, as they did, to try to signal they were trustworthy only gave them enormous downside risk and challenged malicious external hackers to try to bring them down.

Next, the ABS did not tell people immediately about their actions by saying something at 7:45pm like "We've detected attempts to access Census data and, in the interests of Australians, taken the decision to take our system offline until we're sure your data will continue to be as safe and secure as possible". Instead there was no real public communication of the ABS's decision (right though it was) until Wednesday morning.

This was a major, major, engagement failure. People were engaged and trying to complete the process the ABS had asked them to complete. Yet the ABS did not have the courtesy to tell them that they could not complete it until hours of frustration afterwards (though the ABS did tell the government).

People had trusted the ABS, given up their evenings to 'take a pause' for Australia - and the ABS then left them hanging and wasting time - unsure if they would be fined.

This did more than damage trust in the ABS and government. It destroyed respect.

To other government agencies - trust and respect must be mutual. They are built slowly over time, but can be destroyed in an instant. The ABS just did that, and the impacts will be felt, by the ABS, by the Turnbull Government and by other agencies for years to come.

The impacts will be subtle. Every major IT project will be met with scepticism. Statements by Ministers and senior public servants will be disregarded and lampooned. What has been lost will take years to recover.

Now the ABS has to pick up the pieces and move forward - so what's the best way to do this?

This is a classic crisis recovery scenario writ large. Transparency is the key. The ABS has to own the issues and be proactive about engaging media and the public on what happened and why the ABS acted as it did.

It did get off to an OK start this morning with David Kalisch's interview on ABC radio. Now the community needs a continuing stream of engagement clarifying the steps the ABS has taken and what will happen next.

The last tweet from @ABSCensus was a 9:53am. it's now 1:51pm. The ABS realistically need to be communicating at least hourly on what is going on while peoples' attention is still focused on #CensusFail.

The ABS should reframe its communications campaign (using any budget it has left) to focus on how it put public interest ahead of its own reputation, pulling the Census site offline until it was certain it could guarantee the security it promised Australians.

The ABS needs to commit to deeper and longer engagement with the community and specifically privacy advocates around any future Census changes - potentially even step back from it's position on linking data, committing to working more closely to communicate the benefits and safeguards and listening to the concerns of the public.

In short the ABS has to eat humble pie.

If the ABS takes these steps it will, over time, recover the trust and respect of the public, rebuild its reputation and regain its position as one of the most respected and trusted government institutions.

However if the government and ABS can't get their story straight (already an issue) senior egos or politics get in the way, the ABS decides to be 'selectively transparent' and only share details it thinks are important to the people it deems important - the strategy will fail.

I have enormous respect and trust in the people who work at the ABS and hope they take the steps necessary to turn this tragedy into a resurgence. The responsibility for their future success is on their shoulders, and I hope they bear it well.


PS: Who attacked the Census and Australia?

I consider that there's four potential groups who may have led the attacks that led the ABS to take the website offline. Note that it's even possible that several groups attacked around the same time - seeing the same opportunity.

These are:
  • Random opportunity hackers - people who saw an opportunity to bignote themselves by claiming the Australian Census as a 'scalp'. With the ABS touting themselves as totally secure and a five hour window with maximised traffic, it wouldn't take much for an individual or group of hackers to decide to take down the Census for LOLs and credibility.
  • Organised crime - groups who systematically hack organisations for data they sell or use for fraudulent activities. These groups would see the Census as an enormous honeypot of data they can sell and resell for years to come, containing all the vital information for identity and credit card fraud. Again these groups would easily be able to pick the best time to hack the data as the ABS flagged it in their advertising, Census night when millions of households attempted to use the service. 
  • Privacy groups - 'organisations' like Anonymous often go after organisations they see as crossing the line on privacy - as the ABS was seen to be doing by a number of people. Their goal would simply be to disrupt the process as a political awareness tool.
  • State-sponsored hackers - a number of governments (China and Russia chief amongst them) use state-sponsored hackers to attack foreign governments, companies and other organisations that oppose their perspectives, embarrass them or as a tool in geopolitical positioning. In this case the Census was a high profile event for Australia, with a new one-seat majority government in place. Disrupting the Census would cause political damage and fallout, potentially even causing the government to fall, but at minimum reducing the trust Australians have in their government and creating distrust that could be exploited later. Given than Australia has been considered by China as embarrassing China on several recent occasions - over China's East China Sea installations and at the Olympics, an attack disrupting the Census might be considered proportionate retribution to 'teach Australia its place'. China has been known to perform similar acts against Australian non-government organisations, such as film festivals showing politically sensitive films - attacking the Census would be a bold escalation, but a plausible one. These hackers may also look to access data on individuals as a secondary bonus - as a foreign government could use it for years to come for commercial and political gain.
My view on the above scenarios - random hackers are least likely. Australia isn't that significant and the Census, while a public target, isn't a 'hard' one by international standards. Military targets are more often targets for credibility.

Privacy groups would have flagged the attack. Their gain is in public exposure and ridicule and, while there's been plenty of that for the ABS, they'd want to clearly own the hack and make their point.

Organised crime is more likely, but attacking on Census night is potentially too early as not all the data is in. It would be more probable these groups would target the ABS once the data is collected, or would attempt a social attack (paying an IBM or ABS staff member) as this is likely easier and less of a public signal than a brute force attack. These groups don't want public attention, so this form of public attack is rare.

In my view the most likely scenario is that state-sponsored hackers targeted the Census to embarrass the Australian Government in return for a perceived slight. I reckon the Chinese have the most reason right now to do so, although the Russians, when annoyed at Tony Abbott's 'shirtfront' comments, did send a significant navy force to Australia's coast using a G20 meeting to signal their disapproval and defiance.

There's few other nations that publicly have an issue with Australia (and the capacity to carry out this attack) - but very remote possibilities are North Korea, or the failing IS (though IS would have taken credit right away).

Read full post...

Friday, August 05, 2016

Is it time for governments to extend digital security protections to all parliamentary candidates & parties?

Over the last few years we've seen increasing attention on the use of personal technology by politicians.

From our current Prime Minister, Malcolm Turnbull, who uses Wickr, to Hilary Clinton's use of a personal email server, and even the struggle President Barack Obama faced to use an iPhone, politicians - like the rest of us - are increasingly using a diverse range of technologies to conduct both personal and official business.

Not all of these technologies are officially approved or secured. Many are newer technologies with both known and unknown security concerns.

However politicians, like the rest of us, continue to use them either because we perceived the benefits (convenience, flexibility, speed, utility) far outweigh the risks we accept, or because the risks are not clearly understood by non-technical people.

This becomes a particular issue for politicians, political parties and individual candidates for parliament when state-sponsored agents, organised crime or unscrupulous businesses attempt to access their information.

There's many motivations for 'political hacking' - commercial advantage where particular information or decisions are obtained before the market knows, political advantage, blackmail or an improved capability to 'groom' politicians to a given perspective supportive of a particular desired goal or outlook, or opposing an undesired reform or initiative.

In fact I think it can be said that political power doesn't only originate from the muzzle of guns, but now political power also emerges from the keyboard.

Information is power, and the best source for information about an individual's views and decisions can be their private email and social accounts.

With the revelations of Russian state-sponsored hackers penetrating the Democratic National Convention and Clinton's Presidential campaign data stores, it's clear that state-sponsored and other organised hackers are increasingly seeing unelected potential parliamentarians as targets.

This is a logical development. It's in the interest of foreign nations to understand the views and decision-making approaches of powerful national leaders. Combine this with the likelihood that the security deployed by a political party is far easier to penetrate than the security deployed by a national government, and the fallout if caught is far less and it becomes a no-brainer for nations and large commercial interests to conduct hacking before an election locks away leaders behind tighter firewalls.

So, now we know that there's a reasonable to high risk that electoral candidates and parties will be hacked - particularly if they have a good chance at election - there's a question for governments to consider.

Should governments extend their security expertise and protections to all electoral candidates, placing them behind state-supported firewalls and security provisions, as soon as candidates nominate for electoral roles? And should this protection be extended to all political parties as well?

Given that even medium-sized governments, such as Australia's, secure hundreds of thousands of devices and people through their security regime, extending this to a few hundred more would be a technically manageable exercise.

The approach would help protect more of Australia's governance institutions from foreign and commercial influence, though likely would only be a partial measure as traditional intelligence gathering and governance influencing methods (background research, infiltrators, electoral donations and hosted trips and tours) would still be available to interest groups and countries.

Individual politicians and candidates would still have personal digital accounts vulnerable to hacking, with which they may engage with the public, the media, each other, business partners, friends, family and, occasionally and hopefully discreetly, with potential sexual partners.

So perhaps the step would provide partial protection - avoiding situations like the one the US Democrats have found themselves in, where the long-term ramifications are as yet unclear.

However even government systems are not totally impervious to cyberattacks, and the limitations of working within a government firewalled system might be too invasive or restrictive for some in the political world.

Also in a world where no security is perfect, partial protection can provide an illusion of security where none should be assumed, with the potential that protecting candidate correspondence could lead to more significant information theft or leaks from either hacking or internal disgruntled staff - or the misuse of candidate data by a future unscrupulous government to influence an electoral result.

On balance I think we're going to have to take our changes over whether political parties and individual candidates are hacked by foreign or corporate interests.

No security solution will ever be perfect and so Australia, and other nations, need to focus less on hiding potentially damaging information and focus more on developing transparent and fair agendas, with individual candidates and politicians being as honest and forthright as they claim their opponents should be.

Read full post...

Wednesday, August 03, 2016

The consequences of dropping the ball in digital engagement - The ABS and Australian Census 2016

Next week Australia will be holding its 17th national census (since 1911), led by the Australian Bureau of Statistics, which is itself celebrating its 110th anniversary as an agency (albeit with a name change midway).

This is an auspicious occasion for another reason. While it has been possible to complete the census online in both 2011 and 2006, when the ABS first trialled an online completion system - 2016 will mark the first occasion when the ABS expects a majority of households to complete their census surveys online.

In fact, Duncan Young, head of the 2016 Census process, is on record stating that the ABS expect 65% - two-thirds - of households to complete the Australian Census online, rather than in paper form.

This is a fantastic achievement and speaks highly to the ABS's commitment to quality data collection and maintaining a forward-facing approach to trialling and adopting new technologies.

This commitment has also been typified by the ABS BetaWorks Blog (sadly now defunct), ABS CodePlay (sadly not repeated) and the work the ABS has done to expose data in open and machine-readable formats, including ABS.Stat and APIs such as for the Population Clock.

Data collected by the ABS, particularly via the Australian Census, underpins an enormous amount of evidence-based decisions made by all levels of Australian government, as well as by companies who access the information to guide their commercial decisions.

The census is also an enormous undertaking. To quote Wikipedia quoting the 2011 Census site, "the 2011 Census was the largest logistical peacetime operation ever undertaken in Australia, employing over 43,000 field staff to ensure approximately 14.2 million forms were delivered to 9.8 million households." The cost was $440 million.

That makes the census a prime target for budget cuts - with the idea of reducing the frequency of the Australian Census to every ten years, or reducing its complexity, thrown around last year before being dropped.

The impact of not having regularly collected census data, collected in a compulsory manner from all households, can be hard for Australians to imagine.

However in countries like Lebanon, which hasn't had a census since 1932, the lack of accurate data leads to opinion-based government decision-making, which is generally viewed as a poor alternative to fact-based policy decisions.

The need for compulsory collection of census data was highlighted by the decision by the former Canadian government to make their long-form census voluntary in 2011, resulting in a massive drop in participation and corresponding degradation of data quality.

Called "a disaster for policy makers", unfortunately it suited the Canadian government of the day to not have accurate data in order to provide them greater room for making ideological decisions, rather than decisions that were based on facts. The net result was a drop in participation from 95% to 68%, a more expensive Census process (due to increased mailout of forms to prompt engagement), the resignation of several of the most experienced and competent senior officers in Canada's statistical agency, ongoing issues for national, provincial and local Canadian governments in identifying disadvantage, population numbers, statistical population changes and reduced capability for companies to make appropriate commercial decisions without investing in further expensive research.

The current Canadian government reinstated the compulsory long-form census, which completed collection in May this year.

So regular compulsory censuses are a BIG DEAL for a nation, and Australia has a very strong statistical foundation to build on.

The ABS has also demonstrated leadership in how it has marketed and communicated past Australian Censuses. In particular in 2011 the ABS demonstrated global leadership in the use of digital channels and tools to promote the importance of the Census and lift participation.

Through quirky best practice engagement on Twitter and Facebook, which made the Australian Census front-page news for all the right reasons, the development of an interactive online service allowing people to 'place' themselves within Australia, and a mobile game which allowed people (particularly kids) to see how census data was used in civic decision-making, the ABS knocked it out of the park in terms of its communication strategy and implementation.

That's a fantastic base for the ABS to build from. I think a number of people were expecting the same, or better, engagement from the ABS in 2016.

Alas, it was not to be. In 2016 little of the previous engagement brilliance is evident from the ABS.

While the ABS has repeated a level of their communication via Twitter, it's basically a shadowy repeat of their 2011 strategy - as though new management said "repeat the good stuff from five years ago, but don't update anything or take any risks".

The ABS is also remaining stalwart and largely silent in the face of several decisions which have left census collection exposed.

Their online service has been exposed as using an older and less secure security standard in order to support older browsers, rather than taking an approach which warns people and encourages them to upgrade to a more secure technology.

For non-technical people, an analogy would be the police waving past someone without headlights on a dark night onto a crowded and unlit highway in order to not slow down the traffic flow.

On another front, the ABS is confronting a surge of privacy concerns around its decision to keep names and other personal details connected to census data for at least four years. Taken without consultation with the public, this decision has raised alarm bells with privacy advocates and organisations such as Electronic Frontiers Australia, as well as with former senior officials of the ABS.

While the ABS has been fighting back to some degree, they've not really addressed the concerns in an effective way.

#Censusfail is continuing to grow as a hashtag, with a number of people considering ways to circumvent responding to the census, avoiding providing personal information or considering providing false information.

Should enough people take one of these steps it would reduce the value of the census to Australia.

I must admit that I've also become concerned about the ABS's approach, and unconvinced by the ABS's engagement on this front to-date.

I totally support and value the ABS as an organisation, and all the people that work there - however they are burning much of the goodwill they established in 2011 and potentially devaluing the census, and hurting all Australian governments through their lack of effective engagement on the issues above.

The worst thing for me is that the ABS has been a shining light in Australian government. The organisation has consistently been a leader in open data and the use of digital and social media to engage with the public.

This is important not simply for the egos of the leadership at the ABS, but is essential for good governance and effective commercial decision-making in Australia. The ABS's success serves all of us - and its failure would hurt us all.

I hope the ABS recovers from this and Australia continues to be well-served by the statistics the organisation collects.

However it would have been far better for the ABS, and all Australia, if the ABS hadn't put itself in this position of needing to recover at all.

Read full post...