'Governments do not like freedom of information': the war on Australia's privacy and information watchdog

When Australian’s freedom of information and privacy watchdog was launched at federal parliament in November 2010 it was seen as the beginning of a new era of open government.

The office of the Australian information commissioner (OAIC) was supposed to herald Australia’s new era of transparency and accountability. The then home affairs minister, Brendan O’Connor, said at the launch that the office would reign over a new dawn of “pro-disclosure culture” in government.

Almost five years after it began, the story is very different. The federal government has decided to kill off the OAIC, but the office has continued on with limited funding. Uncertainty over its future lingers. The first Australian information commissioner, John McMillan, left his office in July 2015. It is possible he will be the last. He spent the last several months in office working from home.

The freedom of information commissioner, James Popple, also left in December 2014, leaving just the privacy commissioner, Timothy Pilgrim, who has now been appointed acting information commissioner until the end of this month. This uncertainty has come at a time when the privacy functions of the OAIC have arguably never been more important, and it has now been tasked with an even greater responsibility to oversee parts of the mandatory data retention scheme.

A team somewhat diminished in size, but no less committed, is now doing more than ever under Pilgrim’s leadership to enforce Australians’ privacy and freedom of information rights in what he says is a “great testament” to the commitment of his staff.

The uncertainty around the OAIC is the culmination of many factors. But it comes back to one constant, which McMillan outlined in his first interview with Guardian Australia: governments do not like freedom of information.

The mission: an independent champion of information policy

The information commissioner’s office was set up to draw distinct but complementary areas of government information policy into a single space. It’s an exciting – and quite rare – opportunity to build an entirely new office in government.

Freedom of information was a big part. The office would create guidelines, take on reviews and hear complaints about FOI matters. This would be at low cost, with no charge for review applications, in contrast to the $800 application fee to the administrative appeals tribunal. It was also less formal, with cases decided on written submissions. An emphasis was placed on informal dispute resolution.

But the OAIC’s reach was far greater. It makes guidelines for credit reporting with financial institutions. It forces companies and web services to write privacy policies. It helps medical organisations work out how best to store Australian’s medical files. It receives a greater number of privacy related complaints each year.

The office came into existence at a time when similar offices were being established by state governments. Deirdre O’Donnell, who was New South Wales’s first information commissioner, says information commissioners are “important in a very profound way” for the way governments operate.

“It was an exciting time to be there and it was a privilege to be there. There’s no doubt that the legislation was courageous. It had a vision and it was pretty inspiring,” she says.

O’Donnell says one of the crucial factors is that the NSW government of the day wholly supported it. The then premier, Barry O’Farrell, even opened an open government event held by the office.

“It’s really important that citizens are confident that government is being exercised in an accountable way. One mechanism for doing that is freedom of information,” she says. “It shouldn’t be hard, it shouldn’t be a last resort, it should be an easy way for people to know about what’s going on.”

In the early days of the OAIC there was also strong support high up in the federal Labor government for open government. Former Labor special minister of state John Faulkner was an important voice in pushing for change, as was senator Joe Ludwig.

But McMillan – who now serves as the acting NSW ombudsman – says that it became clear sometimes that some parts of government were pushing back.

“It’s all led by the tone at the top. When we started we had wonderful support at senior levels of government, so you got a real culture change,” he says. “But after about a year or so it became clear – and this is during Labor – that government doesn’t like FOI and it’s acceptable, it’s culturally acceptable, to thwart FOI requests.”

The office found itself facing growing pushback from some more temperamental agencies. In the last few years, it has become increasingly common for government heads to publicly criticise FOI. The Australian government solicitors even have an email list called the “FOI Chatspace”, where decisions of the OAIC are occasionally criticised by some of the government’s most senior lawyers.

Perhaps as a consequence of some of this pushback, the OAIC did not quite receive the funding that some believed it should have, despite estimates it would need 100 staff when it opened. It operated at its peak with about 80 staff. Efficiency dividends were levelled on it under Labor, that reduced staffing further. All the while, applications were building up. These delays faced growing criticism in 2012 and 2013 from frustrated applicants, including a number of journalists.

The efficiency claims used to attack the office

Ultimately, it was the perceived inefficiencies of the OAIC in these early days that were used to justify the plans to abolish the office. The federal government announced it would abolish it in May 2014.

In October 2014 it introduced a bill into federal parliament, which explained it was necessary to reduce the complexity and unnecessary delays of the office. Under the arrangements, the privacy commissioner’s office was to be moved back to the Australian Human Rights Commission – in a way that Gillian Triggs, the commission’s president, described as “unworkable”.

But are the criticisms the office faced justified? McMillan defends the office, but acknowledges there were problems in the early days. He says it was always expected it would take about three years to bed down the procedures, work out what needed to be fixed internally – and also to address structural problems in the FOI act.

“I was very clear from the start that I would accept that government wanted to restructure, but I wasn’t going to let them do so on the basis that we had failed in our task,” he says. “And I’m still firm in my view, and I told them that informally and formally. I said I would not dispute their plan to restructure government, but don’t use us as the fall guy.”

The figures from the OAIC have demonstrated a marked improvement in performance. In fact, since the abolition was announced, the smaller team of staff now there have made remarkable achievements in processing FOI applications. Only $1.7m in funding has been allocated to FOI functions.

Despite this, Pilgrim says the OAIC’s performance has markedly improved.

“We’ve made some really good inroads into improving the timeliness of the way we’re dealing with FOI,” he says. “The turnaround times have improved, they’re about on average three months … the legacy backlog of FOI matters is only about 16 cases that are 12 months old.”

The smaller FOI team initiated and completed an own-motion investigation into the Department of Human Services in just two months.

Pilgrim says he understood the concerns from the community and stakeholders early on. But he also says the changes to FOI and privacy law meant that agencies, stakeholders and the agency itself needed to develop processes. Those changes took time to resolve, which did in some areas lead to backlogs.

Peter Timmins, an expert in Australian freedom of information policy, says there are some legitimate criticisms of the office. He points particularly at the relatively small number of own motion investigations they undertook into FOI practices in government.

“There certainly was some validity in that. Whether it was the inefficiency within the office is another matter. But right from the start the office may have been underfunded,” he says. “The initial estimate I’ve seen is that they would need 100 staff to do the three functions that were allocated to them. I don’t think they got anywhere near that.”

These problems identified in the early days may also have stymied a stronger response to the abolition of the OAIC from news organisations. The ABC’s freedom of information editor, Michael McKinnon, appeared before a Senate inquiry into the bill in his role representing the Right to Know coalition of news organisations.

McKinnon is one of Australia’s most seasoned FOI practitioners, and had found the appeals process with the OAIC frustrating. In his oral submission to the Senate he said the Right to Know coalition supported the position of an information commissioner. But he adds the “major discontent” with the organisation was the lengthy delays, and urged a more fulsome review of the office.

Could the office have been more aggressive? This was a perennial aspect of the criticism around it. McMillan says he asks himself that question all the time.

“It was difficult. You don’t get cultural change by telling everybody their behaviour is appalling,” he says. “Our strategy was to work with government to change things, and that was one reason. But another was that because we had this unusual mixture of functions in the IC review functions, I had to do independent tribunal type review, but then I also had to perform an advocacy role.”

Timmins supports the role of the office, but agrees there may be some tweaks needed. He says there needs to be a specialist independent office like the OAIC to champion information culture, but urged the need for a full review and modernisation of the FOI system more broadly.

The questions over perceived inefficiencies will no doubt continue, but Pilgrim stresses the work of the OAIC is continuing and it was now tending to a vast range of privacy functions.

He adds: “We also have to look at the breadth of responsibility that the office has. In the current financial year that’s just finished we had some 373 applications for information commissioner review in the FOI side, but at the same time we had 2,841 privacy complaints coming in that we needed to look at, plus a raft of other functions.”

An unprecedented abolition

The government’s plan to abolish the OAIC appears to be a world first.

Timmins says he was not aware of another country that had embarked on a similar course of action. The decision to abolish the office sets a worrying trend around the global for the principles of open government and transparency.

Australia’s standing on the global stage of governance is also facing growing criticism in another area. Under the previous government, Australia appeared to be willing to sign on to the Open Government Partnership to encourage transparency in government. Signing on to the agreement would see Australia join other nations like the US, Canada, Sweden, Finland and scores of others.

But that hasn’t happened. The Australian has reported the attorney general has quite seriously considered not joining up at all. McMillan says he thought the decision was “a no-brainer.”

“Australia had a stellar reputation from introducing a lot of these laws. We ticked all the boxes on the open government partnership,” he says. “But there was no enthusiasm anywhere in government. And this is the same for Labor and Liberal, there was no enthusiasm at the ministerial level, there was no enthusiasm within agencies.”

“There was just this reluctance to ever have a discussion with the heading open government. And that was because of the tone from the top.”

Taking a really close look at the FOI system has been something that both Liberal and Labor governments have resisted. The previous government commissioned Dr Allan Hawke to undertake a review of FOI laws. Two years on the Labor and Liberal governments took no action to implement the recommendations.

“The [Hawke] report was a real disappointment,” Macmillan says. Its major recommendation is that we need another big review of FOI. That hasn’t happened. There’s this great disinclination to have a public discussion about FOI. Partly it’s because it’s become politicised, but that’s why we’re where we are 16 months later,” he says.

Australia now finds itself with an FOI system in need of reform, without a fully staffed and funded information commissioner’s office, and without the strong international leadership on open government that many believe the country should be putting forward.

Breaking the stalemate

The OAIC remains in limbo a year after the federal government put forward its bill to abolish the office. In that time, the attorney general George Brandis has failed to gain the support of the Senate to pass the legislation.

Labor and the Greens both oppose the bill. On the crossbench, independent senator Nick Xenophon, Liberal Democrat senator John Leyonhjelm, independent senator John Madigan, Palmer United party senator Dio Wang and Family First senator Bob Day have all indicated they don’t support the bill.

Rarely does a government bill have such a degree of opposition to legislation. But what options are available?

The federal government could simply continue with the current uncertainty. But the Labor party, the Greens and independent senators could also band together and force a vote on the bill. The procedure to achieve this is difficult, and would likely take an entire sitting day. But the government would lose this vote. It would send a resounding and very public message that the abolition of the office would not be tolerated.

The longer the current crisis continues, the greater the loss of experience at the OAIC, and the more government agencies will continue to flout the principles of open government. There is almost unanimous support for the existence of an OAIC.

For McMillan, the potential loss of the OAIC is a very personal issue. He says the office in its current form “is doing a wonderful job”. But the impact on staff is hard.

“It can’t continue. It’s very hard to maintain staff morale. It’s very hard to recruit really talented people on an ongoing basis when they don’t know whether the office is going to disappear from one month to the next.” he says.

He adds: “That’s why they’ve got to come up with a plan.”

Pilgrim stresses he didn’t want to talk on behalf of the staff, and he says it has unquestionably been a “challenging time”. But he is proud of their effectiveness and resilience.

“I am extraordinarily amazed at how committed the staff have been to getting the work done and getting it done in an effective way through what has been really challenging times for them,” he says.

There may yet be some hope for those who wish to salvage the office. The prime minister, Malcolm Turnbull, has made pointed comments about the need for open government. He has brought “public data policy” into his own departmental portfolio. The prime minister’s office did not respond to questions from Guardian Australia about whether they considered the OAIC to fall within this new policy area, but the interest has certainly drawn the attention of many of those who work in the field.

Whatever happens, Australia’s privacy and freedom of information watchdog lives on for now. It is doing more than it ever has, with less than it ever has. But it deserves far more than that.