Only a month ago the outcome of the US presidential election was seen my most commentators as being a question of the margin of President Barack Obama’s victory. But then the television debates intervened. Obama’s lacklustre performance in the first leaders debate, a democratic innovation that began with Kennedy and Nixon in 1960, made the president appear complacent and lacking focus.

From then, whether in the subsequent debates or any other intervention, the narrative was set for the last weeks of the campaign. Would Governor Romney’s “bounce” sustain? Could the president regain his standing? Few would now argue those debates weren’t important.

Politics has many characteristics, but when you get to the pointy end of the electoral process in the United States, it is certainly high drama. But debates also create key moments where the electorate can observe alternative leaders responding to focused questioning. With each candidate on the same platform and at the same time, debates provide structure to a campaign: a time to scratch beneath the surface of the well rehearsed lines and phrases and shed light on the style and character of the candidate.

They are, as is the case in other liberal democracies in Canada, France, the United Kingdom and – until the 2010 campaign – here in Australia, generally positive contributions to the democratic process.

Engaging the audience

In my role at the Institute for Democracy and Human Rights I have spoken with those involved in broadcasting and negotiating leaders debates in four jurisdictions. I have spent hours watching Gore, Hawke, Bush, Bush (again), Clinton, Mitterrand, Reagan, Cameron, Howard and Gillard.

The high points are few: John Hewson and Paul Keating in 1993 having a sharp exchange regarding their respective views of the role of the government supporting a public health system; Ronald Reagan making President Carter appear smaller and smaller as each moment passed in 1980; and the innovation of the first prime ministerial debates in the UK two years ago. But what we learn from the debates themselves is less important than what we learn from the process surrounding them.

Leaders debates fulfil a more important role than an hour and a half of television. Competing policies and characters are presented and assessed in the wider media.

And people watch. In the UK, more than eleven million people watch each of the three prime ministerial debates: more than hit show Strictly Come Dancing. Around 67 million people tuned in to the first Obama–Romney debate.

These numbers don’t reveal how many people engage with the substance of the debates through newspapers, radio, television, or on their smart phones and computers. In 2010 all the major UK newspapers ran pages of analysis following each debate. In the US, numerous news sites ran immediate fact checks on what was said and developed engaging ways to explain differing perspectives and policy positions.

Breaking eggs to make an omelette

Of course, the situation in the US is not the same as in Australia. In the US the electorate has very little exposure to their prospective leaders prior to the debates. In Australia, we get to see our alternative leaders in Question Time whenever parliament is sitting.

In the 2010 campaign, our one televised debate on free to air television made international news. Sadly this was because it was moved to ensure that it did not compete with the MasterChef final. Why let questioning the prospective leaders of the nation compete with preparing a simple three-egg omelette?

In Australia, decisions on how many debates there are, and what format they take, have been conditioned by largely self-serving negotiations between the broadcasters and political parties. In 2010, Julia Gillard thought a small target strategy would serve her best, and she agreed to a short, single debate. Only when the election appeared closer than expected did the prime minister seek to debate Tony Abbott at every opportunity. This resulted in ill-prepared public forums where the main questions to emerge were whether jackets stayed on, white was worn or politicians talked from the stage.

Countering the sideshow

We can, and I believe must, do better. The US and UK experience shows us that these debates matter. They may not decide elections, but in the spirit of Hans Christian Andersen, they do allow us to view our prospective emperors and assess the thread of their clothes.

Whether it is Kevin Rudd calling for greater civility in politics; Malcolm Turnbull arguing for greater truth, leadership and responsibility or Malcolm Fraser’s pithy description of both political parties “rattling around at the bottom of the barrel trying to find something derogatory they can say about the other party”, there is a growing argument that Australia must find new ways of engaging the public on the substance of politics and policy.

The tired clichés, hugged babies, tested platitudes, and apparent nastiness of the political art are wearing thin. There is a hunger for something different. A series of more effective prime ministerial debates overseen by a non-partisan Electoral Debates Commission would be one counter to the “sideshow”.

The most effective form of debate is largely a matter of subjective taste, but there are a few elements that would make a good start: agreement between the parties and broadcasters to three debates of an hour and half each; using different formats for each; using a creative process for establishing the questions and engaging new media.

Achieving this will be hard. The stakes in any democracy at election time are high. Decisions will be taken in a highly politicised environment involving the parties, politicians and broadcasters. Yet it should be the people who are best served by effective democratic engagement. Through more credible prime ministerial debates, they can be.