Can Liz Truss survive? The humiliating U-turn over the top rate tax cut has won her time in the party and in the markets. But within hours of the first U-turn, her premiership was unravelling further in almost every direction.

In the crammed bars of the Birmingham Hyatt hotel, seasoned Tories were, during their party’s conference, discussing their future, using phrases such as “death spiral” and “doom loop” as well as fruitier Anglo-Saxon terms unsuitable for family publications like this one.

Tory MPs, up to and including ministers, were making it clear they would not vote through real-terms benefit cuts; a weakening of environmental protection in enterprise zones; fracking; or a planning free-for-all. There is a growing move against any radical dilution by Jacob Rees-Mogg, the Business Secretary, of employment rights.

Two obvious conclusions follow. First, never mind the disastrous mini-Budget: within two days of Liz Truss’s first conference as prime minister, there is not much left of her much-vaunted growth package. Second – since this is all about parliamentary numbers – it suggests that the Conservative Party’s system of imposing by vote of its ordinary members a leader its MPs haven’t elected is fundamentally flawed and needs to be scrapped.

What has been happening in Birmingham is all about that indefinable, unmistakable and essential quality in leadership: authority. Neither Truss nor Kwasi Kwarteng, the Chancellor, are going to be taken as seriously as they were before the Budget debacle. (And that wasn’t very.)

The tax U-turn happened because, in the oily hubbub of the Tory Birmingham conference, it was becoming clear that a putsch was under way. Michael Gove’s defiance on live television on Sunday October 2, in making it known, as he sat only yards away from the Prime Minister, that he was prepared to vote against her finance bill, set off behind-the-palm discussions among anti-Truss MPs. A stream of senior party figures told her directly and privately that she would have to eat humble pie. She stared back, expressionless, but she was listening.

This may seem mad – but others were talking quite seriously of ways to remove the new PM without staging another leadership election. That ancient and obscure Tory behaviour of “taking soundings” might be employed to present her with some kind of alternative “unity candidate”. So far no one seems able to agree on who that might be.

Meanwhile, 10 polling companies could not agree about the Labour lead. Some said just 19 points; two others went as high as 33. The Daily Mail went from a front-page splash on Saturday September 24 of “At last! A true Tory Budget” to, by Tuesday October 4, the bellowed appeal “Get a Grip!”.

So at this time, above all, we need some perspective. Taking a longer view, the problems so glaringly evident in Birmingham started around six years ago, because the Conservatives, having created the circumstances for Brexit, could not agree about the meaning of what had happened. Was the vote for Brexit a reassertion of British traditionalism, or was it a revolutionary break? Or was it for many an expression of mass disaffection?

Almost everything that has gone wrong for the Tories since the referendum goes back to that central confusion. One small example: traditionally we have had a progressive tax system, so Gove called the scrapping of the top rate of income tax un-Conservative. Yet for the market revolutionaries in his party, it was the most “Conservative” move possible.

I was filming around the country at the time of the 2016 referendum and was struck by the discordance between what the people I was talking to said they wanted from leaving the EU and what the Brexit high command claimed they wanted.

In almost every Tory-leaning conversation, radical nostalgia came through. It wasn’t ridiculous or nasty. There was a wistful impatience about the modern world, a conviction that manners, dutifulness and efficiency had deteriorated and that this was somehow to do with the imposition of overseas authority. (I think almost none of this was true, but I’m reporting a widespread feeling.)

Some leading Brexiters agreed. Nigel Farage certainly shared a lot of that, marinated in a golf-clubby, cricket-fixated mindset that looked back to imperial measures and the old City where his father, whom he idolised, worked. Getting out of “Europe” meant returning to aspects of an idealised childhood. Or something like that.

One could almost touch and smell this nostalgic urge in small towns in Hampshire or Kent, where men affected yachting caps and raspberry-coloured corduroys and where the shops sold scale models of Spitfires and picture books of the same town in Edwardian photographs.

Much of Brexit Britain was still the England of high privet hedges, cryptic crosswords and unpopular vergers – a 21^st century take on a mindset George Orwell or Louis MacNeice would have recognised.

Was it racist? Yes, often enough in its jokes, though seldom face to face. It was cheerfully ignorant of other cultures except, of course, those of the Dordogne and the Algarve. “Abroad” for Brexit nostalgists meant the gentler parts of the Caribbean and European peninsulas reachable from P&O cruise liners.

• The New Statesman report / Andrew Marr