How do authoritarian rulers survive in the context of democratic institutions? This is a long-standing puzzle that has become more pressing with the rise of authoritarianism in the 21st century.

In theory, democratic institutions should allow citizens to vote out elected officials who don’t pursue the public interest, or hold them accountable via other measures, like an independent judiciary.

Yet, most of today’s authoritarian regimes hold regular elections, have a formal separation of powers and a relatively independent press. Scholars have called these hybrid or electoral authoritarian regimes.

I set out to research this puzzle of authoritarian rule. I focused on Uganda because it provides a clear case of an electoral authoritarian regime. President Yoweri Museveni has held power since 1986 under the ruling party, the National Resistance Movement. The state is seen as increasingly repressive. However, it holds regular elections and meets some other basic criteria of a democracy. This includes a nominally independent judiciary, inclusive suffrage and a fairly free press.

 In studying Uganda, I identified a type of governance that uses unpredictability to combine democratic institutions with authoritarian power. I call this ‘institutionalised arbitrariness’.

Uganda is unique in many ways. Nevertheless, my research offers some insights for contemporary practices of authoritarianism worldwide. It also offers insights into the workings of post-liberation African states like Ethiopia, Rwanda and Zimbabwe.

I carried out fieldwork in Uganda, studying local violent actors. I held more than 300 interviews with local security actors. These included vigilantes and community police, community members and government representatives.

I attended public events like community meetings, and collected village-level bylaws and other documentation that helped triangulate what people told me.

I studied the interactions between state authorities and informal security actors to understand who can use violence and how, and the implications for state authority. For instance, I looked at how vigilante groups were formed in different communities and what they did. This included how they enforced local order and when they were seen to overstep their mandate.

I also studied Uganda’s Crime Preventer programme. My aim was to understand who joined it, what activities crime preventers were asked to do, and how the programme joined national-level politics to the grassroots.

Uganda has limited capacity to fully monopolise violence in its territory or provide basic services to its citizens. It relies on repression, but its limited capacity means that it cannot silence dissent systematically and reliably.

My research analysed the interactions between state authorities and informal violent actors. What I found was surprising. First, state actors encouraged the formation of these groups and gave them the job of using violence to police their communities. This was an active outsourcing of violence to non-state actors.

• The Conversation report