We all do it. Make little snap judgments about everyday strangers as we go about our lives. Without giving it a second’s thought, we sketch minibiographies of the people we pass on the sidewalk, the guy seated across from us on the train, or the woman in line in front of us at the grocery store.

We wonder: Who are they? Where are they from? How do they make a living? Lately, though, such passing encounters tend to leave me with a sense of suspicion, a wariness tinged with grim curiosity. I think to myself: Is he or she one of them?

By them, I mean one of the tens of thousands, if not hundreds of thousands, of “people” I encountered during my many forays into the darkest recesses of the Internet. Despite the staggering amount of time many of us spend online – more than six-and-a-half hours a day, according to recent research – we tend to haunt the same websites and social media platforms (Facebook, YouTube, CNN, Reddit, Google) again and again.

Not me, though. Over the past five years, I’ve spent more hours than I wish to count exploring the subterranean hideaways and uncensored gathering spaces for some of the most unhinged communities on the Internet.

Call it an occupational hazard. Only recently, I published my first book, A Death on W Street: The Murder of Seth Rich and the Age of Conspiracy, an investigative political thriller that opens with the 2016 street murder of a 27-year-old who had worked for the Democratic National Committee.

In the absence of a culprit, Seth Rich’s killing got swept into the fast-flowing conspiratorial currents of that year’s presidential race, a contest that pitted an unabashed conspiracy theorist, Donald Trump, against a candidate, Hillary Clinton, who had been the subject of decades’ worth of elaborately sinister claims (with no basis in reality).

For my book, I set out to understand how a senseless crime that took the life of a beloved but hardly famous mid-level political staffer became a national and then international news story, a viral phenomenon of ever more twisted conspiracy theories that reached millions and all too soon became a piece of modern folklore.

To do so, I traced the arc of those Rich conspiracy theories back to their origins. In practical terms, that meant hundreds of late nights spent huddled over my desk, eyes fixed on my computer screen, clicking and scrolling my way through a seemingly endless trail of tweets, memes, posts and videos.

The Internet is, in some ways, like an ancient city, its latest incarnation resting atop the ruins of so many civilisations past. I came to think of myself then as an online archaeologist digging my way through the digital eons, sifting through archived websites and seeking out long-vanished posts in search of clues and answers.

Or maybe I was a waste handler, holding my nose as I picked through piles (or do I mean miles?) of toxic detritus that littered old versions of social media sites you’d know like Twitter and Reddit and others you probably don’t, like 4chan, 8kun, and Telegram. It was there that I encountered so many of them, those faceless users, the ones I might have passed on the street, who, with the promise of anonymity, had felt unburdened to voice their unfiltered, often deeply disturbing selves. It was all id, all the time.

Who were these people? I couldn’t help but wonder whether they actually believed the stuff they wrote. Or was it all about the thrill of saying it? In an unnervingly boundless online world, were they testing the boundaries of the acceptable by one-upping each other with brazen displays of racism, misogyny, or antisemitism (just to start down the list)?

Firing up my laptop and venturing into those noxious places was like entering an inside-out world impervious to logic and critical thinking. They had their own language – losers were “cucks,” loyal foot soldiers “pedes,” and Hillary Clinton was Hillary “Klanton” – and they operated with their own sets of elaborate but twisted rules and hierarchies. After a few hours of studying such “conversations,” a form of vertigo would set in, a spinning sensation that made me get up from my desk and clear my head with a walk or a conversation with a real human being.

Now that the book is published, I don’t spend much time in those disturbing online worlds. Still, every once in a while, I can’t help checking in – old habits die hard – despite the horrors I saw there while gathering material for my book. What nags at me even now – in fact, it haunts me in some way – is the knowledge that there were real people behind those toxic accounts.

The same people you might sit next to on a bus without having the slightest suspicion of just how disturbed they were and what a disturbing world they were helping create or elaborate. That knowledge still weighs on me.

A confession: on a few of those late nights spent in the online ruins, I caught myself starting to nod along with some of the wild-eyed nonsense I was reading. Maybe I found a particular Reddit thread surprisingly convincing. Maybe the post in question had sprinkled a few verifiable facts amid the nonsense to make me think, Huh?

Maybe my sixth cup of coffee and lack of sleep had so weakened my mental safeguards that madness itself began to seem at least faintly reasonable. When I felt such heretical thoughts seep into my stream of consciousness, I took it as a sure sign that I should log off and go to bed.

Thinking back on those moments, I admit that the first feeling I have is pure and utter embarrassment. I’m an investigative reporter. I make a living dealing in facts, data and vetted information. Heck, my first job in journalism was as a full-time, trained fact-checker. I should be impervious to the demented siren song of conspiracy theories, right?

The correct answer is indeed: right. And yet…

I realize now that, on those disturbing long nights at the computer, I was more than an avid journalistic explorer of online content. I had immersed myself – and immersion is what the Internet does best. It’s the gateway point to a seemingly infinite number of rabbit holes. Who hasn’t clicked on a Wikipedia entry about, say, the making of the atomic bomb only to check the time, realise that two hours had slipped by, and you’re now watching a YouTube video about the greatest comebacks in baseball history with no memory of how you got here in the first place?

• A TomDispatch report / By Andy Kroll