Joe Weisberg left the CIA after three and a half years, still feeling positively toward it, he says, although a review of his 2008 novel An Ordinary Spy in the CIA’s house organ, Studies in Intelligence, suggests otherwise.

“A nasty and poorly executed look at our world,” snarls the reviewer, a veteran CIA agent specialising in counterintelligence. Quoting a le Carré character’s statement that what spies do – however unscrupulously – is vital to the “safety of ordinary, crummy people like you and me,” the reviewer insists this is a truth “few people in the intelligence profession would dispute.”

An Ordinary Spy disputes precisely that. The first-person account of Mark Ruttenberg, a bookish, sweaty, newly minted CIA case officer not entirely unlike Weisberg, it’s also rather a takedown. Mark, though no Lothario (he hasn’t had sex for a year), ends up in bed with Daisy, an embassy worker he’d been trying and failing to recruit. And is then left in deep shit after she imparts a useful piece of postcoital intel. Unfortunately for Mark, this is not the daring world of sexy spies familiar from movies and airport paperbacks; the real CIA (as depicted in the novel) is a rule-bound bureaucracy where crossing lines or bedding a “developmental” gets you summarily fired.

Weisberg’s other realist gesture was covering the pages with blacked-out redactions – his having worked at the CIA meant the book did actually have to be vetted by its publications review board (as would every Americans script) – the effect of which is a sly indictment of institutional ass-covering about a botched operation.

Overall, the novel struck me as far more cynical about the mission of the intelligence services than even le Carré tends to be. When I pressed Weisberg about the cynicism, he said he thinks le Carré is sceptical about the goals of espionage while still respecting his characters’ competence; his own book, he acknowledges, is cynical even about the competence. For both, the cost of intelligence gathering means not infrequently wrecking informants’ lives and livelihoods, and sometimes getting them killed. For le Carré it’s a necessary trade-off; in An Ordinary Spy the value of any intelligence gained is minuscule, also entirely unreliable.

If you’re a case officer in the field, a shockingly high percentage of your informants are lying to you, and there’s frequently no way to tell. One of his main characters, another CIA agent, gets scammed by an 11-year-old.

The novel didn’t sell a lot of copies, but Hollywood loves spies, Weisberg had sort of been one and could also write dialogue, which led to a well-known agent approaching him about writing for TV. Weisberg sold a show about a CIA station in Bulgaria to FX, which didn’t get made but led to relationships with producers at DreamWorks, which led to him writing some episodes of their sci-fi show Falling Skies. When the Russian illegals were arrested in 2010, the DreamWorks producers called and said, ‘Do you want to do a show about this?’

Weisberg spent a couple of weeks wandering around and thinking about it, and decided the story should be set in the 1980s and be told from the point of view of the KGB spies. And it should be about a family. Weisberg was by then a father himself and something that had stuck with him from his CIA days was how many people there lied to their kids about what they really did for a living.

After Weisberg wrote the Americans pilot and it got picked up, he joined forces with the more experienced Joel Fields to co-executive produce the series. Weisberg describes working with Fields – son of a rabbi, studied moral philosophy in college – as transformative. Fields is also the product of a lot of talk therapy; the two soon realised that they wanted to make a show where the drama derives less from plot twists than how the characters navigate them emotionally.

When I asked Fields about their creative coupledom – what I really wanted to know was what they fight about – he said they used to joke on The Americans that like Philip and Elizabeth, they had an arranged marriage. They’re also both too conflict-averse to fight.

Among their goals was having the spy-craft be as realistic as possible, and much of it is entirely real. One of their consultants, an expert on the Soviet illegals, had a personal collection of KGB gizmos and gadgets – the actual stuff that actual spies used. Even the props were marinated in history, the same history that had fired Weisberg’s obsessions, which I suspect somehow filters into the emotional texture of the show.

His political trajectory still puzzled me, though. In my youth, people who needed a geopolitical enemy looked for foes closer to home: US imperialism, capitalist pillage. They swung left, not right. Maybe Joe was wilier – it’s not like becoming a CIA agent was something kids of Chicago lakefront liberals were encouraged to do, especially when your parents are active in local Democratic politics and your lawyer-dad works part-time for the ACLU, and your mother…

Yes, let’s pause to discuss Joe’s mother – although I come late to the undertaking, as her story was previously related by Malcolm Gladwell in a 1999 New Yorker article (Six Degrees of Lois Weisberg) and his subsequent mega-bestseller The Tipping Point.

“Everyone who knows Lois Weisberg has a story about meeting Lois Weisberg,” opens Gladwell. Chain-smoking, coffee-addicted, frizzy-haired, five-foot-nothing, Lois was the type of person Gladwell calls a “connector,” someone with a weird genius for sweeping people from entirely different worlds into their orbits. Somehow Lois knew everyone – Lenny Bruce, Dizzy Gillespie, Ralph Ellison, Isaac Asimov. Gladwell’s theory is that people like Lois may actually run the world.

I count myself a beneficiary of the Lois effect, having casually known Joe’s one-year-older brother Jacob since back when I used to write for Slate in the 2000s. As its boss, and being Lois’ offspring, Jacob regularly convened assorted Slate writers for meals and occasionally far-flung outings, which included once beckoning me, maybe 20 years ago, to Lois’ Chicago apartment for a family dinner when he was in town, where Joe was also in attendance.

This was in his post-CIA malaise – he’d taken a leave to help care for his dying father, briefly returned, then resigned. (He didn’t want to live abroad, he says now.) I recall him being remote and difficult to talk to. Someone I know who met him around then describes him as “vaguely desperate.” His father’s death had torpedoed him; soon after, he entered therapy, urged by his brother and friends. (When I reminded Joe that we’d met once long ago, he claimed to remember, although I chalk this up to the Weisberg affability.) These days Jacob, alongside Gladwell, runs Pushkin Industries, a podcast company.

Now it was my turn to summon Jacob to dinner, to grill him about Joe. Joe was not a happy child, I learned, an outsider at school – “a little awkward or funny-looking,” said Jacob, quickly backtracking to add that “funny-looking” was unfair. He just wasn’t comfortable with kids his age, thus lonely, also the outlier in the family. All Joe wanted was to read comic books and watch TV; his bibliophile father hated television so much that he may have once said, depending on which Weisberg brother you ask, that it was worse than the atom bomb, and permitted only two hours of it per week.

Jacob, who describes himself as a far less interesting person than Joe, didn’t have conflicts with their parents, and didn’t much want to watch TV. To him it seemed like a wonderful family life. “I accepted the terms of the imprisonment pretty well,” he said. When Joe went into therapy and started characterising their homelife as difficult and repressive, Jacob’s initial reaction was, “What? I was there too. It wasn’t like that.”

Jacob told me that Lois was the kind of mother who’d say, “Why don’t you go join the circus?” I assumed she’d meant it in a cruel-mom way, as in “You don’t like your dinner, go join the circus.” No, she’d meant it literally, I learned from Joe. Lois was then in charge of special events for the city of Chicago, and when the Ringling Brothers circus came to town, Lois (being Lois) had gotten to know the guy in charge and one night during dinner said, “Joseph, I think you should join the circus.” He was in his teens. She said she’d introduce him to someone who could probably find him a job and take him with them when they left, which would be an amazing experience. “She was right that it would have been a great experience,” Joe says now, “though also wrong and crazy.” He’d always seen it as a funny and benevolent story, but later wondered if there was also a part of her that wanted to get rid of him. “I think one has to face that interpretation of the story too.”

One late summer afternoon, Weisberg and I met at a midtown tourist museum called Spyscape, which gauges its visitors’ potential spy abilities via a series of interactive exhibits and tests. Long on what’s known in writers’ rooms as “hangability,” Weisberg gamely played along, although barraged with a lot of whooshing sound effects and flashing lights upon entering asked, “Does the fact that these are making me nauseous mean I wouldn’t be a good spy?”

What was great about this field trip was that the museum promised to do my job for me: construct a profile of the person I was supposed to be profiling. We were tested about whether we were good liars, good at detecting lies, and willing to take risks. In “Special Ops” we were faced with an infernally complicated challenge involving pushing a lot of glowing white buttons on a wall while dodging a meshwork of laser beams.

Weisberg leaped athletically to the task, determined to beat that day’s record, exclaiming afterward, “I thought that looked dumb, but it was great!” Squirting himself with Purell at one of the stations thoughtfully located around the museum, he joked, “Here’s where you really fall down in their assessment – if you use the hand sanitizer.” In the Surveillance exhibit we had our first fight, over Edward Snowden, about whom Weisberg was decidedly negative and I insisted had been a patriot.

Then it was time for Weisberg’s spy evaluation. “You have high emotional intelligence, which helps you understand people in social situations and are empathetic,” pronounced a creepy omniscient robot. “You take risks after careful consideration,” it added. “Joe Weisberg, you are going to be an intelligence operative!” This didn’t thrill him. “The real question is, do I want for it to say that I’d be a good spy or a bad spy?” he mulled. “The truth is I don’t want to be a good spy anymore.” But maybe old habits die hard.

On his personality assessment, when asked if he was willing to be unethical if it would help him succeed, he’d rated himself a 1, the lowest score. Asked if he’d say anything to get what he wants, he’d given himself a 2. “Obviously that’s what you’d say if you were saying anything to get what you want!” I pointed out.

After all, he was the one who’d earlier said that the whole thing you learn to do in the CIA is manipulate people. Is unlearning that really possible? The question of who was manipulating whom had been a meta thing in our conversations from the beginning, with jokey badinage about the power of interviewers and the vulnerability of their subjects.

Not long after our field trip, Weisberg – a foodie who spends much of his free time patrolling lower Manhattan in quest of Chinatown’s most electrifying dumpling – suggested by email that we hop on the Long Island Rail Road to Flushing, Queens, for “sour fish”; he knew a restaurant that served a half dozen varieties. The accompanying photo displayed a bowl of lethal-looking chilies the size of hand grenades. I wrote back: “Fearing unflattering portrayal, profile subject poisons unwitting profiler with capsaicin overdose.”

Weisberg rejoined a second later: “Pathologists were shocked to discover the poison delivered simultaneously with a subcutaneous patch and ingested along with, judging by the contents of the victim’s stomach, sour fish.”

He was funny, I was charmed, but then so was poor lonely Martha Hanson in The Americans – secretary to the head of the FBI’s DC counterintelligence unit – skillfully charmed by Philip in a great demonstration of what a powerful interpersonal weapon nerdy vulnerability can be. Spoiler alert: it doesn’t end well for Martha.

• A Wired report