ALERT: Subscriber Acquisition Master Class early registration deadline is Friday, click here for agenda

Radical collaboration could provide the solution news publishers are looking for

By Paula Felps

INMA

United States

As AI accelerates both the opportunities and pressures facing news media organisations, collaboration is emerging as the best option for publishers. This week’s Webinar, presented by the INMA Product & Tech Initiative, explored the idea of “radical collaboration” as a way to share resources, reduce duplication, and build common infrastructure. The conversation between Jodie Hopperton, INMA Product & Tech Initiative lead; Ryan Pitts, managing director of Big Local News at Stanford; and Scott Klein, founder of Open Journalism, offered an insightful look into what radical collaboration looks like in real time.

One clear message emerged during the conversation: the industry is entering a moment where shared tools, shared data, and shared knowledge are no longer optional. They are forming the backbone of journalism’s future in the AI era.

INMA’s Jodie Hopperton was joined by Ryan Pitts, managing director of Big Local News at Stanford, and Scott Klein, formerly of ProPublica and founder of Open Journalism, for a Webinar on “radical collaboration.”
INMA’s Jodie Hopperton was joined by Ryan Pitts, managing director of Big Local News at Stanford, and Scott Klein, formerly of ProPublica and founder of Open Journalism, for a Webinar on “radical collaboration.”

Making the case for radical collaboration

Hopperton began by laying out the realities news publishers face today. Although AI has paved the way for new capabilities, it has also introduced new costs, new dependencies, and new pressures. “We are seeing more and more open source,” she said, explaining that “open source” can mean different things to different publishers depending upon their needs.

She reinforced that by asking what members would like to see used more, and the responses covered a broad range: fact‑checking databases, prompts, agents, technology frameworks, and more. The diversity of answers reflected the diverse needs across the industry.

And, Hopperton said, the extent of those needs proves that collaboration is not simply beneficial, but necessary.

She noted that open source AI has undergone a dramatic shift. “We are in this phase where open source LLMs and open source foundational models are weeks behind. They’re no longer months or years behind,” she said. That causes additional financial strain for publishers struggling with rising AI costs and the burden of maintaining both legacy systems and new AI‑driven workflows.

Some organisations have already begun capping internal AI usage to keep expenses under control, Hopperton said, “because the costs can get pretty high.” One emerging solution is the use of open-source models and “middleware” strategies — approaches that rely on cheaper, good‑enough models for most tasks while reserving high‑cost frontier systems only for work that truly requires them. It’s a pragmatic shift that could help publishers manage the rising cost curve without sacrificing innovation.

Middleware offers an opportunity for publishers to use cheaper, “good enough” tools for tasks that don’t demand high-cost systems.
Middleware offers an opportunity for publishers to use cheaper, “good enough” tools for tasks that don’t demand high-cost systems.

“We haven’t got enough tools,” Hopperton said. “We’re not far enough advanced where we can get rid of the existing software and move all to AI… but at the moment, for a lot of tech leaders, they’re trying to cope with both sides.”

The result is a growing interest in shared infrastructure — tools, data pipelines, and standards that can be built once and used by many.

Building shared data infrastructure

Ryan Pitts, who leads Big Local News at Stanford, has spent his career building exactly that kind of shared infrastructure. His organisation supports local newsrooms by collecting, cleaning, documenting, and distributing complex datasets that would otherwise be inaccessible to smaller outlets.

Pitts described Big Local News’ framework, DART, which stands for Data, Algorithms, Recipes, and Training. It’s a matrix designed to meet newsrooms where they are. A small newsroom with no data expertise may need training and reporting recipes. A mid‑sized newsroom may benefit from tools that simplify complex datasets. A large newsroom may simply want raw data pipelines.

“We think of ourselves as an infrastructure provider for the local news ecosystem,” Pitts said. “We run a lot of scrapers, we maintain a lot of databases… and then we can give access to that, to newsrooms at whatever level they’re ready to use it.”

He highlighted three projects that illustrate how collaboration and open source can expand the impact of journalism:

1. DataTalk uses AI to unlock complex data

DataTalk is a research tool that uses LLMs to translate natural‑language questions into SQL queries for U.S. federal campaign finance data. The FEC database is notoriously complex — Pitts said few reporters can write SQL queries against it — but DataTalk simplifies it.

“It may be hard for me to write a query against that FEC database, but it’s really, really easy for me to articulate a question,” he said. “The LLM is really good at translating that into a query and pulling data down that then I can move forward with as a reporter.”

DataTalk uses AI only for translation and summarisation — not for data retrieval — to avoid hallucinations and maintain trust.

The tool, originally prototyped in 2024, has been updated for the 2026 election cycle and is available free of charge to newsrooms.

2. Open‑sourcing a cross‑border investigation

The second project was a collaboration between Lighthouse Reports, the Financial Times, and El País on “brain waste,” or the underemployment of highly skilled immigrants in Europe. The investigation found that nearly half of immigrants in the EU were overqualified for their jobs, compared to one‑third of native residents.

The Brain Waste project mined data to reveal the underemployment of skilled immigrants in Europe.
The Brain Waste project mined data to reveal the underemployment of skilled immigrants in Europe.

Big Local News took the project further by cleaning the underlying Eurostat dataset, documenting the methodology, and hosting the data in Stanford’s digital repository. It also created reporting recipes so any newsroom could replicate the analysis locally.

“This is open source not just for code,” Pitts said, “but for data.”

3. Opioid Overdose project

The third project began with a single reporter in Baltimore and grew into a national, award‑winning investigation. Working with the New York Times Local Investigations Fellowship, Big Local News helped analyse CDC data on opioid overdose deaths. That revealed a surprising pattern in which Black men born between the 1950s and 1970 were dying at disproportionately high rates.

The team opened the data and invited other newsrooms to replicate the analysis. Six more outlets joined, producing local stories across the country and contributing to a broader national investigation published in the New York Times.

“This project ended up winning a Pulitzer Prize for local reporting a couple of years ago,” Pitts said. “And it all started because we were collaborating with this local investigations fellowship and looking at data together, helping a reporter work on a data project that they found interesting.”

After studying the initial data on opioid overdose death, Big Local invited other newsrooms to replicate the analysis, resulting in a national investigation — and a Pulitzer Prize.
After studying the initial data on opioid overdose death, Big Local invited other newsrooms to replicate the analysis, resulting in a national investigation — and a Pulitzer Prize.

The decline and revival of open source

After spending years at ProPublica building open-source tools and fostering a community of newsroom developers who shared code freely, Klein began noticing a troubling trend in recent years: open-source activity seemed to be declining.

Along with journalist Ben Welsh, he conducted a study of GitHub repositories created by newsrooms between 2012 and 2024 and found that open-source repository creation had dropped by 80% from its peak in 2016–17. Repositories, or a package of code uploaded to the GitHub Web site, track the code that teams collaborate on and preserve the full history of how that code evolves over time. That structure allows multiple people — even entire newsrooms — to work on the same project simultaneously without overwriting one another’s work.

“It wasn’t just code,” Klein said, noting that across the board, “People were participating far less in the show‑your‑work initiatives that were really popular in the mid‑2010s.”

But in addition to sounding the alarm about the decline of open source, Klein has begun looking for places where it is growing again. Much of that growth, he said, is coming from new newsrooms.

Klein shared examples of several organisations that are embracing open source with new energy, including Spotlight PA, which “open sources radically” — even making articles available before publication — and The Pudding, where “open source is everything.”

He also offered several standout examples of open‑source tools that demonstrate how collaborative development can scale across the industry. Schibsted’s VideoFy, which converts articles into vertical video, has been widely forked by other publishers looking to adapt the format for their own workflows. (A fork, he explained, “is when another newsroom or anybody uses the code for themselves and modifies the code for themselves.”) Documenters’ City Scrapers has become another model of shared civic‑tech infrastructure, with more than 300 forks powering automated monitoring of public meetings across the United States.

Open-source tools are showing publishers how collaborative development can scale across the industry.
Open-source tools are showing publishers how collaborative development can scale across the industry.

He also highlighted Deutsche Welle’s investigative repositories, which not only release code but include structured data, methodology, and documentation — an approach that makes complex investigative work replicable for any newsroom willing to build on it.

AI is changing what “open source” means

Klein also pointed out that AI is reshaping open source itself. In 2026, one in seven newsroom repositories included AI artefacts such as prompts, skills, and agent files. Just one year earlier, only one in 17 newsrooms could make that claim.

“We’re seeing newsrooms open source not just code,” he said, “but the instructions that make AI tools work.”

This shift could democratise AI development across the industry— but only if publishers embrace it.

“We’re seeing the use of AI very, very broadly in the creation of this stuff, and we’re seeing more and more people include credit for that AI agent or the artefacts that got created to make that code available when they release the code,” he said. “So it’s truly exciting times.

“We’re kind of all in this together, and I think even if we compete on stories, I think there’s no reason for us to compete on writing Web scrapers.”

Banner photo: Adobe stock by Deemerwha studio

About Paula Felps