Three investigative journalists shared with the IJF16’s audience their professional joys and challenges of working with big data. Crofton Black and Abigail Fielding-Smith, respectively investigator and reporter at the Bureau of Investigative Journalism, used their work to highlight some practical problems of looking for useful information in massive public databases. Megan Lucero, data journalism editor of The Times, stressed how data lead her team to important front-page stories. Alessandro Accorsi and Emanuele Midolo, students at City University London, moderated the panel.
In light of the recent Panama papers leak, Accorsi opened the discussion mentioning that more and more data-driven investigative stories are generated every day. He added that data journalists today are a hybrid of tech-savvy people and traditional reporters: they both analyse and build data sets. “Reporters have always used documents to turn simple researching to stories. What is different now is the scale: the size of these databases, and the number of documents,” Midolo said.
“The stories are in data now”
Computational investigative reporting means making complex stories accessible, Lucero said in the beginning of her presentation. She presented historic and recent coverage from the Times to show that data journalism – which is often connected just to visualization – existed decades ago.
“There will always be good journalism and bad journalism,” she said. “And I think that data journalism is just an example of good journalism; people going to the places where the stories are, and the stories are in data now.”
Black and Fielding-Smith shared lessons learnt from their years of investigative journalism. They used their five-year search to track down the companies that supported the US government in their drone warfare to explain how they effectively used data to tell an important story.
“This is a kind of a story about how using data enabled us to break through the wall of secrecies surrounding this area. So, we started with an idea of something we wanted to find about and we used data to do that, rather than starting with a dataset, although the fact that such a huge dataset existed gave us some ideas of how it might be used,“ said Fielding-Smith.
The team started looking at government spending records from the US Department of Defense. They said that the data were publicly available, but there were certain challenges to accessing them. “We felt that somewhere in the side of this massive information was the stuff we wanted, but the government wasn’t really providing tools that could enable us to get it. So, we had to build a new system to access this data that already existed,” said Crofton.
Their solution was to download all the data, load it into a new database, index it, interrogate it using Structured Query Language (SQL) and break it down into smaller units.
They ended up with 8.3 million records that included miscellaneous information, which they have to survive, they both joked. “There is a danger in data journalism,” said Fielding-Smith, who comes from a more traditional journalism background. “You can let data guide your story too much.” She added that they always reminded themselves to what extent their story would be interesting. So, she supplemented data crunching with interviews in order to find the core of the story. Their investigation was published at the Guardian and picked up by a lot of US outlets.
Journalist should understand coding
In an interview following the panel discussion, Lucero shared how she pursued her specialisation in the Times’ newsroom. She started as an intern five years ago, verifying stories and doing fact checking for the foreign desk. She soon saw the mass potential for digital storytelling in her newsroom. For a year, she said, she worked as a digital story producer helping colleagues tell important stories with video, audio and even social media. Lucero gradually evolved into a data journalist. “I taught myself a bit of coding, I built some interactives…I just learned a lot of different things,” she added.
Her advice to young journalists? “I don’t want to say that everyone should be a programmer. I do not think this is the right advice. You think about what you want to do and you find the tools to get there….I think everyone should have that literacy and understand coding, at least to know its potential, but I wouldn’t be too prescriptive about what you can do.”