Al Jazeera Journalism Review

Outside image

Using open-source intelligence in journalism

Where once journalists relied on sources for information - also known as ‘human intelligence’ (HUMINT) - they now increasingly rely on ‘open-source’ intelligence (OSINT) gathered from the internet, satellite imagery, corporate databases and much, much more 


Many of us know the scene in All the President’s Men, Hollywood’s interpretation of the Watergate scandal, when “Deep Throat” is standing in a car park basement in Washington DC. The man we now know to be the former Associate Director of the FBI, Mark Felt, was the secret informant who gave clues such as “follow the money” to the Washington Post journalist, Bob Woodward. 

Finding the evidence that brought down US President Richard Nixon was a watershed event in investigative journalism and has rightly entered its folklore. To break investigative stories in the 1970s, you needed to develop sources. 

The skill sets needed for success were once described by the late Nick Tomalin as “ratlike cunning, a plausible manner, and a little literary ability”. Tomalin was killed while reporting the Arab-Israeli war in 1973, a year after Woodward broke the Watergate story. Investigative journalists were usually required to obtain confidential documents from people who did not hand them over without a great deal of persuasion. 

The investigator had to nurture sources over time. They’d met in restaurants, coffee shops or late at night in bars. They persuaded whistle blowers to do the right thing; they gained their trust so that the identity of a source would not be revealed. Managing a source was a critical skill for an investigator. 

A historical marker stands outside the parking garage underneath the Oakhill Office Building in Rosslyn, Virginia, where Washington Post reporter Bob Woodward would meet his source - known as 'Deep Throat' - to exchange notes about the Watergate scandal [Chip Somodevilla/Getty Images]

HUMINT - or human intelligence - was the cornerstone of investigative journalism and obtaining information that no one else had was essential for an exclusive. A journalist usually carried only a notebook and a tape recorder. When I started in journalism, there were no mobile phones. There was little methodology to investigative journalism. Success depended on who you knew and how effectively you exploited them. Then along came the computer and everything changed. Technology altered journalism, both its practice and its consumption.

The roots of OSINT lie in Computer Assisted Research. CAR began by exploring and analysing databases. In doing this we can discover patterns, trends and anomalies that may be useful in producing new information. The practical use of this methodology emerged with the Freedom of Information Act in the United States, which was introduced in the 1960s to open the workings of government to public scrutiny. 

Philip Meyer, a pioneer of CAR, called it “precision journalism”. It was inspired by the methodology of social sciences where a journalist used evidence to prove his assertion. A methodology was born that inspired a distinct storytelling style that distinguishes investigative from conventional journalism. Conventional journalism is reactive and observational. It describes the world as it is seen, usually after something has happened. 

Investigative journalism, by contrast, seeks to prove that some aspect of what the public thinks it knows about the world is wrong. Like a policeman or prosecutor, an investigator will discover a lead or obtain prima facie evidence that supports a hypothesis that “X is lying” or “X is corrupt”. The investigation will aim to prove this supposition. If it can’t, the investigation is dropped. This investigative methodology, known as hypothesis-based narrative, replaced conventional character-based or travelogue storytelling. Evidence gathering became the glue that holds the narrative together. 

Reporter Nicholas Tomalin is pictured 10 minutes before he was killed by a bomb on the Golan Heights on October 17, 1973 during the Yom Kippur war [Fred Ihrt/LightRocket via Getty Images]

Decades ago, Philip Meyer made the prophetic statement: “When information was scarce, most of our efforts were devoted to hunting and gathering. Now that information is abundant, processing is more important.” In the last decade, open-source intelligence (OSINT) has emerged as a journalistic science, as the vast resource of data collected from social networks and internet-connected devices is mined for information beyond just databases.

The volume of data created, captured and consumed globally is projected to be around 200 billion gigabytes a year in 2025 (up from 70 billion in 2020). Every minute on Facebook, around half a million comments are posted and 150,000 photos are uploaded. More than four million hours of content is uploaded to YouTube every day. Add to that, 700 million tweets per day Investigative journalism will increasingly rely on tapping these sources. 

We are not discovering truths that are strictly hidden from us - they are not confidential - but we are assembling information in a fashion that reveals new truths. We are unpicking the resources available online to tell the story behind the picture, the story that the metadata provides, or the story that shipping or flight data tells us about an event. 

For filmmakers dealing with investigative content, there are new challenges. There will be more use of computer-generated imagery to tell the story and less use of video. There will be a need to harmonise different sources, such as vertical aspect ratio imagery, publicly generated and low-definition content with professional standards. 


Graphic designers, data scientists and filmmakers will need to work together in ways that presently rarely exist. The model of television production needs to adapt to a new method of storytelling. With the amount of information in the public domain, investigative journalists of the future may be less concerned with obtaining secret data than finding ways to make sense of public data, and tell stories based on that. More complex computer-based tools, such as data mining programmes, geographic information systems, demographic databases and so forth can be used to identify patterns, anomalies and discrepancies in data. 

Much of the new technology surrounding open source intelligence will involve machine learning, that is when a computer model is trained to analyse data much faster than a human being. In effect, you train a computer to do the hunting for you. It means that investigative journalists no longer need to only learn how to write and turn on a tape recorder or camera. They will need to learn the tools of the internet. 

While computer scientists will write the programmes, journalists will need to understand the science of OSINT. OSINT is not a substitute but a complement for HUMINT. For most investigations, journalists need to use human sources as well as data. 

Investigative journalists still need “ratlike cunning, a plausible manner, and a little literary ability”. But they also need to understand how to get value from the abundance of information on the Internet. 

The Al Jazeera Media Institute Open-Source Investigation Handbook provides an invaluable guide to achieve this. 


More Articles

Analysis: The media’s coverage of the Pakistan cable car incident

It was a roller coaster ride with news organisations all over the world giving minute-by-minute reports on the daring rescue. How does the media create suspense and is this sort of coverage useful?

Anam Hussain
Anam Hussain Published on: 21 Sep, 2023
How to use data to report on earthquakes

Sifting through data sounds clinical, but journalists can use it to seek out the human element when reporting on natural disasters such as earthquakes

Arwa Kooli Published on: 19 Sep, 2023
‘I had no idea how to report on this’ - local journalists tackling climate change stories

Local journalists are key to informing the public about the devastating dangers of climate change but, in India, a lack of knowledge, training and access to expert sources is holding them back

Saurabh Sharma
Saurabh Sharma Published on: 13 Sep, 2023
Ethical reporting - how to cover suicide responsibly

Sensationalist reporting of suicide cases has been shown to cause a rise in the numbers of people taking their own lives in affected communities. Journalists must take great care

Abeer Ayyoub
Abeer Ayyoub Published on: 7 Sep, 2023
‘Don’t let someone else narrate your stories for you’ - travel journalists in the global south

THE LONG READ: Life as a travel journalist isn’t just for privileged Westerners ‘discovering’ quaint parts of south-east Asia and Africa

Anam Hussain
Anam Hussain Published on: 1 Sep, 2023
‘People need to stop blindly obeying the law’ - journalists fighting on the fringes in Vietnam

THE LONG READ: Imprisoned, exiled and forced to base themselves overseas, independent journalists in Vietnam are punished harshly if they publish the ‘wrong’ sort of content. Some, such as Luật Khoa tạp chí, are fighting back

Al Jazeera Journalism Review Correspondent Published on: 25 Aug, 2023
Ethics and safety in OSINT - can you believe what you see?

OSINT is increasingly important for journalists in a digital world. We take a look at ethics, safety on the internet and how to spot a ‘deepfake’

Sara Creta Published on: 15 Aug, 2023
‘Other journalists jeer at us’ – life for mobile journalists in Cameroon

Journalists in Cameroon are using their phones in innovative ways to report the news for many different types of media, but major news organisations have still not caught up

Akem Nkwain Published on: 1 Aug, 2023
‘Life of journalists is cheap’ - how the Philippines became deadly for reporters

Forging ahead with a career in journalism is fraught with difficulty in the Philippines - and many are walking away. What went so wrong?

Ana P Santos Published on: 19 Jul, 2023
Analysis: Could AI replace humans in journalism?

Recent advances in AI are mind-blowing. But good journalism requires certain skills which, for now at least, only humans can master

Mei Shigenobu
Mei Shigenobu Published on: 17 Jul, 2023
Analysis: Comparing coverage of the Titanic submersible and migrant boat disasters

Two disasters costing human lives have occurred at sea in the past two weeks, but the media coverage of each was markedly different. How and why?

Anam Hussain
Anam Hussain Published on: 28 Jun, 2023
The silencing of Sudan's journalists - again

THE LONG READ: Detained, beaten and their cameras smashed - Sudan’s journalists are enduring a renewed crackdown on the media

Philip Obaji Jr
Philip Obaji Jr Published on: 7 Jun, 2023
'Rebuilt memory by memory' - recreating a Palestinian village 75 years after the Nakba

REPORTER'S NOTEBOOK: How it took the collective memories of several generations, painstaking interviews and a determined search through tall grass and prickly plants to recreate a destroyed community

Amandas Ong Published on: 4 Jun, 2023
Suffering in silence - the Kashmiri journalists facing a mental health crisis

THE LONG READ: Al Jazeera Journalism Review has interviewed more than 20 journalists in India-controlled Kashmir who are facing exhausting, insurmountable obstacles to doing their jobs safely every single day

Adil Akhoon
Adil Amin Akhoon, Khalid Bashir Gura Published on: 29 May, 2023
How to analyse satellite imagery

When you have a story, but still need to tie up loose ends to answer where or when a particular event occurred, satellite imagery can point you in the right direction

Sara Creta Published on: 25 May, 2023
OSINT: Tracking ships, planes and weapons

Tracking ships and planes is an increasingly valuable technique in open-source investigations carried out by journalists. In part 4 of our special series, we examine how it works

Sara Creta Published on: 18 May, 2023
How social media bans have crippled journalism in India’s Punjab

The Indian government has ordered social media platforms to block hundreds of accounts of journalists and activists

Meer Faisal
Meer Faisal Published on: 16 May, 2023
Tear gas and internet blackouts - reporting on protests in Pakistan

REPORTER'S NOTEBOOK: Following the arrest of former Prime Minister Imran Khan, violence has erupted across Pakistan. For journalists, it is like reporting from the centre of a storm

Anam Hussain
Anam Hussain Published on: 14 May, 2023
Remembering Shireen; my colleague and a 'role model for professionalism'

On the eve of the first anniversary of the killing of Shireen Abu Akleh, Al Jazeera's Senior Correspondent in Palestine, her colleague commemorates the compassion, integrity and professionalism which made her the extraordinary journalist and human being that she was

Walid Omary Published on: 10 May, 2023
Planning and carrying out an open-source investigation

Part three of our special series of articles on using OSINT in journalism. This time, follow our four steps to completing an open-source investigation

Sara Creta Published on: 9 May, 2023
What is an open-source investigation?

In the second part of our special series on using open-source intelligence in journalism, we look at what constitutes and open-source investigation

Sara Creta Published on: 4 May, 2023
How misinformation in the media wreaked havoc on an Indian village

When journalists - and social media ‘influencers’ claiming to be journalists - aimed for sensationalism and did not check their facts about the outbreak of the HIV virus in an Indian village, the results were devastating

Saurabh Sharma
Saurabh Sharma Published on: 1 May, 2023
Why is life so dangerous for Pakistani journalists?

Pakistani journalists face huge danger in the course of carrying out their work. Why is so little being done to address this?

Anam Hussain
Anam Hussain Published on: 6 Apr, 2023
Investigating the assassination of my own father

As a journalist, reporting on the murder of my father meant answering questions about my own position as an objective observer

Diana Lopez
Diana Lopez Zuleta Published on: 3 Apr, 2023