Data is only one part of the investigative story. In Part 5 of our series on investigative journalism, we look at different methods of gathering evidence

More than 130 countries have adopted laws encouraging the sharing of information and 76 of them have become part of the Open Government Partnership. This has improved the flow of data from governments to the media and onto the internet. 

The same applies to companies and individuals. The world produces vast quantities of data every day estimated at 2.5 quintillion bytes (a number with 18 zeros). It is the journalist’s job to sift through this data and confirm its accuracy and analyse and collate it in order to produce high-quality investigations serving the public interest. 

Data does not only mean numbers. It means any raw material - numbers, letters, symbols - whether it has been processed or not, even if it is simply individual facts obtained in different contexts. 

Database it!

To put together a database that will help produce a quality investigation, you need to be aware of where to find that data, whether it is descriptive or numerical, and how best to get at, develop and verify it. Much published or available data will fail to totally meet your needs.

For this reason, you will need to put together a database tailored to your specific needs. You can do this by using existing databases, collecting the missing data yourself or with the help of others and conducting surveys, interviews and field visits. In the absence of more effective ways of getting at data, this will be your best option, despite the effort and cost involved. 

In a 2016 investigation published on the Jordanian website Amman.net and overseen by the Arab Investigative Journalism Network (ARIJ), journalists demonstrated that members of the House of Representatives had been taking home millions in illegal and unconstitutional tendering money. 

Read more:

Investigative journalism, Part 1: What should you investigate?

Investigative journalism, Part 2: Hypothesis-based investigations

Investigative journalism, Part 3: How to develop and manage your sources

Investigative journalism, Part 4: Going undercover

Investigative journalism in the digital age

There was no database giving the names of all the members of the 17th House of Representatives who had taken government contracts or the value of those contracts, or showing their voting, oversight and legislative behaviours after winning these contracts. Nor did requests for information help the investigative team to obtain the names of the companies that had won those contracts or find out their financial value. 

This made the first part of the investigation very difficult. With some effort, however, the team managed to find a list of all government contracts given to companies on the Government Contracts Office’s website. They also found data on the tenders put out by the Public Procurement Bureau and the Greater Amman Municipality. This formed the kernel of the investigation. 

The team also drew on an open database of companies and their shareholders as well as the House of Representatives website, which provides members’ names. This allowed them to make sure that they had the right representatives and there had been no mix-ups because of similar names. 

A leaked list of Civil Status records listing names, spouses and children also provided some clues. Using all this information, the team came up with a spreadsheet format appropriate to the investigation, which they filled in with the names of representatives, their dates of birth, their election dates, the names of the companies in which they held shares, these companies’ purposes and capital, the government contracts they had won, the value and date of those contracts, and other relevant information.

A lot was achieved in this investigation by analysing, collating, indexing and arranging data extracted from publicly available documents. This data helped to create a map of companies, people associated with them, and government contracts. This allowed the team to take the first steps towards producing and publishing this information about 15 days before the next set of elections. 

The time frame made this investigation the most important in the country at the time, and revealed a number of headline figures: 

• A quarter of a billion Jordanian Dinars ($352.5 million) of government money went to companies belonging to HoR member Hazem Majali and his son 
• 128 million went to the siblings of the Atyeh brothers, both HoR members 
• 150 million went to companies owned by the brothers of HoR member Atef Tarawneh 
• 92 members of the 17th House of Representatives held shares in companies with a combined capital of one and a half billion JD.

Data alone is not enough

Nonetheless, data cannot take the place of traditional journalism, and even the best data analysis cannot replace fieldwork and investigation. The team also submitted some 27 Freedom of Information requests to various ministries and institutions asking about government contracts that had not been published on the three official websites. 

They also interviewed experts on the constitutional and legal aspects of the case, on transparency and on companies, and confronted representatives who had violated the constitution and the law. 

Verifying data protects journalists from misleading the public. Data is at least as dishonest as people: at the end of the day, it is collected and collated by human beings. As such, data cannot be trusted absolutely, because it is influenced by the biases of those who collect it. 

You should always make an effort to verify your data and ask questions to those who collected it in order to avoid publishing inaccurate observations. We advise journalists to ask government bodies for raw data, because analysis of raw data helps produce a successful investigation. 

It is also often necessary, however, to access information that the government does not want to reveal. In this case you will have to negotiate with officials, work out how to classify it, digitalise it and enter it into a database. And of course, you will have to sift through it carefully and review it in order to guarantee that it is free of mistakes. 

While data provided by government bodies is essential, you should also try to work out how to get access to other additional or supporting data in order to make sure the investigation is as watertight as possible. 

You could try following the accounts of people assumed to be close to the issue or who are influential or involved in public affairs, whether because they hold an administrative or legislative office or because of their position in the world of finance and business. Following their posts and the people who interact with them may reveal new information or lines of investigation. 

You may also find it useful to look for relevant data by analysing or reviewing old media publications on the subject. This can also give you an insight into public opinion on the issue at hand or on similar issues in the past. After this, you can sift through the data, in order to get a new angle that will give it an added value not present in traditional analysis. 

Written evidence 

Written and documentary evidence is an important source for any journalist seeking to access and prove facts in order to use them as evidence in their investigation. The official documents used should confirm the incidents and acts that the journalist wants to prove. They should be related to the facts and the characters involved. Documents should not be used simply for decoration. 

An investigation produced by the Syrian Investigative Journalism Unit (SIRAJ) showed that five patients had died and a sixth had lost a hand because of negligence and inadequate precautions taken by doctors. 

Both the doctors and the hospitals that they worked at had escaped being held accountable because of the lack of a medical negligence law in Syria. The investigation used autopsies listing negligence as the cause of death as well as death certificates in order to demonstrate this. 

The investigation team made sure to verify that the documents were accurate before incorporating them into the investigation, closely examining the form, the content, the handwriting and the signatures and consulting sources mentioned in the reports. 

A similar approach was taken by the Guardian team that analysed the so-called Palestine Papers (documents pertaining to the Palestine-Israel peace negotiations), which were leaked to Al Jazeera and published in partnership with the British newspaper. Before publishing the documents, they consulted independent sources, former participants in talks and various diplomatic and intelligence sources, who all confirmed that they were accurate.

Official documents issued by responsible government bodies are some of the strongest proof you can muster, so long as they do not turn out to be forged. 

There is a legal principle that an identical copy is as effective as the original unless it can be proven invalid. This principle applies equally to investigative journalism: evidence collected by journalists is no less effective than that collected by detectives, particularly given how often investigations end up in court (either seeking justice for victims or because the journalist is being sued). If the evidence cited is weak, then this can cause you real legal trouble. 

Verifying documentary evidence is essential. You should assess the form and content, the dates, the stamps, the handwriting, the signatures, the letterhead, the CCs and BCCs.

Documents can be just as imprecise as human sources, and may even have been forged or issued by someone with no right to do so. All this means that they can be misleading. Take care to ensure that they are accurate before using them. 

If the document is an official letter, then there will be a sender and a recipient, as well as CCs for reference - each of them giving names and job titles. You can contact these people, present them with the document and ask them to confirm that it is real. Documents that are not covered by secrecy laws may also be subject to Freedom of Information requests.

When power struggles take place between different elites or factions within the regime itself, many documents will be issued, some entirely valid and some less so. Journalists may find themselves being used as puppets in these conflicts. Be particularly careful in these circumstances: publishing a fake document may end your career.

Leaks that do not come in document form but rather as copy-pasted or encoded text require particular care.

Anyone whose name appears in such a leak can easily dispute or deny their involvement, and journalists and their employers may be sued for publishing unsubstantiated information. When no original copy is provided, we have no way of knowing whether sentences or particular words have been lost or added, or whether this has been done deliberately, or whether the figures and dates or the spelling of names is correct.

In the case of the Panama Papers, for example, the leaks came in the form of documents, passports, emails, signatures and emails, which helped to confirm their veracity (the public

admission by Fonseca that the documents were real and had been linked as the result of a hack also lent the documents great credence). Government institutions also took steps to verify the documents. All this transformed them from documents into solid proof. 

The same applies to the famous HSBC leaks, whose veracity was confirmed by the bank itself after a former employee shared them with the French Tax Authority. But none of this means that journalists are exempted from engaging in their own traditional examination and verification of documents that they want to use in their investigations.

There are several kinds of written evidence. In law, these documents have varying probative value:

• Documents produced by government bodies, which are assumed to be proof of their contents unless they are proven to be forged and can be marshalled as evidence of anything and against anyone. These documents include passports, birth certificates, university and school graduation certificates, government contracts, official communications and letters, anything published in the official gazette (so long as it has not subsequently been amended or repealed), and documents originating elsewhere that have been signed or stamped by a competent government official.
• Documents produced and ratified by organisations or companies and issued by a competent employee. These can be used as proof against their originating organisations, so long as they are not proven to be forgeries. Documents of this kind include budgets, decisions taken by the board of directors, other administrative decisions, payrolls, etc. 
• Documents produced and ratified by natural persons (ie signed, stamped or fingerprinted by them). These can be used as proof against their originator, so long as they are not proven to be forgeries, but cannot serve as proof against others. Documents of this kind include personal letters, wills and diaries. 
• Documents produced by organisations or natural persons that have not been ratified or signed can also be cited as supporting evidence, ie supporting a case already substantiated with other forms of proof. 
• Emails can be used as evidence against their sender, so long as they cannot prove that someone else sent the email. 
• Government emails, so long as they are not proven to be forged, can always be used as proof. 
• Final court judgements can be used as proof against anyone with regard to their content and the facts of the case. Non-final judgements and investigation files are not proof be - cause they may include confessions extracted by force, but they can be used as supporting evidence.
• The minutes of parliamentary sessions and meetings of municipal councils, local committees, party committees, associations, unions and publicly traded companies can be used as proof of what was said, even if it is defamatory, as well as any decisions reached and the general progress of the session. 

Personal experience 

By “personal experience”, we mean a journalist undertaking an entirely legal action, without concealing their identity, in order to test compliance to the law by others and uncover mistakes or violations taking place because of the system or its responsible people. 

For example, you might buy a prepaid sim card and verify that companies calculate the price of a minute correctly as they claim. Or you might go and play a particular sport that requires protective gear and discover that the relevant authority is not interested in the safety of players and allows them to play in normal clothing. 

Personal experience is one of the most important sources of proof, particularly in investigations that involve careful inspection of legal procedures or routine official practice. Eyewitnesses, no matter how good they are at telling you the facts, often do not pay enough attention to the detail of legally and procedurally complex processes to provide you with all the information you need. 

A journalist, on the other hand, can familiarise themselves with the process through careful research and then test it out. Under these circumstances, they may end up having to conceal their identity. 

Majdoline Alan, for example, produced a report on government failures to implement the Jordanian Freedom of Information Law, published in Al Arab Al Yawm in 2010. All she had to do was submit Freedom of Information requests to the relevant government offices and see whether the procedure was completed as stipulated by law - that is, whether the request was accepted and processed by the deadline imposed by the law and the form and content of the response. 

She then submitted a complaint to the Information Office, and brought a lawsuit against the Land and Surveys Bureau for failing to respond to a request for information on the sale of government land. By carefully documenting every stage of the process, she was able to demonstrate that the law suffered from loopholes and was not being enforced. 

The team behind a 2014 investigation into misleading adverts that tricked girls into working in prostitution wanted to find out how adverts of this kind were finding their way into respectable newspapers, and how much effort newspapers put into verifying advertising submissions. 

They thus submitted misleading and vague adverts offering work for girls to two national newspapers. The newspapers did not request any proof of personal or organisational identity in order to confirm the veracity of the adverts. After the publication of the story, the government introduced legislation obliging newspapers to run checks on advertisers. 

You might reasonably ask what the difference is between personal experience and disguise. In the latter case, a journalist changes their identity. They go into a place that would be difficult to enter honestly under false pretences, because without doing so they would be unable to prove their claims, and because if they were discovered, they would not get the information they were looking for. 

Personal experience, meanwhile, does not involve a change of identity. The journalist goes in like any other normal person, exercising their right to do so. Their identity being revealed would have no effect on the operation. Nor would it put them in any danger. In this case they are a service user or customer like any other. Secret operations are more complicated and subject to more conditions. 

You should always think carefully before using personal experience and make sure that it is going to be useful and add value to the investigation. It should contribute to uncovering the truth and proving the hypothesis. 

When gathering information through personal experience: 

• Do not conceal your identity. If asked, you should answer honestly. But you should not volunteer this information without being asked, as it may skew the results. 
• Do engage with the system as a normal citizen - as a normal service user or customer - in an entirely legal way, in order to find out how it works and identify problems. 
• Do repeat the experience multiple times in order to prove that the problem is systemic and cannot be attributed to bad apples. 
• Do conduct in-depth, careful research, in which you familiarise yourself with the relevant regulations and procedures before engaging with them. 
• Do be careful to engage with the process like any other person. 
• Do be careful not to attract attention. 
• Do document everything that you do and everything that takes place. Ask for and hold onto receipts, cards and documents, take photos and record films, or write everything down in detail as soon as you finish. 

Lab analysis 

Journalists use lab analysis to confirm that violations have taken place that cannot be proven by other methods. Lab analysis constitutes very solid evidence, especially if it is produced by a neutral, specialised, recognised party. Although it is costly and time-consuming and requires concentration and precision, it can be exactly what an investigation needs to make it watertight. 

Al Jazeera used lab analysis in its famous investigation into the death of Yasser Arafat, in which analysis conducted by a Swiss laboratory uncovered high levels of polonium - a radioactive poison - in the former PLO leader’s belongings. Arafat’s remains were subsequently disinterred and samples taken for testing in France, Russia and Switzerland. 

Lab testing is used to investigate deaths, attempted murders and sexual assaults. It can also be used when investigating food safety, water quality, agricultural practices and consumer protection issues, when you need to establish the extent to which roads, power grids or children’s toys (for example) meet industry standards. 

Lab testing requires careful and professional study. In its aforementioned investigation into Arafat’s death, the Al Jazeera investigation team was helped by the family of the deceased, who had held onto various items of his belongings and were able to supply hair and urine samples from his clothes. If journalists had not been able to convince Arafat’s widow, Soha, to participate, the investigation could not have gone ahead. 

A journalist who wants to produce an investigation drawing on lab analysis will need to familiarise themselves with the nature of the analysis required and the types of measurements that it is likely to produce. It is never a bad idea to consult experts on these sorts of questions. 

Lab analysis also requires a precise and solid methodology in order to rule out any doubt in the results and their credibility. You should always stick to highly respected labs that have technical and scientific accreditation. 

Lab analysis results alone do not constitute an investigation. You will also need to establish the background and the context, as well as working out what they mean from a scientific perspective and what effect they have on the victims. You will also need to work out how the results of the analysis are connected to the victims and to the event. 

In Al Jazeera’s investigation, journalists had to begin with Arafat’s DNA, making sure that it matched with the hair they found in his hat and the urine samples they took from his underwear. They then had to look for poisonous chemicals, and then biological and ultimately radioactive material in order to get results. This was a long and costly process, but it was worth it. 

Investigations into the safety of food, medicine, water etc require mechanical study. This begins with taking samples correctly. You might want to seek the help of a technical expert in taking the samples. If so, you can record the process on video. You should also record the payment as well as the process of taking samples to the laboratory and handing them over.

You will also need to explain the results to the public simply and straightforwardly, emphasising why the results are important for them. The investigation may require testing multiple samples of the same kind over an extended period of time in order to demonstrate that contamination, for example, is not an isolated case but part of a systematic and ongoing problem. 

An example of this kind of investigation was one undertaken by journalists at the Jordanian magazine 7iber, which discovered that Jordanians were unwittingly buying genetically modified wheat from traders who were circumventing the law.  

Technical expertise 

In many investigations, a journalist will not be able to produce convincing proof on their own because of the technical nature of the issue. In this case you will have no option but to turn to an expert. If you are investigating, for example, the forging of official documents, diplomas or signatures, then you will have to seek the help of a technical expert who can help you analyse and compare them in order to work out whether they have been forged or not.

Technical expertise is also useful when working out property pricing or identifying ill-gotten wealth or financial corruption. In this case you might want to take an assessor recognised by the courts or by government bodies to the property and ask them to write an expert report. You should then get another two independent experts to do the same thing and compare the results, so as to ensure that they are neutral and independent. 

Expert assistants can also be of use in working out whether infrastructure or buildings meet technical specifications - the thickness and height of walls, for example, or the external shape of a building. 

You should always choose the best experts - the ones used by the courts or other official bodies - and make sure that they are credible. When they finish their work, they should write a full report on what they have seen and, if possible, sign and stamp it. A report of this kind can serve as proof in court. 

If the investigation is going to be broadcast on TV, you should also interview the expert on camera. Al Jazeera’s Extrajudicial Execution report showed that nine prominent Muslim Brotherhood leaders who died in a flat in 6 October City (Cairo, Egypt) - who the Interior Ministry claimed had been shot during a firefight with the security forces - had actually been executed. 

The investigation found that there had been no such firefight, and that the nine victims had been shot deliberately at point blank range after being taken into custody. Proving that no firefight had taken place and that the men had been killed deliberately required forensic expertise. Victims’ entry and exit wounds were examined, allowing experts to establish the range at which they had been shot as well as other details: if an entry wound was on the dead man’s back, for example, then that meant that he could not have been attacking the security forces - who were supposedly firing up from the ground at a flat on a higher floor. 

The team behind the investigation used the services of a group of independent forensic pathologists including Tasneem Koroud, head of forensic pathology at Istanbul University, the Iraqi pathologist Abdul Nasser Kilani, and the security expert Mazen Al Samarrai. These experts were able to totally disprove the security forces’ story using scientific and technical evidence, demonstrating that one of the victims was leaning against a door facing away from his assailants (rather than facing towards them and resting the gun on the door as he would have been if he was attacking). 

Just as with lab analysis, technical expertise alone does not produce an investigation. It is part of the process of proof and must be supported with context, background, source testimony and, if possible, documentary evidence.

Gathering testimony 

Some journalists reach out directly to the public for information. 

In a Boston Globe investigation that revealed sexual harassment in Catholic churches in Boston, a team of journalists followed up the publication of their main investigation by calling for input from their readership and the broader public, in cooperation with local radio stations. This meant that many victims were able to provide further information and documentary evidence to the paper, allowing them to keep the investigation alive for as long as possible. 

Thanks to rapid technological development, gathering data in this way is now easier than ever - but it is also more dangerous. Data provided voluntarily by the public needs a great deal of work to verify and corroborate before it is incorporated into the investigation.

Bellingcat was able to use some 60 sources found using social media to track down a suspect who had fled the Dutch police, identifying the locations of photos he had shared on Instagram. Thanks to these efforts, the police were able to find and arrest the suspect and charge him with two murders and the theft of 175,000 euros. 

Gathering testimony like this requires a solid relationship with the audience based on trust rooted in the journalist’s independence, professionality and desire to serve the public interest. 

The website Help Me Investigate has also used this technique to produce some very important investigations. 

This article has been adapted from the AJMI Investigative Journalism Handbook