What Journalists Need to Know About APIs for Data Reporting

For a journalist, the powerful capabilities of Application Programming Interfaces (APIs) can be a valuable asset. In this guide, you’ll learn the basic principles on how to use APIs to assist with your data research and reporting, as well as tips and tricks to get the most out of their implementation.

Journalists looking for their next data-driven story might find an API just as useful, if not more so, than a traditional spreadsheet. What are APIs and how can they help in creating reports? Paul Bradshaw explains.Paul Bradshaw. To watch a video tutorial by Bradshaw on using APIs, go to the end of this article.

If you’re looking for data as a journalist, there’s a good chance you’ll find it in an API. The meaning of those three letters (Application Programming Interface) isn’t that important: what reallyè matters for journalists is that an API is a way to ask questions and get answers.

What is an API?

An API (Application Programming Interface) is a way to enable communication between different applications or websites. Simply put, it allows apps to talk to each other by exchanging data. APIs are most commonly used to simplify complex operations and increase the efficiency of processes where information is exchanged between different computer systems.

So is an API just a type of database? Well, yes and no. Some APIs work like a database, where you can specify a keyword or category, select parameters for your search, and get results in a data format like CSV, JSON, or XML.

OpenCorporates is an example of a website that offers an API like this: you provide a name to a part of the API and it gives you data on company directors who match (this API was used for Global Witness’s investigation into the jade trade). OpenCorporates offers free accounts to journalists and you’ll find videos like this one where they explain how to use the platform for investigations.

The UK’s police API, for example, will provide data on crimes near a certain location. To get that data, you need to form a URL: one example provided in the API “documentation” is https://data.police.uk/api/crimes-at-location?date=2017-02&lat=52.629729&lng=-1.131592

That URL basically encodes a particular query. It breaks down into several parts:

• The base URL, which points to the API itself. In this case: https://data.police.uk/api/
• The type of question being asked (an API can support different types of questions). In this case: crimes at location
• A question mark followed by any ingredient used in your query. In this case the example indicates that the API expects a date, a latitude, and a longitude: date=2017-02&lat=52.629729&lng=-1.131592

If I break them up at each ampersand you can see more clearly what’s being asked:

date=2017-02  &  lat=52.629729  &  lng=-1.131592

The items after each equal sign (and before any following ampersand) are the specific values of your query, so in this case you could change the value after ‘lat=’ and ‘lng=’ to get data for a different location, and the date value in the format YYYY-MM to get data for a different period (the API no longer returns data from 2017, so you’ll need to update it in this specific example anyway).

When you load that URL – use Firefox or Chrome for best results – you will be provided with data in response to that query. It won’t look like a normal web page because the data will most likely be in JSON format (some APIs will let you specify which format you prefer, and some provide data in XML), which uses brackets and curly braces to structure the data.

Handling that JSON (or XML) is a whole separate challenge and is beyond the scope of this article. It usually requires using code (Python, R, and JavaScript all work well with JSON and XML) and ultimately, that’s what an API is designed for: to make it easier to write scripts that can retrieve and work with data.

However, if you just want data from one query, you can save the JSON or XML (using your browser’s File > Save option) and search for “JSON to CSV converter” or “XML to CSV converter” to find one of the many free tools that will convert the data into a spreadsheet (such as those on ConvertCSV).

Using APIs for image or other media research

But other APIs let you ask a question in the form of an image: the Google Cloud Vision API, when an image is provided, it will return the text it identifies in the image, list objects, and give assessments regarding concepts like “bold” or “violence.” The Microsoft Emotion API, for example, was used by Periscopic to produce an analysis of Donald Trump’s facial expressions during his most important speeches.

The other big difference between a database and an API is that some APIs do not return data in response to a query but other media like images and sounds: QuickChart is just one of several APIs that will generate a chart when given data. Other APIs will return audio (music or speech), and the Google Maps API will provide you with a map when queried with a lat-long coordinate.

A final difference, crucial for journalists, is that APIs are often updated much more regularly than published datasets. Some APIs provide access to real-time data: Twitter’s API, for example, tells you what’s on the site right now. This means APIs can be used to power interactive visualizations that update whenever data is — there’s no need to download the latest spreadsheet, repeat your analysis, and create a new chart: instead, you can write a script that updates the charts whenever the data changes.

Building stories with APIs

APIs open up new storytelling possibilities that aren’t always available with traditional datasets. Spotify’s API provides data on music and listening behavior that isn’t accessible in any other way. At the BBC Data Unit, I used it as part of a report on gender balance at music festivals , while for a story about the most expensive train tickets the Google Maps Distance Matrix API was used, which allowed me to find out the distance of a train journey itself, something you don’t get on Google Maps itself.

APIs can be particularly useful for augmenting existing data: a number of APIs, when given a list of names, will return a likely gender for each, along with a confidence level. The Parserator API will break down an address into its separate components; and there are several APIs that will provide you with a list of administrative bodies for a given postcode or zip code, helping you categorize granular data by area.

Where to find APIs

Many data sources will provide an API alongside traditional spreadsheets: it’s always worth paying attention and checking if it offers any additional functionality or information. It’s also a good idea to include the word “API” when searching for data more generally.

Once you find an API, look for the “documentation”: this is a page or a collection of pages that typically explains how the API works. Normally, it is aimed at people familiar with programming, so there will be some jargon to deal with, including:

• Functions and methods : these are the queries you can make in an API. The OpenCorporates API, for example, has a “method call” for searching company officers and another for searching by company number, among others.
• Arguments : these are parameters you might include when making a query to an API. For example, you might need to include an argument to specify the file format in which you want the results, or the postcode for which you want to obtain the data.
• API keys : a “key” is a password that some APIs may require you to include in your query. To get a key, you’ll normally need to register on the site, and to use it you typically include it as one of the “arguments” (see above).
• Limits and quotas : some APIs will limit the number of queries (or “requests”) you can make per hour, day, or month.
• Endpoint : an endpoint is a URL you generate to ask your question. Once loaded, the URL should provide the answer to your question, in the form of data (often in JSON format).

Not all APIs are equal: some are poorly documented. Some may even be discontinued, so don’t be afraid to abandon one API and try others until you’re satisfied.

There’s a certain learning curve involved in using APIs: you need to know a bit of programming or ask a developer for help. So, if you’re just getting started, look for tutorials that guide you through the process of working with a particular API, and try to choose APIs that provide sample URLs you can adapt (the UK police API, for example, offers a ‘sample request’ for each method) or interactive tools that allow you to generate a URL.