In the United Kingdom, Prevention of Future Deaths forms (PFDs) play a crucial role in ensuring public safety. This is a special type of coroner report that documents more than just the circumstances of an individual’s death. PFDs are issued when a coroner investigates a death and rules that a specific risk or systemic failure—deemed as preventable—has played a significant role in said death.

While these forms do have a structure in so much as they each have sections that must be filled out by coroners, these sections are filled out by coroners in natural language, making analysis of these forms (until now) very time consuming, with each report having to be read by a human.

Wolfram Language’s extensive list of built-in functions allows calls to various different large language models (LLMs) to be made from inside the Wolfram kernel. Implementing LLMs inside of Wolfram means that extracting unstructured data, such as the contents of a coroner report, is conducted in a fraction of the time. We can then use Wolfram’s data analysis tools to process what we’ve gathered.

Collecting the Data

The UK Courts and Tribunals Judiciary posts a sample of these PFDs on their website. Unfortunately, they don’t have a public API for accessing these files, meaning the only way to view the files is by visiting the page and finding each file. This would take a very long time to do by hand, so we’ll need to make a web scraper to go through and automatically download the PFDs:

Note: all data is taken from the Courts and Tribunal Judiciary Prevention of Future Death Reports under Open Government Licence v3 .0.

Engage with the code in this post by downloading the Wolfram Notebook

Let’s test that this code works by getting the first two pages of links:

Brilliant! Now let’s use it to pull from more pages:

Let’s now import all of them to get the text of the document:

Data Extraction

With the data now collected, an interesting application is to review the length of these investigations plotted over time. The traditional way to do this would be to have someone read all of these reports and manually input the start and end dates of an investigation into a spreadsheet. This sounds very time consuming (and boring). LLMs can be extremely helpful here, having enough knowledge to be able to read the report and extract just the two dates, while taking nowhere near as long as a human would.

One drawback of using LLMs is that a lot of prompting often has to go into them to constrain their behavior. With imprecise or vaguely worded prompting, the LLM often ends up being very unhelpful and produces unexpected results. Thankfully, Wolfram has a good way of combatting this drawback. LLMExampleFunction not only takes standard prompting as an argument, but also allows you to pass in a list of examples for the LLM to follow:

This piece of code uses LLMExampleFunction to create a function that will take imported PDFs as input and will give a list containing the start and end dates of the investigations:

A timeline plot of a random sample of the results shows that it returned what was expected (each line on the plot represents an investigation):

Real-World Applications

Previous academic research from Alison Leary et al. has investigated the main areas of concern that coroners express in their reports. Here, we use the resulting categories of that research and apply it to our own data. With that, we are able to combine previous insights from academia, the computational power of Wolfram Language and the fluency of LLMs to gather insights on a much larger corpus of data:

Categorization Code

We then list the main concerns identified by Leary, we pass those concerns to an LLMFunction and we prompt the language model to apply the categories to each file in the dataset:

Plotting

By plotting each category for each year that we have reports for in a bar chart, we can see the most common PFDs:

A stacked bar chart is an alternative way of visualizing the same data that allows us to focus on the proportion of each category within each year. While roughly consistent across the years, we can spot some temporal trends, for example, the peak in communication issues in 2020. The bar for 2024 is much smaller since the data was collected in the summer of 2024, when most reports for that year hadn’t been submitted yet:

To visualize the current trends better, we can make the stacked bar chart proportional to 100% of the year’s concerns. In that, we see that communication issues are on track to have a higher proportion of the total share compared to previous years, potentially reaching the levels that they had in 2020:

Interestingly, these results mostly mirror the ones found in Leary’s work. This suggests that employing LLMs in the initial stages of tasks that aim to extract insights from natural language—such as thematic analyses—can be a valuable first step in getting meaning out of unstructured data. That is likely to be especially true in cases where broad categories have already been defined by previous works, and these definitions can be passed down as instructions to the LLMs.

Going Forward

Using Wolfram tech, we can quickly gather and prepare data to spend more time making analyses and finding solutions to improve practices in the future. For extra help learning to computationalize your workflow, be sure to check out the new Wolfram Notebook Assistant!