Applying to EA Global

In which I attempt to answer the question: “Please briefly describe your current approach to improving the world”.

I work at a good university in a full time research role in AI for digital diagnostics, and am also doing a PhD in AI interpretability. When my PhD is complete I hope to either stay in academia in a similar research role, or move into industry (which would increase my ability to give). I’m unsure which is the most impactful choice.
It is my hope that my work will lead to a) improved safety through better understanding of deep neural networks, b) the ability to leverage this understanding for knowledge discovery as well as automation, and c) high-impact clinical applications.


Recent work includes:

  • A new saliency mapping algorithm which is 20x faster than SoTA and completely model-agnostic, enabling very fast local attribution even for very large inputs or models.
  • Using deep learning and interpretability methods to identify previously unknown and highly discriminative morphological features in immune cells from Hoechst stained slides, which should have significant clinical impact as it has potential to decrease immune-profiling costs by an order of magnitude, allowing many more patients to benefit from tailored immunotherapy and more accurate prognosis.
  • A broadly applicable neural network training protocol, with which I hope to improve the intrinsic interpretability of DNNs by optimising for feature clustering and branching. (In progress.)

Crap drawings

We own so many beautiful things, and also so much crap. Here is my gorgeous and wickedly sharp folded steel cleaver, but also my macbook charger, which, savaged by a cat I know, now works intermittently at best. Old glue, used by a former flatmate to fix his boots, moved house with us and will now languish in a drawer for years as it gradually solidifies. Approximately ten thousand pencils. And elderly vegetables, which we watch rot while eating mortadella sandwiches. I try to be stricter and keep only what’s beautiful, but there’s always the possibility I might find myself in need of sofrito.

Hoechst Is All You Need: Lymphocyte Classification with Deep Learning

Multiplex immunofluorescence and immunohistochemistry benefit patients by allowing cancer pathologists to identify several proteins expressed on the surface of cells, enabling cell classification, better understanding of the tumour microenvironment, more accurate diagnoses, prognoses, and tailored immunotherapy based on the immune status of individual patients. However, they are expensive and time consuming processes which require complex staining and imaging techniques by expert technicians. Hoechst staining is much cheaper and easier to perform, but is not typically used in this case as it binds to DNA rather than to the proteins targeted by immunofluorescent techniques, and it was not previously thought possible to differentiate cells expressing these proteins based only on DNA morphology. In this work we show otherwise, training a deep convolutional neural network to identify cells expressing three proteins (T lymphocyte markers CD3 and CD8, and the B lymphocyte marker CD20) with greater than 90% precision and recall, from Hoechst 33342 stained tissue only. Our model learns previously unknown morphological features associated with expression of these proteins which can be used to accurately differentiate lymphocyte subtypes for use in key prognostic metrics such as assessment of immune cell infiltration, and thereby predict and improve patient outcomes without the need for costly multiplex immunofluorescence.

Full Text

Oil Portraits

In Scotland I had a lovely phase of painting silly oil portraits of our friends to adorn Matt‘s office. I have a hard time finding similar opportunities these days — if anybody out there would like one, let me know.

Believe The HiPe: Hierarchical Perturbation for Fast and Robust Explanation of Black Box Models

Understanding the predictions made by Artificial Intelligence (AI) systems is becoming more and more important as deep learning models are used for increasingly complex and high-stakes tasks. Saliency mapping – an easily interpretable visual attribution method – is one important tool for this, but existing formulations are limited by either computational cost or architectural constraints. We therefore propose Hierarchical Perturbation, a very fast and completely model-agnostic method for explaining model predictions with robust saliency maps. Using standard benchmarks and datasets, we show that our saliency maps are of competitive or superior quality to those generated by existing black-box methods – and are over 20x faster to compute.

Preprint

Learning to Describe: A New Approach to Computer Vision Based Ancient Coin Analysis

In recent years, a range of problems under the broad umbrella of computer vision based analysis of ancient coins have been attracting an increasing amount of attention. Notwithstanding this research effort, the results achieved by the state of the art in published literature remain poor and far from sufficiently well performing for any practical purpose. In the present paper we present a series of contributions which we believe will benefit the interested community. We explain that the approach of visual matching of coins, universally adopted in existing published papers on the topic, is not of practical interest because the number of ancient coin types exceeds by far the number of those types which have been imaged, be it in digital form (e.g., online) or otherwise (traditional film, in print, etc.). Rather, we argue that the focus should be on understanding the semantic content of coins. Hence, we describe a novel approach—to first extract semantic concepts from real-world multimodal input and associate them with their corresponding coin images, and then to train a convolutional neural network to learn the appearance of these concepts. On a real-world data set, we demonstrate highly promising results, correctly identifying a range of visual elements on unseen coins with up to 84% accuracy.

Full Text

Understanding Ancient Coin Images

In recent years, a range of problems within the broad umbrella of automatic, computer vision based analysis of ancient coins has been attracting an increasing amount of attention. Notwithstanding this research effort, the results achieved by the state of the art in the published literature remain poor and far from sufficiently well performing for any practical purpose. In the present paper we present a series of contributions which we believe will benefit the interested community. Firstly, we explain that the approach of visual matching of coins, universally adopted in all existing published papers on the topic, is not of practical interest because the number of ancient coin types exceeds by far the number of those types which have been imaged, be it in digital form (e.g. online) or otherwise (traditional film, in print, etc.). Rather, we argue that the focus should be on the understanding of the semantic content of coins. Hence, we describe a novel method which uses real-world multimodal input to extract and associate semantic concepts with the correct coin images and then using a novel convolutional neural network learn the appearance of these concepts. Empirical evidence on a real-world and by far the largest data set of ancient coins, we demonstrate highly promising results.

Full Text Preprint

Weakly Supervised Deep Learning of Artistically Depicted Concepts on Ancient Coins

Ancient coins are difficult to classify as they exhibit a large amount of variation due to centering issues, differences in shape, colour and orientation, varying depictions of semantically identical elements, and patination or wear on the surface of the coin.

Most current identification methods are local feature based, but the loss of spatial relationships often results in bad performance. To overcome this limitation, approaches which divide a coin into segments have been tried, but these assume that coins have perfect centring, are registered accurately, and are nearly circular in shape, none of which are reliable assumptions. 

All current work in ancient coin recognition relies on matching existing coin types. This limits the application of existing classification systems, as assuming that the unknown coin type is present in the training set is unrealistic – there are hundreds of thousands of different types of coin. Because of this, it is more useful to be able to describe what is depicted upon the coin, rather than classify it by type.

This study trains a deep convolutional neural network to predict if a given concept (e.g. a patera, a shield, a cornucopia) is depicted on an unknown coin. To validate whether the network is successfully identifying the salient pixels, heatmaps are generated using an occlusion technique to visualise which areas of the image are most important for the prediction made.

Full Text