AI Joe Linogao

In 2021 Joe was awarded a Laidlaw scholarship. This is a prestigious award that funds undergraduate students in Trinity to conduct academic research with a professor during the college term and over the summer months. I am nominating Joe based on the work that he performed over the course of this program to date. Joe’s research concerned the automated counting of bacterial colonies on petri dishes.

This is a common task in infection control, where swabbing surfaces for the presence of microorganisms is an important requirement. Counting colonies on a plate is a time consuming and error prone task for microbiologists – it can take several minutes to count the number of colonies on just one plate. AI classifiers exist for automatically counting colonies, however, these typically only work for homogeneous colonies of the same microbe grown under controlled lab conditions.

However, these classifiers perform poorly with petri dishes containing heterogeneous colony growth (which is common when you swab surfaces in hospitals or other clinical settings), where the dish contains different microorganisms and appears much more irregularly.

Joe was initially given a set of high-quality images of petri dishes with varying amounts of bacterial colonies and colony clusters growing. With these images, he first created a labelled dataset which he would then use to train deep learning models for object detectors. These object detectors would be used to detect and identify bacteria colonies on new images of contact plates.

He then trained two separate object detectors, one for detecting individual colonies and another for detecting clusters of colonies. The idea was to pass new images through each detector to reduce the risk of false positives and overcounting. Where there were individual colonies detected, the amount detected would simply be added to a global counter. Where there were colony clusters detected, the resulting bounding boxes would be cropped out and isolated for further processing. Using watershed segmentation, an estimated bacteria count was determined for each colony cluster and was then added to the global counter.

Joe utilised Keras as his deep learning environment. It provided a high-level API for Tensorflow 2 and provided high-quality documentation and quick prototyping for new ideas. However, the main technology utilised was Mask R-CNN or, more specifically, the Python and Kera port by Waleed Abdulla. This port proved beneficial as I was most comfortable coding in Python, provided both training and object detection functions, and was very easy to customise for my dataset. For creating the labelled dataset, he used ImageJ which was an open-source, Python-based software that allowed for simple bounding box and label creation.

The main technical issue was with the image set provided. While many deep-learning models require around 10,000 images to be accurate, the image set we had only contained 200 images. After filtering out unusable images, such as empty plates and fully covered plates, we were left with only 95 images to work with. As such, the challenge was to create a reasonably accurate object detector using a very small dataset. However, by using transfer learning and the CSRNet model weights (which was originally used for head counting for crowds), he was able to train useable models for the program.

The project has produced a comprehensive, automatic bacteria colony counter. The counter was able to take new images in, process them, and provide a result at an average time of 20 – 30 seconds. The “individual bacteria colony” detector and the “colony cluster” detector had accuracies of 90% and 70% respectively, which is comparable with human errors associated. When testing with a new set of images, it was found that the counter provided only a slight amount of overcounting, and performed much better than current state of the art systems, such as OpenCFU.