‘Facial Recognition’ for Cells

Anyone suspected to have cancer wants to know the situation as soon as possible – is the diagnosis confirmed? Is the treatment working? Has it come back? But getting accurate information can be expensive and time-consuming. Labels, or biomarkers, have to be created before samples are screened, to tell the machine what to look for.

But now, a much faster, more cost-effective solution using generative AI has been developed by Professor Kevin Tsia of the Department of Electrical and Electronic Engineering, in collaboration with clinicians from the Li Ka Shing Faculty of Medicine (HKUMed) and Queen Mary Hospital.

Rather than trying to match cells to labels, the Cyto-Morphology Adversarial Distillation (CytoMAD) takes pictures of the cells and uses AI to determine their shape, mechanical properties and molecular information. Anomalies or abnormalities are flagged for clinicians to investigate.

CytoMAD was tested on about 10 lung cancer patients using both blood samples and tissue samples for clinicians to assess each patient’s tumour at the individual cell level and their risk of metastasis. The accuracy was on a par with existing detection methods, but faster and cheaper. Tens of millions of cell images from one patient could be processed in a single day.

“Our technology is very similar to facial recognition technologies that are used to identify individual faces. In our case, we’re focussed on what cells look like, whether their appearance is normal or not, and whether we can identify any disease,” Professor Tsia said.

“The key advantage is our ultra-high-speed imaging capability. By capturing a huge amount of cell images within a very short period of time, we generate enormous, invaluable data that trains our AI models to pinpoint diseases and abnormalities more effectively and efficiently. Our lab also engineered specialised computing hardware that further accelerates the AI computation of the data we collect.”

Creating ‘cell atlases’

Professor Tsia’s work on imaging has been a decade in development, following foundational work such as capturing the dynamic communication of individual neurons in the brain of a mouse that was awake and imaging blood cells racing through the brain at thousands of images per second – this is the kind of speed that his super-fast imaging technology can operate at.

In addition to CytoMAD, he recently also made advances using AI technology to decode the development of zebrafish embryos from a single cell to a complete organism. The zebrafish shares about 70 per cent of its genes with humans so it is of much interest in the scientific community.

Professor Tsia and his team, which included biomedical scientists from HKUMed, developed a large-scale computational framework called StaVia that can process genomic, proteomic and other ‘omics’ information from cells, as well as their morphology, to map in detail how cells change and interact over time, which could help explain disease progression. They worked with one of the largest datasets in the world, from the Chan Zuckerberg Biohub, processing millions of cells and creating catalogues for each cell to uncover the zebrafish’s embryonic development.

“Our tool works very well with large-scale cell atlases, maintaining all the intricate details embedded in omics data, to make it easy and intuitive for scientists to understand complex cell behaviours, from fertilisation to the complex formation of organs,” Professor Tsia said.

Hurdles to overcome

The focus for StaVia at the moment is on developmental biology, given it is a more defined field than cancer, but they also hope it can be applied to diseases. An important feature of StaVia is its ability to do trajectory prediction, meaning it could be harnessed to predict the course of diseases and ageing over time.

“Our hope is that one day, CytoMAD and StaVia could together be a one-stop solution for imaging and doing computations to better understand human disease,” he said.

There are some hurdles to overcome, though. Although CytoMAD can process one patient’s blood and tumour samples in one day, this is still rather slow for larger scale clinical trials. If hundreds of people are involved, it would take the team hundreds of days to process using their present equipment and processing power, so they are looking for partners in the industry and elsewhere to help them scale up.

Professor Tsia added that the work would not be possible without the involvement of clinicians and biomedical scientists. He is also Programme Director of HKU’s Biomedical Engineering programme and has seen enthusiasm grow for establishing more partnerships.

“I have benefitted from cross-faculty, cross-discipline collaboration and I am keen to promote it on campus. HKU researchers are very open-minded and willing to try new things. I strongly advocate to our students that they should embrace these kinds of opportunities. There is a lot of innovation to be unlocked,” he said.