Advancing Intelligence

Professor Can Li, Assistant Professor in the Department of Electrical and Electronic Engineering was awarded the Croucher Tak Wah Mak Innovation Award last year, for his work on advanced computer systems and emerging nano-electronic devices. His research involves tackling the limits of traditional digital computers by working to create advanced artificial intelligence hardware systems that are inspired by the human brain. The analogue systems that he is developing amalgamate how the human brain remembers and recognises information – namely the associative memory, and the randomness of decision-making – a necessary ingredient for advanced intelligence.

These tasks require hardware that can process analogue signals at the exact location where the information is stored, which has bottlenecked the performance of digital computers when handling a massive amount of data, for example, the neural network model for artificial intelligence. In achieving the task, Professor Li is exploring the potential of the memristor, an emerging nano-electronic device that can not only process information in the memory device, but also emulate the non-linear behaviour of biological synapses and neurons through full analogue processing.

“The crux of the matter is that AI capability is increasing faster than the hardware tech can cope,” said Professor Li. “Before it doubled every two years, now the trend is for it to double every three to four months. This means the software is not sustainable because the hardware cannot support it. Hence, it is time to start inventing again!

“What is fundamental is that traditional computers work on Boolean logics – all zeros and ones – and they are very good at some things like mathematical calculation. AI is different in that it is a brain-inspired algorithm. So we need to design the computer in a new way. The data involved is endless, so bottlenecks occur, and brain-inspired ‘neural networks’ are inefficient on digital computers.”

Brain-inspired

Professor Li’s research asks what we can learn from the brain and from what Mother Nature can do, and finds parallels, such as: they compute in the analogue domain; they massively parallel compute on easy tasks; and there is some stochasticity.

Stochasticity is the quality of lacking a predictable order or plan – that is, things being random or haphazard. Human brains are high-density and work at high speed, but mistakes occur. Professor Li’s argument is that random mistakes can lead to discovery – and if this stochasticity can be worked into analogue computers new things may unfold. “Sometimes stochasticity is the source of intelligence,” he said, “and we want to harness this unpredictability as a feature to increase functionality.”

In the past 10 years, he has been using a new device called a memristor – which has very high density and very high speed and can be scaled down to smaller than two nanometres – to help build a brain-inspired computing machine. His team have built a prototype computer – “3D like the human brain” – integrating a memristor.

“Humans also memorise things differently from computers,” said Professor Li. “Humans learn from experience, but computers cannot do that – not even ChatGPT! A computer stores the pattern in the memory, so when we want to retrieve the pattern we need to know the address of where it’s stored. Humans can use associative memory – ‘maybe I have experienced something similar to that’ – so we are trying to introduce associative memory paths into the computer.

“We are imitating that in artificial neural networks using the memristor crossbar array. A 3D crossbar has the potential to provide human-brain-level complexity.”

Physical systems

The second focus of the research is what can be learnt from physical systems. “First, how the computing system can learn from the crystallisation process or annealing process. For example, how atoms in a solid-state system find their location that minimises the system’s energy when the system cools down,” said Professor Li. “Second, how can we learn from quantum computers and apply that knowledge to our classical analogue computers?

“We start with a very simple configuration and the system will find the minimum energy very quickly. And then we change the energy configuration slowly to a more and more complicated configuration,” said Professor Li. “So this is something we learn from quantum computing because the most mature commercial application for the computer right now is to find the combinatorial optimisation problem.”

In conclusion, Professor Li said that, while researchers know that quantum computing is superior, there are problems. One is the need for a cryogenic system to produce very low temperatures – “highly problematic because it consumes a lot of energy and is not portable”.

“We have learnt some things from the quantum system that we can use for analogue computers in the classical domain, and therefore we are implementing a quantum element in the classical analogue computer,” he said. “We are also exploring ways to use other natural processes to perform that. If you compare our solution with the quantum computers you can see that we are actually better, especially when you consider the energy needed for a cryogenic system in the comparison.”