If you kick a football forward, you will have a fairly good idea about its trajectory. For instance, you will be able to anticipate that the football will roll forward in the direction it was pushed and even where it might end up after it bounces off an obstacle.
Owing to a lifetime of experience with the planet’s gravity and laws of physics, as well as millions of years of evolution hardwired into our brains, humans are pretty good at inferring causal relationships. You kick a ball, it will start moving forward. And if you were to freeze the movement of the ball in space and time, you would be good at predicting where the ball ends up, frame by frame, with each passing second.
But believe it or not, even supposedly ‘smart’ machine learning algorithms would find this task challenging. While AIs are good at spotting patterns of correlation, they can be very poor at predicting an event triggered by a subsequent event a number of steps into the future. For an AI, predicting that a ball moves in reverse rather than forward once you kick wouldn’t be nonsense.
In order to improve causal reasoning in AIs, researchers at Imperial College London thought outside the box and took inspiration from Einstein‘s theory of special relativity. What’s the connection, you might ask?
One of the consequences of Einstein’s theory of special relativity is that nothing can travel faster than the speed of light. But this also means that there are boundaries of cause and effect in spacetime, which can be mathematically described. The resulting mathematical structures are known as light cones.
Spacetime is constructed by taking instantaneous snapshots of space at successive instants of time and stacking them up — think of a movie where each frame represents such an instance of time.
To get an idea of how light cones work, imagine an explosion of TNT. Light will propagate out from the TNT in an expanding spherical shell. In a two dimensional space, it will look like an expanding circle, whose radius can never expand more than the distance traveled by light for a given timeline.
If we stack up these spatial snapshots, you end up with a spacetime diagram that corresponds to a cone, where the pointy end corresponds to the origin of an event and the subsequent ever-expanding circles represent causal events. Light cones are usually represented as an hourglass figure, with the bottom cone representing spacetime in reverse: if you follow the circles you can gain information about possible pasts that could have led to the original event.
In order to predict events, such as the next frame in a movie, AIs are typically fed millions of clips that allow the algorithm to generate new frames that are similar to the preceding ones, ultimately picking the one that best suits the context. That works well enough for some applications such as bionic limbs developed by the likes of Tej Kohli or Open Bionics, but there’s always room for improvmeent.
Taking inspiration from light cones, researchers led by Athanasios Vlontzos at Imperial College London used two datasets, one consisting of handwritten digits, the other showing people walking and waving their arms in a time series, to generate frames similar to the ones in the dataset.
What sets this approach apart from other algorithms is that frames from the datasets were not shown in sequence. Instead, the AI grouped frames by similarity and then used the light-cone algorithm to draw a boundary.
Although this algorithm was not trained to fill in the blanks in the sequence, it did a good job of predicting which frames came next.
“We utilize the concept of a light cone from special relativity to restrict and traverse the latent space of an arbitrary model. We demonstrate successful applications in causal image synthesis and future video frame prediction on a dataset of images. Our framework is architecture- and task-independent and comes with strong theoretical guarantees of causal capabilities,” the researchers wrote.
AIs that can predict causal events like humans would be highly desirable for a number of applications. Self-driving cars would be more accurate at predicting the motion of a cyclist or other vehicles on the road, thereby reducing traffic-related accidents, same goes for industrial robots that interact with human workers on the assembly line. In medicine, causal algorithms would be more sensible at recommending treatments that need to run their course step by step.
The light cone algorithm was described in a study published in the preprint server arXiv.
Article updated on September 7 for clarity. A paragraph describing how light cones are represented incorrently stated that a reverse cone indicates possible pasts ‘triggered by the original event’ rather than ‘possible pasts that could have led to the original event’.
