The amazing camera that can see around corners

It’s just your average corner, but a far from average camera. Gariepy et al./Heriott-Watt

How can a person see around a blind corner? One answer is to develop X-ray vision. A more mundane approach is to use a mirror. But if neither are an option, a group of scientists led by Genevieve Gariepy have developed a state-of-the-art detector which, with some clever data processing techniques, can turn walls and floors into a “virtual mirror”, giving the power to locate and track moving objects out of direct line of sight.

The shiny surface of a mirror works by reflecting scattered light from an object at a well-defined angle towards your eye. Because light scattered from different points on the object is reflected at the same angle, your eye sees a clear image of the object. In contrast, a non-reflective surface scatters light randomly in all directions, and creates no clear image.

However, as the researchers at Heriot-Watt University and the University of Edinburgh recognised, there is a way to tease out information on the object even from apparently random scattered light. Their method, published in Nature Photonics, relies on laser range-finding technology, which measures the distance to an object based on the time it takes a pulse of light to travel to the object, scatter, and travel back to a detector.

Laser light fired at the floor is scattered in a spherical wave. It reflects from the hidden object and makes its way back to the SPAD detector, where the noisy data is processed to reveal the hidden object. Gariepy et al./Nature Photonics

In principle, the measurement is quite simple. A laser pulse is bounced off the floor and scatters in all directions. A small fraction of the laser light strikes the object, and the back-scattered light is recorded on a patch of floor – the “virtual mirror” – next to the spot the laser strikes. Because the speed of light is known and constant, by measuring the time interval between the start of the laser pulse and the scattered light reaching the patch of floor, the position of the object can be triangulated.

However, the devil is in the detail. The timing measurement needs to be accurate to within around 50 thousand billionths of a second (5x10-11, or 50 picoseconds), and the light levels that must be detected are extremely low. Overcoming both of these obstacles requires some serious laser and detector technology. The laser pulses used for the timing measurement are just ten femtoseconds (10 million-billionths of a second, or 10-15) long, and each pixel in the ultra-sensitive “camera” (known as a single-pixel avalanche diode array, or SPAD) used to image the patch of floor is essentially an ultrafast stopwatch that records the arrival time of the scattered light pulse to within a few hundred-billionths of a second.

The complications do not end there. Light scattered from the object of interest reaches the virtual mirror of the floor, but so does light scattered from every other object in the vicinity. The success of this technique requires that the two be separated, the “signal” of the hidden object from the background noise of everything else.

This is achieved by using the fact that the hidden object the device is trying to detect is moving, while other nearby objects are not. Because the moving object generates a signal in the virtual mirror that changes with time, it can be filtered from the constant background signal produced by the stationary objects of the surroundings.

A histogram is created of photon arrival times for every pixel, which must then be processed to remove background objects from the target object. Gariepy et al./Nature Photonics

The final complication is that the timing measurement for scattered light arriving at a single point on the virtual mirror and recorded by a single pixel in the detector unfortunately doesn’t locate the object to a single unique position. A similar time delay could result from objects located at any number of different positions located an appropriate distance from the virtual mirror.

While the timing data from a single pixel only locates the object to a range of positions, the range is different for each pixel. However, it turns out that there is only a single position at which the timing condition is satisfied simultaneously for all pixels, and this allows the object to be unambiguously identified from the background signals.

The prototype camera system allows the object’s position behind the wall to be localised to within a centimetre or two, and by making measurements every few seconds the camera can also detect the speed of a moving object. In contrast to previous methods, which required long data processing times, the new method can track moving objects in real time. At present it’s limited to locating objects up to 60cm away from the virtual mirror on the floor, but this should improve to around ten metres, as well as to more closely detect the shapes of hidden objects as well as their positions.

So while it’s not quite as promising, or as convenient, as the science-fiction powers of X-ray vision, the study’s authors note that the technology has interesting future applications in areas such as surveillance – to detect a moving person behind a wall, for example – or in car safety systems to detect incoming vehicles approaching around corners.