If an empty bag of chips was lying in a room where someone was talking, do you think the bag is hearing too? If that questions sounds insane, it is not. In a fascinating talk by computer scientist Michael Rubinstein who along with a team developed the motion microscope at MIT labs says it is, “…a video camera and image processing to reveal to us the tiniest motions and colour changes in objects and people, changes that are impossible for us to see with our naked eyes.”
So as a first he shows us how with each throb of our pulse, the colour drains and fills our face. It is amazing to see that when the colour rushes to any part of the body it turns red and then pales the next second when it is pumped out. Rubinstein says, “Our skin, for example, changes its colour very slightly when the blood flows under it. That change is incredibly subtle, which is why, when you look at other people, when you look at the person sitting next to you, you don’t see their skin or their face changing colour.” As you see the colour change with each throb, you can also measure the small movement. You can see the veins throb, something that you could only feel till now but could not see with the naked eye. Any video you have taken can be upgraded to see the small micro movements. How is this done?
“We basically analyze the changes in the light that are recorded at every pixel in the video over time, and then we crank up those changes. We make them bigger so that we can see them.
The tricky part is that those signals, those changes that we’re after, are extremely subtle, so we have to be very careful when you try to separate them from noise that always exists in videos. So we use some clever image processing techniques to get a very accurate measurement of the colour at each pixel in the video, and then the way the colour changes over time, and then we amplify those changes. We make them bigger to create those types of enhanced videos, or magnified videos that actually show us those changes,” says Rubinstein, showing us clips of how through this technique, the pulse rate can be measured.
What is amazing is that when we think we are sitting still, we are actually moving in many kinds of ways. “Even when a person is sitting still, there’s a lot of information we can extract about their breathing patterns, small facial expressions. May be we could use those motions to tell us something about our thoughts or our emotions. We can also magnify small mechanical movements, like vibrations in engines that can help engineers detect and diagnose machinery problems, or see how our buildings and structures sway in the wind and react to forces.
Those are all things that our society knows how to measure in various ways, but measuring those motions is one thing, and actually seeing those motions as they happen is a whole different thing.” Rubinstein demonstrates how it is possible to measure at what frequency singing can shatter glass!
“But this made us think. It gave us this crazy idea. Can we actually invert this process and recover sound from video by analyzing the tiny vibrations that sound waves create in objects, and essentially convert those back into the sounds that produced them.
In this way, we can turn everyday objects into microphones….” Rubinstein shows how that was done and even an object a little distance away goes through some vibrations with sound and when that is reconverted into sound, you can hear what went on in a room from the tell-tale empty bag of chips!
sudhamahi@gmail.com
Web link: >See invisible motion, hear silent sounds
