Seeing the human pulse

Researchers at MIT's Computer Science and Artificial Intelligence Laboratory have developed a new algorithm that can accurately measure the heart rates of people depicted in ordinary digital video by analysing imperceptibly small head movements that accompany the rush of blood caused by the heart's contractions.

In tests, the algorithm gave pulse measurements that were consistently within a few beats per minute of those produced by electrocardiograms (EKGs). It was also able to provide useful estimates of the time intervals between beats, a measurement that can be used to identify patients at risk for cardiac events.

Guha Balakrishnan, a graduate student in MIT's Department of Electrical Engineering and Computer Science, and his two advisors - John Guttag, the Dugald C. Jackson Professor of Electrical Engineering and Computer Science and director of MIT's Data-Driven Medicine Group, and professor of computer science and engineering Fredo Durand - describe the new algorithm in a paper appearing this summer at the Institute of Electrical and Electronics Engineers' Computer Vision and Pattern Recognition conference.

A video-based pulse-measurement system could be useful for monitoring newborns or the elderly, whose sensitive skin could be damaged by frequent attachment and removal of EKG leads. But, Guttag says, ''From a medical perspective, I think that the long-term utility is going to be in applications beyond just pulse measurement.''

For instance, Guttag says, an arterial obstruction could cause the blood to flow unevenly to the head. ''Can you use the same type of techniques to look for bilateral asymmetries?'' he asks. ''What would it mean if you had more motion on one side than the other?''

Similarly, Guttag says, the technique could, in principle, measure cardiac output, or the volume of blood pumped by the heart, which is used in the diagnosis of several types of heart disease. Indeed, he says, before the advent of the echocardiogram, cardiac output was estimated by measuring exactly the types of mechanical forces that the new algorithm registers: In a technique called ballistocardiography, a heart patient would lie on a table with a low-friction suspension system; with every heartbeat, the table would move slightly, with a displacement corresponding to cardiac output.

''I think this should be viewed as proof of concept,'' Guttag says. ''It opens up a lot of potential flexibility.''

Heart signals
The algorithm works by combining several techniques common in the field of computer vision. First, it uses standard face recognition to distinguish the subject's head from the rest of the image. Then it randomly selects 500 to 1,000 distinct points, clustered around the subjects' mouths and noses, whose movement it tracks from frame to frame. ''I avoided the eyes, because there's blinking, which you don't want,'' Balakrishnan says.

Next, it filters out any frame-to-frame movements whose temporal frequency falls outside the range of a normal heartbeat - roughly 0.5 to 5 hertz, or 30 to 300 cycles per minute. That eliminates movements that repeat at a lower frequency, such as those caused by regular breathing and gradual changes in posture.

Finally, using a technique called principal component analysis, the algorithm decomposes the resulting signal into several constituent signals, which represent aspects of the remaining movements that don't appear to be correlated with each other. Of those signals, it selects the one that appears to be the most regular and that falls within the typical frequency band of the human pulse.

Balakrishnan also created a variation of the algorithm that doesn't use face recognition. Although its output was slightly less accurate, it was able to produce a reasonable approximation of pulse rate from video of the back of a subject's head.

The accuracy of the algorithm could also be improved by combining it with other video-analysis techniques, such as an algorithm that Guttag, Durand and several colleagues described last year, which amplifies otherwise imperceptible color changes between frames of video. ''The signals can complement each other,'' Balakrishnan says.