Engineers at the University of Central Florida are devising a way to use video to detect abnormal behavior in large crowds in an effort to help keep people safe from terrorist attacks.

Professor Mubarak Shah heads the team that is using video streams, algorithms and a unique analysis to figure out how crowds behave and what might be suspicious behavior.

Video cameras monitoring people in public areas are now commonplace, and in large settings such as the Super Bowl, political rallies or royal weddings, the stakes are even higher because there are more people, which means more potential casualties.

Several research efforts are under way to develop systems that cue security personnel to individuals or events of interest in crowded scenes. Most of that now is handled by people who can easily miss something because it is impossible for them to monitor every change in the midst of a large crowd.

Shah is trying to automate most of the work with computer algorithms, and he’s modeling his work on how liquids behave in motion.

“While human psychology and individual quirks complicate the analysis, in essence people in high-density crowds appear to move with the flow of the crowd, like particles in a liquid flow,” Shah said.

So his team is building a program that analyzes behavior in a crowd much like analyzing particles and how they act in a fluid state. But there is one variation.

“We are saying one difference between crowds and fluid is that people have some destination they are going towards. That’s why crowds can be called ‘thinking fluid,’” Shah said.

The team’s work is promising and was the cover story last month in Communications of ACM, a computer science periodical.

“We’re still years away from perfecting this,” Shah said. “But by using the basics of hydrodynamics we are developing a really good tool here.”

Most work in automatic-video surveillance is done with crowds of no more than 20 people, Shah said, but if perfected could be helpful in crowds of hundreds or thousands. Shah’s research team used public video from large events such as the New York Marathon, a political rally in Los Angeles and pilgrims circling the Kaaba shrine in Mecca.

Computational and applied mathematics are critical for visual analysis of crowds, Shah added.

Pixel information from the screen is translated into particle trajectories used to understand crowd flow and then examined in various ways to recognize crowd behaviors, track individuals and detect unusual behavior.

Shah is an expert in the field of visual analysis. When he joined UCF in 1986 he founded the Computer Vision Lab, which is a leader in developing technology for use in crowd surveillance, visual tracking, Unmanned Aerial Vehicle video analysis and analysis of crowded scenes.

Shah has a Ph.D. from Wayne State University and is a fellow of IEEE, AAAS, IAPR and SPIE, the largest professional organizations in his field. Several professional organizations have designated Shah as a distinguished speaker. He also has published hundreds of articles.  According to Academic Search, Shah is the sixth most frequently cited author in the world in Computer Vision in the past five years. He’s been honored with several awards, including Pegasus Professor, the highest award given by UCF to a faculty member that has made a significant impact on the university. He also is a member of UCF’s Millionaires club. He teaches several courses, is director of the Computer Vision Lab and is the Agere Chair Professor of Computer Science.

Shah’s team includes Brian Moore, an assistant professor of mathematics at UCF; Saad Ali, a UCF Computer Vision Lab alumnus and a computer scientist at SRI International in Princeton, N.J., and Ramin Mehran, who completed his Ph.D.  last month at UCF and is joining Microsoft in January.