A $1.3 million grant from the National Institute of Justice is funding a new two-year project that may revolutionize the way police monitor and analyze crime-scene surveillance video footage with technology developed at the University of Central Florida.

For the first time, UCF computer scientists will develop and test computer-vision technology that will automate the process of monitoring and reviewing thousands of hours of video streams fed from multiple cameras. The technology will be developed to work quickly to handle the large volume of data generated by the cameras, and will significantly reduce the burden placed on human investigators who perform the work and may produce faster leads for some criminal investigations.

Computer vision is a field within computer science that uses computers to quickly recognize and analyze patterns, gestures, facial features and objects in images such as photographs and videos. Cameras are already commonplace in public areas from airports to streets and the video feeds are constant.

The research team, led by Mubarak Shah, UCF Trustee Chair professor of computer science and director of the Center for Research in Computer Vision, also includes Raymond Surette, professor of criminal justice at UCF, and researchers from Columbia University.

The team will develop the technology using archived and live video clips supplied by the Orlando Police Department to build algorithms so that computers will have the ability to recognize and flag out-of-the-norm actions, gestures, events and behaviors that could indicate criminal activity.

For example, in video footage of the 2013 Boston Marathon bombing, the suspect was the only person in the large crowd who did not look back when an explosion ignited behind him.

The project is believed to the first attempt to develop comprehensive technology that goes beyond facial recognition in collaboration with a law enforcement agency. Prior law enforcement computer vision uses have concentrated in facial recognition programs and license plate readers, Surette said. Computer vision has not been extensively tapped and public-safety camera systems remain dependent upon a human to watch or review them.

 “Today there are too many surveillance cameras and too few human monitors,” Shah said. “Watching multiple live-video camera feeds or retroactively reviewing long hours of video streams is a mind-numbing, error-prone task.”

The technology will be applied in pre-event, live-event and post-event video timeframes, and will also employ cutting-edge search functions to speed up the amount of time it takes to review crime-scene videos for particular images.

“Utilizing the most advanced technology and tools to fight crime and keep our community safe is a top priority for the City of Orlando and this partnership will enhance these ongoing efforts,” said Orlando Police Chief John Mina. “The more eyes we have—whether they belong to officers or are created by technology—will further our mission to keep residents and visitors of Orlando safe and protected.”

The two-year project will be implemented in two phases.

During the first phase, UCF and Columbia University researchers will develop computer vision capabilities for the surveillance work and develop user-friendly interfaces based on the needs of police investigators and data supplied by OPD. 

In the second phase, the technology will be field-tested in OPD’s camera-monitoring room with a demonstration visual-analytics workstation. The system will be linked to a new 11-camera system already active in a local neighborhood. Six cameras will be computer monitored, and five cameras will be human monitored. Surette and his team will evaluate the impact of the technology on OPD’s operations and personnel.

“With the rapid advances in computer vision in recent years, this is the perfect time to tackle this challenge,” said Shih-Fu Chang, Richard Dicker Professor and senior executive vice dean at Columbia University. “We are particularly thrilled to have the chance of combining our fast video-search solutions with the world-class activity-recognition technologies developed at UCF, in close collaboration with criminal justice experts at UCF and the Orlando PD.” 

The actual video-surveillance technology for OPD is in development.