New Smartphone App Interprets Gang Graffiti

An Indianapolis police officer came across a smartphone application in 2010 that caught his interest. The Mobile Emergency Response Guidebook (MERGE), developed at Purdue University, helps emergency responders recognize and interpret hazardous material symbols on damaged tanker trucks or rail cars. First responders can take a picture of the hazmat symbol with their smartphone and send it to a MERGE database, where the image is interpreted. The application then tells responders what the chemical is and what precautions they should take to avoid unintended risks to their own and citizens' lives.

After learning of MERGE's capabilities, the officer thought of a much different—and more complex—use for this application: gang graffiti. He contacted Purdue's Dr. Ed Delp to ask if the application could be modified to interpret the markings and symbols gangs use to mark their territory, challenge rivals and advertise a gang's status or power.

Gang graffiti can provide credible information that law enforcement can use to understand gang activities, membership and growth, and perhaps prevent violent incidents. The ability to track gang movements also helps communities develop strategies to mitigate criminal activities. However, unlike hazmat symbols, graffiti is not standardized. There is no digital guidebook with instructions, and one symbol, drawn by different people on different surfaces, can vary widely.

This is just the kind of technical challenge Delp and his team like to take on. A computer scientist and researcher at the U.S Department of Homeland Security's University Center of Excellence for Visualization and Data Analytics (CVADA), Delp welcomed the opportunity to assist local law enforcement. His team had previously developed several other mobile applications in the areas of language translation and healthcare.

Working closely with the Indianapolis Metropolitan Police Department (IMPD), CVADA researchers created the Gang Graffiti Automatic Recognition and Interpretation (GARI) smartphone application and image database, which can quickly translate the meaning of thousands of graffiti images. "We started working with the IMPD in early 2011," Delp says. "We showed them an initial version of GARI and they provided us with feedback that we used to improve the system. This included the user interface design and how the images were displayed."

Indianapolis and a dozen other local law enforcement agencies in the Indiana Gang Network (INGangNetwork) are now actively using this application to track and counter gang activities. "This has been a very productive relationship," Delp says. "We are also working with the Cook County Sheriff's Department in Illinois. Both the INGangNetwork and Cook County interactions have allowed us to greatly improve GARI."

The GARI database is growing every week as law enforcement agencies from around the region add content. Currently, more than 50 law enforcement officials are using or field-testing GARI in Indiana, Ohio and Cook County, Ill.

How Does GARI Work?

GARI users take a picture of the graffiti and load it, along with its GPS coordinates and date/time information, to the central database. The user then receives an analysis of the gang affiliation, an interpretation of the message in that graffiti, and the geographic location of other similar images. By analyzing and comparing the results, law enforcement can track and identify gang activity over time throughout a region.

So is GARI really "reading" graffiti? Delp stresses that analyzing gang graffiti and gang tattoo images is a very difficult and challenging problem. "GARI uses very sophisticated image processing and analysis methods to detect structures and symbols in the graffiti image," he says. "From that we can ‘understand' what is in the graffiti." The system is also used to analyze gang tattoo images.

GARI's tools that index and analyze the images are key to the accuracy of the results. To ensure accuracy, Delp's team extracts information from the images uploaded by first responders and uses it to test GARI's image-analysis methods. "Our method for finding similar images achieves an accuracy of more than 90%," he says. "And our method for classifying gang graffiti components achieves an accuracy of more than 80%. For gang tattoo images we can match 84% of the images."

The next step for GARI: transitioning this technology to the broader homeland security enterprise. Implementations to date have been directed to a particular agency or organization; now, DHS is currently seeking a formal transition partner to commercialize GARI and make it available to all gang detectives and law enforcement agencies as well as other federal partners, such as ICE, CBP and the FBI.

Delp notes that increasing the number of users will only help GARI become more accurate and useful to LEOs. "The more people use GARI, the more images we will have in our database," he says, "which means: 1) The maps of gang graffiti locations are more populated, and we can then visualize gang activity patterns in clusters of crime activity in space and time, and 2) We have more data to train our algorithms and make our graffiti interpretation system more accurate. Also, with more users we can get more feedback to improve the GARI system."

Additional Law Enforcement Partnerships

The GARI project shows how the DHS University Centers of Excellence (COEs) operate—by allowing research faculty and students to work in close quarters with emergency responders and security professionals in their operating environments. To date, the COEs have fielded or piloted more than 75 technologies for federal, state and local homeland security. The COEs have also benefitted from long-standing partnerships with former senior law enforcement officers such as Mike Tobia, formerly with the Port Authority of New York and New Jersey Police Department; Erroll Southers, a former FBI agent; and Mike Young, formerly with the U.S. Secret Service and the Transportation Security Administration. These experienced law enforcement partners have helped the COEs create several new technologies, including:

1. Tobia worked with Rutgers University's CVADA Center of Excellence and New Jersey's MetLife stadium to develop best a practices guide and training for stadium security. This guide is a security handbook for the NFL, which hosts more than 17 million attendees at football games annually.
2. Southers helped to develop and implement multiple applications of CREATE's ARMOR suite of "intelligent" patrol optimization software. Versions of ARMOR (with different names) are in use at Los Angeles Airport (LAX), the Federal Air Marshals, TSA (air and rail), the U.S Coast Guard, and several others. At LAX, law enforcement saw a sharp increase in interceptions of weapons and drugs at the airport perimeter after implementing ARMOR. The U.S. Coast Guard has said that implementing its PROTECT version of ARMOR roughly doubled the deterrence effectiveness of harbor patrols.
3. Young helped the ALERT COE develop a successful test bed at Cleveland International Airport for the Video Anomaly Sensing and Tracking (VAST) video analytics threat-detection technology, to prevent "counter-flow" intrusions, which can shut down airports and cost millions of dollars. Young also worked with Purdue, the Ohio State Highway Patrol, and the Ohio Fusion Center to implement law enforcement-focused data management tools, including JIGSAW for investigative analysis of multiple documents, and VALET, the visual analytics law enforcement toolkit.

Looking for Law Enforcement Partners

Through an end-to-end program management strategy, the DHS Science and Technology Directorate's Office of University Programs increases researchers' interactions with homeland security practitioners at every phase of the research and development process. COE partnerships with law enforcement agencies work for both sides. According to the COEs, the research developed or tested in an operational setting is more interesting and personally rewarding to work on than something they could have developed in isolation. This research leads to more breakthroughs, scientific publications and student theses. At the same time, research results get into use more quickly and with greater certainty, which gives both students and faculty more satisfaction than just publishing papers in academic journals and presenting findings at conferences.

Adding new state and local partners expands the COEs' capabilities. Increasingly, law enforcement agencies are realizing they can combine the capabilities of several COEs to solve a law enforcement challenge. OUP is there to help make connections and provide administrative and logistical support.

Law enforcement organizations interested in working with the DHS COEs should contact the DHS S&T Office of University Programs at 202-254-8680, or university [email protected] or visit www.hsuniversityprograms.org for a searchable database of projects.

The list of technologies developed through collaborations between COEs and law enforcement organizations is getting long. A February 2014 COE technology showcase targeting law enforcement applications included the 14 COE technologies shown in the table on p. 32. Many of these technologies are in the prototype stage and would benefit from further real-world applications. Additionally, DHS is planning a Research Summit for Fall 2014. Check the OUP website this Spring for details.

Force Multipliers

Technology holds exciting possibilities for law enforcement, and as resources are stretched thin in many departments, the force-multiplying ability of applications like GARI will become an essential tool for LEOs. By working closely with today's top researchers, police agencies can help develop applications and programs that are customized to their needs. Expect to see more of these technologies quickly become important tools in crime prediction and prevention.

RESOURCES

If your agency is interested in using GARI or receiving a demonstration, contact the research team at Purdue at [email protected].