Affordable Public Safety: Leveraging Open Source Software to Support Law Enforcement Surveillance Tools

Surveillance systems are specialized data analytics tools leveraging many of the processes and components found in commercial enterprises, defense organizations and the intelligence community.  The recent tragedy in Boston ensures an increased demand for such systems.  Fortunately, many of the systems’ core components can be satisfied by enterprise grade open source software that comes as part of a unified platform.  By eliminating both licensing costs and improving platform productivity, total cost of ownership (TCO) is significantly reduced, allowing access to modern security tools and techniques can be extended to smaller agencies and jurisdictions. 

Commenting on the April 15th Boston Marathon bombing during an interview with MSNBC’s Andrea Mitchell, US Representative Peter King (R-NY) expressed a belief that Americans are going to have to get used to many more surveillance cameras in public spaces:

So, I do think we need more cameras. We have to stay ahead of the terrorists and I do know in New York, the Lower Manhattan Security Initiative, which is based on cameras, the outstanding work that results from that. So yes, I do favor more cameras. They're a great law enforcement method and device. And again, it keeps us ahead of the terrorists, who are constantly trying to kill us.

Questions of domestic policy and civil liberties aside, Representative King’s inclination toward additional surveillance mechanisms has a number of interesting systemic ripple effects.  Understanding these effects requires closer examination of a surveillance system’s constituent components and the nature of the value it provides.

A generic surveillance system consists of four core capabilities (Such systems can, of course, be further decomposed.)  These include:

Collection:  The acquisition of data about the locale or subject of interest.  Representative King’s cameras are one type of collection mechanism, gathering geospatially and temporally referenced imagery and video data.  Other collection mechanisms might acquire radio-frequency data, including such things as cell phone conversations, text messages or emails sent over Wi-Fi and mobile data networks and laser spectrometers, collecting information about what people have done or eaten based on residues on skin and clothing.

Analysis:  Unanalyzed data, like an unmined vein of gold, is little more than potential value.  Analysis tools, like the crushing and precipitation mechanisms in a gold mining operation, both identify relevant events within the overall set of collected data and make sense of the identified events within an operational context.  Analysis, by transforming data into information and information into knowledge, provides the critical element of “what does this mean to me at this time.”

Decision Support:  Most law enforcement and emergency response organizations have doctrines and policies outlining the expected nature and scope of a response to a given type of incident.  Once  analysis has identified the type and magnitude of an event, it’s simply a matter of applying logic consistent with the organization’s business rules to arrive at  a doctrinally valid recommended course of action.

Dissemination:   The best analysis and business rules engines are useless if results and recommendations aren’t placed in the hands of people and organizations with the means to influence events in a timely manner.  Dissemination mechanisms not only ensure timely delivery of critical information, but also preserve the core attributes of information security.  It must ensure that the information being distributed is available only to authorized entities (confidentiality), that it is not altered or corrupted in any way while in transit (integrity), that it can be retrieved when necessary (availability), that both sides of the dissemination transaction have confidence in the identity of the other (authenticity) and that an undeniable audit trail of the transaction exists (non-repudiation).

Using the generic system as a vantage point, it’s easy to see that Representative King’s desire for more cameras exposes only the tip of the security and surveillance iceberg.  An effective surveillance system must solve all four problems concurrently if it is to successfully fulfill its operational requirements.  Having more cameras addresses only the collection issue.  Additionally, fielding a greatly augmented collection capability prior to developing robust analysis, decision support and dissemination capabilities can overwhelm analyst resources and frustrate timely data analysis and dissemination. 

As an illustration, suppose that Representative King gets his way and the number of cameras for a given area is greatly increased, without concomitant improvements to the back end analysis, decision support and dissemination capabilities.  For a system deploying 100 cameras, 2,400 hours of video are collected every day (and 16,800 every week).

Boston’s police department, among the 20 largest in the United States, has about 2,800 uniformed and civilian personnel.  Theoretically, all the video could be reviewed in the course of a single eight hour shift – assuming that the city was willing to withdraw every single police employee from the street and dedicate them to the task, that every employee was a qualified imagery analyst and that only a single analysis pass was necessary.  Realistically, the requirement to manually analyze that much data could overwhelm even the New York Police Department’s much larger forensics investigation division.  (This problem is not unique to law enforcement.  In 2011, US Air Force surveillance systems collected approximately 720 years of video data over Afghanistan.)

However, even a significantly augmented analyst force doesn’t address the fact that current surveillance architectures are inherently reactive. That is, they provide excellent investigative and forensic tools to establish the nucleus of operative facts after an event has taken place but are not preventative or prophylactic in nature.  Law enforcement’s goal with respect to mass casualty events is to ensure that they remain inchoate; that terrorist plans are never realized.  Based on this, we can safely speculate that what Representative King is really seeking is a significantly improved surveillance architecture, of which the collection hardware is only part.  Such an architecture might include image pattern recognition software capable of identifying backpacks or duffel bags or laser spectrometer capable of detective explosives residue from hundreds of feet away.  Categorized by capability, other architectural components include:

Analysis

• A pattern recognition tool;
• A real-time data analytics engine; and
• A storage mechanism capable of handling large data sets that come in at a very high velocity.

Decision Support

• A business rules processor capable of storing rule sets representing doctrine and executing rules in the context of analyzed data; and
• A business process engine capable of implementing processes indicated by the business rules engine.

Dissemination

• An integration and transport mechanism capable of delivering decision support data to a diverse set of applications and endpoints; and
• A security mechanism ensuring that information can only be transmitted, stored or acted upon by authenticated and authorized system entities.

As can be seen, effective surveillance systems have a number of infrastructural middleware sub-components operating in parallel.  The attendant software development effort isn’t trivial; the sheer volume and variety of components is a significant cost driver.  Each sub-component can require specific expertise, which in turn can require employees with special (and expensive) skills and knowledge.  Additionally, each sub-component  may come with a discrete licensing fee.  Requirements for specialized knowledge and licensing fees combine to create a TCO that may be beyond the budgetary means of many agencies.

Part of the answer lies in building the surveillance system around a highly productive, highly integrated platform that provides dedicated products leveraging a consistent, composable core.  For example, if the integration/transport, security and business rules mechanisms share a common core providing key enterprise service oriented architecture (SOA) functionality (e.g., mechanisms to provide and consume services, mediation, service orchestration, service governance, business process management, service monitoring and support for open standards such as WS-* and REST), less expertise on individual products is required, and fewer expensive experts are needed on the payroll. 

There are additional platform characteristics that can mitigate TCO:

• By using an open source platform, licensing fees are eliminated;
• By using a platform based on open standards, expensive vendor lock-in is avoided and innovation, adaptability and flexibility are promoted; and
• Configurable components offer greater productivity than those requiring custom integration code.

Theory and Practice

Fortunately for law enforcement and the security industry, open source enterprise middleware based on a common, composable and configurable platform exists in practice as well as in theory, and it’s possible to map the requirements outlined above to existing, available and – importantly – supported software products:

Requirement	Example Software Product	Notes
Analysis
Pattern recognition	OpenPR; Genia Photonics Picosecond Programmable Laser*	Open Pattern Recognition project shares algorithms of image processing, computer vision, natural language processing, pattern recognition, machine learning and  related fields.   *Not open source
Real time analytics	WSO2 Complex Event Processor
High volume data storage	Accumulo; Cassandra	Accumulo is a NoSQL database developed by the NSA and open sourced through the Apache Software Foundation.  It offers cell level security.
Decision Support
Business rule management	WSO2 Business Rules Server
Business process management	WSO2 Business Process Server
Dissemination
Integration and Transport	WSO2 Enterprise Service Bus
Security & Identity	WSO2 Identity Server

Conclusion

The terrible events in Boston, and the subsequent identification of the suspects testify to the requirement for and effectiveness of surveillance systems.  Two issues become clear:  The need to improve the processing of surveillance data in a manner that helps prevent terrorist incidents from taking place, and the need to provide systems that are affordable to agencies of all sizes and budgets.  Fortunately, technical advances, coupled with the proliferation of high quality open source software offers the promise of achieving both in the near future.