FEATURED IN TECHNOLOGY AND COMMUNICATIONS
Along with the information revolution has come a completely new set of problems and challenges for any organization that creates, modifies and stores data. The most obvious challenge is also the one that most people think least about—that is, of course, until they need to. Then they’re surprised.
Here’s a manifestation of the challenge: A Boston-area company called EMC is so successful and its staff so busy that, inside its campus, you’ll find a Starbucks, a fitness center, a couple of restaurants and a dry cleaner. The place has to be like a mall, because its employees rarely get to leave.
What do they do? EMC makes hard drives.
Law enforcement agencies, like everyone else, find themselves generating more data than ever. Each officer at the 18,500 U.S. agencies generates many megabytes of email every year; each agency generates gigabytes of records and, now, hundreds of gigabytes of video and audio recordings. We’re now measuring storage requirements in terms of terabytes and, in larger agencies, petabytes.
Where do we put it all?
What We Do Now
For thousands of small agencies, storage involves a couple of external hard drives. Prices of disk storage have plummeted in recent years as reliability has skyrocketed. The ability to go down to Fry’s and pick up a couple terabytes of storage for less than $500 means that the storage issue hasn’t been, if you ask most agencies, much of a problem.
But this is mainly an untested hypothesis. That kind of storage strategy is designed not for mission-critical law enforcement applications, but rather for family photos and music. Indexing, maintenance, encryption, search and recovery tools are distinctly lacking—and of course, if someone steals the drive, you’re out of luck.
For these reasons, as well as maintenance, reliability and cost, many agencies have begun to look to the cloud. Simply speaking, the “cloud” is a place to put your data that’s accessible via the Internet. Services such as Dropbox, Box.net, Mozy.com, Amazon AWS and many, many others advertise cheap and reliable storage, and that’s true. Storing in the cloud can result in massive savings, plus the certainty that your data will be accessible when you need it.
We need someplace to put all this stuff, but unlike corporate America, we don’t have the budgets to go out and buy large networks of storage appliances. Your IT organization (if you even have one), at some point in the next 36 months, is going to look to cloud storage.
There are three main questions you must ask to figure out your storage requirements:
1. What have you got now and how much will you have next year?
2. Where do you keep it now, and where will you keep it next year?
3. How will you secure it?
The “How much data will you have next year?” part of the question is your best indication of how soon you have to make decisions. For many agencies, the ramp-up to more data than you can store locally is not yet here, but it will be in the next 18–36 months. So you have some time—and right now, innovations in secure storage mean that time is absolutely on your side (more on that later). If you can wait, you should.
Where Is “The Cloud,” Anyway?
One of the most fundamental questions you should consider when looking at cloud storage is just what that means, in terms of the location of your data. We would never consider mailing those external hard drives to Kuala Lampur and asking someone there to hang on to them for a while, but many cloud storage providers do indeed set up their data centers internationally. Many of the less-expensive cloud storage companies are less expensive precisely because their data centers are in third-world countries.
This brings us to the biggest consideration outside “Can I ever get my data back?” when it comes to cloud storage: security—and, by extension, compliance. Take nothing for granted.
The reason for price disparity among cloud storage providers isn’t about the volume of data you seek to store; it’s entirely about data security. For reasons of compliance and basic common sense, law enforcement data must be encrypted whenever it leaves the relative security of your network, and depending on how much data you have, and how often you need to access it, the need to encrypt in the cloud comes down to key management.
Encrypting data is easy. Governing exactly who can access it, and under what conditions the data can be un-encrypted and accessed later, is the difficult part. This is hard for corporations, hard for the NSA and CIA, and it’s hard for you.
When someone unencrypts something, for example, should they unencrypt the entire stash of data? Just one or two files? For how long? What happens 15 years from now when everyone who currently works in your IT department is gone—how can you recover keys? What retention periods do you currently have for your data, and how can you automate destruction at the end of those periods?
The answers to all these questions and many more like them will govern a lot of your decisions, and all of them require your lawyer.
All of the cheap cloud storage vendors describe security as “easy,” but in fact the questions are very hard. The easiest way to be certain that you can retain control over your data is to compress it into chunks, encrypt it and then send it to cloud storage. That way you know that no one else can see your data, and only you control the keys. Unfortunately, that makes it really, really hard to manage, and you’re getting the worst of all worlds when it comes to accessing your data later.
In addition, many cloud storage vendors do dangerous things with encryption, such as breaking the fundamental rule of encrypted data and storing the keys on the same drive as the data itself. It’s the digital equivalent of leaving the key under the flower pot on the porch. There are other compliance requirements as well, such as those concerning CJIS data and where it can—and can’t—live.
So while cheap storage may sound fantastic to the city manager, it’s best to check with the city attorney and your IT director or a consultant before you pay 99 cents for unlimited storage at Beijing-Storage-a-Go-Go.
Waiting Is the Best Part
The reason we mentioned that time is on your side is this: The storage and data problem is so acute, this is an area of explosive and disruptive innovation. Generational advances are being made monthly, and security is at the fore, just behind capacity. Recently, industry leader Amazon launched something called CloudHSM, which has made it possible to do encryption in the cloud but maintain secure control of the keys in the cloud through something called a hardware security module. This has reduced the cost tremendously, but it’s still in the range of $20,000 a year, putting it outside the reach of small and even some mid-sized agencies.
But vendors like Amazon and others are working on products that will allow splitting up HSMs, which will bring secure storage of encryption keys well within the reach of everyone. The $20,000 that you’ll spend today on a robust, secure cloud storage system will plummet to $5,000 or even $1,000 a year in the next few years.
By 2016, if you’re not storing most everything on the cloud, something will be very amiss with your agency. The time to learn about this is now, mainly so that you won’t miss out on the savings as soon as they become available to your size and storage needs.