FEATURED IN TECHNOLOGY AND COMMUNICATIONS
Along with the information revolution has come a completely new set of problems and challenges for any organization that creates, modifies and stores data. The most obvious challenge is also the one that most people think least about—that is, of course, until they need to. Then they’re surprised.
Here’s a manifestation of the challenge: A Boston-area company called EMC is so successful and its staff so busy that, inside its campus, you’ll find a Starbucks, a fitness center, a couple of restaurants and a dry cleaner. The place has to be like a mall, because its employees rarely get to leave.
What do they do? EMC makes hard drives.
Law enforcement agencies, like everyone else, find themselves generating more data than ever. Each officer at the 18,500 U.S. agencies generates many megabytes of email every year; each agency generates gigabytes of records and, now, hundreds of gigabytes of video and audio recordings. We’re now measuring storage requirements in terms of terabytes and, in larger agencies, petabytes.
Where do we put it all?
What We Do Now
For thousands of small agencies, storage involves a couple of external hard drives. Prices of disk storage have plummeted in recent years as reliability has skyrocketed. The ability to go down to Fry’s and pick up a couple terabytes of storage for less than $500 means that the storage issue hasn’t been, if you ask most agencies, much of a problem.
But this is mainly an untested hypothesis. That kind of storage strategy is designed not for mission-critical law enforcement applications, but rather for family photos and music. Indexing, maintenance, encryption, search and recovery tools are distinctly lacking—and of course, if someone steals the drive, you’re out of luck.
For these reasons, as well as maintenance, reliability and cost, many agencies have begun to look to the cloud. Simply speaking, the “cloud” is a place to put your data that’s accessible via the Internet. Services such as Dropbox, Box.net, Mozy.com, Amazon AWS and many, many others advertise cheap and reliable storage, and that’s true. Storing in the cloud can result in massive savings, plus the certainty that your data will be accessible when you need it.
We need someplace to put all this stuff, but unlike corporate America, we don’t have the budgets to go out and buy large networks of storage appliances. Your IT organization (if you even have one), at some point in the next 36 months, is going to look to cloud storage.
There are three main questions you must ask to figure out your storage requirements:
1. What have you got now and how much will you have next year?
2. Where do you keep it now, and where will you keep it next year?
3. How will you secure it?
The “How much data will you have next year?” part of the question is your best indication of how soon you have to make decisions. For many agencies, the ramp-up to more data than you can store locally is not yet here, but it will be in the next 18–36 months. So you have some time—and right now, innovations in secure storage mean that time is absolutely on your side (more on that later). If you can wait, you should.
Where Is “The Cloud,” Anyway?
One of the most fundamental questions you should consider when looking at cloud storage is just what that means, in terms of the location of your data. We would never consider mailing those external hard drives to Kuala Lampur and asking someone there to hang on to them for a while, but many cloud storage providers do indeed set up their data centers internationally. Many of the less-expensive cloud storage companies are less expensive precisely because their data centers are in third-world countries.
This brings us to the biggest consideration outside “Can I ever get my data back?” when it comes to cloud storage: security—and, by extension, compliance. Take nothing for granted.
The reason for price disparity among cloud storage providers isn’t about the volume of data you seek to store; it’s entirely about data security. For reasons of compliance and basic common sense, law enforcement data must be encrypted whenever it leaves the relative security of your network, and depending on how much data you have, and how often you need to access it, the need to encrypt in the cloud comes down to key management.
Encrypting data is easy. Governing exactly who can access it, and under what conditions the data can be un-encrypted and accessed later, is the difficult part. This is hard for corporations, hard for the NSA and CIA, and it’s hard for you.
When someone unencrypts something, for example, should they unencrypt the entire stash of data? Just one or two files? For how long? What happens 15 years from now when everyone who currently works in your IT department is gone—how can you recover keys? What retention periods do you currently have for your data, and how can you automate destruction at the end of those periods?
The answers to all these questions and many more like them will govern a lot of your decisions, and all of them require your lawyer.