Security is a Big, Big Data Issue

25 Mar 2014

Today everyone is aware of Big Data. World over, organisations have begun to use Big Data to solve varied business problems. We also hear a lot of talk around Big Data security. Well there are two sides to this coin. One- With the Big Data at hand, organisations and even governments can use analytics tools to analyse, predict and reduce security incidents. And the other is the actual Big Data security issue, which refers to securing an organisation’s data or keeping customers information safe. Most organisations are however only just beginning to think about this. Though important, as has happened with other new technologies and trends, security always seems to be an afterthought.

Deploying Big Data for Data Security

The deployment of Big Data for security is attractive to many organisations. The overheads of managing the output of traditional Security Incident and Event Management (SIEM) and logging systems are proving too much for most IT departments and Big Data is seen as a potential saviour. There are commercial replacements available for existing log management systems, or the technology can be deployed to provide a single data store for security event management and enrichment.

One of them, Zettaset Orchestrator, provides an enterprise-class security solution for big data that is embedded in the data cluster itself, moving security as close to the data as possible, and providing protection that perimeter security devices such as firewalls cannot deliver.

Taking the idea a step further, the challenge of detecting and preventing advanced persistent threats may be answered by using Big Data style analysis. These techniques could play a key role in helping detect threats at an early stage, using more sophisticated pattern analysis, and combining and analysing multiple data sources.

Until an incident occur, logs are never being noticed. And that too, today logs are often ignored as there is more focus on handling the data. Big Data provides the opportunity to consolidate and analyse logs automatically from multiple sources rather than in isolation. This could provide insight that individual logs cannot, and potentially enhance Intrusion Detection Systems (IDS) and Intrusion Prevention Systems (IPS) through continual adjustment and effectively learning “good” and “bad” behaviours. This presents the possibility of significantly more advanced detection of fraud and criminal activities.

These are the solutions that have been specifically designed to meet the security requirements of the distributed architectures which predominate in big data environments. We know that organisational databases often reduce the effectiveness of security systems, so businesses must be aware that the potential effectiveness of Big Data style analysis can also be diluted unless these issues are addressed.

Data Security for Organizations

A growing number of organizations are using technology to store and analyse online content to gain better insights about their customers and their business. As a result, information classification becomes even more critical; and information ownership must be addressed to facilitate any reasonable classification. And thus, managing Big Data with a security perspective is imperitive to overcome the privacy of the customers.

Most organisations already struggle with implementing these concepts, making this a significant challenge. They will need to identify owners for the outputs of Big Data processes, as well as the raw data. Thus data ownership will be distinct from information ownership which perhaps IT owning the raw data and business units taking responsibility for the outputs.

Very few organisations are likely to build a Big Data environment in-house, and so get help to build their own cloud. As many organizations are adopting, storing data in the cloud they must realise that this does not remove their responsibility for protecting it from both a regulatory and a commercial perspective.

Techniques such as Attribute Based Encryption may be necessary to protect sensitive data and apply access controls. Many of these concepts are yet to be implemented in present organizations.

In short Big Data Security must become a Big, Big Data issue that organizations recognize and act on.

Interested in a career in Big Data or Data Science? Check out Jigsaw Academy’s courses and find out how you can get started:

Data Scientist Course

Big Data Specialist Course

Are we Mis-using Big Data?
What are the Essential Big Data Skills a Data Scientist Needs?