Big Data Security Analytics

Thanks to our guest blogger, Jayanthi Manikandan for her blog this week on Big Data Security Analytics. Jayanthi has an undergrad degree in Computer Science and a Master’s degree in Information Systems with a specialization in Information Assurance from Walsh College, Detroit, MI. She enjoys writing about Java, Information Security and Big Data.

Introduction

With the innate sense to share across multiple platforms being the norm in today’s Facebook generation, huge volumes of data are being generated. We are part of the data and we are the cause of this data. Nowhere in time has man communicated so much and this much amount of communication been recorded.

The output from all social media platforms such as Facebook, Twitter,  and other data like emails, pictures, videos, data from DBMS, etc are all examples of structured, unstructured, and semi-structured data. What we do with this data is the crucial point of Big Data discussions. How will we leverage this data in a reduced span of time to produce desired results is important for businesses, retail, health care, law enforcement, credit card companies, academic institutions, and so on.
Just like “Java” revolutionized the 90’s with “write once, run anywhere” reality, “Big Data” and its technologies are revolutionizing the current age with the ability to process structured, unstructured data and semi-structured data without the need for a common schema.
This blog post will examine two aspects of Big Data analytics in relation to Information Security:
a)     The huge volume of data related to Information Security
b)    Security aspects when dealing with Big Data

The huge volume of Security Data:

Just like with other disciplines, Big Data technologies are used to tap the huge volume of security-relevant data relating to log files, network traffic, network events, software application events, firewalls, etc. Earlier, all this security data could only be stored for a limited period of time (say 60 days) and not much analysis could be done with them. It wasn’t monetarily feasible to store security data for longer periods of time and it wasn’t easy to perform traditional data mining. Complex queries also took a considerable amount of time to be generated. And the most difficult part of traditional data warehouses was the need for homogenous data formats.  Now thanks to the increased processing power of CPUs combined with a lower cost of RAMs, queries can be executed faster without the need for a common schema. This is done by using Big Data technologies.
Putting this in relation to a security perspective, security alarms and network statistics are aggregated and presented as information through a dashboard for security analysts. (Big Data Analytics for Security, 2014)
This historical security data can be mined to detect:

1. Security breaches that might happen in due course
2. Bank fraud detection
3. Intrusion Detection
4. Credit card fraud detection

Illustrating this fact, Zions Bancorporation has stated that by adopting Big data technologies like Hadoop, huge volumes of security data can now be parsed more quickly than traditional tools. Queries that normally took between 20 minutes and an hour now take about a minute, thanks to working with Hive. (Big Data Analytics for Security Intelligence, 2013)
Illustrating this fact, Zions Bancorporation has stated that by adopting Big data technologies like Hadoop, huge volumes of security data can now be parsed more quickly than traditional tools. Queries that normally took between 20 minutes and an hour now take about a minute, thanks to working with Hive. (Big Data Analytics for Security Intelligence, 2013)

Security aspects when dealing with Big Data

After having discussed the different security components of Big Data and how Big Data will help predict the future of security environments, we next turn to the obvious aspect of Big Data and Security  – what about the security of Big Data itself? For example, is the data mined from different social media interactions really private?  As you can guess, it is very easy for privacy violations to occur in the Big Data age. When large data sets are being mined, they will be subject to tampering and hence certain laws are needed to avoid “data reuse”. In the US certain laws (like HIPAA) are present to prevent misuse of the huge amount of data. (Big Data Analytics for Security Intelligence, 2013)

Organizations dealing with Big Data security

More and more organizations these days are working to make sure that Big Data and security go hand in hand.

1. As an example, IBM’s Big Data products allow us to collect, analyze and report on security related data. These Big Data products are also enriched with forensic capabilities for in-depth evidence of malicious activities. (IBM Security Intelligence with Big Data)
2. In another development, Cloudera bought “Big Data encryption outfit” Gazzang for $900 million in March. (Cloudera buys big data encryption outfit Gazzang, 2014) Gazzang offers encryption when data is at rest. This shows the importance of security when deploying Big Data on production environments.

Having seen the different aspects of Big Data security analytics, we next see the prospects of Big Data Security in the industry. Some of the job roles relating to Big Data Security are:

1. Big Data Security Platform Security Architect
2. Senior Engineer, Big Data Security

Most of these job roles will broadly demand skills pertaining to Big Data technologies like Hadoop, data models and encryption models.  Considering all these factors, the future of Big Data Security analytics is bright and interesting and poised to grow stupendously.
Interested in a career in Big Data? Check out Jigsaw Academy’s Big Data courses and see how you can get trained to become a Big Data specialist.

Bibliography

Big Data Analytics for Security. (2014, Feb 11). Retrieved Aug 26, 2014, from InfoQ: https://www.infoq.com/articles/bigdata-analytics-for-security
Big Data Analytics for Security Intelligence. (2013, September). Retrieved August 26, 2014, from Cloud Security Alliance: https://www.research.att.com/techdocs/TD_101024.pdf
Cloudera buys big data encryption outfit Gazzang. (2014, June 5). Retrieved Sept 1, 2014, from zdnet.com: https://www.zdnet.com/cloudera-buys-big-data-encryption-outfit-gazzang-7000030250/
IBM Security Intelligence with Big Data. (n.d.). Retrieved Sept 2, 2014, from IBM: https://www-03.ibm.com/security/solution/intelligence-big-data/
 

Related Articles:
Are we Mis-using Big Data?
Security is a Big, Big Data Issue
How Important is The Human Element in Big Data?