The deployment of Big Data and analytics techniques to reinforce security


Share on LinkedIn

Big data is immensely popular these days. Today, many business organizations are using data analytics for streamline their processes and to make them more accessible to users.

Big data’s a hugely popular talking point these days. A lot of businesses already are using big data for research and marketing, but may not have the right fundamentals, especially form a security viewpoint. The same as with new technologies, security appears to be an afterthought at best. Also, breaches in data would be big too, with the potential of even more serious damage to reputation and legal repercussions.

Not a lot of organizations would likely create big data in-house, thus big data and cloud would be linked inextricably. As most businesses are aware, data storage in the cloud do not remove their responsibility to protect it, from a commercial and regulatory perspective. Techniques, like an attribute based encryption could be necessary to protect sensitive data as well as apply access controls. Most of these concepts are foreign to businesses these days.

Big data deployment for detection of fraud and security data analytics is attractive to a lot of enterprises. The overhead costs of managing output of logging systems and traditional SIEM prove to be too much for most information technology departments and big data is believed to be a potential savior. There are available commercial replacements for existing log management systems or tech could be deployed to give a single data store for security enrichment and management.
Going a step further, the concern in the detection and prevention of advanced persistent threats could be solved with Big Data style analysis. The techniques may play a major role to determine threats early on, with the use of an analysis pattern that’s more sophisticated, mixing and analyzing data resources. Furthermore, there’s also the possibility of anomaly identification using feature extraction.

Big data is a new technology and thus, as a new technology also could introduce new vulnerabilities. Security data analytics aims to minimize the risks associated with the technology. Implementations of data analytics typically include an open source code, with a potential for back doors and default credentials that are not recognized.

Big data in reality is more on processing techniques as well as outputs than the data size itself, thus specific skills are needed to use the technology effectively. There’s a general shortage of specialist skills for analysis, particularly in terms of using some less mature technologies. The growing Hadoop use and other related technologies is driving demand for specific skills.

Cyber security requires risk management and actionable intelligence that’s common from big data analysis. Although it’s great to have tools that could analyze data, the key is automating the tasks so data would be more quickly available and analysis is forwarded on time to the right people. This enables analysts to classify as well as categorize cyber threats minus lengthy delays that can make data irrelevant to attack.

Securing data comes with unique challenges besides being a high-value target. It is not that security is different from traditional security. Challenges in analytics security arise due to the incremental differences and not fundamental ones. Differences of big data and traditional data environments include the following:
– Data gathered, aggregated and analyzed for data analysis
– Technologies applies for analyzing both structured and unstructured data
– Infrastructure for storing and housing big data


There are various ways of addressing the security challenges that are presented by big data and analytics.
1. Application software security. Use several versions open-source software versions. Using open source techs such as Apache Accumulo or the 20.20x Hadoop version or above could help meet the challenge. Furthermore, proprietary technologies such as DataStax Enterprise and Cloudera Sentry provide improved security at the app layer.
2. Maintenance, monitor and analysis of audit logs. Implement audit logging techs to understand and monitor data clusters. Technologies such as Apache Oozie could help implement the feature. It is necessary to make certain that auditing, maintaining and analyzing logs are consistently performed across the organization.
3. Account control and monitoring. Require robust passwords, deactivate accounts that are inactive and impose a maximum permitted amount of failed log-in attempts. Monitoring account access could help minimize the possibility of a successful compromise from within.
4. Secure for hardware and software configurations. Create servers that are based on secure images for all systems in the big data architecture of the organization. Make sure patching is updated on the machines and administrative privileges are limited to a limited number of users.

Cyber criminals will never stop being offensive, and with such a huge target to protect, it’s prudent for any organization to use big data technologies to be as proactive as much as possible in data security.

Data and analytics would enable companies to identify advanced attack vectors and anomalies. It is important to be able to detect an unusual activity and respond to it when needed.

Dhrumit Shukla
Dhrumit Shukla has been working as Business Development Manager in a software development company named TatvaSoft since 5 years. He is profoundly skilled and well experienced in providing software development services on various technologies ranging from Microsoft .NET to JAVA, SalesForce, BizTalk, SharePoint, PHP, Open Source, iOS, Android, Pentaho and the list goes on.


Please use comments to add value to the discussion. Maximum one link to an educational blog post or article. We will NOT PUBLISH brief comments like "good post," comments that mainly promote links, or comments with links to companies, products, or services.

Please enter your comment!
Please enter your name here