Data Reduction In Data Mining: A Simple And Concise Guide

Introduction

In today’s data-heavy systems, where everything is captured for future use, you get a mammoth data set on every load. This data set could be big in terms of observations or quite minuscule in terms of the number of features or columns, or both. Data mining becomes tedious in such cases, with only a few important features contributing to the value that you can take out of the data. Complex queries might take a long time to go through such huge data sets too. In such cases, a quick alternative is data reduction. Data reduction consciously allows us to categorize or extract the necessary information from a huge array of data to enable us to make conscious decisions.

In this article, we’ll explore

1. What is Data Reduction?
2. Data Reduction Techniques

1. What Is Data Reduction?

According to Wikipedia, a formal definition of Data Reduction goes like this, “Data reduction is the transformation of numerical or alphabetical digital information derived empirically or experimentally into a corrected, ordered, and simplified form.” In simple terms, it simply means large amounts of data are cleaned, organized and categorized based on prerequisite criteria to help in driving business decisions. 

2. Data Reduction Techniques

There are two primary methods of Data Reduction, Dimensionality Reduction and Numerosity Reduction.

A) Dimensionality Reduction

Dimensionality Reduction is the process of reducing the number of dimensions the data is spread across. It means, the attributes or features, that the data set carries as the number of dimensions increases the sparsity. This sparsity is critical to clustering, outlier analysis and other algorithms. With reduced dimensionality, it is easy to visualize and manipulate data. There are three types of Dimensionality reduction.

• Wavelet Transform

Wavelet Transform is a lossy method for dimensionality reduction, where a data vector X is transformed into another vector X’, in such a way that both X and X’ still represent the same length. The result of wavelet transform can be truncated, unlike its original, thus achieving dimensionality reduction. Wavelet transforms are well suited for data cube, sparse data or data which is highly skewed. Wavelet transform is often used in image compression.

• Principal Component Analysis

This method involves the identification of a few independent tuples with ‘n’ attributes that can represent the entire data set. This method can be applied to skewed and sparse data.

• Attribute Subset Selection

Here, attributes irrelevant to data mining or redundant ones are not included in a core attribute subset. The core attribute subset selection reduces the data volume and dimensionality.

B) Numerosity Reduction

This method uses alternate, small forms of data representation, thus reducing data volume. There are two types of Numerosity reduction, Parametric and Non-Parametric.

• Parametric

This method assumes a model into which the data fits. Data model parameters are estimated, and only those parameters are stored, and the rest of the data is discarded. For example, a regression model can be used to achieve Parametric reduction if the data fits the Linear Regression model. 

Linear Regression models a linear relationship between two attributes of the data set. Let’s say we need to fit a linear regression model between two attributes, x and y, where y is the dependent attribute, and x is the independent attribute or predictor attribute. The model can be represented by the equation y=wx b. Where w and b are regression coefficients. A multiple linear regression model lets us express the attribute y in terms of multiple predictor attributes.

Another method, the Log-Linear model discovers the relationship between two or more discrete attributes. Assume, we have a set of tuples in n-dimensional space; the log-linear model helps to derive the probability of each tuple in this n-dimensional space.

• Non-Parametric

A non-parametric numerosity reduction technique does not assume any model. The non-Parametric technique results in a more uniform reduction, irrespective of data size, but it may not achieve a high volume of data reduction like the Parametric one. There are at least four types of Non-Parametric data reduction techniques, Histogram, Clustering, Sampling, Data Cube Aggregation, Data Compression.

C) Histogram

A histogram can be used to represent dense, sparse, skewed or uniform data, involving multiple attributes, effectively up to 5 together.

D) Clustering

In Clustering, the data set is replaced by the cluster representation, where the data is split between clusters depending on similarities to each other within-cluster and dissimilarities to other clusters. The more the similarity within-cluster, the closer they appear within the cluster. The quality of the cluster depends on the maximum distance between any two data items in the cluster.

 

E) Sampling

Sampling is capable of reducing large data set into smaller sample data sets, reducing it to a representation of the original data set. There are four types of sampling data reduction methods.

• Simple Random Sample Without Replacement of sizes
• Simple Random Sample with Replacement of sizes
• Cluster Sample
• Stratified Sample

F) Data Cube Aggregation

Data Cube Aggregation is a multidimensional aggregation that uses aggregation at various levels of a data cube to represent the original data set, thus achieving data reduction. Data Cube Aggregation, where the data cube is a much more efficient way of storing data, thus achieving data reduction, besides faster aggregation operations.

G) Data Compression

It employs modification, encoding or converting the structure of data in a way that consumes less space. Data compression involves building a compact representation of information by removing redundancy and representing data in binary form. Data that can be restored successfully from its compressed form is called Lossless compression while the opposite where it is not possible to restore the original form from the compressed form is Lossy compression.

Conclusion

Data reduction achieves a reduction in volume, making it easy to represent and run data through advanced analytical algorithms. Data reduction also helps in the deduplication of data reducing the load on storage and the algorithms serving data science techniques downstream. It can be achieved in two principal ways. One by reducing the number of data records, or features and the other by generating summary data and statistics at different levels.

If you are interested in making a career in the Data Science domain, our 9-month online PG Certificate Program in Data Science and Machine Learning course can help you immensely in becoming a successful Data Science professional. 

Also Read

• What is Data Processing? An Important Overview In 6 Points