Concept drift for big data
Combating Security Challenges in the Age of Big Data
School of Science / ECU Security Research Institute
The term “concept drift” refers to a change in statistical distribution of the data. In machine learning and predictive analysis, a fundamental assumption exits which reasons that the data is a random variable which is being generated independently from an underlying stationary distribution. In this chapter we present discussions on concept drifts that are inherent in the context big data. We discuss different forms of concept drifts that are evident in streaming data and outline different techniques for handling them. Handling concept drift is important for big data where the data flow occurs continuously causing existing learned models to lose their predictive accuracy. This chapter will serve as a reference to academicians and industry practitioners who are interested in the niche area of handling concept drift for big data applications.