Data mining and statistics

I have recently found an interesting paper about the connection between data mining and statistics. It is written by Diego Kuonen, who is now working at Statoo Consulting in Switzerland. The basic question that leads his paper is whether data mining is statistical déjà vu.

After explaining what is statistics and why it is needed, he explains data mining using several definitions. He points out an interesting fact by writing that defining and understanding the business process are most important parts of data mining tasks. He argues that:

"Even the most advances algorithms cannot figure out what is most important."

He also refers to the garbage in, garbage out issue that has been previously discussed on Data Mining Research. He then concludes that data mining cannot be ignored by companies since the advantages of knowledge extraction for businesses are enormous. I would like to quote a sentence I liked where he emphasizes differences between data miners, statisticians and clients:

"[...] computer scientists focus upon database manipulations and processing algorithms; statisticians focus upon identifying and handling uncertainties; and clients focus upon integrating knowledge into the knowledge domain."

If you're interested, feel free to read the article.

Continue reading... Sphere: Related Content

10 years of data mining

In a recent paper in the Data Mining and Knowledge Discovery Journal, Gregory Piatetsky-Shapiro has an interesting paper about recent tendency in data mining. He wrote a survey based on the point of view of KDnuggets, a must-known company sharing news and more for the data mining community. Below is the abstract of his paper:

I survey the transformation of the data mining and knowledge discovery field over the last 10 years from the unique vantage point of KDnuggets as a leading chronicler of the field. Analysis of the most frequent words in KDnuggets News leads to revealing observations.

In this paper, Gregory points out the main transformations that have affected the domain. He provides several application domains where data mining has succeeded, as well as successful methods such as Support Vector Machines (SVM). After some text mining on KDnuggets news, Gregory draws a few conclusions. The main one is a move of data mining from university to business. The greatest demand for data miners is from business analytics, according to his study.

I think that the paper is only accessible if you have access to the journal content.

Continue reading... Sphere: Related Content