On ethical issues of randomized experiments

After reading Super Crunchers, a recent book by Ian Ayres, I have thought of ethical aspects of some experiments he mentioned (note that he is not involved in these experiments). In a few words, the idea of so called randomized experiments is to run an experiment with different parameters on the same population (in a somehow large scale) to see the influence of such parameters.

Ayres gives several examples of randomized experiments in his book. Of course, on some of these examples, there is no real ethical issue. For example, one can imagine a company such as Amazon performing randomized experiments. As written in Ayres book, a company may be interested in the impact of the way they threat their customers. For example, the company can decide to apply three different actions on customers who received their order lately. Of course, some customers will be treated with less attention than others. In this case, this way of doing is alright for two reasons. First, customers usually don't know about such a process. Second, if they know it, they can certainly understand that this is a random experiment of the company.

Now, imagine the more serious case (some examples are presented in the book) of treating illness or influencing education. Performing random experiments will, in certain situations, directly affect the life of people. Would you agree that companies (or governments) performs randomized experiments that can influence your education (or the one of your children)? I think that the experiments mentioned in Ayres' book can bring important issues about ethical aspects of data mining or statistics. Maybe you have a particular opinion about it?

Continue reading... Sphere: Related Content

Small book review: Super Crunchers

As written in an earlier post, Super Crunchers is a new book about data mining by Ian Ayres. Super crunching, according to Ayres is the action of applying data mining algorithms to real situations in order to make better decisions from data. I will make it clear right now: Super Crunchers will not give you examples of complex data mining techniques in real situations. Most of the book shows the use of randomized experiments (there are also a few pages on neural network, but that's all).

This book is nevertheless a very interesting reading for many reasons. First, the author did a very good job in introducing the basic ideas behind data mining for non-specialist readers. In addition, Ayres has collected a bunch of small, and very interesting, stories about people crunching data (wine quality prediction, baseball, etc.). In every situations, the author shows how crunching numbers help people make decisions but also how difficult it is to make non-expert believe in your results. This is, to my opinion, the most interesting aspect of the book.

Super Crunchers is very well written (I'm realizing now that I write that for most of my book reviews, but believe me you'll enjoy reading this book). After giving some examples, Ayres describes the actors of this industry (super crunching). He then introduces the idea of randomized experiments. There is also a nice chapter about the confrontation "Experts Versus Equations". He concludes by explaining why this enthusiasm for super crunching is happening only now and not before.

Finally, although the action of super crunching is certainly more about applying statistic methods (rather than data mining) to real situations, this is a must-have book, even for specialists in data mining. For interested reader, an interview of Ian Ayres is accessible here.

Ayres, I., Super Crunchers, Why Thinking-by-Numbers Is the New Way to be Smart, Bantam Books, 2007, 260p.

Continue reading... Sphere: Related Content

Super Crunchers

I have recently bought the book Super Crunchers: Why Thinking-by- Numbers Is the New Way to Be Smart by Ian Ayres. The book gives examples of data mining applications in nowadays companies. Instead of focusing on equations and algorithms, Ayres gives insights into data mining and statistics through real case studies. I will write a review about this book on Data Mining Research when I finish it.

If you're already interested, you can see the website of the book and a few words on the book on Newsweek.

[End of post]

Continue reading... Sphere: Related Content