tag:blogger.com,1999:blog-29117034.post2332681423213586801..comments2024-01-08T12:39:20.176+01:00Comments on Data Mining Research - dataminingblog.com: Readability of Decision TreesSandro Saittahttp://www.blogger.com/profile/17682082649770875583noreply@blogger.comBlogger6125tag:blogger.com,1999:blog-29117034.post-39896257965608920082009-01-08T17:05:00.000+01:002009-01-08T17:05:00.000+01:00Thanks for your comment James!Thanks for your comment James!Sandro Saittahttps://www.blogger.com/profile/17682082649770875583noreply@blogger.comtag:blogger.com,1999:blog-29117034.post-14109398715999316882009-01-08T01:07:00.000+01:002009-01-08T01:07:00.000+01:00Random forests have the advantage of opening up pr...Random forests have the advantage of opening up problems that a single decision tree might not deal with well, such as when a class of interest is relatively rare. I prefer boosted trees for this situation, though.James Pearcehttps://www.blogger.com/profile/12523949311049795913noreply@blogger.comtag:blogger.com,1999:blog-29117034.post-86945526477216598782008-12-01T17:12:00.000+01:002008-12-01T17:12:00.000+01:00Thanks for your contribution Shane!Thanks for your contribution Shane!Sandro Saittahttps://www.blogger.com/profile/17682082649770875583noreply@blogger.comtag:blogger.com,1999:blog-29117034.post-21438314426240098482008-11-30T23:30:00.000+01:002008-11-30T23:30:00.000+01:00I think random forest are both useful and powerful...I think random forest are both useful and powerful... with some caveats... you need lots of memory for big data (so not practical for all tasks) and readability is also a problem. <A HREF="http://rattle.togaware.com/" REL="nofollow">Rattle</A> solves the random forest readability issues by producing an <A HREF="http://datamining.togaware.com/survivor/rattle-audit-model-rf-importance.png" REL="nofollow">importance chart</A>.Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-29117034.post-3342061928296216752008-11-28T14:31:00.000+01:002008-11-28T14:31:00.000+01:00Hi Tim,Thanks for your comment!1) Binning the data...Hi Tim,<BR/><BR/>Thanks for your comment!<BR/><BR/>1) Binning the data into buckets is a nice way to avoid this "unreadability" problem. I have always used normalization or standardization, but never used binning. Also the fact that you have the same number of occurrences in each bin avoid the issue of outliers.<BR/><BR/>2) Regarding random forests, I definitely agree on the issues when using this technique (that's why I don't use random forests). However, I really like the concept of several models voting for the output class.Sandro Saittahttps://www.blogger.com/profile/17682082649770875583noreply@blogger.comtag:blogger.com,1999:blog-29117034.post-64008087296503713452008-11-28T06:10:00.000+01:002008-11-28T06:10:00.000+01:00Hi Sandro,I have a few thoughts.a) Hey, this bit;"...Hi Sandro,<BR/><BR/>I have a few thoughts.<BR/><BR/>a) Hey, this bit;<BR/>"In most projects, data have to be normalized before using decision tree. Therefore, once you plot the tree, values are meaningless."<BR/> - I reckon not necessarily true! <BR/><BR/>Depends on your normalisation. You can normalise your data with meaning! <BR/><BR/>I like binning into 100 buckets, each with the same numbers of occurrences (say, customer rows). I do this for a few reasons, one being that I can then report customers as being in the top 5% buckets etc. It is also an easy and fast way to rescale lots of data in SQL. CART or C5.0 models using this type of normalised data is actually quite easy to make sense of (eg, “if stock price is above 70% bucket” etc etc).<BR/><BR/>b) Random forest doesn't work well with big datasets (millions rows). I use fairly easy CART or C5.0. Sometimes I build a handful of models on subsets samples and average the models, but I'm not convinced hundred of models is the best way to go. I always take time creating new derived 'information rich columns' and using these as additional inputs to a decision tree or neural net.<BR/><BR/><BR/>I agree with the problems you describe and, for those reasons you mention, I don't follow the steps you describe. Maybe I'm jaded, but I believe Random Forests is a classic example of mad academia over practicality (and yes, I know that's controversial considering the brilliant guy who created random forests...). <BR/><BR/> - TimTim Mannshttps://www.blogger.com/profile/17405266346372888597noreply@blogger.com