From d9614dfbe8b631df52ad2021459abf80e9e05c33 Mon Sep 17 00:00:00 2001 From: phunterlau Date: Wed, 24 Feb 2016 17:36:20 -0800 Subject: [PATCH 1/3] Awesome-XGBoost, first commit --- demo/README.md | 72 ++++++++++++++++++++++++++++++++++++++++---------- 1 file changed, 58 insertions(+), 14 deletions(-) diff --git a/demo/README.md b/demo/README.md index 229ffc6ff..284df4b62 100644 --- a/demo/README.md +++ b/demo/README.md @@ -1,13 +1,19 @@ -XGBoost Code Examples -===================== -This folder contains all the code examples using xgboost. +#Awesome XGBoost + +Welcome to the wonderland of XGBoost. This page contains a curated list of awesome XGBoost examples, tutorials and blogs. It is inspired by [awesom-MXnet](https://github.com/dmlc/mxnet/blob/master/example/README.md), [awesome-php](https://github.com/ziadoz/awesome-php) and [awesome-machine-learning](https://github.com/josephmisiti/awesome-machine-learning). + +## Contributing * Contribution of examples, benchmarks is more than welcome! * If you like to share how you use xgboost to solve your problem, send a pull request:) +* If you want to contribute to this list and the examples, please open a new pull request. + +##List of examples + +### Features Walkthrough -Features Walkthrough --------------------- This is a list of short codes introducing different functionalities of xgboost packages. + * Basic walkthrough of packages [python](guide-python/basic_walkthrough.py) [R](../R-package/demo/basic_walkthrough.R) @@ -36,23 +42,61 @@ This is a list of short codes introducing different functionalities of xgboost p [python](guide-python/predict_leaf_indices.py) [R](../R-package/demo/predict_leaf_indices.R) -Basic Examples by Tasks ------------------------ +### Basic Examples by Tasks + Most of examples in this section are based on CLI or python version. However, the parameter settings can be applied to all versions + * [Binary classification](binary_classification) * [Multiclass classification](multiclass_classification) * [Regression](regression) * [Learning to Rank](rank) * [Distributed Training](distributed-training) -Benchmarks ----------- +### Benchmarks + * [Starter script for Kaggle Higgs Boson](kaggle-higgs) * [Kaggle Tradeshift winning solution by daxiongshu](https://github.com/daxiongshu/kaggle-tradeshift-winning-solution) -Machine Learning Challenge Winning Solutions --------------------------------------------- -* XGBoost helps Vlad Mironov, Alexander Guschin to win the [CERN LHCb experiment Flavour of Physics competition](https://www.kaggle.com/c/flavours-of-physics). Check out the [interview from Kaggle](http://blog.kaggle.com/2015/11/30/flavour-of-physics-technical-write-up-1st-place-go-polar-bears/). -* XGBoost helps Mario Filho, Josef Feigl, Lucas, Gilberto to win the [Caterpillar Tube Pricing competition](https://www.kaggle.com/c/caterpillar-tube-pricing). Check out the [interview from Kaggle](http://blog.kaggle.com/2015/09/22/caterpillar-winners-interview-1st-place-gilberto-josef-leustagos-mario/). -* XGBoost helps Halla Yang to win the [Recruit Coupon Purchase Prediction Challenge](https://www.kaggle.com/c/coupon-purchase-prediction). Check out the [interview from Kaggle](http://blog.kaggle.com/2015/10/21/recruit-coupon-purchase-winners-interview-2nd-place-halla-yang/). +## Machine Learning Challenge Winning Solutions + +"Over the last six months, a new algorithm has come up on Kaggle __winning every single competition__ in this category, it is an algorithm called __XGBoost__." -- Anthony Goldbloom, Founder & CEO of Kaggle (from his presentation "What Is Winning on Kaggle?" [youtube link](https://youtu.be/GTs5ZQ6XwUM?t=7m7s)) + +* XGBoost helps Marios Michailidis, Mathias Müller and HJ van Veen to win (1st place) the [Dato Truely Native? competition](https://www.kaggle.com/c/dato-native). Check out the [interview from Kaggle](http://blog.kaggle.com/2015/12/03/dato-winners-interview-1st-place-mad-professors/). +* XGBoost helps Vlad Mironov, Alexander Guschin to win (1st place) the [CERN LHCb experiment Flavour of Physics competition](https://www.kaggle.com/c/flavours-of-physics). Check out the [interview from Kaggle](http://blog.kaggle.com/2015/11/30/flavour-of-physics-technical-write-up-1st-place-go-polar-bears/). +* XGBoost helps Josef Slavicek to win (3rd place) the [CERN LHCb experiment Flavour of Physics competition](https://www.kaggle.com/c/flavours-of-physics). Check out the [interview from Kaggle](http://blog.kaggle.com/2015/11/23/flavour-of-physics-winners-interview-3rd-place-josef-slavicek/). +* XGBoost helps Mario Filho, Josef Feigl, Lucas, Gilberto to win (1st place) the [Caterpillar Tube Pricing competition](https://www.kaggle.com/c/caterpillar-tube-pricing). Check out the [interview from Kaggle](http://blog.kaggle.com/2015/09/22/caterpillar-winners-interview-1st-place-gilberto-josef-leustagos-mario/). +* XGBoost helps Qingchen Wang to win (1st place) the [Liberty Mutual Property Inspection](https://www.kaggle.com/c/liberty-mutual-group-property-inspection-prediction). Check out the [interview from Kaggle](http://blog.kaggle.com/2015/09/28/liberty-mutual-property-inspection-winners-interview-qingchen-wang/). +* XGBoost helps Chenglong Chen to win (1st place) the [Crowdflower Search Results Relevance](https://www.kaggle.com/c/crowdflower-search-relevance). Check out the [Winning solution](https://www.kaggle.com/c/crowdflower-search-relevance/forums/t/15186/1st-place-winner-solution-chenglong-chen/). +* XGBoost helps Alexandre Barachant (“Cat”) and Rafał Cycoń (“Dog”) to win (1st place) the [Grasp-and-Lift EEG Detection](https://www.kaggle.com/c/grasp-and-lift-eeg-detection). Check out the [interview from Kaggle](http://blog.kaggle.com/2015/10/12/grasp-and-lift-eeg-winners-interview-1st-place-cat-dog/). +* XGBoost helps Halla Yang to win (2nd place) the [Recruit Coupon Purchase Prediction Challenge](https://www.kaggle.com/c/coupon-purchase-prediction). Check out the [interview from Kaggle](http://blog.kaggle.com/2015/10/21/recruit-coupon-purchase-winners-interview-2nd-place-halla-yang/). +* XGBoost helps Owen Zhang to win (1st place) the [Avito Context Ad Clicks competition](https://www.kaggle.com/c/avito-context-ad-clicks). Check out the [interview from Kaggle](http://blog.kaggle.com/2015/08/26/avito-winners-interview-1st-place-owen-zhang/). +* There are many other great winning solutions and interviews, but this list is [too small](https://en.wikipedia.org/wiki/Fermat%27s_Last_Theorem) to put all of them here. Please send pull requests if important ones appear. + + +## List of Tutorials + +* "[Open Source Tools & Data Science Competitions](http://www.slideshare.net/odsc/owen-zhangopen-sourcetoolsanddscompetitions1)" by Owen Zhang - XGBoost parameter tuning tips +* "[Tips for data science competitions](http://www.slideshare.net/OwenZhang2/tips-for-data-science-competitions)" by Owen Zhang - Page 14 +* "[XGBoost - eXtreme Gradient Boosting](http://www.slideshare.net/ShangxuanZhang/xgboost)" by Tong He +* "[How to use XGBoost algorithm in R in easy steps](http://www.analyticsvidhya.com/blog/2016/01/xgboost-algorithm-easy-steps/)" by TAVISH SRIVASTAVA ([Chinese Translation 中文翻译](https://segmentfault.com/a/1190000004421821) by [HarryZhu](https://segmentfault.com/u/harryprince)) +* "[Kaggle Solution: What’s Cooking ? (Text Mining Competition)](http://www.analyticsvidhya.com/blog/2015/12/kaggle-solution-cooking-text-mining-competition/)" by MANISH SARASWAT +* "Better Optimization with Repeated Cross Validation and the XGBoost model - Machine Learning with R)" by Manuel Amunategui ([Youtube Link](https://www.youtube.com/watch?v=Og7CGAfSr_Y)) ([Github Link](https://github.com/amunategui/BetterCrossValidation)) +* "[XGBoost Rossman Parameter Tuning](https://www.kaggle.com/khozzy/rossmann-store-sales/xgboost-parameter-tuning-template/run/90168/notebook)" by [Norbert Kozlowski](https://www.kaggle.com/khozzy) +* "[Featurizing log data before XGBoost](http://www.slideshare.net/DataRobot/featurizing-log-data-before-xgboost)" by Xavier Conort, Owen Zhang etc +* "[West Nile Virus Competition Benchmarks & Tutorials](http://blog.kaggle.com/2015/07/21/west-nile-virus-competition-benchmarks-tutorials/)" by [Anna Montoya](http://blog.kaggle.com/author/annamontoya/) +* "[Ensemble Decision Tree with XGBoost](https://www.kaggle.com/binghsu/predict-west-nile-virus/xgboost-starter-code-python-0-69)" by [Bing Xu](https://www.kaggle.com/binghsu) +* "[Notes on eXtreme Gradient Boosting](http://startup.ml/blog/xgboost)" by ARSHAK NAVRUZYAN ([iPython Notebook](https://github.com/startupml/koan/blob/master/eXtreme%20Gradient%20Boosting.ipynb)) + +## List of Tools with XGBoost + +* [BayesBoost](https://github.com/mpearmain/BayesBoost) - Bayesian Optimization using xgboost and sklearn API + +## List of Services Powered by XGBoost + +* [Seldon predictive service powered by XGBoost](http://docs.seldon.io/iris-demo.html) +* [ODPS by Alibaba](https://yq.aliyun.com/articles/6355) (in Chinese) + +## List of Awards + +* [John Chambers Award](http://stat-computing.org/awards/jmc/winners.html) - 2016 Winner: XGBoost, by Tong He (Simon Fraser University) and Tianqi Chen (University of Washington) From ef7d26eb077c687e404d1316fe70c391a559021b Mon Sep 17 00:00:00 2001 From: phunterlau Date: Wed, 24 Feb 2016 21:56:51 -0800 Subject: [PATCH 2/3] add TOC, simplied text in the solution section --- demo/README.md | 59 ++++++++++++++++++++++++++++++++++---------------- 1 file changed, 40 insertions(+), 19 deletions(-) diff --git a/demo/README.md b/demo/README.md index 284df4b62..68ba3cb2c 100644 --- a/demo/README.md +++ b/demo/README.md @@ -1,15 +1,33 @@ -#Awesome XGBoost + + +**Table of Contents** *generated with [DocToc](https://github.com/thlorenz/doctoc)* -Welcome to the wonderland of XGBoost. This page contains a curated list of awesome XGBoost examples, tutorials and blogs. It is inspired by [awesom-MXnet](https://github.com/dmlc/mxnet/blob/master/example/README.md), [awesome-php](https://github.com/ziadoz/awesome-php) and [awesome-machine-learning](https://github.com/josephmisiti/awesome-machine-learning). +- [Awesome XGBoost](#awesome-xgboost) + - [Contributing](#contributing) + - [Examples](#examples) + - [Features Walkthrough](#features-walkthrough) + - [Basic Examples by Tasks](#basic-examples-by-tasks) + - [Benchmarks](#benchmarks) + - [Machine Learning Challenge Winning Solutions](#machine-learning-challenge-winning-solutions) + - [Tutorials](#tutorials) + - [Tools with XGBoost](#tools-with-xgboost) + - [Services Powered by XGBoost](#services-powered-by-xgboost) + - [Awards](#awards) -## Contributing + +Awesome XGBoost +====== +Welcome to the wonderland of XGBoost. This page contains a curated list of awesome XGBoost examples, tutorials and blogs. It is inspired by [awesome-MXNet](https://github.com/dmlc/mxnet/blob/master/example/README.md), [awesome-php](https://github.com/ziadoz/awesome-php) and [awesome-machine-learning](https://github.com/josephmisiti/awesome-machine-learning). + +Contributing +---- * Contribution of examples, benchmarks is more than welcome! * If you like to share how you use xgboost to solve your problem, send a pull request:) * If you want to contribute to this list and the examples, please open a new pull request. -##List of examples - +Examples +---- ### Features Walkthrough This is a list of short codes introducing different functionalities of xgboost packages. @@ -62,19 +80,22 @@ However, the parameter settings can be applied to all versions "Over the last six months, a new algorithm has come up on Kaggle __winning every single competition__ in this category, it is an algorithm called __XGBoost__." -- Anthony Goldbloom, Founder & CEO of Kaggle (from his presentation "What Is Winning on Kaggle?" [youtube link](https://youtu.be/GTs5ZQ6XwUM?t=7m7s)) -* XGBoost helps Marios Michailidis, Mathias Müller and HJ van Veen to win (1st place) the [Dato Truely Native? competition](https://www.kaggle.com/c/dato-native). Check out the [interview from Kaggle](http://blog.kaggle.com/2015/12/03/dato-winners-interview-1st-place-mad-professors/). -* XGBoost helps Vlad Mironov, Alexander Guschin to win (1st place) the [CERN LHCb experiment Flavour of Physics competition](https://www.kaggle.com/c/flavours-of-physics). Check out the [interview from Kaggle](http://blog.kaggle.com/2015/11/30/flavour-of-physics-technical-write-up-1st-place-go-polar-bears/). -* XGBoost helps Josef Slavicek to win (3rd place) the [CERN LHCb experiment Flavour of Physics competition](https://www.kaggle.com/c/flavours-of-physics). Check out the [interview from Kaggle](http://blog.kaggle.com/2015/11/23/flavour-of-physics-winners-interview-3rd-place-josef-slavicek/). -* XGBoost helps Mario Filho, Josef Feigl, Lucas, Gilberto to win (1st place) the [Caterpillar Tube Pricing competition](https://www.kaggle.com/c/caterpillar-tube-pricing). Check out the [interview from Kaggle](http://blog.kaggle.com/2015/09/22/caterpillar-winners-interview-1st-place-gilberto-josef-leustagos-mario/). -* XGBoost helps Qingchen Wang to win (1st place) the [Liberty Mutual Property Inspection](https://www.kaggle.com/c/liberty-mutual-group-property-inspection-prediction). Check out the [interview from Kaggle](http://blog.kaggle.com/2015/09/28/liberty-mutual-property-inspection-winners-interview-qingchen-wang/). -* XGBoost helps Chenglong Chen to win (1st place) the [Crowdflower Search Results Relevance](https://www.kaggle.com/c/crowdflower-search-relevance). Check out the [Winning solution](https://www.kaggle.com/c/crowdflower-search-relevance/forums/t/15186/1st-place-winner-solution-chenglong-chen/). -* XGBoost helps Alexandre Barachant (“Cat”) and Rafał Cycoń (“Dog”) to win (1st place) the [Grasp-and-Lift EEG Detection](https://www.kaggle.com/c/grasp-and-lift-eeg-detection). Check out the [interview from Kaggle](http://blog.kaggle.com/2015/10/12/grasp-and-lift-eeg-winners-interview-1st-place-cat-dog/). -* XGBoost helps Halla Yang to win (2nd place) the [Recruit Coupon Purchase Prediction Challenge](https://www.kaggle.com/c/coupon-purchase-prediction). Check out the [interview from Kaggle](http://blog.kaggle.com/2015/10/21/recruit-coupon-purchase-winners-interview-2nd-place-halla-yang/). -* XGBoost helps Owen Zhang to win (1st place) the [Avito Context Ad Clicks competition](https://www.kaggle.com/c/avito-context-ad-clicks). Check out the [interview from Kaggle](http://blog.kaggle.com/2015/08/26/avito-winners-interview-1st-place-owen-zhang/). -* There are many other great winning solutions and interviews, but this list is [too small](https://en.wikipedia.org/wiki/Fermat%27s_Last_Theorem) to put all of them here. Please send pull requests if important ones appear. +XGBoost has helped on these winning solutions: + +* Marios Michailidis, Mathias Müller and HJ van Veen, 1st place of the [Dato Truely Native? competition](https://www.kaggle.com/c/dato-native). Link to [the Kaggle interview](http://blog.kaggle.com/2015/12/03/dato-winners-interview-1st-place-mad-professors/). +* Vlad Mironov, Alexander Guschin, 1st place of the [CERN LHCb experiment Flavour of Physics competition](https://www.kaggle.com/c/flavours-of-physics). Link to [the Kaggle interview](http://blog.kaggle.com/2015/11/30/flavour-of-physics-technical-write-up-1st-place-go-polar-bears/). +* Josef Slavicek, 3rd place of the [CERN LHCb experiment Flavour of Physics competition](https://www.kaggle.com/c/flavours-of-physics). Link to [the Kaggle interview](http://blog.kaggle.com/2015/11/23/flavour-of-physics-winners-interview-3rd-place-josef-slavicek/). +* Mario Filho, Josef Feigl, Lucas, Gilberto, 1st place of the [Caterpillar Tube Pricing competition](https://www.kaggle.com/c/caterpillar-tube-pricing). Link to [the Kaggle interview](http://blog.kaggle.com/2015/09/22/caterpillar-winners-interview-1st-place-gilberto-josef-leustagos-mario/). +* Qingchen Wang, 1st place of the [Liberty Mutual Property Inspection](https://www.kaggle.com/c/liberty-mutual-group-property-inspection-prediction). Link to [the Kaggle interview] (http://blog.kaggle.com/2015/09/28/liberty-mutual-property-inspection-winners-interview-qingchen-wang/). +* Chenglong Chen, 1st place of the [Crowdflower Search Results Relevance](https://www.kaggle.com/c/crowdflower-search-relevance). [Link to the winning solution](https://www.kaggle.com/c/crowdflower-search-relevance/forums/t/15186/1st-place-winner-solution-chenglong-chen/). +* Alexandre Barachant (“Cat”) and Rafał Cycoń (“Dog”), 1st place of the [Grasp-and-Lift EEG Detection](https://www.kaggle.com/c/grasp-and-lift-eeg-detection). Link to [the Kaggle interview](http://blog.kaggle.com/2015/10/12/grasp-and-lift-eeg-winners-interview-1st-place-cat-dog/). +* Halla Yang, 2nd place of the [Recruit Coupon Purchase Prediction Challenge](https://www.kaggle.com/c/coupon-purchase-prediction). Link to [the Kaggle interview](http://blog.kaggle.com/2015/10/21/recruit-coupon-purchase-winners-interview-2nd-place-halla-yang/). +* Owen Zhang, 1st place of the [Avito Context Ad Clicks competition](https://www.kaggle.com/c/avito-context-ad-clicks). Link to [the Kaggle interview](http://blog.kaggle.com/2015/08/26/avito-winners-interview-1st-place-owen-zhang/). + +There are many other great winning solutions and interviews, but this list is [too small](https://en.wikipedia.org/wiki/Fermat%27s_Last_Theorem) to put all of them here. Please send pull requests if important ones appear. -## List of Tutorials +## Tutorials * "[Open Source Tools & Data Science Competitions](http://www.slideshare.net/odsc/owen-zhangopen-sourcetoolsanddscompetitions1)" by Owen Zhang - XGBoost parameter tuning tips * "[Tips for data science competitions](http://www.slideshare.net/OwenZhang2/tips-for-data-science-competitions)" by Owen Zhang - Page 14 @@ -88,15 +109,15 @@ However, the parameter settings can be applied to all versions * "[Ensemble Decision Tree with XGBoost](https://www.kaggle.com/binghsu/predict-west-nile-virus/xgboost-starter-code-python-0-69)" by [Bing Xu](https://www.kaggle.com/binghsu) * "[Notes on eXtreme Gradient Boosting](http://startup.ml/blog/xgboost)" by ARSHAK NAVRUZYAN ([iPython Notebook](https://github.com/startupml/koan/blob/master/eXtreme%20Gradient%20Boosting.ipynb)) -## List of Tools with XGBoost +## Tools with XGBoost * [BayesBoost](https://github.com/mpearmain/BayesBoost) - Bayesian Optimization using xgboost and sklearn API -## List of Services Powered by XGBoost +## Services Powered by XGBoost * [Seldon predictive service powered by XGBoost](http://docs.seldon.io/iris-demo.html) * [ODPS by Alibaba](https://yq.aliyun.com/articles/6355) (in Chinese) -## List of Awards +## Awards * [John Chambers Award](http://stat-computing.org/awards/jmc/winners.html) - 2016 Winner: XGBoost, by Tong He (Simon Fraser University) and Tianqi Chen (University of Washington) From 80595d6cc59b6f14006e1a7bf90b90e2f1d43847 Mon Sep 17 00:00:00 2001 From: phunterlau Date: Thu, 25 Feb 2016 11:32:11 -0800 Subject: [PATCH 3/3] move TOC under title --- demo/README.md | 13 +++---------- 1 file changed, 3 insertions(+), 10 deletions(-) diff --git a/demo/README.md b/demo/README.md index 68ba3cb2c..2864887a4 100644 --- a/demo/README.md +++ b/demo/README.md @@ -1,8 +1,7 @@ - - -**Table of Contents** *generated with [DocToc](https://github.com/thlorenz/doctoc)* +Awesome XGBoost +====== +Welcome to the wonderland of XGBoost. This page contains a curated list of awesome XGBoost examples, tutorials and blogs. It is inspired by [awesome-MXNet](https://github.com/dmlc/mxnet/blob/master/example/README.md), [awesome-php](https://github.com/ziadoz/awesome-php) and [awesome-machine-learning](https://github.com/josephmisiti/awesome-machine-learning). -- [Awesome XGBoost](#awesome-xgboost) - [Contributing](#contributing) - [Examples](#examples) - [Features Walkthrough](#features-walkthrough) @@ -14,12 +13,6 @@ - [Services Powered by XGBoost](#services-powered-by-xgboost) - [Awards](#awards) - - -Awesome XGBoost -====== -Welcome to the wonderland of XGBoost. This page contains a curated list of awesome XGBoost examples, tutorials and blogs. It is inspired by [awesome-MXNet](https://github.com/dmlc/mxnet/blob/master/example/README.md), [awesome-php](https://github.com/ziadoz/awesome-php) and [awesome-machine-learning](https://github.com/josephmisiti/awesome-machine-learning). - Contributing ---- * Contribution of examples, benchmarks is more than welcome!