Hendrik/xgboost
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

fix Dart::NormalizeTrees (#1265)

Browse Source
This commit is contained in:
Yoshinori Nakano 2016-06-10 07:28:24 +09:00 committed by Tianqi Chen
parent 949d1e3027
commit 7cfeb5f012
2 changed files with 4 additions and 4 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
doc/parameter.md
View File

@ -84,10 +84,10 @@ Additional parameters for Dart Booster
* normalize_type [default="tree] * normalize_type [default="tree]
- type of normalization algorithm. - type of normalization algorithm.
- "tree": New trees have the same weight of each of dropped trees. - "tree": New trees have the same weight of each of dropped trees.
weight of new trees are learning_rate / (k + learnig_rate) weight of new trees are 1 / (k + learnig_rate)
dropped trees are scaled by a factor of k / (k + learning_rate) dropped trees are scaled by a factor of k / (k + learning_rate)
- "forest": New trees have the same weight of sum of dropped trees (forest). - "forest": New trees have the same weight of sum of dropped trees (forest).
weight of new trees are learning_rate / (1 + learning_rate) weight of new trees are 1 / (1 + learning_rate)
dropped trees are scaled by a factor of 1 / (1 + learning_rate) dropped trees are scaled by a factor of 1 / (1 + learning_rate)
* rate_drop [default=0.0] * rate_drop [default=0.0]
- dropout rate. - dropout rate.

4
src/gbm/gbtree.cc
View File

@ -703,7 +703,7 @@ class Dart : public GBTree {
weight_drop[i] *= factor; weight_drop[i] *= factor;
} }
for (size_t i = 0; i < size_new_trees; ++i) { for (size_t i = 0; i < size_new_trees; ++i) {
weight_drop.push_back(lr * factor); weight_drop.push_back(factor);
} }
} else { } else {
// normalize_type 0 // normalize_type 0
@ -712,7 +712,7 @@ class Dart : public GBTree {
weight_drop[i] *= factor; weight_drop[i] *= factor;
} }
for (size_t i = 0; i < size_new_trees; ++i) { for (size_t i = 0; i < size_new_trees; ++i) {
weight_drop.push_back(1.0 * lr / (num_drop + lr)); weight_drop.push_back(1.0 / (num_drop + lr));
} }
} }
} }