From 331cd3e4f7c44e5ed4c7eef3097d74c46924f6e7 Mon Sep 17 00:00:00 2001 From: Philip Hyunsu Cho Date: Fri, 8 Mar 2019 10:05:39 -0800 Subject: [PATCH] Document limitation of one-split-at-a-time Greedy tree learning heuristic (#4233) --- doc/tutorials/model.rst | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/doc/tutorials/model.rst b/doc/tutorials/model.rst index 0694efebf..5c394724d 100644 --- a/doc/tutorials/model.rst +++ b/doc/tutorials/model.rst @@ -255,6 +255,10 @@ For real valued data, we usually want to search for an optimal split. To efficie A left to right scan is sufficient to calculate the structure score of all possible split solutions, and we can find the best split efficiently. +.. note:: Limitation of additive tree learning + + Since it is intractable to enumerate all possible tree structures, we add one split at a time. This approach works well most of the time, but there are some edge cases that fail due to this approach. For those edge cases, training results in a degenerate model because we consider only one feature dimension at a time. See `Can Gradient Boosting Learn Simple Arithmetic? `_ for an example. + ********************** Final words on XGBoost **********************