Hendrik/xgboost
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

[jvm-packages] add example to handle missing value other than 0 (#5677)

Browse Source 
add example to handle missing value other than 0 under Dealing with missing values section
This commit is contained in:
Naveed Ahmed Saleem Janvekar 2020-10-28 17:24:35 -07:00 committed by GitHub
parent 74ea82209b
commit 608bda7052
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 24 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

25
doc/jvm/xgboost4j_spark_tutorial.rst
View File

@ -161,7 +161,30 @@ Example of setting a missing value (e.g. -999) to the "missing" parameter in XGB
1. Explicitly convert the Vector returned from VectorAssembler to a DenseVector to return the zeros to the dataset. If
doing this with missing values encoded as NaN, you will want to set ``setHandleInvalid = "keep"`` on VectorAssembler
in order to keep the NaN values in the dataset. You would then set the "missing" parameter to whatever you want to be
treated as missing. However this may cause a large amount of memory use if your dataset is very sparse.
treated as missing. However this may cause a large amount of memory use if your dataset is very sparse. For example:
.. code-block:: scala
val assembler = new VectorAssembler().setInputCols(feature_names.toArray).setOutputCol("features").setHandleInvalid("keep")
// conversion to dense vector using Array()
val featurePipeline = new Pipeline().setStages(Array(assembler))
val featureModel = featurePipeline.fit(df_training)
val featureDf = featureModel.transform(df_training)
val xgbParam = Map("eta" -> 0.1f,
"max_depth" -> 2,
"objective" -> "multi:softprob",
"num_class" -> 3,
"num_round" -> 100,
"num_workers" -> 2,
"allow_non_zero_for_missing" -> "true",
"missing" -> -999)
val xgb = new XGBoostClassifier(xgbParam)
val xgbclassifier = xgb.fit(featureDf)
2. Before calling VectorAssembler you can transform the values you want to represent missing into an irregular value
that is not 0, NaN, or Null and set the "missing" parameter to 0. The irregular value should ideally be chosen to be