The functions featureValueOfSparseVector or featureValueOfDenseVector could return a Float.NaN if the input vectore was containing any missing values. This would make fail the partition key computation and most of the vectors would end up in the same partition. We fix this by avoid returning a NaN and simply use the row HashCode in this case.
We added a test to ensure that the repartition is indeed now uniform on input dataset containing values by checking that the partitions size variance is below a certain threshold.
Signed-off-by: Anthony D'Amato <anthony.damato@hotmail.fr>
* add SHAP summary plot using ggplot2
* Update xgb.plot.shap
* Update example in xgb.plot.shap documentation
* update logic, add tests
* whitespace fixes
* whitespace fixes for test_helpers
* namespace for sd function
* explicitly declare variables that are automatically evaluated by data.table
* Fix R lint
Co-authored-by: Philip Hyunsu Cho <chohyu01@cs.washington.edu>
* fixed some endian issues
* Use dmlc::ByteSwap() to simplify code
* Fix lint check
* [CI] Add test for s390x
* Download latest CMake on s390x
* Fix a bug in my code
* Save magic number in dmatrix with byteswap on big-endian machine
* Save version in binary with byteswap on big-endian machine
* Load scalar with byteswap in MetaInfo
* Add a debugging message
* Handle arrays correctly when byteswapping
* EOF can also be 255
* Handle magic number in MetaInfo carefully
* Skip Tree.Load test for big-endian, since the test manually builds little-endian binary model
* Handle missing packages in Python tests
* Don't use boto3 in model compatibility tests
* Add s390 Docker file for local testing
* Add model compatibility tests
* Add R compatibility test
* Revert "Add R compatibility test"
This reverts commit c2d2bdcb7dbae133cbb927fcd20f7e83ee2b18a8.
Co-authored-by: Qi Zhang <q.zhang@ibm.com>
Co-authored-by: Hyunsu Cho <chohyu01@cs.washington.edu>
* [CI] Add RMM as an optional dependency
* Replace caching allocator with pool allocator from RMM
* Revert "Replace caching allocator with pool allocator from RMM"
This reverts commit e15845d4e72e890c2babe31a988b26503a7d9038.
* Use rmm::mr::get_default_resource()
* Try setting default resource (doesn't work yet)
* Allocate pool_mr in the heap
* Prevent leaking pool_mr handle
* Separate EXPECT_DEATH() in separate test suite suffixed DeathTest
* Turn off death tests for RMM
* Address reviewer's feedback
* Prevent leaking of cuda_mr
* Fix Jenkinsfile syntax
* Remove unnecessary function in Jenkinsfile
* [CI] Install NCCL into RMM container
* Run Python tests
* Try building with RMM, CUDA 10.0
* Do not use RMM for CUDA 10.0 target
* Actually test for test_rmm flag
* Fix TestPythonGPU
* Use CNMeM allocator, since pool allocator doesn't yet support multiGPU
* Use 10.0 container to build RMM-enabled XGBoost
* Revert "Use 10.0 container to build RMM-enabled XGBoost"
This reverts commit 789021fa31112e25b683aef39fff375403060141.
* Fix Jenkinsfile
* [CI] Assign larger /dev/shm to NCCL
* Use 10.2 artifact to run multi-GPU Python tests
* Add CUDA 10.0 -> 11.0 cross-version test; remove CUDA 10.0 target
* Rename Conda env rmm_test -> gpu_test
* Use env var to opt into CNMeM pool for C++ tests
* Use identical CUDA version for RMM builds and tests
* Use Pytest fixtures to enable RMM pool in Python tests
* Move RMM to plugin/CMakeLists.txt; use PLUGIN_RMM
* Use per-device MR; use command arg in gtest
* Set CMake prefix path to use Conda env
* Use 0.15 nightly version of RMM
* Remove unnecessary header
* Fix a unit test when cudf is missing
* Add RMM demos
* Remove print()
* Use HostDeviceVector in GPU predictor
* Simplify pytest setup; use LocalCUDACluster fixture
* Address reviewers' commments
Co-authored-by: Hyunsu Cho <chohyu01@cs.wasshington.edu>
* Added plugin with DPC++-based predictor and objective function
* Update CMakeLists.txt
* Update regression_obj_oneapi.cc
* Added README.md for OneAPI plugin
* Added OneAPI predictor support to gbtree
* Update README.md
* Merged kernels in gradient computation. Enabled multiple loss functions with DPC++ backend
* Aligned plugin CMake files with latest master changes. Fixed whitespace typos
* Removed debug output
* [CI] Make oneapi_plugin a CMake target
* Added tests for OneAPI plugin for predictor and obj. functions
* Temporarily switched to default selector for device dispacthing in OneAPI plugin to enable execution in environments without gpus
* Updated readme file.
* Fixed USM usage in predictor
* Removed workaround with explicit templated names for DPC++ kernels
* Fixed warnings in plugin tests
* Fix CMake build of gtest
Co-authored-by: Hyunsu Cho <chohyu01@cs.washington.edu>
* Allow non-zero for missing value when training.
* Fix wrong method names.
* Add a unit test
* Move the getter/setter unit test to MissingValueHandlingSuite
Co-authored-by: Hyunsu Cho <chohyu01@cs.washington.edu>
* [CI] Assign larger /dev/shm to NCCL
* Use 10.2 artifact to run multi-GPU Python tests
* Add CUDA 10.0 -> 11.0 cross-version test; remove CUDA 10.0 target
* [CI] Move lint to a separate script
* [CI] Improved lintr launcher
* Add lintr as a separate action
* Add custom parsing logic to print out logs
* Fix lintr issues in demos
* Run R demos
* Fix CRAN checks
* Install XGBoost into R env before running lintr
* Install devtools (needed to run demos)
* [jvm-packages] add gpu_hist tree method
* change updater hist to grow_quantile_histmaker
* add gpu scheduling
* pass correct parameters to xgboost library
* remove debug info
* add use.cuda for pom
* add CI for gpu_hist for jvm
* add gpu unit tests
* use gpu node to build jvm
* use nvidia-docker
* Add CLI interface to create_jni.py using argparse
Co-authored-by: Hyunsu Cho <chohyu01@cs.washington.edu>
