3726 Commits

Author SHA1 Message Date
Philip Hyunsu Cho
b5f7cbfadf
[CI] Cache two R build Docker containers (#4458) 2019-05-11 10:54:00 -07:00
Rong Ou
be0f346ec9 mgpu predictor using explicit offsets (#4438)
* mgpu prediction using explicit sharding
2019-05-11 09:35:06 +12:00
Rory Mitchell
d16d9a9988
Correctly determine cuda version (#4453) 2019-05-10 19:46:57 +12:00
Philip Hyunsu Cho
6ff994126a [BLOCKING][CI] Upgrade to Spark 2.4.3 (#4414)
* [CI] Upgrade to Spark 2.4.2

* Pass Spark version to build script

* Allow multiple --build-arg in ci_build.sh

* Fix syntax

* Fix container name

* Update pom.xml

* Fix container name

* Update Jenkinsfile

* Update pom.xml

* Update Dockerfile.jvm_cross
2019-05-09 21:36:59 -07:00
Shaochen Shi
18e4fc3690 [jvm-packages] Automatically set maximize_evaluation_metrics if not explicitly given in XGBoost4J-Spark (#4446)
* Automatically set maximize_evaluation_metrics if not explicitly given.

* When custom_eval is set, require maximize_evaluation_metrics.

* Update documents on early stop in XGBoost4J-Spark.

* Fix code error.
2019-05-09 12:49:44 -07:00
Jiaming Yuan
8da4907e89
Enable building with shared NCCL. (#4447)
* Add `BUILD_WITH_SHARED_NCCL` to CMake.
2019-05-09 23:19:58 +08:00
Philip Hyunsu Cho
ade3f30237
Fix list formatting in missing value tutorial in XGBoost4J-Spark 2019-05-06 14:24:02 -07:00
Philip Hyunsu Cho
b511638ca1
Fix list formatting in missing value tutorial in XGBoost4J-Spark 2019-05-06 14:21:49 -07:00
Daniel Hen
eabcc0e210 [jvm-packages] Tutorial on handling missing values (#4425)
Add tutorial on missing values and how to handle those within XGBoost.
2019-05-06 13:57:18 -07:00
Jiaming Yuan
5de7e12704
Change obj name to reg:squarederror in learner. (#4427)
* Change memory dump size in R test.
2019-05-06 21:35:35 +08:00
Xin Yin
8d1098a983 In AUC and AUCPR metrics, detect whether weights are per-instance or per-group (#4216)
* In AUC and AUCPR metrics, detect whether weights are per-instance or per-group

* Fix C++ style check

* Add a test for weighted AUC
2019-05-04 00:53:04 -07:00
Philip Hyunsu Cho
9252b686ae
Make AUCPR work with multiple query groups (#4436)
* Make AUCPR work with multiple query groups

* Check AUCPR <= 1.0 in distributed setting
2019-05-03 10:34:44 -07:00
ras44
2be85fc62a max_digits10 guarantees float decimal roundtrip (#4435)
2 additional digits are not needed to guarantee that casting the decimal representation will result in the same float, see https://github.com/dmlc/xgboost/issues/3980#issuecomment-458702440
2019-05-02 20:01:26 -07:00
Rong Ou
feb6ae3e18 Initial support for external memory in gpu_predictor (#4284) 2019-05-03 13:01:27 +12:00
ras44
54980b8959 Fix typo in xgboost_R.h (#4432) 2019-05-02 19:18:34 +08:00
Philip Hyunsu Cho
c1e4a0f2c6 Upgrade dmlc-core (#4430) 2019-05-02 18:38:22 +08:00
Philip Hyunsu Cho
bfddc2c42c Make CMakeLists.txt compatible with CMake 3.3 (#4420)
* Make CMakeLists.txt compatible with CMake 3.3; require CMake 3.11 for MSVC

* Use CMake 3.12 when sanitizer is enabled

* Disable funroll-loops for MSVC

* Use cmake version in container name

* Add missing arg

* Fix egrep use in ci_build.sh

* Display CMake version

* Do not set OpenMP_CXX_LIBRARIES for MSVC

* Use cmake_minimum_required()
2019-05-02 11:49:32 +08:00
Philip Hyunsu Cho
17df5fd296
Simplify bound checking in feature interaction constraints (#4428) 2019-05-01 16:59:53 -07:00
Xu Xiao
4c74336384 Use feature interaction constraints to narrow search space for split candidates (#4341)
* Use feature interaction constraints to narrow search space for split candidates.

* fix clang-tidy broken at updater_quantile_hist.cc:535:3

* make const

* fix

* try to fix exception thrown in java_test

* fix suspected mistake which cause EvaluateSplit error

* try fix

* Fix bug: feature ID and node ID swapped in argument

* Rename CheckValidation() to CheckFeatureConstraint() for clarity

* Do not create temporary vector validFeatures, to enable parallelism
2019-04-30 20:59:58 -07:00
Philip Hyunsu Cho
ba98e0cdf2
Add additional Python tests to test training under constraints (#4426) 2019-04-30 18:23:39 -07:00
Rong Ou
eaab364a63 More explict sharding methods for device memory (#4396)
* Rename the Reshard method to Shard

* Add a new Reshard method for sharding a vector that's already sharded
2019-05-01 11:47:22 +12:00
Xu Xiao
797ba8e72d [jvm-packages] fix compatibility problem of spark version (#4411)
* fix compatibility problem of spark version on MissingValueHandlingSuite.scala

* call setHandleInvalid by runtime reflection
2019-04-30 09:13:05 -07:00
Nan Zhu
253fdd8a42
[jvm-packages] fix the split of input (#4417) 2019-04-29 18:52:40 -07:00
tqchen
91c513a0c1 fix doc 2019-04-29 17:50:46 -07:00
Rory Mitchell
5e582b0fa7
Combine thread launches into single launch per tree for gpu_hist (#4343)
* Combine thread launches into single launch per tree for gpu_hist
algorithm.

* Address deprecation warning

* Add manual column sampler constructor

* Turn off omp dynamic to get a guaranteed number of threads

* Enable openmp in cuda code
2019-04-29 09:58:34 +12:00
Ravi Kalia
146e83f3b3 Fix typo in model.rst (#4393) 2019-04-27 14:22:07 -07:00
Bozhao
5dfb27fb2d Update demo readme's use case section with BentoML (#4400) 2019-04-27 14:21:17 -07:00
Jiaming Yuan
77c03538b0
Fix node reuse. (#4404)
* Reinitialize `_sindex` when reallocating a deleted node.
2019-04-27 13:03:23 +08:00
Nan Zhu
37dc82c3ff
[jvm-packages] allow partial evaluation of dataframe before prediction (#4407)
* allow partial evaluation of dataframe before prediction

* resume spark test

* comments

* Run unit tests after building JVM packages
2019-04-26 21:02:40 -07:00
Philip Hyunsu Cho
ea850ecd20
[CI] Refactor Jenkins CI pipeline + migrate all Linux tests to Jenkins (#4401)
* All Linux tests are now in Jenkins CI
* Tests are now de-coupled from builds. We can now build XGBoost with one version of CUDA/JDK and test it with another version of CUDA/JDK
* Builds (compilation) are significantly faster because 1) They use C5 instances with faster CPU cores; and 2) build environment setup is cached using Docker containers
2019-04-26 18:39:12 -07:00
Nan Zhu
995698b0cb
[BREAKING][jvm-packages] fix the non-zero missing value handling (#4349)
* fix the nan and non-zero missing value handling

* fix nan handling part

* add missing value

* Update MissingValueHandlingSuite.scala

* Update MissingValueHandlingSuite.scala

* stylistic fix
2019-04-26 11:10:33 -07:00
Xu Xiao
2d875ec019 [BLOCKING][jvm-packages] fix non-deterministic order within a partition (in the case of an upstream shuffle) on prediction (#4388)
* [jvm-packages][hot-fix] fix column mismatch caused by zip actions at XGBooostModel.transformInternal

* apply minibatch in prediction

* an iterator-compatible minibatch prediction

* regressor impl

* continuous working on mini-batch prediction of xgboost4j-spark

* Update Booster.java
2019-04-26 11:09:20 -07:00
Philip Hyunsu Cho
503cc42f48 [CI] Fix Windows tests (#4403)
* Install binary igraph

* Include Graphviz in PATH
2019-04-25 20:25:43 -07:00
Rong Ou
2c61f02add fix broken python test (#4395) 2019-04-23 16:01:23 -07:00
Philip Hyunsu Cho
bbe0dbd7ec
Migrate pylint check to Python 3 (#4381)
* Migrate lint to Python 3

* Fix lint errors

* Use Miniconda3 to use Python 3.7

* Use latest pylint and astroid
2019-04-21 01:01:54 -07:00
James Lamb
5e97de6a41 fixed typos in R package docs (#4345)
* fixed typos in R package docs

* updated verbosity parameter in xgb.train docs
2019-04-21 15:54:11 +08:00
Nan Zhu
65db8d0626
[jvm-packages] support spark 2.4 and compatibility test with previous xgboost version (#4377)
* bump spark version

* keep float.nan

* handle brokenly changed name/value

* add test

* add model files

* add model files

* update doc
2019-04-17 11:33:13 -07:00
Egor Smirnov
711397d645 Optimizations of pre-processing for 'hist' tree method (#4310)
* oprimizations for pre-processing

* code cleaning

* code cleaning

* code cleaning after review

* Apply suggestions from code review

Co-Authored-By: SmirnovEgorRu <egor.smirnov@intel.com>
2019-04-16 17:36:19 -07:00
Jiaming Yuan
207f058711 Refactor CMake scripts. (#4323)
* Refactor CMake scripts.

* Remove CMake CUDA wrapper.
* Bump CMake version for CUDA.
* Use CMake to handle Doxygen.
* Split up CMakeList.
* Export install target.
* Use modern CMake.
* Remove build.sh
* Workaround for gpu_hist test.
* Use cmake 3.12.

* Revert machine.conf.

* Move CLI test to gpu.

* Small cleanup.

* Support using XGBoost as submodule.

* Fix windows

* Fix cpp tests on Windows

* Remove duplicated find_package.
2019-04-15 10:08:12 -07:00
Jiaming Yuan
84d992babc
GPU multiclass metrics (#4368)
* Port multi classes metrics to CUDA.
2019-04-15 17:47:47 +08:00
James Lamb
be7bc07ca3 added files from local R build to gitignore (#4346) 2019-04-13 03:02:02 +08:00
James Lamb
edae664afb [r-package] cut CI-time dependency on craigcitro/r-travis (fixes #4348) (#4353)
* [r-package] cut CI-time dependency on craigcitro/r-travis (fixes #4348)

* Install R

* Install R on OSX

* Remove gfortran symlink

* Specify CRAN repo

* added more R dependencies needed for testing

* removed heavy R dependencies in CI

* fixed bug in env var, removed unnecessary apt installs of R

* fix to R installs
2019-04-12 00:22:48 -07:00
Rong Ou
f4521bf6aa refactor tests to get rid of duplication (#4358)
* refactor tests to get rid of duplication

* address review comments
2019-04-12 00:21:48 -07:00
Xu Xiao
3078b5944d add OpenMP option in CMakeLists.txt (#4339) 2019-04-10 17:35:06 -07:00
Adam Pocock
a448a8320c [jvm-packages] Fixing the NativeLibLoader on Java 9+ (#4351)
The old NativeLibLoader had a short-circuit load path which modified
java.library.path and attempted to load the xgboost library from outside
the jar first, falling back to loading the library from inside the jar.
This path is a no-op every time when using XGBoost outside of it's
source tree. Additionally it triggers an illegal reflective access
warning in the module system in 9, 10, and 11.

On Java 12 the ClassLoader fields are not accessible via reflection
(separately from the illegal reflective acces warning), and so it fails
in a way that isn't caught by the code which falls back to loading the
library from inside the jar.

This commit removes that code path and always loads the xgboost library
from inside the jar file as it's a valid technique across multiple JVM
implementations and works with all versions of Java.
2019-04-10 12:41:44 -07:00
Jean-Francois Zinque
956e73f183 Fix matrix attributes not sliced (#4311) 2019-04-10 11:14:44 -07:00
Jiaming Yuan
5c2575535f
Fix Histogram allocation. (#4347)
* Fix Histogram allocation.

nidx_map is cleared after `Reset`, but histogram data size isn't changed hence
histogram recycling is used in later iterations.  After a reset(building new
tree), newly allocated node will start from 0, while recycling always choose
the node with smallest index, which happens to be our newly allocated node 0.
2019-04-10 19:21:26 +08:00
Rong Ou
81c1cd40ca add a test for cpu predictor using external memory (#4308)
* add a test for cpu predictor using external memory

* allow different page size for testing
2019-04-10 13:25:10 +12:00
James Lamb
b72eab3e07 Added travis logo (#4344) 2019-04-08 21:20:15 -07:00
Mayank Suman
360f25ec27 Added language classifier for python (#4327)
* Added language classifier for python

* Removed python2 language classifier

* Fix formatting
2019-04-08 11:13:26 -07:00