146 Commits

Author SHA1 Message Date
Bobby Wang
1b103e1f5f
[CI] make container be able to re-attached (#7848)
When re-starting the container, it will fail in entrypoint.sh which
will exit when adding an existing group or user
2022-04-29 19:00:35 -07:00
Jiaming Yuan
fd78af404b
Drop support for deprecated CUDA architectures. (#7774)
Co-authored-by: Philip Hyunsu Cho <chohyu01@cs.washington.edu>
2022-03-31 21:42:23 +08:00
Philip Hyunsu Cho
e8eff3581b
[CI] Enable faulthandler to show details when 0xC0000005 error occurs (#7771) (#7775) 2022-03-31 17:40:06 +08:00
Xiaochang Wu
613ec36c5a
Support building SimpleDMatrix from Arrow data format (#7512)
* Integrate with Arrow C data API.
* Support Arrow dataset.
* Support Arrow table.

Co-authored-by: Xiaochang Wu <xiaochang.wu@intel.com>
Co-authored-by: Jiaming Yuan <jm.yuan@outlook.com>
Co-authored-by: Zhang Zhang <zhang.zhang@intel.com>
2022-03-15 13:25:19 +08:00
Philip Hyunsu Cho
1b25dd59f9
Use CUDA 11 in clang-tidy (#7701)
* Show command args when clang-tidy fails

* Add option to specify CUDA args

* Use clang-tidy 11

* [CI] Use CUDA 11
2022-02-24 15:15:07 -08:00
Philip Hyunsu Cho
0149f81a5a
[CI] Fix S3 upload (#7662) 2022-02-16 01:35:27 -08:00
Philip Hyunsu Cho
34a238ca98
[CI] Clean up Python wheel build pipeline (#7626)
* [CI] Always upload artifacts to [branch_name]/

* [CI] Move detailed setup inside build_python_wheels.sh

* Fix typo
2022-02-03 00:55:44 -08:00
Philip Hyunsu Cho
f6e6d0b2c0
[CI] Build Python wheels for MacOS (x86_64 and arm64) (#7621)
* Build Python wheels for OSX (x86_64 and arm64)

* Use Conda's libomp when running Python tests

* fix

* Add comment to explain CIBW_TARGET_OSX_ARM64

* Update release script

* Add comments in build_python_wheels.sh

* Document wheel pipeline
2022-02-02 17:35:48 -08:00
Jiaming Yuan
9f20a3315e
Test with latest numpy. (#7573) 2022-01-19 00:46:23 +08:00
Jiaming Yuan
a1bcd33a3b
[breaking] Change internal model serialization to UBJSON. (#7556)
* Use typed array for models.
* Change the memory snapshot format.
* Add new C API for saving to raw format.
2022-01-16 02:11:53 +08:00
Bobby Wang
e8c1eb99e4
[jvm-package] Clean up the legacy gpu support tests (#7523) 2021-12-21 09:15:51 +08:00
Jiaming Yuan
820e1c01ef
Fix macos package upload. (#7475)
* Split up the tests.
2021-11-24 03:43:49 +08:00
Philip Hyunsu Cho
2adf222fb2
[CI] CI cost saving (#7407)
* [CI] Drop CUDA 10.1; Require 11.0

* Change NCCL version

* Use CUDA 10.1 for clang-tidy, for now

* Remove JDK 11 and 12

* Fix NCCL version

* Don't require 11.0 just yet, until clang-tidy is fixed

* Skip MultiClassesSerializationTest.GpuHist
2021-11-17 21:02:20 -08:00
Jiaming Yuan
3b0b74fa94
[doc] Use RTD theme. (#7346) 2021-10-19 23:49:19 -07:00
Jiaming Yuan
ca17f8a5fc
Dispatch thrust versions and upgrade rmm. (#7254)
Co-authored-by: Philip Hyunsu Cho <chohyu01@cs.washington.edu>
2021-09-25 03:43:23 +08:00
Bobby Wang
0ee11dac77
[jvm-packages][xgboost4j-gpu] Support GPU dataframe and DeviceQuantileDMatrix (#7195)
Following classes are added to support dataframe in java binding:

- `Column` is an abstract type for a single column in tabular data.
- `ColumnBatch` is an abstract type for dataframe.

- `CuDFColumn` is an implementaiton of `Column` that consume cuDF column
- `CudfColumnBatch` is an implementation of `ColumnBatch` that consumes cuDF dataframe.

- `DeviceQuantileDMatrix` is the interface for quantized data.

The Java implementation mimics the Python interface and uses `__cuda_array_interface__` protocol for memory indexing.  One difference is on JVM package, the data batch is staged on the host as java iterators cannot be reset.

Co-authored-by: jiamingy <jm.yuan@outlook.com>
2021-09-24 14:25:00 +08:00
Philip Hyunsu Cho
3060f0b562
[CI] Automatically build GPU-enabled R package for Windows (#7185)
* [CI] Automatically build GPU-enabled R package for Windows

* Update Jenkinsfile-win64

* Build R package for the release branch only

* Update install doc
2021-08-25 02:11:01 -07:00
Philip Hyunsu Cho
d04312b9c0
[CI] Fix hanging Python setup in Windows CI (#7186) 2021-08-24 22:03:51 -07:00
Philip Hyunsu Cho
f1a4a1ac95
[CI] Upgrade build image to CentOS 7 + GCC 8; require CUDA 10.1 and later (#7141) 2021-07-29 10:54:33 -07:00
Jiaming Yuan
345796825f
Optional find dependency in installed cmake config. (#7099)
* Find dependency only when xgboost is built as static library.
* Resolve msvc warning.
* Add test for linking shared library.
2021-07-11 17:20:55 +08:00
Jiaming Yuan
f937f514aa
Remove lz4 compression with external memory. (#7076) 2021-07-06 14:46:43 +08:00
Philip Hyunsu Cho
b2d300e727
[CI] Upgrade to CMake 3.14 (#7060)
* [CI] Upgrade to CMake 3.14

* Add FATAL_ERROR directive, for users with CMake 2.x
2021-06-24 18:07:24 -07:00
Jiaming Yuan
86715e4cd4
Support categorical data for dask functional interface and DQM. (#7043)
* Support categorical data for dask functional interface and DQM.

* Implement categorical data support for GPU GK-merge.
* Add support for dask functional interface.
* Add support for DQM.

* Get newer cupy.
2021-06-18 13:06:52 +08:00
Jiaming Yuan
dcd84b3979
[CI] Configure RAPIDS, dask, modin (#7033)
Co-authored-by: Philip Hyunsu Cho <chohyu01@cs.washington.edu>
2021-06-18 10:27:51 +08:00
Jiaming Yuan
7beb2f7fae
Hide symbols in CI build + hide symbols for C and CUDA (#6798)
* Hide symbols in CI build.
* Hide symbols for other languages.
2021-06-04 02:35:46 +08:00
Philip Hyunsu Cho
05db6a6c29
[CI] Upgrade cuDF and RMM to 21.06 nightly (#7012)
* [CI] Upgrade cuDF and RMM to 21.06 nightly

* Trim outdated test cases

* Pin Dask version to 2021.05.0 for now
2021-06-02 11:59:30 -07:00
Jiaming Yuan
ee4f51a631
Support for all primitive types from array. (#7003)
* Change C API name.
* Test for all primitive types from array.
* Add native support for CPU 128 float.
* Convert boolean and float16 in Python.

* Fix dask version for now.
2021-06-01 08:34:48 +08:00
Jiaming Yuan
29d6a5e2b8
[CI] Move appveyor tests to action (#6986)
* Drop support for VS14, use VS15 instead.
* Drop support for mingw.
* Remove debug build.
* Split up jvm tests.
* Split up Python tests.
2021-05-27 04:49:45 +08:00
Philip Hyunsu Cho
c6d87e5e18
[CI] Remove stray build artifact to avoid error in artifact packaging (#6994) 2021-05-25 19:48:27 +08:00
Philip Hyunsu Cho
90cd724be1
[CI] Fix CI/CD pipeline broken by latest auditwheel (4.0.0) (#6951) 2021-05-10 22:43:15 -07:00
Jiaming Yuan
34df1f588b
Reduce Travis environment setup time. (#6912)
* Remove unused r from travis.
* Don't update homebrew.
* Don't install indirect/unused dependencies like libgit2, wget, openssl.
* Move graphviz installation to conda.
2021-04-30 09:02:40 +08:00
Philip Hyunsu Cho
ea7a6a0321
[CI] Pack R package tarball with pre-built xgboost.so (with GPU support) (#6827)
* Add scripts for packaging R package with GPU-enabled libxgboost.so

* [CI] Automatically build R package tarball

* Add comments

* Don't build tarball for pull requests

* Update the installation doc
2021-04-07 21:15:34 -07:00
Philip Hyunsu Cho
366f3cb9d8
Add use_rmm flag to global configuration (#6656)
* Ensure RMM is 0.18 or later

* Add use_rmm flag to global configuration

* Modify XGBCachingDeviceAllocatorImpl to skip CUB when use_rmm=True

* Update the demo

* [CI] Pin NumPy to 1.19.4, since NumPy 1.19.5 doesn't work with latest Shap
2021-03-09 14:53:05 -08:00
Jiaming Yuan
4656b09d5d
[breaking] Add prediction fucntion for DMatrix and use inplace predict for dask. (#6668)
* Add a new API function for predicting on `DMatrix`.  This function aligns
with rest of the `XGBoosterPredictFrom*` functions on semantic of function
arguments.
* Purge `ntree_limit` from libxgboost, use iteration instead.
* [dask] Use `inplace_predict` by default for dask sklearn models.
* [dask] Run prediction shape inference on worker instead of client.

The breaking change is in the Python sklearn `apply` function, I made it to be
consistent with other prediction functions where `best_iteration` is used by
default.
2021-02-08 18:26:32 +08:00
Philip Hyunsu Cho
55ee2bd77f
[CI] Add ARM64 test to Jenkins pipeline (#6643)
* Add ARM64 test to Jenkins pipeline

* Check for bundled libgomp

* Use a separate test suite for ARM64

* Ensure that x86 jobs don't run on ARM workers
2021-01-27 21:51:17 +09:00
Jiaming Yuan
610ee632cc
[Breaking] Rename data to X in predict_proba. (#6555)
New Scikit-Learn version uses keyword argument, and `X` is the predefined
keyword.

* Use pip to install latest Python graphviz on Windows CI.
2020-12-28 21:36:03 +08:00
Jiaming Yuan
cb207a355d
Add script for generating release tarball. (#6544) 2020-12-23 16:08:10 +08:00
Philip Hyunsu Cho
bf6cfe3b99
[Breaking] Upgrade cuDF and RMM to 0.18 nightlies; require RMM 0.18+ for RMM plugin (#6510)
* [CI] Upgrade cuDF and RMM to 0.18 nightlies

* Modify RMM plugin to be compatible with RMM 0.18

* Update src/common/device_helpers.cuh

Co-authored-by: Mark Harris <mharris@nvidia.com>

Co-authored-by: Mark Harris <mharris@nvidia.com>
2020-12-16 10:07:52 -08:00
Philip Hyunsu Cho
b8044e6136
[CI] Use manylinux2010_x86_64 container to vendor libgomp (#6485) 2020-12-10 07:37:15 -08:00
Philip Hyunsu Cho
05e5563c2c
[CI] Fix CentOS 6 Docker images (#6467) 2020-12-04 21:33:11 -08:00
Philip Hyunsu Cho
84b726ef53
Vendor libgomp in the manylinux Python wheel (#6461)
* Vendor libgomp in the manylinux2014_aarch64 wheel

* Use vault repo, since CentOS 6 has reached End-of-Life on Nov 30

* Vendor libgomp in the manylinux2010_x86_64 wheel

* Run verification step inside the container
2020-12-03 19:55:32 -08:00
Philip Hyunsu Cho
4dbbeb635d
[CI] Upgrade cuDF and RMM to 0.17 nightlies (#6434) 2020-11-24 13:21:41 -08:00
Philip Hyunsu Cho
7f101d1b33
[CI] Remove R check from Jenkins (#6372)
* Remove R check from Jenkins

* Print stacktrace when CRAN test fail in GitHub Actions

* Add verbose flag in tests/ci_build/print_r_stacktrace.sh

* Fix path in tests/ci_build/print_r_stacktrace.sh
2020-11-10 22:46:54 -08:00
Philip Hyunsu Cho
f6169c0b16
[CI] Use separate Docker cache for each CUDA version (#6305) 2020-10-28 11:07:00 -07:00
James Lamb
e1de390e6e
[ci] replace 'egrep' with 'grep -E' (#6287) 2020-10-27 12:05:48 -07:00
Jiaming Yuan
b5c2a47b20
Drop single point model recovery (#6262)
* Pass rabit params in JVM package.
* Implement timeout using poll timeout parameter.
* Remove OOB data check.
2020-10-21 15:27:03 +08:00
Jiaming Yuan
81c37c28d5
Time the CPU tests on Jenkins. (#6257)
* Time the CPU tests on Jenkins.
* Reduce thread contention.
* Add doc.
* Skip heavy tests on ARM.
2020-10-20 17:19:07 -07:00
Philip Hyunsu Cho
7f6ed5780c
[CI] Build a Python wheel for aarch64 platform (#6253) 2020-10-18 22:35:19 -07:00
Philip Hyunsu Cho
c991eb612d
[jvm-packages] Fix up build for xgboost4j-gpu, xgboost4j-spark-gpu (#6216)
* [CI] Clean up build for JVM packages

* Use correct path for saving native lib

* Fix groupId of maven-surefire-plugin

* Fix stashing of xgboost4j_jar_gpu

* [CI] Don't run xgboost4j-tester with GPU, since it doesn't use gpu_hist
2020-10-09 14:08:15 -07:00
Philip Hyunsu Cho
f121f2738f
[CI] Fix Docker build for CUDA 11 (#6202) 2020-10-05 17:54:14 -07:00