Commit Graph
288 Commits
Author SHA1 Message Date
Dmitry RazdoburdinandDmitry Razdoburdin <> ce97de2a7c replace channel for sycl dependencies (#10576)
Co-authored-by: Dmitry Razdoburdin <>
2024-07-12 18:28:54 +08:00
Jiaming Yuan 00264eb72b [EM] Basic distributed test for external memory. (#10492) 2024-07-06 01:15:20 +08:00
Jiaming Yuan a39fef2c67 [fed] Fixes for the encrypted GRPC backend. (#10503) 2024-07-02 15:15:12 +08:00
Philip Hyunsu Cho 4b88dfff24 [CI] Temporarily pin pylint to 3.2.3 (#10494)
* [CI] Temporarily pin pylint to 3.2.3

* Add quotes

* Correct env
2024-06-26 14:08:49 -07:00
Philip Hyunsu Cho 08658b124d [CI] Add CI pipeline to build libxgboost4j.so targeting Linux ARM64 (#10487) 2024-06-26 01:43:15 -07:00
Philip Hyunsu Cho d4dee25eb3 [CI] Set up pipeline to build manylinux2014 wheels (#10478) 2024-06-24 12:25:26 -07:00
Philip Hyunsu Cho 8689f0b562 [CI] Stop vendoring libomp.dylib in MacOS Python wheels (#10440) 2024-06-18 19:17:02 -07:00
Philip Hyunsu Cho bc3747bdce [CI] Migrate to rockylinux8 / manylinux_2_28_x86_64 (#10399)
* [CI] Migrate to rockylinux8 / manylinux_2_28_x86_64

* Scrub all references to CentOS 7

* Fix

* Remove use of yum

* Use gcc-10 in cpu

* Temporarily disable -Werror

* Use GCC 9 for now

* Roll back gRPC

* Scrub all references to manylinux2014_x86_64

* Revise rename_whl.py to handle no-op rename

* Change JDK_VERSION back to 8

* Reviewer's comment

* Use GCC 10

* Use Spark 3.5.1, same as in pom.xml

* Fix JAR install
2024-06-17 12:07:49 -07:00
Richard (Rick) ZamoraandJiaming Yuan dc14f98f40 Avoid default tokenization in Dask (#10398)
---------

Co-authored-by: Jiaming Yuan <jm.yuan@outlook.com>
2024-06-14 19:44:54 +08:00
Philip Hyunsu Cho 4057f861c1 [CI] Add nightly CI job to test against dev version of deps (#10351)
* [CI] Add nightly CI job to test against dev version of deps

* Update build-containers.sh

* Add build step

* Wait for build artifact

* Try pinning dask

* Address reviewers' comments

* Fix unbound variable error

* Specify dev version exactly

* Pin dask=2024.1.1
2024-06-03 19:28:55 -07:00
Jiaming Yuan e6eefea5e2 [coll] Move the rabit poll helper. (#10349) 2024-05-31 08:02:21 +08:00
Jiaming YuanandPhilip Hyunsu Cho 7354955cbb Test federated plugin using GitHub action. (#10336)
Co-authored-by: Philip Hyunsu Cho <chohyu01@cs.washington.edu>
2024-05-29 02:28:14 +08:00
7ae5c972f9 [CI] Upgrade github workflows to use latest Conda setup action (#10320)
Co-authored-by: Christian Clauss <cclauss@me.com>
Co-authored-by: Jiaming Yuan <jm.yuan@outlook.com>
2024-05-28 10:23:07 -07:00
Bobby WangandHyunsu Cho 9def441e9a [CI] add script to generate meta info and upload to s3 (#10295)
* [CI] add script to generate meta info and upload to s3

* Write Python script to generate meta.json

* Update other pipelines

* Add wheel_name field

* Add description

---------

Co-authored-by: Hyunsu Cho <phcho@nvidia.com>
2024-05-24 10:03:28 -07:00
Jiaming Yuan a5a58102e5 Revamp the rabit implementation. (#10112)
This PR replaces the original RABIT implementation with a new one, which has already been partially merged into XGBoost. The new one features:
- Federated learning for both CPU and GPU.
- NCCL.
- More data types.
- A unified interface for all the underlying implementations.
- Improved timeout handling for both tracker and workers.
- Exhausted tests with metrics (fixed a couple of bugs along the way).
- A reusable tracker for Python and JVM packages.
2024-05-20 11:56:23 +08:00
Philip Hyunsu ChoandJiaming Yuan edb945d59b [CI] Use native arm64 worker in GHAction to build M1 wheel (#10225)
* [CI] Use native arm64 worker in GHAction to build M1 wheel

* Set up Conda

* Use mamba

* debug

* fix

* fix

* fix

* fix

* fix

* Temporarily disable other tests

* Fix prefix

* Use micromamba

* Use conda-incubator/setup-miniconda

* Use mambaforge

* Fix

* Fix prefix

* Don't use deprecated set-output

* Add verbose output from build

* verbose

* Specify arch

* Bump setup-miniconda to v3

* Use Python 3.9

* Restore deleted files

* WAR.

---------

Co-authored-by: Jiaming Yuan <jm.yuan@outlook.com>
2024-04-26 10:16:55 -07:00
Dmitry RazdoburdinandDmitry Razdoburdin <> 58513dc288 [SYCL] Add sampling initialization (#10216)
---------

Co-authored-by: Dmitry Razdoburdin <>
2024-04-25 04:35:52 +08:00
Jiaming Yuan 59d7b8dc72 [doc] Add typing to dask demos. (#10207) 2024-04-23 00:57:05 +08:00
Philip Hyunsu Cho 32be4669fb [jvm-packages] Ombinus patch to update all minor dependencies (#10188)
* Fold in #10184

* Fold in #10176

* Fold in #10168

* Fold in #10165

* Fold in #10164

* Fold in #10155

* Fold in #10062

* Fold in #9984

* Fold in #9843

* Upgrade to Maven 3.6.3
2024-04-16 19:43:04 -07:00
2925cebdca [CI] Use latest RAPIDS; Pandas 2.0 compatibility fix (#10175)
* [CI] Update RAPIDS to latest stable

* [CI] Use rapidsai stable channel; fix syntax errors in Dockerfile.gpu

* Don't combine astype() with loc()

* Work around https://github.com/dmlc/xgboost/issues/10181

* Fix formatting

* Fix test

---------

Co-authored-by: hcho3 <hcho3@users.noreply.github.com>
Co-authored-by: Hyunsu Cho <chohyu01@cs.washington.edu>
2024-04-15 13:38:53 -07:00
Jiaming Yuan ca4801f81d Work with IPv6 in the new tracker. (#10125) 2024-03-20 05:19:23 +08:00
Philip Hyunsu Cho 23a37dcaf9 [CI] Test R package with CMake (#10087)
* [CI] Test R package with CMake

* Fix

* Fix

* Update test_r_package.py

* Fix CMake flag for R package

* Install system deps

* Fix

* Use sudo
2024-03-04 12:32:44 -08:00
Jiaming Yuan 8ea705e4d5 Support sample weight in sklearn custom objective. (#10050) 2024-02-21 00:43:14 +08:00
Philip Hyunsu ChoandJiaming Yuan 4dfbe2a893 [CI] Test building for 32-bit arch (#10021)
* [CI] Test building for 32-bit arch

* Update CMakeLists.txt

* Fix yaml

* Use Debian container

* Remove -Werror for 32-bit

* Revert "Remove -Werror for 32-bit"

This reverts commit c652bc6a037361bcceaf56fb01863210b462793d.

* Don't error for overloaded-virtual warning

* Ignore some warnings from dmlc-core

* Fix compiler warnings

* Fix formatting

* Apply suggestions from code review

Co-authored-by: Jiaming Yuan <jm.yuan@outlook.com>

* Add more cast

---------

Co-authored-by: Jiaming Yuan <jm.yuan@outlook.com>
2024-01-31 13:20:51 -08:00
Jiaming Yuan 54b71c8fba Fix with black 24.1.1. (#10014) 2024-01-30 17:24:11 +08:00
Jiaming Yuan d12cc1090a Refactor tests for training continuation. (#9997) 2024-01-24 16:07:19 +08:00
Philip Hyunsu Cho 2c8fa8b8b9 [CI] Skip MSVC when building R package (#9995)
* [CI] Skip MSVC when building R package

* [CI] Stop building binary tarball for Windows

* Remove unused script
2024-01-18 08:09:53 -08:00
Jiaming Yuan 0798e36d73 [breaking] Remove deprecated parameters in the skl interface. (#9986) 2024-01-15 20:40:05 +08:00
Philip Hyunsu Cho 1168a68872 [jvm-packages] Update release scripts (#9983)
* [jvm-packages] Add Scala version suffix to xgboost-jvm package (#9776)

* Update JVM script (#9714)

* Revamp pom.xml

* Update instructions in prepare_jvm_release.py

* Fix formatting

* [jvm-packages] Fix POM for xgboost-jvm metapackage (#9893)

* [jvm-packages] Fix POM for xgboost-jvm metapackage

* Add script for updating the Scala version

* Update change_scala_version.py to also change scala.version property (#9897)

* Remove 'release-cpu-only' profile

* Remove scala-2.13 profile; enable gpu package for Scala 2.13
2024-01-12 10:37:55 -08:00
Jiaming Yuan b3eb5d0945 Use UBJ in Python checkpoint. (#9958) 2024-01-09 03:22:15 +08:00
Jiaming Yuan 38dd91f491 Save model in ubj as the default. (#9947) 2024-01-05 17:53:36 +08:00
Jiaming YuanandPhilip Hyunsu Cho 621348abb3 Fix multi-output with alternating strategies. (#9933)
---------

Co-authored-by: Philip Hyunsu Cho <chohyu01@cs.washington.edu>
2024-01-04 16:41:13 +08:00
Jiaming Yuan 9f73127a23 Cleanup Python GPU tests. (#9934)
* Cleanup Python GPU tests.

- Remove the use of `gpu_hist` and `gpu_id` in cudf/cupy tests.
- Move base margin test into the testing directory.
2024-01-04 13:15:18 +08:00
Philip Hyunsu Cho ef8bdaa047 [CI] Update machine images (#9932) 2023-12-29 11:15:38 -08:00
Jiaming Yuan 4bc1f3a388 [R] Bump requirement to 4.3.0. (#9847) 2023-12-07 00:12:45 +08:00
381f1d3dc9 Add support inference on SYCL devices (#9800)
---------

Co-authored-by: Dmitry Razdoburdin <>
Co-authored-by: Nikolay Petrov <nikolay.a.petrov@intel.com>
Co-authored-by: Alexandra <alexandra.epanchinzeva@intel.com>
2023-12-04 16:15:57 +08:00
Jiaming Yuan e78b46046e [CI] Update R version on Linux. (#9835) 2023-12-02 11:03:17 +08:00
Jiaming Yuan 0715ab3c10 Use dlopen to load NCCL. (#9796)
This PR adds optional support for loading nccl with `dlopen` as an alternative of compile time linking. This is to address the size bloat issue with the PyPI binary release.
- Add CMake option to load `nccl` at runtime.
- Add an NCCL stub.

After this, `nccl` will be fetched from PyPI when using pip to install XGBoost, either by a user or by `pyproject.toml`. Others who want to link the nccl at compile time can continue to do so without any change.

At the moment, this is Linux only since we only support MNMG on Linux.
2023-11-22 19:27:31 +08:00
Jiaming Yuan 82828621d0 [doc] Add doc for linters and simplify c++ lint script. (#9750) 2023-11-07 05:03:30 +08:00
Jiaming Yuan 3ca06ac51e [doc] Mention data consistency for categorical features. (#9678) 2023-10-24 10:11:33 +08:00
Philip Hyunsu Cho 5e6cb63a56 [CI] Set up CI for Mac M1 (#9699) 2023-10-22 23:33:19 -07:00
Rong Ou 6fbe6248f4 More in-memory input support for column split (#9685) 2023-10-20 16:02:36 +08:00
Chuck Atkins 83cdf14b2c CMake LTO and CUDA arch (#9677) 2023-10-20 13:01:37 +08:00
Philip Hyunsu Cho a5e07a01f8 [CI] Pull CentOS 7 images from NGC (#9666) 2023-10-13 12:11:54 +08:00
d1dee4ad99 [CI] Update RAPIDS to latest stable (#9654)
* [CI] Update RAPIDS to latest stable

* Remove slashes from Docker tag

---------

Co-authored-by: hcho3 <hcho3@users.noreply.github.com>
Co-authored-by: Hyunsu Cho <chohyu01@cs.washington.edu>
2023-10-11 23:26:09 -07:00
James LambandJiaming Yuan 51e32e4905 [CI] add cmakelint to C++ linting task (#9641)
Co-authored-by: Jiaming Yuan <jm.yuan@outlook.com>
2023-10-11 16:04:10 +08:00
Jiaming Yuan 4e5a7729c3 Fix lint errors. (#9634) 2023-10-09 19:04:31 +08:00
James Lamb 799f8485e2 [R] [CI] enforce lintr::function_left_parentheses_linter check (#9631) 2023-10-08 09:42:09 +08:00
Jiaming Yuan cac2cd2e94 [R] Set number of threads in demos and tests. (#9591)
- Restrict the number of threads in IO.
- Specify the number of threads in demos and tests.
- Add helper scripts for checks.
2023-09-23 21:44:03 +08:00
Jiaming YuanandPhilip Hyunsu Cho e6cf7a1278 Deprecate the command line interface. (#9485)
---------

Co-authored-by: Philip Hyunsu Cho <chohyu01@cs.washington.edu>
2023-08-21 06:47:48 +08:00