* [CI] Migrate to rockylinux8 / manylinux_2_28_x86_64
* Scrub all references to CentOS 7
* Fix
* Remove use of yum
* Use gcc-10 in cpu
* Temporarily disable -Werror
* Use GCC 9 for now
* Roll back gRPC
* Scrub all references to manylinux2014_x86_64
* Revise rename_whl.py to handle no-op rename
* Change JDK_VERSION back to 8
* Reviewer's comment
* Use GCC 10
* Use Spark 3.5.1, same as in pom.xml
* Fix JAR install
* [CI] Add nightly CI job to test against dev version of deps
* Update build-containers.sh
* Add build step
* Wait for build artifact
* Try pinning dask
* Address reviewers' comments
* Fix unbound variable error
* Specify dev version exactly
* Pin dask=2024.1.1
* [CI] add script to generate meta info and upload to s3
* Write Python script to generate meta.json
* Update other pipelines
* Add wheel_name field
* Add description
---------
Co-authored-by: Hyunsu Cho <phcho@nvidia.com>
This PR adds optional support for loading nccl with `dlopen` as an alternative of compile time linking. This is to address the size bloat issue with the PyPI binary release.
- Add CMake option to load `nccl` at runtime.
- Add an NCCL stub.
After this, `nccl` will be fetched from PyPI when using pip to install XGBoost, either by a user or by `pyproject.toml`. Others who want to link the nccl at compile time can continue to do so without any change.
At the moment, this is Linux only since we only support MNMG on Linux.
* Create pyproject.toml
* Implement a custom build backend (see below) in packager directory. Build logic from setup.py has been refactored and migrated into the new backend.
* Tested: pip wheel . (build wheel), python -m build --sdist . (source distribution)
* Update to C++17
* Turn off unity build
* Update CMake to 3.18
* Use MSVC 2022 + CUDA 11.8
* Re-create stack for worker images
* Allocate more disk space for Windows
* Tempiorarily disable clang-tidy
* RAPIDS now requires Python 3.10+
* Unpin cuda-python
* Use latest NCCL
* Use Ubuntu 20.04 in RMM image
* Mark failing mgpu test as xfail
* Fix quantile tests running on multi-gpus
* Run some gtests with multiple GPUs
* fix mgpu test naming
* Instruct NCCL to print extra logs
* Allocate extra space in /dev/shm to enable NCCL
* use gtest_skip to skip mgpu tests
---------
Co-authored-by: Hyunsu Philip Cho <chohyu01@cs.washington.edu>
* [jvm-packages] Bump rapids version to 22.12.0
This PR bumps spark version to 3.1.1 and the rapids version
to 22.12.0, which results in the latest xgboost can't run
with the old rapids packages.
* Reduce clutter in log of Python test
* Set up BuildKite test analytics
* Add separate step for building containers
* Enable incremental update of CI stack; custom agent IAM policy
Hyunsu Cho
Bobby WangandHyunsu Cho
dependabot[bot]
Boris
Rong OuandHyunsu Philip Cho