0715ab3c10
This PR adds optional support for loading nccl with `dlopen` as an alternative of compile time linking. This is to address the size bloat issue with the PyPI binary release. - Add CMake option to load `nccl` at runtime. - Add an NCCL stub. After this, `nccl` will be fetched from PyPI when using pip to install XGBoost, either by a user or by `pyproject.toml`. Others who want to link the nccl at compile time can continue to do so without any change. At the moment, this is Linux only since we only support MNMG on Linux.
XGBoost Plugin for Federated Learning
This folder contains the plugin for federated learning. Follow these steps to build and test it.
Install gRPC
Refer to the installation guide from the gRPC website.
Build the Plugin
# Under xgboost source tree.
mkdir build
cd build
cmake .. -GNinja \
-DPLUGIN_FEDERATED=ON \
-DUSE_CUDA=ON\
-DUSE_NCCL=ON
ninja
cd ../python-package
pip install -e .
If CMake fails to locate gRPC, you may need to pass -DCMAKE_PREFIX_PATH=<grpc path> to CMake.
Test Federated XGBoost
# Under xgboost source tree.
cd tests/distributed
# This tests both CPU training (`hist`) and GPU training (`gpu_hist`).
./runtests-federated.sh