Hendrik/xgboost
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
6,443 Commits 32 Branches 59 Tags
2a0ff209ff4ffd1ccadb00674d6c75a7ff223b74
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Rong Ou 3632242e0b Support column split with GPU quantile (#9370) 2023-07-11 12:15:56 +08:00
Rong Ou 15ca12a77e Fix NCCL test hang (#9367) 2023-07-07 11:21:35 +08:00
Rong Ou 3a0f787703 Support column split in GPU predictor (#9343) 2023-07-03 04:05:34 +08:00
Rong Ou f90771eec6 Fix device communicator dependency (#9346) 2023-06-29 10:34:30 +08:00
Rong Ou d8beb517ed Support bitwise allreduce in NCCL communicator (#9300) 2023-06-17 01:56:50 +08:00
Rong Ou e70810be8a Refactor device communicator to make allreduce more flexible (#9295) 2023-06-14 03:53:03 +08:00
Rong Ou ff122d61ff More tests for cpu predictor with column split (#9270) 2023-06-08 22:47:19 +08:00
Rong Ou 962a20693f More support for column split in cpu predictor (#9244) 
- Added column split support to `PredictInstance` and `PredictLeaf`.
- Refactoring of tests.
 2023-06-05 08:05:38 +08:00
Rong Ou 5b69534b43 Support column split in multi-target hist (#9171) 2023-05-26 16:56:05 +08:00
Rong Ou acd363033e Fix running MGPU gtests (#9200) 2023-05-26 05:26:38 +08:00
Rong Ou 603f8ce2fa Support hist in the partition builder under column split (#9120) 2023-05-11 05:24:29 +08:00
Rong Ou 52311dcec9 Fix multi-threaded gtests (#9148) 2023-05-10 19:15:32 +08:00
Rong Ou 250b22dd22 Fix nvflare horizontal demo (#9124) 2023-05-05 16:48:22 +08:00
Rong Ou fb941262b4 Add demo for vertical federated learning (#9103) 2023-04-28 16:03:21 +08:00
Rong Ou 511d4996b5 Rely on gRPC to generate random port (#9102) 2023-04-27 09:48:26 +08:00
Rong Ou a320b402a5 More refactoring to take advantage of collective aggregators (#9081) 2023-04-26 03:36:09 +08:00
Rong Ou 8dbe0510de More collective aggregators (#9060) 2023-04-22 03:32:05 +08:00
Rong Ou 42d100de18 Make sure metrics work with federated learning (#9037) 2023-04-19 15:39:11 +08:00
Rong Ou ba9d24ff7b Make sure metrics work with column-wise distributed training (#9020) 2023-04-18 03:48:23 +08:00
Rong Ou 15e073ca9d Make objectives work with vertical distributed and federated learning (#9002) 2023-04-03 17:07:42 +08:00
Rong Ou d385cc64e2 Fix aft_loss_distribution documentation (#8995) 2023-03-29 19:13:23 -07:00
Rong Ou ff26cd3212 More tests for column split and vertical federated learning (#8985) 
Added some more tests for the learner and fit_stump, for both column-wise distributed learning and vertical federated learning.

Also moved the `IsRowSplit` and `IsColumnSplit` methods from the `DMatrix` to the `MetaInfo` since in some places we only have access to the `MetaInfo`. Added a new convenience method `IsVerticalFederatedLearning`.

Some refactoring of the testing fixtures.
 2023-03-28 16:40:26 +08:00
Rong Ou b240f055d3 Support vertical federated learning (#8932) 2023-03-22 14:25:26 +08:00
Rong Ou 79efcd37f5 Pick up dmlc-core fix for CSV parser (#8897) 2023-03-11 04:51:43 +08:00
Rong OuHyunsu Philip ChoJiaming Yuan
 		2dc22e7aad Take advantage of C++17 features (#8858) 
---------

Co-authored-by: Hyunsu Philip Cho <chohyu01@cs.washington.edu>
Co-authored-by: Jiaming Yuan <jm.yuan@outlook.com>
 2023-03-04 00:24:13 +08:00
Rong Ou a5852365fd Update dmlc-core to get C++17 deprecation warning (#8855) 2023-03-01 12:30:59 -08:00
Rong Ou 7cbaee9916 Support column split in approx tree method (#8847) 2023-03-02 03:59:07 +08:00
Rong Ou d9688f93c7 Support column-split in row partitioner (#8828) 2023-02-26 04:43:35 +08:00
Rong Ou a65ad0bd9c Support column split in histogram builder (#8811) 2023-02-17 22:37:01 +08:00
Rong Ou 74572b5d45 Add convenience method for allgather (#8804) 2023-02-15 11:37:11 +08:00
Rong OuandHyunsu Philip Cho ed91e775ec Fix quantile tests running on multi-gpus (#8775) 
* Fix quantile tests running on multi-gpus

* Run some gtests with multiple GPUs

* fix mgpu test naming

* Instruct NCCL to print extra logs

* Allocate extra space in /dev/shm to enable NCCL

* use gtest_skip to skip mgpu tests

---------

Co-authored-by: Hyunsu Philip Cho <chohyu01@cs.washington.edu>
 2023-02-12 17:00:26 -08:00
Rong Ou cbf98cb9c6 Add Allgather to collective communicator (#8765) 
* Add Allgather to collective communicator
 2023-02-09 11:31:22 +08:00
Rong Ou 66191e9926 Support cpu quantile sketch with column-wise data split (#8742) 2023-02-05 14:26:24 +08:00
Rong Ou 8af98e30fc Use in-memory communicator to test quantile (#8710) 2023-01-27 23:28:28 +08:00
Rong Ou 78396f8a6e Initial support for column-split cpu predictor (#8676) 2023-01-18 06:33:13 +08:00
Rong Ou 3ceeb8c61c Add data split mode to DMatrix MetaInfo (#8568) 2022-12-25 20:37:37 +08:00
Rong Ou 77b069c25d Support bitwise allreduce operations in the communicator (#8623) 2022-12-25 06:40:05 +08:00
Rong Ou 15a88ceef0 Fix deprecated CUB calls in CUDA 12.0 (#8578) 2022-12-12 17:02:30 +08:00
Rong Ou 42e6fbb0db Fix sklearn test that calls a removed field (#8579) 2022-12-09 13:06:44 -08:00
Rong Ou 0caf2be684 Update NVFlare demo to work with the latest release (#8576) 2022-12-09 02:48:20 +08:00
Rong Ou 78d65a1928 Initial support for column-wise data split (#8468) 2022-12-04 01:37:51 +08:00
Rong Ou a8255ea678 Add an in-memory collective communicator (#8494) 2022-12-01 00:24:12 +08:00
Rong Ou 30b1a26fc0 Remove unused page size constant (#8457) 2022-11-17 11:41:39 +08:00
Rong Ou 4449e30184 Always link federated proto statically (#8442) 2022-11-09 07:47:38 +08:00
Rong Ou 8e76f5f595 Use DataSplitMode to configure data loading (#8434) 
* Use `DataSplitMode` to configure data loading
 2022-11-08 16:21:50 +08:00
Rong OuandHyunsu Philip Cho 99fa8dad2d Add back xgboost.rabit for backwards compatibility (#8408) 
* Add back xgboost.rabit for backwards compatibility

* fix my errors

* Fix lint

* Use FutureWarning

Co-authored-by: Hyunsu Philip Cho <chohyu01@cs.washington.edu>
 2022-11-01 21:47:41 -07:00
Rong Ou 521086d56b Make federated client more robust (#8351) 2022-10-18 13:52:44 +08:00
Rong OuandHyunsu Philip Cho 8f3dee58be Speed up tests with federated learning enabled (#8350) 
* Speed up tests with federated learning enabled

* Re-enable timeouts

Co-authored-by: Hyunsu Philip Cho <chohyu01@cs.washington.edu>
 2022-10-17 15:17:04 -07:00
Rong Ou 80e10e02ab Avoid blank lines with federated training (#8342) 2022-10-14 14:55:01 +08:00
Rong Ou b3208aac4e Fix NVFLARE demo (#8340) 2022-10-14 12:18:34 +08:00
Rong Ouandjiamingy 39afdac3be Better error message when world size and rank are set as strings (#8316) 
Co-authored-by: jiamingy <jm.yuan@outlook.com>
 2022-10-12 15:53:25 +08:00
Rong OuandHyunsu Philip Cho 8d4038da57 Don't split input data in federated mode (#8279) 
Co-authored-by: Hyunsu Philip Cho <chohyu01@cs.washington.edu>
 2022-10-05 18:19:28 -08:00
Rong OuandHyunsu Philip Cho 668b8a0ea4 [Breaking] Switch from rabit to the collective communicator (#8257) 
* Switch from rabit to the collective communicator

* fix size_t specialization

* really fix size_t

* try again

* add include

* more include

* fix lint errors

* remove rabit includes

* fix pylint error

* return dict from communicator context

* fix communicator shutdown

* fix dask test

* reset communicator mocklist

* fix distributed tests

* do not save device communicator

* fix jvm gpu tests

* add python test for federated communicator

* Update gputreeshap submodule

Co-authored-by: Hyunsu Philip Cho <chohyu01@cs.washington.edu>
 2022-10-05 14:39:01 -08:00
Rong Ou 7d43e74e71 JNI wrapper for the collective communicator (#8242) 2022-09-21 04:20:25 +08:00
Rong OuandHyunsu Cho a2686543a9 Common interface for collective communication (#8057) 
* implement broadcast for federated communicator

* implement allreduce

* add communicator factory

* add device adapter

* add device communicator to factory

* add rabit communicator

* add rabit communicator to the factory

* add nccl device communicator

* add synchronize to device communicator

* add back print and getprocessorname

* add python wrapper and c api

* clean up types

* fix non-gpu build

* try to fix ci

* fix std::size_t

* portable string compare ignore case

* c style size_t

* fix lint errors

* cross platform setenv

* fix memory leak

* fix lint errors

* address review feedback

* add python test for rabit communicator

* fix failing gtest

* use json to configure communicators

* fix lint error

* get rid of factories

* fix cpu build

* fix include

* fix python import

* don't export collective.py yet

* skip collective communicator pytest on windows

* add review feedback

* update documentation

* remove mpi communicator type

* fix tests

* shutdown the communicator separately

Co-authored-by: Hyunsu Cho <chohyu01@cs.washington.edu>
 2022-09-12 15:21:12 -07:00
Rong Ou b78bc734d9 Fix dask.py lint error (#8216) 2022-09-02 16:30:01 +08:00
Rong Ou d6e2013c5f Set max message size in insecure gRPC (#8203) 2022-08-26 16:33:51 +08:00
Rong Ou ad3bc0edee Allow insecure gRPC connections for federated learning (#8181) 
* Allow insecure gRPC connections for federated learning

* format
 2022-08-19 12:16:14 +08:00
Rong Ou 7a6b711eb8 Remove unused updater basemaker (#8091) 2022-07-19 15:41:27 +08:00
Rong Ou 6eb23353d7 Update nvflare demo for release 2.1.2 (#8038) 2022-06-29 17:58:06 +08:00
Rong Ou 45dc1f818a Make federated plugin work with cmake 3.16.3 (#8029) 2022-06-27 17:26:41 +08:00
Rong Ou 0725fd6081 fix federated learning plugin (#8027) 2022-06-24 08:41:07 +08:00
Rong Ou e5ec546da5 [Breaking] Remove rabit support for custom reductions and grow_local_histmaker updater (#7992) 2022-06-21 15:08:23 +08:00
Rong Ou 31e6902e43 Support GPU training in the NVFlare demo (#7965) 2022-06-02 21:52:36 +08:00
Rong Ou d3429f2ff6 Increase gRPC max receive message size for federated learning (#7958) 2022-06-01 13:21:54 +08:00
Rong Ou 80339c3427 Enable distributed GPU training over Rabit (#7930) 2022-05-31 04:09:45 +08:00
Rong Ou 77d4a53c32 use RabitContext intead of init/finalize (#7911) 2022-05-17 12:15:41 +08:00
Rong Ouandjiamingy af907e2d0d Demo of federated learning using NVFlare (#7879) 
Co-authored-by: jiamingy <jm.yuan@outlook.com>
 2022-05-14 22:45:41 +08:00
Rong Ou 14ef38b834 Initial support for federated learning (#7831) 
Federated learning plugin for xgboost:
* A gRPC server to aggregate MPI-style requests (allgather, allreduce, broadcast) from federated workers.
* A Rabit engine for the federated environment.
* Integration test to simulate federated learning.

Additional followups are needed to address GPU support, better security, and privacy, etc.
 2022-05-05 21:49:22 +08:00
Rong Ou 06320729d4 fix device sketch with weights in external memory mode (#5870) 2020-07-08 08:44:07 +08:00
Rong Ou e21a608552 add pointers to the gpu external memory paper (#5684) 2020-05-19 19:46:16 -07:00
Rong Ou a1085396e2 add reference to gpu external memory (#5490) 2020-04-07 11:15:58 +12:00
Rong Ou d6b31df449 update docs for gpu external memory (#5332) 
* update docs for gpu external memory

* add hist limitation
 2020-02-22 14:57:40 +08:00
Rong Ou e4b74c4d22 Gradient based sampling for GPU Hist (#5093) 
* Implement gradient based sampling for GPU Hist tree method.
* Add samplers and handle compacted page in GPU Hist.
 2020-02-04 10:31:27 +08:00
Rong Ou 0afcc55d98 Support multiple batches in gpu_hist (#5014) 
* Initial external memory training support for GPU Hist tree method.
 2019-11-16 14:50:20 +08:00
Rong Ou 5b1715d97c Write ELLPACK pages to disk (#4879) 
* add ellpack source
* add batch param
* extract function to parse cache info
* construct ellpack info separately
* push batch to ellpack page
* write ellpack page.
* make sparse page source reusable
 2019-10-22 23:44:32 -04:00
Rong Ou 562bb0ae31 remove device shards (#4867) 2019-09-25 13:15:46 +08:00
Rong Ou 006eb80578 ignore vscode and clion files (#4866) 
* ignore vscode and clion files

* ignore all .idea directories
 2019-09-17 21:27:40 -04:00
Rong Ou 125bcec62e Move ellpack page construction into DMatrix (#4833) 2019-09-16 23:50:55 -04:00
Rong Ou 733ed24dd9 further cleanup of single process multi-GPU code (#4810) 
* use subspan in gpu predictor instead of copying
* Revise `HostDeviceVector`
 2019-08-30 05:27:23 -04:00
Rong Ou 38ab79f889 Make HostDeviceVector single gpu only (#4773) 
* Make HostDeviceVector single gpu only
 2019-08-26 09:51:13 +12:00
Rong Ou c5b229632d [BREAKING] prevent multi-gpu usage (#4749) 
* prevent multi-gpu usage

* fix distributed test

* combine gpu predictor tests

* set upper bound on n_gpus
 2019-08-13 09:11:35 +12:00
Rong Ou 19f9fd5de9 remove the qids_ field in MetaInfo (#4744) 2019-08-08 10:01:59 +08:00
Rong Ou 602484e19f Remove some unused functions as reported by cppcheck (#4743) 2019-08-07 02:42:33 -04:00
Rong Ou 851b5b3808 Remove gpu_exact tree method (#4742) 2019-08-07 11:43:20 +12:00
Rong Ou 6edddd7966 Refactor DMatrix to return batches of different page types (#4686) 
* Use explicit template parameter for specifying page type.
 2019-08-03 15:10:34 -04:00
Rong Ou cb9a80ca90 Update dmlc-core (#4726) 2019-08-02 03:54:14 -04:00
Rong Ou 9b9e298ff2 remove RowSet which is no longer being used (#4697) 2019-07-25 17:25:58 -07:00
Rong Ou 30204b50fe fix spark tests on machines with many cores (#4634) 2019-07-07 16:02:56 -07:00
Rong Ou 63ec95623d fix gpu predictor when dmatrix is mismatched with model (#4613) 2019-06-28 11:03:02 +12:00
Rong Ou 77fc28427d fix benchmark_tree.py (#4593) 2019-06-21 11:51:48 +12:00
Rong Ou 6125521caf fix compiler warning (#4588) 2019-06-21 04:06:26 +08:00
Rong Ou e94f85f0e4 Deprecate single node multi-gpu mode (#4579) 
* deprecate multi-gpu training

* add single node

* add warning
 2019-06-19 15:51:38 +12:00
Rong Ou ba1d848767 Remove doc about not supporting cuda 10.1 (#4578) 2019-06-19 10:44:59 +12:00
Rong Ou a9ec2dd295 only copy the model once when predicting multiple batches (#4457) 2019-05-15 11:04:22 +12:00
Rong Ou df2cdaca50 add cuda 10.1 support (#4468) 2019-05-14 18:30:58 +00:00
Rong Ou be0f346ec9 mgpu predictor using explicit offsets (#4438) 
* mgpu prediction using explicit sharding
 2019-05-11 09:35:06 +12:00
Rong Ou feb6ae3e18 Initial support for external memory in gpu_predictor (#4284) 2019-05-03 13:01:27 +12:00
Rong Ou eaab364a63 More explict sharding methods for device memory (#4396) 
* Rename the Reshard method to Shard

* Add a new Reshard method for sharding a vector that's already sharded
 2019-05-01 11:47:22 +12:00
Rong Ou 2c61f02add fix broken python test (#4395) 2019-04-23 16:01:23 -07:00