Hendrik/xgboost
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
5,818 Commits 32 Branches 59 Tags
1a33b50a0de15ebc3147710a92877007f0d27e79
Commit Graph
49 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Rong Ou 31e6902e43 Support GPU training in the NVFlare demo (#7965) 2022-06-02 21:52:36 +08:00
Rong Ou d3429f2ff6 Increase gRPC max receive message size for federated learning (#7958) 2022-06-01 13:21:54 +08:00
Rong Ou 80339c3427 Enable distributed GPU training over Rabit (#7930) 2022-05-31 04:09:45 +08:00
Rong Ou 77d4a53c32 use RabitContext intead of init/finalize (#7911) 2022-05-17 12:15:41 +08:00
Rong Ouandjiamingy af907e2d0d Demo of federated learning using NVFlare (#7879) 
Co-authored-by: jiamingy <jm.yuan@outlook.com>
 2022-05-14 22:45:41 +08:00
Rong Ou 14ef38b834 Initial support for federated learning (#7831) 
Federated learning plugin for xgboost:
* A gRPC server to aggregate MPI-style requests (allgather, allreduce, broadcast) from federated workers.
* A Rabit engine for the federated environment.
* Integration test to simulate federated learning.

Additional followups are needed to address GPU support, better security, and privacy, etc.
 2022-05-05 21:49:22 +08:00
Rong Ou 06320729d4 fix device sketch with weights in external memory mode (#5870) 2020-07-08 08:44:07 +08:00
Rong Ou e21a608552 add pointers to the gpu external memory paper (#5684) 2020-05-19 19:46:16 -07:00
Rong Ou a1085396e2 add reference to gpu external memory (#5490) 2020-04-07 11:15:58 +12:00
Rong Ou d6b31df449 update docs for gpu external memory (#5332) 
* update docs for gpu external memory

* add hist limitation
 2020-02-22 14:57:40 +08:00
Rong Ou e4b74c4d22 Gradient based sampling for GPU Hist (#5093) 
* Implement gradient based sampling for GPU Hist tree method.
* Add samplers and handle compacted page in GPU Hist.
 2020-02-04 10:31:27 +08:00
Rong Ou 0afcc55d98 Support multiple batches in gpu_hist (#5014) 
* Initial external memory training support for GPU Hist tree method.
 2019-11-16 14:50:20 +08:00
Rong Ou 5b1715d97c Write ELLPACK pages to disk (#4879) 
* add ellpack source
* add batch param
* extract function to parse cache info
* construct ellpack info separately
* push batch to ellpack page
* write ellpack page.
* make sparse page source reusable
 2019-10-22 23:44:32 -04:00
Rong Ou 562bb0ae31 remove device shards (#4867) 2019-09-25 13:15:46 +08:00
Rong Ou 006eb80578 ignore vscode and clion files (#4866) 
* ignore vscode and clion files

* ignore all .idea directories
 2019-09-17 21:27:40 -04:00
Rong Ou 125bcec62e Move ellpack page construction into DMatrix (#4833) 2019-09-16 23:50:55 -04:00
Rong Ou 733ed24dd9 further cleanup of single process multi-GPU code (#4810) 
* use subspan in gpu predictor instead of copying
* Revise `HostDeviceVector`
 2019-08-30 05:27:23 -04:00
Rong Ou 38ab79f889 Make HostDeviceVector single gpu only (#4773) 
* Make HostDeviceVector single gpu only
 2019-08-26 09:51:13 +12:00
Rong Ou c5b229632d [BREAKING] prevent multi-gpu usage (#4749) 
* prevent multi-gpu usage

* fix distributed test

* combine gpu predictor tests

* set upper bound on n_gpus
 2019-08-13 09:11:35 +12:00
Rong Ou 19f9fd5de9 remove the qids_ field in MetaInfo (#4744) 2019-08-08 10:01:59 +08:00
Rong Ou 602484e19f Remove some unused functions as reported by cppcheck (#4743) 2019-08-07 02:42:33 -04:00
Rong Ou 851b5b3808 Remove gpu_exact tree method (#4742) 2019-08-07 11:43:20 +12:00
Rong Ou 6edddd7966 Refactor DMatrix to return batches of different page types (#4686) 
* Use explicit template parameter for specifying page type.
 2019-08-03 15:10:34 -04:00
Rong Ou cb9a80ca90 Update dmlc-core (#4726) 2019-08-02 03:54:14 -04:00
Rong Ou 9b9e298ff2 remove RowSet which is no longer being used (#4697) 2019-07-25 17:25:58 -07:00
Rong Ou 30204b50fe fix spark tests on machines with many cores (#4634) 2019-07-07 16:02:56 -07:00
Rong Ou 63ec95623d fix gpu predictor when dmatrix is mismatched with model (#4613) 2019-06-28 11:03:02 +12:00
Rong Ou 77fc28427d fix benchmark_tree.py (#4593) 2019-06-21 11:51:48 +12:00
Rong Ou 6125521caf fix compiler warning (#4588) 2019-06-21 04:06:26 +08:00
Rong Ou e94f85f0e4 Deprecate single node multi-gpu mode (#4579) 
* deprecate multi-gpu training

* add single node

* add warning
 2019-06-19 15:51:38 +12:00
Rong Ou ba1d848767 Remove doc about not supporting cuda 10.1 (#4578) 2019-06-19 10:44:59 +12:00
Rong Ou a9ec2dd295 only copy the model once when predicting multiple batches (#4457) 2019-05-15 11:04:22 +12:00
Rong Ou df2cdaca50 add cuda 10.1 support (#4468) 2019-05-14 18:30:58 +00:00
Rong Ou be0f346ec9 mgpu predictor using explicit offsets (#4438) 
* mgpu prediction using explicit sharding
 2019-05-11 09:35:06 +12:00
Rong Ou feb6ae3e18 Initial support for external memory in gpu_predictor (#4284) 2019-05-03 13:01:27 +12:00
Rong Ou eaab364a63 More explict sharding methods for device memory (#4396) 
* Rename the Reshard method to Shard

* Add a new Reshard method for sharding a vector that's already sharded
 2019-05-01 11:47:22 +12:00
Rong Ou 2c61f02add fix broken python test (#4395) 2019-04-23 16:01:23 -07:00
Rong Ou f4521bf6aa refactor tests to get rid of duplication (#4358) 
* refactor tests to get rid of duplication

* address review comments
 2019-04-12 00:21:48 -07:00
Rong Ou 81c1cd40ca add a test for cpu predictor using external memory (#4308) 
* add a test for cpu predictor using external memory

* allow different page size for testing
 2019-04-10 13:25:10 +12:00
Rong Ou 7ea5b772fb do not filter shared library files (#4303) 2019-03-28 19:40:54 +08:00
Rong Ou 8c8021dfa7 use all cores to build on linux (#4304) 2019-03-27 19:51:08 -07:00
Rong Ou 5aa42b5f11 jenkins build for cuda 10.0 (#4281) 
* jenkins build for cuda 10.0

* yum install nccl2 for cuda 10.0
 2019-03-22 22:35:18 -07:00
Rong Ou 9837b09b20 support cuda 10.1 (#4223) 
* support cuda 10.1

* add cuda 10.1 to jenkins build matrix
 2019-03-08 12:22:12 +13:00
Rong Ou 0944360416 minor fix: log InitDataOnce() only when it is actually called (#4206) 2019-03-08 10:53:09 +13:00
Rong Ou 8e0a08fbcf Update python benchmarking script (#4164) 
* a few tweaks to speed up data generation

* del variable to save memory

* switch to random numpy arrays
 2019-02-21 15:16:09 +13:00
Rong Ou d506a8bc63 [jvm-packages] add verbosity param (#4138) 2019-02-13 20:57:17 -08:00
Rong Ou 3be1b9ae30 reformat benchmark_tree.py to get rid of lint errors (#4126) 2019-02-13 18:54:56 +13:00
Rong Ou 9b917cda4f [jvm-packages] fix simple logic error :) (#4128) 
@CodingCat
 2019-02-11 21:47:30 -08:00
Rong Ou 2a9b085bc8 [jvm-packages] minor fix of params (#4114) 2019-02-08 00:21:59 -08:00