Hendrik/xgboost
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
6,849 Commits 32 Branches 59 Tags
c7e7ce75693f899f5dc9720c5adb58305d2d93b1
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Jiaming Yuan 7a54ca41c9 [CI] Bump checkout action version. (#10305) 2024-05-21 16:38:20 +08:00
Jiaming Yuan a5a58102e5 Revamp the rabit implementation. (#10112) 
This PR replaces the original RABIT implementation with a new one, which has already been partially merged into XGBoost. The new one features:
- Federated learning for both CPU and GPU.
- NCCL.
- More data types.
- A unified interface for all the underlying implementations.
- Improved timeout handling for both tracker and workers.
- Exhausted tests with metrics (fixed a couple of bugs along the way).
- A reusable tracker for Python and JVM packages.
 2024-05-20 11:56:23 +08:00
Jiaming Yuan ba9b4cb1ee Fix pylint. (#10296) 2024-05-17 13:28:39 +08:00
Jiaming Yuan 835e59e538 Use a thread pool for external memory. (#10288) 2024-05-16 19:32:12 +08:00
Jiaming Yuan ca1d04bcb7 Release data in cache. (#10286) 2024-05-14 14:20:19 +08:00
Jiaming Yuan f1f69ff10e [CI] Fixes for using the latest modin. (#10285) 2024-05-14 12:13:35 +08:00
Jiaming Yuan 871fabeee3 [doc][dask] Update notes about k8s. (#10271) 2024-05-14 04:21:02 +08:00
Jiaming YuanandPhilip Hyunsu Cho d81e319e78 Fixes for the latest pandas. (#10266) 
Co-authored-by: Philip Hyunsu Cho <chohyu01@cs.washington.edu>
 2024-05-12 11:15:46 +08:00
Jiaming Yuan 5e816e616a [CI] Upgrade setup-r. (#10267) 2024-05-12 05:25:29 +08:00
Jiaming Yuan 5de57435c7 Be more lenient on floating point error for AUC. (#10264) 2024-05-11 08:48:11 +08:00
Jiaming Yuan 8237920c48 [jvm-packagaes] Freeze spark to 3.4.1 for now. (#10253) 
The newer spark version for CPU conflicts with the more conservative version used by
rapids.
 2024-05-07 09:00:59 -07:00
Jiaming Yuan 73afef1a6e Fixes for numpy 2.0. (#10252) 2024-05-07 03:54:32 +08:00
Jiaming Yuan 5e64276a9b Update nvtx. (#10227) 2024-04-29 06:33:46 +08:00
Jiaming Yuan 837d44a345 Support more sklearn tags for testing. (#10230) 2024-04-29 06:33:23 +08:00
Jiaming Yuan 54754f29dd [pyspark] Sort workers by task ID. (#10220) 2024-04-28 18:05:15 +08:00
Jiaming Yuan a81b78e56b [CI] Test new setup-r. (#10228) 2024-04-26 08:23:31 -07:00
Jiaming Yuan 59d7b8dc72 [doc] Add typing to dask demos. (#10207) 2024-04-23 00:57:05 +08:00
Jiaming Yuan 3fbb221fec [coll] Implement shutdown for tracker and comm. (#10208) 
- Force shutdown the tracker.
- Implement shutdown notice for error handling thread in comm.
 2024-04-20 04:08:17 +08:00
Jiaming Yuan 3f64b4fde3 [coll] Add global functions. (#10203) 2024-04-19 03:17:23 +08:00
Jiaming Yuan 303c603c7d [pyspark] Reuse the collective communicator. (#10198) 2024-04-18 19:09:30 +08:00
Jiaming Yuan 4b10200456 [coll] Improve event loop. (#10199) 
- Add a test for blocking calls.
- Do not require the queue to be empty after waking up; this frees up the thread to answer blocking calls.
- Handle EOF in read.
- Improve the error message in the result. Allow concatenation of multiple results.
 2024-04-18 03:29:52 +08:00
Jiaming Yuan 1022909bbe Fix global config for external memory. (#10173) 
Pass the thread-local configuration between threads.
 2024-04-11 01:29:28 +08:00
Jiaming Yuan f0a138f33a Fix pyspark with verbosity=3. (#10172) 2024-04-09 23:18:56 +08:00
Jiaming Yuan 8bad677c2f Update collective implementation. (#10152) 
* Update collective implementation.

- Cleanup resource during `Finalize` to avoid handling threads in destructor.
- Calculate the size for allgather automatically.
- Use simple allgather for small (smaller than the number of worker) allreduce.
 2024-03-30 18:57:31 +08:00
Jiaming YuanandPhilip Hyunsu Cho 230010d9a0 Cleanup set info. (#10139) 
- Use the array interface internally.
- Deprecate `XGDMatrixSetDenseInfo`.
- Deprecate `XGDMatrixSetUIntInfo`.
- Move the handling of `DataType` into the deprecated C function.

---------

Co-authored-by: Philip Hyunsu Cho <chohyu01@cs.washington.edu>
 2024-03-26 23:26:24 +08:00
Jiaming Yuan e1695775e9 [CI] Fix yml in github action. (#10134) 2024-03-20 16:38:03 +08:00
Jiaming Yuan 2b2aac85f4 [CI] Update scorecard actions. (#10133) 2024-03-20 15:51:38 +08:00
Jiaming Yuan ca4801f81d Work with IPv6 in the new tracker. (#10125) 2024-03-20 05:19:23 +08:00
Jiaming Yuan 53fc17578f Use std::uint64_t for row index. (#10120) 
- Use std::uint64_t instead of size_t to avoid implementation-defined type.
- Rename to bst_idx_t, to account for other types of indexing.
- Small cleanup to the base header.
 2024-03-15 18:43:49 +08:00
Jiaming Yuan 56b1868278 Fix compilation with the latest ctk. (#10123) 2024-03-15 08:04:41 +08:00
Jiaming Yuan 1450aebb74 Fix pairwise objective with NDCG metric along with custom gain. (#10100) 
* Fix pairwise objective with NDCG metric.

- Allow setting `ndcg_exp_gain` for `rank:pairwise`.

This is useful when using pairwise for objective but ndcg for metric.
 2024-03-11 14:54:10 +08:00
Jiaming Yuan 06c9702028 [doc] Fix the default value for lambdarank_pair_method. (#10098) 2024-03-11 14:53:17 +08:00
Jiaming Yuan 2c13f90384 Support graphviz plot for multi-target tree. (#10093) 2024-03-09 05:35:25 +08:00
Jiaming Yuan e14c3b9325 Optional normalization for learning to rank. (#10094) 2024-03-08 12:41:21 +08:00
Jiaming Yuan d07b7fe8c8 Small cleanup for mock tests. (#10085) 2024-03-04 23:32:11 +08:00
Jiaming Yuan 3941b31ade Disable column sample by node for the exact tree method. (#10083) 2024-03-01 14:16:10 +08:00
Jiaming Yuan 8189126d51 Add CUDA iterator to tensor view. (#10074) 2024-03-01 14:15:31 +08:00
Jiaming Yuan 5ac233280e Require context in aggregators. (#10075) 2024-02-28 03:12:42 +08:00
Jiaming Yuan 0ce4372bd4 Use UBJSON for serializing splits for vertical data split. (#10059) 2024-02-25 00:18:23 +08:00
Jiaming YuanandHyunsu Cho eb281ff9b4 [CI] Fix JVM tests on GH Action (#10064) 
---------

Co-authored-by: Hyunsu Cho <chohyu01@cs.washington.edu>
 2024-02-22 14:21:32 -08:00
Jiaming Yuan 2e4ea5ecc0 Support f64 for ubjson. (#10055) 2024-02-21 02:18:42 +08:00
Jiaming Yuan 8ea705e4d5 Support sample weight in sklearn custom objective. (#10050) 2024-02-21 00:43:14 +08:00
Jiaming Yuan 69a17d5114 Fix with None input. (#10052) 2024-02-20 22:34:22 +08:00
Jiaming Yuan d37b83e8d9 Fix UBJSON with boolean value. (#10054) 2024-02-20 22:13:51 +08:00
Jiaming Yuan 54b71c8fba Fix with black 24.1.1. (#10014) 2024-01-30 17:24:11 +08:00
Jiaming Yuan 65d7bf2dfe Handle np integer in model slice and prediction. (#10007) 2024-01-26 04:58:48 +08:00
Jiaming Yuan a76d6c6131 Fix cpp deprecation. (#10010) 2024-01-26 02:13:40 +08:00
Jiaming Yuan d3f2dbe64f [dask] Add seed to demos. (#10009) 2024-01-26 02:09:38 +08:00
Jiaming Yuan d12cc1090a Refactor tests for training continuation. (#9997) 2024-01-24 16:07:19 +08:00
Jiaming Yuan bde20dd897 Remove benchmark scripts. (#9992) 2024-01-17 13:19:34 +08:00
Jiaming Yuan d07e8b503e Fix quantile regression demo. (#9991) 2024-01-17 13:19:08 +08:00
Jiaming Yuan cacb4b1fdd Fix gain calculation in multi-target tree. (#9978) 2024-01-17 13:18:44 +08:00
Jiaming Yuan 85d09245f6 Fix error handling in the event loop. (#9990) 2024-01-17 05:35:35 +08:00
Jiaming Yuan 0798e36d73 [breaking] Remove deprecated parameters in the skl interface. (#9986) 2024-01-15 20:40:05 +08:00
Jiaming Yuan 01c4711556 Check __cuda_array_interface__ instead of cupy class. (#9971) 
* Now XGBoost can directly consume CUDA data from torch.
 2024-01-09 19:59:01 +08:00
Jiaming Yuan 2f57bbde3c Additional tests for attributes and model booosted rounds. (#9962) 2024-01-09 09:54:39 +08:00
Jiaming Yuan b3eb5d0945 Use UBJ in Python checkpoint. (#9958) 2024-01-09 03:22:15 +08:00
Jiaming Yuan fa5e2f6c45 Synthesize the AMES housing dataset for tests. (#9963) 2024-01-09 00:54:23 +08:00
Jiaming Yuan 9a30bdd313 Test loading models with invalid file extensions. (#9955) 2024-01-08 19:26:24 +08:00
Jiaming Yuan 3976455af9 [jvm-packages] Use UBJ for checkpoints. (#9954) 2024-01-08 13:26:12 +08:00
Jiaming Yuan 38dd91f491 Save model in ubj as the default. (#9947) 2024-01-05 17:53:36 +08:00
Jiaming Yuan c03a4d5088 Check support status for categorical features. (#9946) 2024-01-04 16:51:33 +08:00
Jiaming YuanandPhilip Hyunsu Cho 621348abb3 Fix multi-output with alternating strategies. (#9933) 
---------

Co-authored-by: Philip Hyunsu Cho <chohyu01@cs.washington.edu>
 2024-01-04 16:41:13 +08:00
Jiaming Yuan 5f7b5a6921 Add tests for pickling with custom obj and metric. (#9943) 2024-01-04 14:52:48 +08:00
Jiaming Yuan 26a5436a65 [doc] Describe feature info behavior. [skip ci] (#9866) 2024-01-04 14:52:19 +08:00
Jiaming Yuan 9f73127a23 Cleanup Python GPU tests. (#9934) 
* Cleanup Python GPU tests.

- Remove the use of `gpu_hist` and `gpu_id` in cudf/cupy tests.
- Move base margin test into the testing directory.
 2024-01-04 13:15:18 +08:00
Jiaming Yuan a7226c0222 Fix feature names with special characters. (#9923) 2023-12-28 22:45:13 +08:00
Jiaming Yuan 6a5f6ba694 [CI] Add timeout for distributed GPU tests. (#9917) 2023-12-24 00:09:05 +08:00
Jiaming Yuan 9d122293bc [doc] Fix typo. [skip ci] (#9904) 2023-12-20 09:17:00 +08:00
Jiaming Yuan 0edd600f3d [doc] Brief introduction to base_score. (#9882) 2023-12-17 13:34:34 +08:00
Jiaming Yuan 1c6e031c75 [R] Fix clang warning. (#9874) 2023-12-15 01:30:43 +08:00
Jiaming Yuan 125bc812f8 [doc] Reference enable_categorical doc in sklearn. (#9884) 2023-12-14 23:29:19 +08:00
Jiaming Yuan 1aa8c8d9be Support more scipy types. (#9881) 2023-12-14 18:28:37 +08:00
Jiaming Yuan faf0f2df10 Support dataframe data format in native XGBoost. (#9828) 
- Implement a columnar adapter.
- Refactor Python pandas handling code to avoid converting into a single numpy array.
- Add support in R for transforming columns.
- Support R data.frame and factor type.
 2023-12-12 09:56:31 +08:00
Jiaming Yuan b3700bbb3f Flexible find protobuf. (#9867) 2023-12-12 07:34:01 +08:00
Jiaming Yuan 1094d6015d [py] Use the first found native library. (#9860) 2023-12-08 17:23:16 +08:00
Jiaming Yuan 42de9206fc Support multi-target, fit intercept for hinge. (#9850) 2023-12-08 05:50:41 +08:00
Jiaming Yuan 39c637ee19 Use array interface in Python prediction return. (#9855) 2023-12-08 03:42:14 +08:00
Jiaming Yuan 4bc1f3a388 [R] Bump requirement to 4.3.0. (#9847) 2023-12-07 00:12:45 +08:00
Jiaming Yuan e78b46046e [CI] Update R version on Linux. (#9835) 2023-12-02 11:03:17 +08:00
Jiaming Yuan 2d8c67d6dc [jvm-packages] Bump dependencies. (#9827) 
- #9811
- #9814
- #9826
- #9830
- #9833
- #9832
- #9831
- #9834
 2023-12-02 07:34:56 +08:00
Jiaming Yuan e2e089ce12 [jvm-packages] Bump rapids version. (#9820) 2023-11-29 15:51:07 +08:00
Jiaming Yuan 59684b2db6 [doc] Draft for language binding consistency. [skip ci] (#9755) 2023-11-29 13:13:40 +08:00
Jiaming Yuan 34a2616696 [jvm-packages] Update dependencies. (#9809) 
- scalatest: 3.2.17
- maven-checkstyle-plugin: 3.3.1
- maven-surefire-plugin: 3.2.2
- maven-project-info-reports-plugin: 3.5.0
- maven-javadoc-plugin: 3.6.2
 2023-11-27 20:09:25 +08:00
Jiaming Yuan e9f149481e [sklearn] Fix loading model attributes. (#9808) 2023-11-27 17:19:01 +08:00
Jiaming Yuan 3f4e22015a Mark NCCL python test optional. (#9804) 
Skip the tests if XGBoost is not compiled with dlopen.
 2023-11-25 11:25:47 +08:00
Jiaming Yuan 8fe1a2213c Cleanup code for distributed training. (#9805) 
* Cleanup code for distributed training.

- Merge `GetNcclResult` into nccl stub.
- Split up utilities from the main dask module.
- Let Channel return `Result` to accommodate nccl channel.
- Remove old `use_label_encoder` parameter.
 2023-11-25 09:10:56 +08:00
Jiaming Yuan e9260de3f3 [breaking] Remove dense libsvm parser plugin. (#9799) 2023-11-23 00:12:39 +08:00
Jiaming Yuan 1877cb8e83 Change default metric for gamma regression to deviance. (#9757) 
* Change default metric for gamma regression to deviance.

- Cleanup the gamma implementation.
- Use deviance instead since the objective is derived from deviance.
 2023-11-22 21:17:48 +08:00
Jiaming Yuan 0715ab3c10 Use dlopen to load NCCL. (#9796) 
This PR adds optional support for loading nccl with `dlopen` as an alternative of compile time linking. This is to address the size bloat issue with the PyPI binary release.
- Add CMake option to load `nccl` at runtime.
- Add an NCCL stub.

After this, `nccl` will be fetched from PyPI when using pip to install XGBoost, either by a user or by `pyproject.toml`. Others who want to link the nccl at compile time can continue to do so without any change.

At the moment, this is Linux only since we only support MNMG on Linux.
 2023-11-22 19:27:31 +08:00
Jiaming Yuan fedd9674c8 Implement column sampler in CUDA. (#9785) 
- CUDA implementation.
- Extract the broadcasting logic, we will need the context parameter after revamping the collective implementation.
- Some changes to the event loop for fixing a deadlock in CI.
- Move argsort into algorithms.cuh, add support for cuda stream.
 2023-11-17 04:29:08 +08:00
Jiaming Yuan ada377c57e [coll] Reduce the scope of lock in the event loop. (#9784) 2023-11-15 14:16:19 +08:00
Jiaming Yuan 6fd4a30667 [coll] Increase timeout for allgather test. (#9777) 2023-11-09 05:26:40 +08:00
Jiaming Yuan 44099f585d [coll] Add C API for the tracker. (#9773) 2023-11-08 18:17:14 +08:00
Jiaming Yuan 06bdc15e9b [coll] Pass context to various functions. (#9772) 
* [coll] Pass context to various functions.

In the future, the `Context` object would be required for collective operations, this PR
passes the context object to some required functions to prepare for swapping out the
implementation.
 2023-11-08 09:54:05 +08:00
Jiaming Yuan 6c0a190f6d [coll] Add comm group. (#9759) 
- Implement `CommGroup` for double dispatching.
- Small cleanup to tracker for handling abort.
 2023-11-07 11:12:31 +08:00
Jiaming Yuan c3a0622b49 Fix using categorical data with the score function of ranker. (#9753) 2023-11-07 07:29:11 +08:00
Jiaming Yuan 82828621d0 [doc] Add doc for linters and simplify c++ lint script. (#9750) 2023-11-07 05:03:30 +08:00
Jiaming Yuan 98238d63fa [dask] Change document to avoid using default import. (#9742) 
This aligns dask with pyspark, users need to explicitly call:

```
from xgboost.dask import DaskXGBClassifier
from xgboost import dask as dxgb
```

In future releases, we might stop using the default import and remove the lazy loader.
 2023-11-07 02:44:39 +08:00
Jiaming Yuan 4da4e092b5 [coll] Improvements and fixes for tracker and allreduce. (#9745) 
- Allow the tracker to wait.
- Fix allreduce type cast
- Return args from the federated tracker.
 2023-11-02 04:06:46 +08:00