Hendrik/xgboost
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
6,222 Commits 32 Branches 59 Tags
Commit Graph

161 Commits

Author SHA1 Message Date
Jiaming Yuan
4d665b3fb0
Restore clang tidy test. (#8861) 2023-03-03 13:47:04 -08:00
Rory Mitchell
69a50248b7
Fix scope of feature set pointers (#8850) 
---------

Co-authored-by: Jiaming Yuan <jm.yuan@outlook.com>
 2023-03-02 12:37:14 +08:00
Jiaming Yuan
282b1729da
Specify the number of threads for parallel sort. (#8735) 
* Specify the number of threads for parallel sort.

- Pass context object into argsort.
- Replace macros with inline functions.
 2023-02-16 00:20:19 +08:00
Rory Mitchell
7214a45e83
Fix different number of features in gpu_hist evaluator. (#8754) 2023-02-06 23:15:16 +08:00
Jiaming Yuan
0e61ba57d6
Fix GPU L1 error. (#8749) 2023-02-04 03:02:00 +08:00
Jiaming Yuan
c6a8754c62
Define CUDA Context. (#8604) 
We will transition to non-default and non-blocking CUDA stream.
 2022-12-20 15:15:07 +08:00
Jiaming Yuan
43a647a4dd
Fix inference with categorical feature. (#8591) 2022-12-15 17:57:26 +08:00
Jiaming Yuan
3e26107a9c
Rename and extract Context. (#8528) 
* Rename `GenericParameter` to `Context`.
* Rename header file to reflect the change.
* Rename all references.
 2022-12-07 04:58:54 +08:00
Rory Mitchell
210915c985
Use integer gradients in gpu_hist split evaluation (#8274) 2022-10-11 12:16:27 +02:00
Rong Ou
668b8a0ea4
[Breaking] Switch from rabit to the collective communicator (#8257) 
* Switch from rabit to the collective communicator

* fix size_t specialization

* really fix size_t

* try again

* add include

* more include

* fix lint errors

* remove rabit includes

* fix pylint error

* return dict from communicator context

* fix communicator shutdown

* fix dask test

* reset communicator mocklist

* fix distributed tests

* do not save device communicator

* fix jvm gpu tests

* add python test for federated communicator

* Update gputreeshap submodule

Co-authored-by: Hyunsu Philip Cho <chohyu01@cs.washington.edu>
 2022-10-05 14:39:01 -08:00
Rory Mitchell
8f77677193
Use quantised gradients in gpu_hist histograms (#8246) 2022-09-26 17:35:35 +02:00
Jiaming Yuan
b5eb36f1af
Add max_cat_threshold to GPU and handle missing cat values. (#8212) 2022-09-07 00:57:51 +08:00
Rory Mitchell
1be09848a7
Refactor split valuation kernel (#8073) 2022-07-21 15:41:50 +02:00
Jiaming Yuan
abaa593aa0
Fix compiler warnings. (#8059) 
- Remove unused parameters.
- Avoid comparison of different signedness.
 2022-07-14 05:29:56 +08:00
Rory Mitchell
794cbaa60a
Fuse split evaluation kernels (#8026) 2022-07-05 10:24:31 +02:00
Rory Mitchell
bc4f802b17
Batch UpdatePosition using cudaMemcpy (#7964) 2022-06-30 17:52:40 +02:00
Jiaming Yuan
1a33b50a0d
Fix compiler warnings. (#7974) 
- Remove unused parameters. There are still many warnings that are not yet
addressed. Currently, the warnings in dmlc-core dominate the error log.
- Remove `distributed` parameter from metric.
- Fixes some warnings about signed comparison.
 2022-06-06 22:56:25 +08:00
Rory Mitchell
f6babc814c
Do not initialise data structures to maximum possible tree size. (#7919) 2022-05-19 19:45:53 +02:00
Rory Mitchell
71d3b2e036
Fuse gpu_hist all-reduce calls where possible (#7867) 2022-05-17 13:27:50 +02:00
Rory Mitchell
7ef54e39ec
Small refactor to categoricals (#7858) 2022-05-05 17:47:02 +02:00
Jiaming Yuan
317d7be6ee
Always use partition based categorical splits. (#7857) 2022-05-03 22:30:32 +08:00
Rory Mitchell
90cce38236
Remove single_precision_histogram for gpu_hist (#7828) 2022-05-03 14:53:19 +02:00
Jiaming Yuan
fdf533f2b9
[POC] Experimental support for l1 error. (#7812) 
Support adaptive tree, a feature supported by both sklearn and lightgbm.  The tree leaf is recomputed based on residue of labels and predictions after construction.

For l1 error, the optimal value is the median (50 percentile).

This is marked as experimental support for the following reasons:
- The value is not well defined for distributed training, where we might have empty leaves for local workers. Right now I just use the original leaf value for computing the average with other workers, which might cause significant errors.
- Some follow-ups are required, for exact, pruner, and optimization for quantile function. Also, we need to calculate the initial estimation.
 2022-04-26 21:41:55 +08:00
Jiaming Yuan
9a400731d9
Replace device sync with stream sync. (#7737) 2022-03-19 23:22:23 +08:00
Jiaming Yuan
0d0abe1845
Support optimal partitioning for GPU hist. (#7652) 
* Implement `MaxCategory` in quantile.
* Implement partition-based split for GPU evaluation.  Currently, it's based on the existing evaluation function.
* Extract an evaluator from GPU Hist to store the needed states.
* Added some CUDA stream/event utilities.
* Update document with references.
* Fixed a bug in approx evaluator where the number of data points is less than the number of categories.
 2022-02-15 03:03:12 +08:00
Jiaming Yuan
2369d55e9a
Add tests for prediction cache. (#7650) 
* Extract the test from approx for other tree methods.
* Add note on how it works.
 2022-02-15 00:28:00 +08:00
Jiaming Yuan
5d7818e75d
Remove omp_get_max_threads in tree updaters. (#7590) 2022-01-26 19:55:47 +08:00
Jiaming Yuan
deab0e32ba
Validate out of range categorical value. (#7576) 
* Use float in CPU categorical set to preserve the input value.
* Check out of range values.
 2022-01-18 20:16:19 +08:00
Jiaming Yuan
e5e47c3c99
Clarify the behavior of invalid categorical value handling. (#7529) 2022-01-13 16:11:52 +08:00
Jiaming Yuan
7f399eac8b
Use double for GPU Hist node sum. (#7507) 2021-12-22 08:41:35 +08:00
Jiaming Yuan
d4274bc556
Fix typo. (#7433) 2021-11-15 01:28:11 +08:00
Jiaming Yuan
b06040b6d0
Implement a general array view. (#7365) 
* Replace existing matrix and vector view.

This is to prepare for handling higher dimension data and prediction when we support multi-target models.
 2021-11-05 04:16:11 +08:00
Jiaming Yuan
4100827971
Pass infomation about objective to tree methods. (#7385) 
* Define the `ObjInfo` and pass it down to every tree updater.
 2021-11-04 01:52:44 +08:00
Jiaming Yuan
a55d43ccfd
Add test for invalid categorical data values. (#7380) 
* Add test for invalid categorical data values.

* Add check during sketching.
 2021-11-02 18:00:52 +08:00
Jiaming Yuan
ac9bfaa4f2
Handle missing values in dataframe with category dtype. (#7331) 
* Replace -1 in pandas initializer.
* Unify `IsValid` functor.
* Mimic pandas data handling in cuDF glue code.
* Check invalid categories.
* Fix DDM sketching.
 2021-10-28 03:33:54 +08:00
Jiaming Yuan
a7d0c66457
Remove unused code. (#7293) 2021-10-12 15:04:41 +08:00
Jiaming Yuan
22d56cebf1
Encode pandas categorical data automatically. (#7231) 2021-09-17 11:09:55 +08:00
Jiaming Yuan
3515931305
Initial support for external memory in gradient index. (#7183) 
* Add hessian to batch param in preparation of new approx impl.
* Extract a push method for gradient index matrix.
* Use span instead of vector ref for hessian in sketching.
* Create a binary format for gradient index.
 2021-09-13 12:40:56 +08:00
Jiaming Yuan
7a1d67f9cb
[breaking] Use integer atomic for GPU histogram. (#7180) 
On GPU we use rouding factor to truncate the gradient for deterministic results. This PR changes the gradient representation to fixed point number with exponent aligned with rounding factor.

    [breaking] Drop non-deterministic histogram.
    Use fixed point for shared memory.

This PR is to improve the performance of GPU Hist. 

Co-authored-by: Andy Adinets <aadinets@nvidia.com>
 2021-08-28 05:17:05 +08:00
Jiaming Yuan
bf562bd33c
Remove unused code. (#7175) 2021-08-18 14:02:19 +08:00
Jiaming Yuan
bd1f3a38f0
Rewrite sparse dmatrix using callbacks. (#7092) 
- Reduce dependency on dmlc parsers and provide an interface for users to load data by themselves.
- Remove use of threaded iterator and IO queue.
- Remove `page_size`.
- Make sure the number of pages in memory is bounded.
- Make sure the cache can not be violated.
- Provide an interface for internal algorithms to process data asynchronously.
 2021-07-16 12:33:31 +08:00
Jiaming Yuan
1c8fdf2218
Remove use of device_idx in dh::LaunchN. (#7063) 
It's an unused parameter, removing it can make the CI log more readable.
 2021-06-29 11:37:26 +08:00
ShvetsKS
57c732655e
Merge lossgude and depthwise strategies for CPU hist (#7007) 
* fix java/scala test: max depth is also valid parameter for lossguide

Co-authored-by: Kirill Shvets <kirill.shvets@intel.com>
 2021-06-03 01:49:43 +08:00
Andrew Ziem
3e7e426b36
Fix spelling in documents (#6948) 
* Update roxygen2 doc.

Co-authored-by: fis <jm.yuan@outlook.com>
 2021-05-11 20:44:36 +08:00
Jiaming Yuan
556a83022d
Implement unified update prediction cache for (gpu_)hist. (#6860) 
* Implement utilites for linalg.
* Unify the update prediction cache functions.
* Implement update prediction cache for multi-class gpu hist.
 2021-04-17 00:29:34 +08:00
Jiaming Yuan
4f75f514ce
Fix GPU RF (#6755) 
* Fix sampling.
 2021-03-17 06:23:35 +08:00
Jiaming Yuan
bed7ae4083
Loop over thrust::reduce. (#6229) 
* Check input chunk size of dqdm.
* Add doc for current limitation.
 2020-10-14 10:40:56 +13:00
Jiaming Yuan
444131a2e6
Add categorical data support to GPU Hist. (#6164) 2020-09-29 11:27:25 +08:00
Jiaming Yuan
2fcc4f2886
Unify evaluation functions. (#6037) 2020-08-26 14:23:27 +08:00
Jiaming Yuan
4d99c58a5f
Feature weights (#5962) 2020-08-18 19:55:41 +08:00