Hendrik/xgboost
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
5,484 Commits 32 Branches 59 Tags
11f8b5cfcd3a3eacf45589ff8e137140cfefa17a
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Jiaming Yuan 11f8b5cfcd [backport] Support building with CTK11.5. (#7379) (#7411) 
* Support building with CTK11.5.

* Require system cub installation for CTK11.4+.
* Check thrust version for segmented sort.
 2021-11-10 19:23:29 +08:00
Jiaming Yuan e7ac2486eb [backport] [R] Fix global feature importance and predict with 1 sample. (#7394) (#7397) 
* [R] Fix global feature importance.

* Add implementation for tree index.  The parameter is not documented in C API since we
should work on porting the model slicing to R instead of supporting more use of tree
index.

* Fix the difference between "gain" and "total_gain".

* debug.

* Fix prediction.
 2021-11-06 00:07:36 +08:00
Jiaming Yuan a3d195e73e Handle OMP_THREAD_LIMIT. (#7390) (#7391) 2021-11-03 20:25:51 +08:00
Jiaming YuanandHyunsu Cho fab3c05ced Move macos test to github action. (#7382) (#7392) 
Co-authored-by: Hyunsu Cho <chohyu01@cs.washington.edu>
 2021-11-03 18:39:47 +08:00
Jiaming Yuan 584b45a9cc Release 1.5.0. (#7317) 2021-10-15 12:21:04 +08:00
Jiaming Yuan 30c1b5c54c [backport] Fix prediction with cat data in sklearn interface. (#7306) (#7312) 
* Specify DMatrix parameter for pre-processing dataframe.
* Add document about the behaviour of prediction.
 2021-10-12 18:49:57 +08:00
Jiaming Yuan 36e247aca4 Fix weighted samples in multi-class AUC. (#7300) (#7305) 2021-10-11 18:00:36 +08:00
Jiaming Yuan c4aff733bb [backport] Fix cv verbose_eval (#7291) (#7296) 2021-10-08 14:24:27 +08:00
Jiaming Yuan cdbfd21d31 [backport] Fix gamma neg log likelihood. (#7275) (#7285) 2021-10-05 23:01:11 +08:00
Jiaming Yuan 508a0b0dbd [backport] [R] Fix document for nthread. (#7263) (#7269) 2021-09-28 14:41:32 +08:00
Jiaming Yuan e04e773f9f Add RC1 tag for building packages. (#7261) 2021-09-28 11:50:18 +08:00
Jiaming Yuan 1debabb321 Change version to 1.5.0. (#7258) 2021-09-26 13:27:54 +08:00
Jiaming Yuan d8a549e6ac Avoid thread block with sparse data. (#7255) 2021-09-25 13:11:34 +08:00
Jiaming YuanandPhilip Hyunsu Cho ca17f8a5fc Dispatch thrust versions and upgrade rmm. (#7254) 
Co-authored-by: Philip Hyunsu Cho <chohyu01@cs.washington.edu>
 2021-09-25 03:43:23 +08:00
Jiaming Yuan fbd58bf190 [jvm-packages] Create demo and test for xgboost4j early stopping. (#7252) 2021-09-25 03:29:27 +08:00
Jiaming Yuan 9472be7d77 Fix initialization from pandas series. (#7243) 2021-09-23 04:43:25 +08:00
Jiaming Yuan 18bd16341a Update Python intro. [skip ci] (#7235) 
* Fix the link to demo.
* Stop recommending text file inputs.
* Brief mention to scikit-learn interface.
* Fix indent warning in tree method doc.
 2021-09-21 02:47:09 +00:00
Jiaming Yuan e48e05e6e2 Add typehint to rabit module. (#7240) 2021-09-17 18:31:02 +08:00
Jiaming Yuan c735c17f33 Disable callback and ES on random forest. (#7236) 2021-09-17 18:21:17 +08:00
Jiaming Yuan c311a8c1d8 Enable compiling with system cub. (#7232) 
- Tested with all CUDA 11.x.
- Workaround cub scan by using discard iterator in AUC.
- Limit the size of Argsort when compiled with CUDA cub.
 2021-09-17 14:28:18 +08:00
Jiaming Yuan b18f5f61b0 Fix pylint (#7241) 2021-09-17 11:50:36 +08:00
Jiaming Yuan 38a23f66a8 Fix typo in release script. [skip ci] (#7238) 2021-09-17 11:14:05 +08:00
Jiaming Yuan 8ad7e8eeb0 [doc] Fix typo. [skip ci] (#7226) 2021-09-17 11:13:49 +08:00
Jiaming Yuan 22d56cebf1 Encode pandas categorical data automatically. (#7231) 2021-09-17 11:09:55 +08:00
Jiaming Yuan 32e0858501 Fix travis. (#7237) 2021-09-17 10:06:23 +08:00
Jiaming Yuan 31c1e13f90 Categorical data support in CPU sketching. (#7221) 2021-09-17 04:37:09 +08:00
Jiaming Yuan 9f63d6fead [jvm-packages] Deprecate constructors with implicit missing value. (#7225) 2021-09-17 04:35:04 +08:00
Jiaming Yuan 0ed979b096 Support more input types for categorical data. (#7220) 
* Support more input types for categorical data.

* Shorten the type name from "categorical" to "c".
* Tests for np/cp array and scipy csr/csc/coo.
* Specify the type for feature info.
 2021-09-16 20:39:30 +08:00
Jiaming Yuan 2942dc68e4 Fix mixed types in GPU sketching. (#7228) 2021-09-16 00:10:25 +08:00
Jiaming Yuan 037dd0820d Implement __sklearn_is_fitted__. (#7230) 2021-09-15 19:09:04 +08:00
Jiaming Yuan d997c967d5 Demo for experimental categorical data support. (#7213) 2021-09-15 08:20:12 +08:00
Jiaming Yuan 3515931305 Initial support for external memory in gradient index. (#7183) 
* Add hessian to batch param in preparation of new approx impl.
* Extract a push method for gradient index matrix.
* Use span instead of vector ref for hessian in sketching.
* Create a binary format for gradient index.
 2021-09-13 12:40:56 +08:00
Jiaming Yuan 804b2ac60f Expose DMatrix API for CUDA columnar and array. (#7217) 
* Use JSON encoded configurations.
* Expose them into header file.
 2021-09-09 17:55:25 +08:00
Jiaming Yuan 68a2c7b8d6 Fix memory leak in demo. (#7216) 2021-09-09 13:51:03 +08:00
Jiaming Yuan b12e7f7edd Add noexcept to JSON objects. (#7205) 2021-09-07 13:56:48 +08:00
Jiaming Yuan 3a4f51f39f Avoid calling CUDA code on CPU for linear model. (#7154) 2021-09-01 10:45:31 +08:00
Jiaming Yuan ba69244a94 Restore the custom double atomic add. (#7198) 2021-08-28 18:30:42 +08:00
Jiaming YuanandAndy Adinets 7a1d67f9cb [breaking] Use integer atomic for GPU histogram. (#7180) 
On GPU we use rouding factor to truncate the gradient for deterministic results. This PR changes the gradient representation to fixed point number with exponent aligned with rounding factor.

    [breaking] Drop non-deterministic histogram.
    Use fixed point for shared memory.

This PR is to improve the performance of GPU Hist. 

Co-authored-by: Andy Adinets <aadinets@nvidia.com>
 2021-08-28 05:17:05 +08:00
Jiaming Yuan e7d7ab6bc3 Better error message for ncclUnhandledCudaError. (#7190) 2021-08-27 10:29:22 +08:00
Jiaming Yuan cdfaa705f3 Fix building on CUDA 11.0. (#7187) 2021-08-25 02:57:53 -07:00
Jiaming Yuan 9c64618cb6 [breaking] Remove CUDA sm_35, add sm_86 (#7182) 2021-08-25 16:04:23 +08:00
Jiaming Yuan ee8d1f5ed8 Fix histogram truncation. (#7181) 
* Fix truncation.

* Lint.

* lint.
 2021-08-24 18:34:32 -07:00
Jiaming Yuan 3290a4f3ed Re-enable feature validation in predict proba. (#7177) 2021-08-22 15:28:08 +08:00
Jiaming Yuan bf562bd33c Remove unused code. (#7175) 2021-08-18 14:02:19 +08:00
Jiaming Yuan 6bcbc77226 [doc] Fix typo. [skip ci] (#7170) 2021-08-13 03:48:16 +08:00
Jiaming Yuan 3f38d983a6 Fix prediction configuration. (#7159) 
After the predictor parameter was added to the constructor, this configuration was broken.
 2021-08-11 16:34:36 +08:00
Jiaming Yuan 9600ca83f3 Remove synchronization in monitor. (#7164) 
* Remove synchronization in monitor.

Calling rabit functions during destruction is flaky.

* Add xgboost prefix to nvtx marker.
 2021-08-11 16:33:53 +08:00
Jiaming Yuan 149f209af6 Extract histogram builder from CPU Hist. (#7152) 
* Extract the CPU histogram builder.
* Fix tests.
* Reduce number of histograms being built.
 2021-08-09 21:15:21 +08:00
Jiaming Yuan 8a84be37b8 Pass scikit learn estimator checks for regressor. (#7130) 
* Check data shape.
* Check labels.
 2021-08-03 18:58:20 +08:00
Jiaming Yuan 8ee127469f [R] Fix nthread in DMatrix constructor. (#7127) 
* Break the R C API for nthread.
 2021-08-03 17:39:25 +08:00
Jiaming Yuan ba47eda61b [doc] Use figure directive. (#7143) 2021-08-03 15:56:25 +08:00
Jiaming Yuan e2c406f5c8 Support min_delta in early stopping. (#7137) 
* Support `min_delta` in early stopping.

* Remove abs_tol.
 2021-08-03 14:29:17 +08:00
Jiaming Yuan 7bdedacb54 Document for process_type. (#7135) 
* Update document for prune and refresh.

* Add demo.
 2021-08-03 13:11:52 +08:00
Jiaming Yuan d080b5a953 Fix model slicing. (#7149) 
* Use correct pointer.
* Remove best_iteration/best_score.
 2021-08-03 11:51:56 +08:00
Jiaming Yuan 36346f8f56 C API demo for inference. (#7151) 2021-08-03 00:46:47 +08:00
Jiaming Yuan 1369133916 [dask] Remove the workaround for segfault. (#7146) 2021-07-30 03:57:53 +08:00
Jiaming Yuan 7017dd5a26 [JVM-Packages] Use Python tracker in XGBoost for JVM package. (#7132) 2021-07-27 16:20:42 +08:00
Jiaming Yuan 48d5de80a2 [R] Fix softprob reshape. (#7126) 2021-07-27 15:25:17 +08:00
Jiaming Yuan 7ee7a95b84 Use upstream URI in distributed quantile tests. (#7129) 
* Use upstream URI in distributed quantile tests.

* Fix test cv `PytestAssertRewriteWarning`.
 2021-07-27 14:09:49 +08:00
Jiaming Yuan e88ac9cc54 [dask] Extend tree stats tests. (#7128) 
* Add tests to GPU.
* Assert cover in children sums up to the parent.
 2021-07-27 12:22:13 +08:00
Jiaming Yuan 778135f657 Fix parameter loading with training continuation. (#7121) 
* Add a demo for training continuation.
 2021-07-23 10:51:47 +08:00
Jiaming Yuan e6088366df Export Python Interface for external memory. (#7070) 
* Add Python iterator interface.
* Add tests.
* Add demo.
* Add documents.
* Handle empty dataset.
 2021-07-22 15:15:53 +08:00
Jiaming Yuan bd1f3a38f0 Rewrite sparse dmatrix using callbacks. (#7092) 
- Reduce dependency on dmlc parsers and provide an interface for users to load data by themselves.
- Remove use of threaded iterator and IO queue.
- Remove `page_size`.
- Make sure the number of pages in memory is bounded.
- Make sure the cache can not be violated.
- Provide an interface for internal algorithms to process data asynchronously.
 2021-07-16 12:33:31 +08:00
Jiaming Yuan 2f524e9f41 [dask] Work around segfault in prediction. (#7112) 2021-07-16 04:27:05 +08:00
Jiaming Yuan abec3dbf6d Fix thread safety of softmax prediction. (#7104) 2021-07-16 02:06:55 +08:00
Jiaming Yuan 8e8232fb4c [CI] Update R cache. (#7102) 2021-07-14 03:15:35 +08:00
Jiaming Yuan 345796825f Optional find dependency in installed cmake config. (#7099) 
* Find dependency only when xgboost is built as static library.
* Resolve msvc warning.
* Add test for linking shared library.
 2021-07-11 17:20:55 +08:00
Jiaming Yuan 77f6cf2d13 Support hessian in host sketch container. (#7081) 
Prepare for migrating approx onto hist's codebase.
 2021-07-08 16:33:58 +08:00
Jiaming Yuan 84d359efb8 Support host data in proxy DMatrix. (#7087) 2021-07-08 11:35:48 +08:00
Jiaming Yuan 5d7cdf2e36 [Breaking] Rename Quantile DMatrix C API. (#7082) 
The role of ProxyDMatrix is going beyond what it was designed.  Now it's used by both
QuantileDeviceDMatrix and inplace prediction.  After the refactoring of sparse DMatrix it
will also be used for external memory.  Renaming the C API to extract it from
QuantileDeviceDMatrix.
 2021-07-08 11:34:14 +08:00
Jiaming Yuan c766f143ab Refactor external memory formats. (#7089) 
* Save base_rowid.
* Return write size.
* Remove unused function.
 2021-07-08 04:04:51 +08:00
Jiaming Yuan 689eb8f620 Check external memory support for exact tree method. (#7088) 2021-07-08 02:12:57 +08:00
Jiaming Yuan 615ab2b03e Extract evaluate splits from CPU hist. (#7079) 
Other than modularizing the split evaluation function, this PR also removes some more functions including `InitNewNodes` and `BuildNodeStats` among some other unused variables.  Also, scattered code like setting leaf weights is grouped into the split evaluator and `NodeEntry` is simplified and made private.  Another subtle difference with the original implementation is that the modified code doesn't call `tree[nidx].Parent()` to traversal upward.
 2021-07-07 15:16:25 +08:00
Jiaming Yuan f937f514aa Remove lz4 compression with external memory. (#7076) 2021-07-06 14:46:43 +08:00
Jiaming Yuan 116d711815 Make SimpleDMatrix ctor reusable. (#7075) 2021-07-06 13:38:24 +08:00
Jiaming Yuan d7e1fa7664 Fix feature names and types in output model slice. (#7078) 2021-07-06 11:47:49 +08:00
Jiaming Yuan ffa66aace0 Persist data in dask test. (#7077) 2021-07-06 11:47:17 +08:00
Jiaming Yuan b56d3d5d5c Fix with latest panda range index. (#7074) 2021-07-03 16:43:52 +08:00
Jiaming Yuan 93f3acdef9 Fix with latest pylint. (#7071) 2021-07-02 21:26:00 +08:00
Jiaming Yuan a5d222fcdb Handle categorical split in model histogram and dataframe. (#7065) 
* Error on get_split_value_histogram when feature is categorical
* Add a category column to output dataframe
 2021-07-02 13:10:36 +08:00
Jiaming Yuan 1cd20efe68 Move GHistIndex into DMatrix. (#7064) 2021-07-01 00:44:49 +08:00
Jiaming Yuan 1c8fdf2218 Remove use of device_idx in dh::LaunchN. (#7063) 
It's an unused parameter, removing it can make the CI log more readable.
 2021-06-29 11:37:26 +08:00
Jiaming Yuan 8fa32fdda2 Implement categorical data support for SHAP. (#7053) 
* Add CPU implementation.
* Update GPUTreeSHAP.
* Add GPU implementation by defining custom split condition.
 2021-06-25 19:02:46 +08:00
Jiaming YuanandPhilip Hyunsu Cho 663136aa08 Implement feature score for linear model. (#7048) 
* Add feature score support for linear model.
* Port R interface to the new implementation.
* Add linear model support in Python.

Co-authored-by: Philip Hyunsu Cho <chohyu01@cs.washington.edu>
 2021-06-25 14:34:02 +08:00
Jiaming Yuan 1d4d345634 Tests for dask skl categorical data support. (#7054) 2021-06-24 16:33:57 +08:00
Jiaming Yuan da1ad798ca Convert numpy float to Python float in feat score. (#7047) 2021-06-21 20:58:43 +08:00
Jiaming Yuan bbfffb444d Fix race condition in CPU shap. (#7050) 2021-06-21 10:03:15 +08:00
Jiaming Yuan 29f8fd6fee Support categorical split in tree model dump. (#7036) 2021-06-18 16:46:20 +08:00
Jiaming Yuan 7968c0d051 Test on s390x. (#7038) 
* Fix && remove unused parameter.
 2021-06-18 14:55:08 +08:00
Jiaming Yuan 86715e4cd4 Support categorical data for dask functional interface and DQM. (#7043) 
* Support categorical data for dask functional interface and DQM.

* Implement categorical data support for GPU GK-merge.
* Add support for dask functional interface.
* Add support for DQM.

* Get newer cupy.
 2021-06-18 13:06:52 +08:00
Jiaming Yuan 7dd29ffd47 Implement feature score in GBTree. (#7041) 
* Categorical data support.
* Eliminate text parsing during feature score computation.
 2021-06-18 11:53:16 +08:00
Jiaming YuanandPhilip Hyunsu Cho dcd84b3979 [CI] Configure RAPIDS, dask, modin (#7033) 
Co-authored-by: Philip Hyunsu Cho <chohyu01@cs.washington.edu>
 2021-06-18 10:27:51 +08:00
Jiaming Yuan d9799b09d0 Categorical data support for cuDF. (#7042) 
* Add support in DMatrix.
* Add support in DQM, except for iterator.
 2021-06-17 13:54:33 +08:00
Jiaming Yuan 5c2d7a18c9 Parallel model dump for trees. (#7040) 2021-06-15 14:08:26 +08:00
Jiaming Yuan 1faad825f4 Remove appveyor badge. [skip ci] (#7035) 2021-06-11 14:37:18 +08:00
Jiaming Yuan b56614e9b8 [R] Use new predict function. (#6819) 
* Call new C prediction API.
* Add `strict_shape`.
* Add `iterationrange`.
* Update document.
 2021-06-11 13:03:29 +08:00
Jiaming YuanandShvets Kirill f79cc4a7a4 Implement categorical prediction for CPU and GPU predict leaf. (#7001) 
* Categorical prediction with CPU predictor and GPU predict leaf.

* Implement categorical prediction for CPU prediction.
* Implement categorical prediction for GPU predict leaf.
* Refactor the prediction functions to have a unified get next node function.

Co-authored-by: Shvets Kirill <kirill.shvets@intel.com>
 2021-06-11 10:11:45 +08:00
Jiaming Yuan 72f9daf9b6 Fix gpu_id with custom objective. (#7015) 2021-06-09 14:51:17 +08:00
Jiaming Yuan 7beb2f7fae Hide symbols in CI build + hide symbols for C and CUDA (#6798) 
* Hide symbols in CI build.
* Hide symbols for other languages.
 2021-06-04 02:35:46 +08:00
Jiaming Yuan c4b9f4f622 Add enable_categorical to sklearn. (#7011) 2021-06-04 02:29:14 +08:00