Hendrik/xgboost
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
5,538 Commits 32 Branches 59 Tags
b06040b6d0e0c5d7f8bd60f0b7094e3ab34c41bc
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Jiaming Yuan b06040b6d0 Implement a general array view. (#7365) 
* Replace existing matrix and vector view.

This is to prepare for handling higher dimension data and prediction when we support multi-target models.
 2021-11-05 04:16:11 +08:00
Jiaming Yuan 232144ca09 Add note about CRAN release [skip ci] (#7395) 2021-11-05 00:34:14 +08:00
Jiaming Yuan 4100827971 Pass infomation about objective to tree methods. (#7385) 
* Define the `ObjInfo` and pass it down to every tree updater.
 2021-11-04 01:52:44 +08:00
Jiaming Yuan ccdabe4512 Support building gradient index with cat data. (#7371) 2021-11-03 22:37:37 +08:00
Jiaming Yuan 57a4b4ff64 Handle OMP_THREAD_LIMIT. (#7390) 2021-11-03 15:44:38 +08:00
Jiaming Yuan e6ab594e14 Change shebang used in CLI demo. (#7389) 
Change from system Python to environment python3.  For Ubuntu 20.04, only `python3` is
available and there's no `python`.  So at least `python3` is consistent with Python
virtual env, Ubuntu and anaconda.
 2021-11-02 22:11:19 +08:00
Jiaming Yuan a55d43ccfd Add test for invalid categorical data values. (#7380) 
* Add test for invalid categorical data values.

* Add check during sketching.
 2021-11-02 18:00:52 +08:00
Jiaming Yuan c74df31bf9 Cleanup the train function. (#7377) 
* Move attribute setter to callback.
* Remove the internal train function.
* Remove unnecessary initialization.
 2021-11-02 18:00:26 +08:00
Jiaming Yuan 154b15060e Move callbacks from fit to __init__. (#7375) 2021-11-02 17:51:42 +08:00
Jiaming Yuan 32e673d8c4 Support building with CTK11.5. (#7379) 
* Support building with CTK11.5.

* Require system cub installation for CTK11.4+.
* Check thrust version for segmented sort.
 2021-11-02 16:22:26 +08:00
Jiaming Yuan a13321148a Support multi-class with base margin. (#7381) 
This is already partially supported but never properly tested. So the only possible way to use it is calling `numpy.ndarray.flatten` with `base_margin` before passing it into XGBoost. This PR adds proper support
for most of the data types along with tests.
 2021-11-02 13:38:00 +08:00
Jiaming Yuan 6295dc3b67 Fix span reverse iterator. (#7387) 
* Fix span reverse iterator.

* Disable `rbegin` on device code to avoid calling host function.
* Add `trbegin` and friends.
 2021-11-02 13:35:59 +08:00
Jiaming Yuan 8211e5f341 Add clang-format config. (#7383) 
Generated using `clang-format -style=google -dump-config > .clang-format`, with column
width changed from 80 to 100 to be consistent with existing cpplint check.
 2021-11-02 13:34:38 +08:00
Jiaming Yuan 0f7a9b42f1 Use double precision in metric calculation. (#7364) 2021-11-02 12:00:32 +08:00
Jiaming YuanandHyunsu Cho 239dbb3c0a Move macos test to github action. (#7382) 
Co-authored-by: Hyunsu Cho <chohyu01@cs.washington.edu>
 2021-10-30 14:40:32 +08:00
Jiaming Yuan c6769488b3 Typehint for subset of core API. (#7348) 2021-10-28 20:47:04 +08:00
Jiaming Yuan 45aef75cca Move skl eval_metric and early_stopping rounds to model params. (#6751) 
A new parameter `custom_metric` is added to `train` and `cv` to distinguish the behaviour from the old `feval`.  And `feval` is deprecated.  The new `custom_metric` receives transformed prediction when the built-in objective is used.  This enables XGBoost to use cost functions from other libraries like scikit-learn directly without going through the definition of the link function.

`eval_metric` and `early_stopping_rounds` in sklearn interface are moved from `fit` to `__init__` and is now saved as part of the scikit-learn model.  The old ones in `fit` function are now deprecated. The new `eval_metric` in `__init__` has the same new behaviour as `custom_metric`.

Added more detailed documents for the behaviour of custom objective and metric.
 2021-10-28 17:20:20 +08:00
Jiaming Yuan 6b074add66 Update setup.py. (#7360) 
* Add new classifiers.
* Typehint.
 2021-10-28 14:58:31 +08:00
Jiaming Yuan 3c4aa9b2ea [breaking] Remove label encoder deprecated in 1.3. (#7357) 2021-10-28 13:24:29 +08:00
Jiaming Yuan d05754f558 Avoid OMP reduction in AUC. (#7362) 2021-10-28 05:03:52 +08:00
Jiaming Yuan ac9bfaa4f2 Handle missing values in dataframe with category dtype. (#7331) 
* Replace -1 in pandas initializer.
* Unify `IsValid` functor.
* Mimic pandas data handling in cuDF glue code.
* Check invalid categories.
* Fix DDM sketching.
 2021-10-28 03:33:54 +08:00
Jiaming Yuan 2eee87423c Remove old custom objective demo. (#7369) 
We have 2 new custom objective demos covering both regression and classification with
accompanying tutorials in documents.
 2021-10-27 16:31:48 +08:00
Jiaming Yuan b9414b6477 Update GPU doc for PR-AUC. [skip ci] (#7368) 2021-10-27 16:31:07 +08:00
Jiaming Yuan d4349426d8 Re-implement PR-AUC. (#7297) 
* Support binary/multi-class classification, ranking.
* Add documents.
* Handle missing data.
 2021-10-26 13:07:50 +08:00
Jiaming Yuan fd61c61071 Avoid omp reduction in rank metric. (#7349) 2021-10-22 14:13:34 +08:00
Jiaming Yuan e36b066344 [doc] Document the status of RTD hosting. [skip ci] (#7353) 2021-10-22 14:12:55 +08:00
Jiaming Yuan 864d236a82 [doc] Remove num_pbuffer. [skip ci] (#7356) 2021-10-22 14:12:32 +08:00
Jiaming YuanandPhilip Hyunsu Cho 7593fa9982 1.5 release note. [skip ci] (#7271) 
Co-authored-by: Philip Hyunsu Cho <chohyu01@cs.washington.edu>
 2021-10-21 13:43:31 +08:00
Jiaming Yuan d1f00fb0b7 Stricter validation for group. (#7345) 2021-10-21 12:13:33 +08:00
Jiaming Yuan 8d7c6366d7 Accept histogram cut instead gradient index in evaluation. (#7336) 2021-10-20 18:04:46 +08:00
Jiaming Yuan 15685996fc [doc] Small improvements for categorical data document. (#7330) 2021-10-20 18:04:32 +08:00
Jiaming Yuan f999897615 [dask] Use nthread in DMatrix construction. (#7337) 
This is consistent with the thread overriding behavior.
 2021-10-20 15:16:40 +08:00
Jiaming Yuan 3b0b74fa94 [doc] Use RTD theme. (#7346) 2021-10-19 23:49:19 -07:00
Jiaming Yuan 376b448015 [doc] Fix broken links. (#7341) 
* Fix most of the link checks from sphinx.
* Remove duplicate explicit target name.
 2021-10-20 14:45:30 +08:00
Jiaming Yuan f53da412aa Add typehint to tracker. (#7338) 2021-10-20 12:49:36 +08:00
Jiaming Yuan 5ff210ed75 Small fix for the release doc and script. [skip ci] (#7332) 
Add Philip as co-maintainer of maven packages.
 2021-10-20 12:49:12 +08:00
Jiaming Yuan c42e3fbcf3 [doc] Fix early stopping document. (#7334) 2021-10-18 11:21:16 -07:00
Jiaming Yuan fbb0dc4275 Remove auto configuration of seed_per_iteration. (#7009) 
* Remove auto configuration of seed_per_iteration.

This should be related to model recovery from rabit, which is removed.

* Document.
 2021-10-17 15:58:57 +08:00
Jiaming Yuan fb1a9e6bc5 Avoid omp reduction in coordinate descent and aft metrics. (#7316) 
Aside from the omp issue, parameter configuration for aft metric is simplified.
 2021-10-17 15:55:49 +08:00
Jiaming Yuan f56e2e9a66 Support categorical data with pandas Dataframe in inplace prediction (#7322) 2021-10-17 14:32:06 +08:00
Jiaming Yuan 8e619010d0 Extract CPUExpandEntry and HistParam. (#7321) 
* Remove kRootNid.
* Check for empty hessian.
 2021-10-17 14:22:25 +08:00
Jiaming Yuan 6cdcfe8128 Improve external memory demo. (#7320) 
* Use npy format.
* Add evaluation.
* Use make_regression.
 2021-10-17 11:25:24 +08:00
Jiaming Yuan e6a142fe70 Fix document about best_iteration (#7324) 2021-10-14 15:30:46 -07:00
Jiaming Yuan 4ddf8d001c Deterministic result for element-wise/mclass metrics. (#7303) 
Remove openmp reduction.
 2021-10-13 14:22:40 +08:00
Jiaming Yuan 406c70ba0e [doc] Fix typo. [skip ci] (#7311) 2021-10-12 19:10:18 +08:00
Jiaming Yuan 0bd8f21e4e Add document for categorical data. (#7307) 2021-10-12 16:10:59 +08:00
Jiaming Yuan a7d0c66457 Remove unused code. (#7293) 2021-10-12 15:04:41 +08:00
Jiaming Yuan 130df8cdda Add tests for tree grow policy. (#7302) 2021-10-12 15:04:06 +08:00
Jiaming Yuan 5b17bb0031 Fix prediction with cat data in sklearn interface. (#7306) 
* Specify DMatrix parameter for pre-processing dataframe.
* Add document about the behaviour of prediction.
 2021-10-12 14:31:12 +08:00
Jiaming Yuan 89d87e5331 Update GPU Tree SHAP (#7304) 2021-10-11 21:39:50 +08:00
Jiaming Yuan 298af6f409 Fix weighted samples in multi-class AUC. (#7300) 2021-10-11 15:12:29 +08:00
Jiaming Yuan 69d3b1b8b4 Remove old callback deprecated in 1.3. (#7280) 2021-10-08 17:24:59 +08:00
Jiaming Yuan 578de9f762 Fix cv verbose_eval (#7291) 2021-10-08 12:28:38 +08:00
Jiaming Yuan f7caac2563 Bump version to 1.6.0 in master. (#7259) 2021-10-07 16:09:26 +08:00
Jiaming Yuan e2660ab8f3 Extend release script with R packages. [skip ci] (#7278) 2021-10-07 16:08:42 +08:00
Jiaming Yuan d8cb395380 Fix gamma neg log likelihood. (#7275) 2021-10-05 16:57:08 +08:00
Jiaming Yuan b3b03200e2 Remove old warning in 1.3 (#7279) 2021-10-01 08:05:50 +08:00
Jiaming Yuan b2d8431aea [R] Fix document for nthread. (#7263) 2021-09-28 11:46:24 +08:00
Jiaming Yuan d8a549e6ac Avoid thread block with sparse data. (#7255) 2021-09-25 13:11:34 +08:00
Jiaming YuanandPhilip Hyunsu Cho ca17f8a5fc Dispatch thrust versions and upgrade rmm. (#7254) 
Co-authored-by: Philip Hyunsu Cho <chohyu01@cs.washington.edu>
 2021-09-25 03:43:23 +08:00
Jiaming Yuan fbd58bf190 [jvm-packages] Create demo and test for xgboost4j early stopping. (#7252) 2021-09-25 03:29:27 +08:00
Jiaming Yuan 9472be7d77 Fix initialization from pandas series. (#7243) 2021-09-23 04:43:25 +08:00
Jiaming Yuan 18bd16341a Update Python intro. [skip ci] (#7235) 
* Fix the link to demo.
* Stop recommending text file inputs.
* Brief mention to scikit-learn interface.
* Fix indent warning in tree method doc.
 2021-09-21 02:47:09 +00:00
Jiaming Yuan e48e05e6e2 Add typehint to rabit module. (#7240) 2021-09-17 18:31:02 +08:00
Jiaming Yuan c735c17f33 Disable callback and ES on random forest. (#7236) 2021-09-17 18:21:17 +08:00
Jiaming Yuan c311a8c1d8 Enable compiling with system cub. (#7232) 
- Tested with all CUDA 11.x.
- Workaround cub scan by using discard iterator in AUC.
- Limit the size of Argsort when compiled with CUDA cub.
 2021-09-17 14:28:18 +08:00
Jiaming Yuan b18f5f61b0 Fix pylint (#7241) 2021-09-17 11:50:36 +08:00
Jiaming Yuan 38a23f66a8 Fix typo in release script. [skip ci] (#7238) 2021-09-17 11:14:05 +08:00
Jiaming Yuan 8ad7e8eeb0 [doc] Fix typo. [skip ci] (#7226) 2021-09-17 11:13:49 +08:00
Jiaming Yuan 22d56cebf1 Encode pandas categorical data automatically. (#7231) 2021-09-17 11:09:55 +08:00
Jiaming Yuan 32e0858501 Fix travis. (#7237) 2021-09-17 10:06:23 +08:00
Jiaming Yuan 31c1e13f90 Categorical data support in CPU sketching. (#7221) 2021-09-17 04:37:09 +08:00
Jiaming Yuan 9f63d6fead [jvm-packages] Deprecate constructors with implicit missing value. (#7225) 2021-09-17 04:35:04 +08:00
Jiaming Yuan 0ed979b096 Support more input types for categorical data. (#7220) 
* Support more input types for categorical data.

* Shorten the type name from "categorical" to "c".
* Tests for np/cp array and scipy csr/csc/coo.
* Specify the type for feature info.
 2021-09-16 20:39:30 +08:00
Jiaming Yuan 2942dc68e4 Fix mixed types in GPU sketching. (#7228) 2021-09-16 00:10:25 +08:00
Jiaming Yuan 037dd0820d Implement __sklearn_is_fitted__. (#7230) 2021-09-15 19:09:04 +08:00
Jiaming Yuan d997c967d5 Demo for experimental categorical data support. (#7213) 2021-09-15 08:20:12 +08:00
Jiaming Yuan 3515931305 Initial support for external memory in gradient index. (#7183) 
* Add hessian to batch param in preparation of new approx impl.
* Extract a push method for gradient index matrix.
* Use span instead of vector ref for hessian in sketching.
* Create a binary format for gradient index.
 2021-09-13 12:40:56 +08:00
Jiaming Yuan 804b2ac60f Expose DMatrix API for CUDA columnar and array. (#7217) 
* Use JSON encoded configurations.
* Expose them into header file.
 2021-09-09 17:55:25 +08:00
Jiaming Yuan 68a2c7b8d6 Fix memory leak in demo. (#7216) 2021-09-09 13:51:03 +08:00
Jiaming Yuan b12e7f7edd Add noexcept to JSON objects. (#7205) 2021-09-07 13:56:48 +08:00
Jiaming Yuan 3a4f51f39f Avoid calling CUDA code on CPU for linear model. (#7154) 2021-09-01 10:45:31 +08:00
Jiaming Yuan ba69244a94 Restore the custom double atomic add. (#7198) 2021-08-28 18:30:42 +08:00
Jiaming YuanandAndy Adinets 7a1d67f9cb [breaking] Use integer atomic for GPU histogram. (#7180) 
On GPU we use rouding factor to truncate the gradient for deterministic results. This PR changes the gradient representation to fixed point number with exponent aligned with rounding factor.

    [breaking] Drop non-deterministic histogram.
    Use fixed point for shared memory.

This PR is to improve the performance of GPU Hist. 

Co-authored-by: Andy Adinets <aadinets@nvidia.com>
 2021-08-28 05:17:05 +08:00
Jiaming Yuan e7d7ab6bc3 Better error message for ncclUnhandledCudaError. (#7190) 2021-08-27 10:29:22 +08:00
Jiaming Yuan cdfaa705f3 Fix building on CUDA 11.0. (#7187) 2021-08-25 02:57:53 -07:00
Jiaming Yuan 9c64618cb6 [breaking] Remove CUDA sm_35, add sm_86 (#7182) 2021-08-25 16:04:23 +08:00
Jiaming Yuan ee8d1f5ed8 Fix histogram truncation. (#7181) 
* Fix truncation.

* Lint.

* lint.
 2021-08-24 18:34:32 -07:00
Jiaming Yuan 3290a4f3ed Re-enable feature validation in predict proba. (#7177) 2021-08-22 15:28:08 +08:00
Jiaming Yuan bf562bd33c Remove unused code. (#7175) 2021-08-18 14:02:19 +08:00
Jiaming Yuan 6bcbc77226 [doc] Fix typo. [skip ci] (#7170) 2021-08-13 03:48:16 +08:00
Jiaming Yuan 3f38d983a6 Fix prediction configuration. (#7159) 
After the predictor parameter was added to the constructor, this configuration was broken.
 2021-08-11 16:34:36 +08:00
Jiaming Yuan 9600ca83f3 Remove synchronization in monitor. (#7164) 
* Remove synchronization in monitor.

Calling rabit functions during destruction is flaky.

* Add xgboost prefix to nvtx marker.
 2021-08-11 16:33:53 +08:00
Jiaming Yuan 149f209af6 Extract histogram builder from CPU Hist. (#7152) 
* Extract the CPU histogram builder.
* Fix tests.
* Reduce number of histograms being built.
 2021-08-09 21:15:21 +08:00
Jiaming Yuan 8a84be37b8 Pass scikit learn estimator checks for regressor. (#7130) 
* Check data shape.
* Check labels.
 2021-08-03 18:58:20 +08:00
Jiaming Yuan 8ee127469f [R] Fix nthread in DMatrix constructor. (#7127) 
* Break the R C API for nthread.
 2021-08-03 17:39:25 +08:00
Jiaming Yuan ba47eda61b [doc] Use figure directive. (#7143) 2021-08-03 15:56:25 +08:00
Jiaming Yuan e2c406f5c8 Support min_delta in early stopping. (#7137) 
* Support `min_delta` in early stopping.

* Remove abs_tol.
 2021-08-03 14:29:17 +08:00
Jiaming Yuan 7bdedacb54 Document for process_type. (#7135) 
* Update document for prune and refresh.

* Add demo.
 2021-08-03 13:11:52 +08:00
Jiaming Yuan d080b5a953 Fix model slicing. (#7149) 
* Use correct pointer.
* Remove best_iteration/best_score.
 2021-08-03 11:51:56 +08:00