Hendrik/xgboost
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
5,621 Commits 32 Branches 59 Tags
9f20a3315e4dd8daae200d29bab60d1305bd8745
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Jiaming Yuan 9f20a3315e Test with latest numpy. (#7573) 2022-01-19 00:46:23 +08:00
Jiaming Yuan bb56bb9a13 Fix merge conflict. (#7577) 2022-01-18 23:01:34 +08:00
Jiaming Yuan cc06fab9a7 Support distributed CPU env for categorical data. (#7575) 
* Add support for cat data in sketch allreduce.
* Share tests between CPU and GPU.
 2022-01-18 21:56:07 +08:00
Jiaming Yuan deab0e32ba Validate out of range categorical value. (#7576) 
* Use float in CPU categorical set to preserve the input value.
* Check out of range values.
 2022-01-18 20:16:19 +08:00
Jiaming Yuan d6ea5cc1ed Cover approx tree method for categorical data tests. (#7569) 
* Add tree to df tests.
* Add plotting tests.
* Add histogram tests.
 2022-01-16 11:31:40 +08:00
Jiaming Yuan 465dc63833 Fix tree param feature type. (#7565) 2022-01-16 04:46:29 +08:00
Jiaming Yuan a1bcd33a3b [breaking] Change internal model serialization to UBJSON. (#7556) 
* Use typed array for models.
* Change the memory snapshot format.
* Add new C API for saving to raw format.
 2022-01-16 02:11:53 +08:00
Jiaming Yuan 13b0fa4b97 Implement get_group. (#7564) 2022-01-16 02:07:42 +08:00
Jiaming Yuan 52277cc3da Rename build info function to be consistent with rest of the API. (#7553) 2022-01-14 00:39:28 +08:00
Jiaming Yuan e94b766310 Fix early stopping with linear model. (#7554) 2022-01-13 21:53:06 +08:00
Jiaming Yuan e5e47c3c99 Clarify the behavior of invalid categorical value handling. (#7529) 2022-01-13 16:11:52 +08:00
Jiaming Yuan 2db808021d Silent some warnings for unused variable. (#7548) 2022-01-11 01:16:26 +08:00
Jiaming Yuan c635d4c46a Implement ubjson. (#7549) 
* Implement ubjson.

This is a partial implementation of UBJSON with support for typed arrays.  Some missing
features are `f64`, typed object, and the no-op.
 2022-01-10 23:24:23 +08:00
Jiaming Yuan 001503186c Rewrite approx (#7214) 
This PR rewrites the approx tree method to use codebase from hist for better performance and code sharing.

The rewrite has many benefits:
- Support for both `max_leaves` and `max_depth`.
- Support for `grow_policy`.
- Support for mono constraint.
- Support for feature weights.
- Support for easier bin configuration (`max_bin`).
- Support for categorical data.
- Faster performance for most of the datasets. (many times faster)
- Support for prediction cache.
- Significantly better performance for external memory.
- Unites the code base between approx and hist.
 2022-01-10 21:15:05 +08:00
Jiaming Yuan ed95e77752 [jvm-packages] Update JNI header. (#7550) 2022-01-10 14:59:40 +08:00
Jiaming Yuan 91c1a1c52f Fix index type for bitfield. (#7541) 2022-01-05 19:23:29 +08:00
Jiaming Yuan 0df2ae63c7 Fix num_boosted_rounds for linear model. (#7538) 
* Add note.

* Fix n boosted rounds.
 2022-01-05 03:29:33 +08:00
Jiaming Yuan 28af6f9abb Remove omp_get_max_threads in gbm and linear. (#7537) 
* Use ctx in gbm.

* Use ctx threads in gbm and linear.
 2022-01-05 03:28:52 +08:00
Jiaming Yuan eea094e1bc Remove some warnings from clang. (#7533) 
* Unused variable.
* Unnecessary virtual function.
 2022-01-05 03:28:21 +08:00
Jiaming Yuan ec56d5869b [doc] Include dask examples into doc. (#7530) 2022-01-05 03:27:22 +08:00
Jiaming Yuan 54582f641a [doc] Use cross references in sphinx doc. (#7522) 
* Use cross references instead of URL.
* Fix auto doc for callback.
 2022-01-05 03:21:25 +08:00
Jiaming Yuan eb1efb54b5 Define feature_names_in_. (#7526) 
* Define `feature_names_in_`.
* Raise attribute error if it's not defined.
 2022-01-05 01:35:34 +08:00
Jiaming Yuan 8f0a42a266 Initial support for multi-label classification. (#7521) 
* Add support in sklearn classifier.
 2022-01-04 23:58:21 +08:00
Jiaming Yuan 68cdbc9c16 Remove omp_get_max_threads in CPU predictor. (#7519) 
This is part of the on going effort to remove the dependency on global omp variables.
 2022-01-04 22:12:15 +08:00
Jiaming Yuan 7f399eac8b Use double for GPU Hist node sum. (#7507) 2021-12-22 08:41:35 +08:00
Jiaming Yuan eabec370e4 [R] Fix single sample prediction. (#7524) 2021-12-21 14:11:07 +08:00
Jiaming Yuan 58a6723eb1 Initial support for multioutput regression. (#7514) 
* Add num target model parameter, which is configured from input labels.
* Change elementwise metric and indexing for weights.
* Add demo.
* Add tests.
 2021-12-18 09:28:38 +08:00
Jiaming Yuan 9ab73f737e Extract Sketch Entry from hist maker. (#7503) 
* Extract Sketch Entry from hist maker.

* Add a new sketch container for sorted inputs.
* Optimize bin search.
 2021-12-18 05:36:56 +08:00
Jiaming Yuan 5b1161bb64 Convert labels into tensor. (#7456) 
* Add a new ctor to tensor for `initilizer_list`.
* Change labels from host device vector to tensor.
* Rename the field from `labels_` to `labels` since it's a public member.
 2021-12-17 00:58:35 +08:00
Jiaming Yuan 6f8a4633b7 Fix Python typehint with upgraded mypy. (#7513) 2021-12-16 23:08:08 +08:00
Jiaming Yuan 70b12d898a [dask] Fix ddqdm with empty partition. (#7510) 
* Fix empty partition.

* war.
 2021-12-16 20:37:29 +08:00
Jiaming Yuan a512b4b394 [doc] Promote dask from experimental. [skip ci] (#7509) 2021-12-16 14:17:06 +08:00
Jiaming Yuan 05497a9141 [dask] Fix asyncio. (#7508) 2021-12-13 01:48:25 +08:00
Jiaming Yuan 01152f89ee Remove unused parameters. (#7499) 2021-12-09 14:24:51 +08:00
Jiaming Yuan 021f8bf28b Fix pylint. (#7498) 2021-12-07 13:23:30 +08:00
Jiaming Yuan eee527d264 Add approx partitioner. (#7467) 2021-11-27 15:22:06 +08:00
Jiaming Yuan 85cbd32c5a Add range-based slicing to tensor view. (#7453) 2021-11-27 13:42:36 +08:00
Jiaming Yuan 557ffc4bf5 Reduce base margin to 2 dim for now. (#7455) 2021-11-27 00:46:13 +08:00
Jiaming Yuan bf7bb575b4 Test CPU histogram with cat data. (#7465) 2021-11-27 00:43:28 +08:00
Jiaming Yuan 820e1c01ef Fix macos package upload. (#7475) 
* Split up the tests.
 2021-11-24 03:43:49 +08:00
Jiaming Yuan 488f12a996 Fix github macos package upload. (#7474) 2021-11-24 00:29:11 +08:00
Jiaming Yuan c024c42dce Modernize XGBoost Python document. (#7468) 
* Use sphinx gallery to integrate examples.
* Remove mock objects.
* Add dask doc inventory.
 2021-11-23 23:24:52 +08:00
Jiaming Yuan b124a27f57 Support scipy sparse in dask. (#7457) 2021-11-23 16:45:36 +08:00
Jiaming Yuan 5262e933f7 Remove unnecessary constexpr. (#7466) 2021-11-23 16:42:08 +08:00
Jiaming Yuan 176110a22d Support external memory in CPU histogram building. (#7372) 2021-11-23 01:13:33 +08:00
Jiaming Yuan d33854af1b [Breaking] Accept multi-dim meta info. (#7405) 
This PR changes base_margin into a 3-dim array, with one of them being reserved for multi-target classification. Also, a breaking change is made for binary serialization due to extra dimension along with a fix for saving the feature weights. Lastly, it unifies the prediction initialization between CPU and GPU. After this PR, the meta info setter in Python will be based on array interface.
 2021-11-18 23:02:54 +08:00
Jiaming Yuan 9fb4338964 Add test for eta and mitigate float error. (#7446) 
* Add eta test.
* Don't skip test.
 2021-11-18 20:42:48 +08:00
Jiaming Yuan b0015fda96 Fix R CRAN failures. (#7404) 
* Remove hist builder dtor.

* Initialize values.

* Tolerance.

* Remove the use of nthread in col maker.
 2021-11-16 10:51:12 +08:00
Jiaming Yuan 55ee272ea8 Extend array interface to handle ndarray. (#7434) 
* Extend array interface to handle ndarray.

The `ArrayInterface` class is extended to support multi-dim array inputs. Previously this
class handles only 2-dim (vector is also matrix).  This PR specifies the expected
dimension at compile-time and the array interface can perform various checks automatically
for input data. Also, adapters like CSR are more rigorous about their input.  Lastly, row
vector and column vector are handled without intervention from the caller.
 2021-11-16 09:52:15 +08:00
Jiaming Yuan e27f543deb Set use_logger in tracker to false. (#7438) 2021-11-16 05:12:42 +08:00
Jiaming Yuan d4274bc556 Fix typo. (#7433) 2021-11-15 01:28:11 +08:00
Jiaming Yuan a7057fa64c Implement typed storage for tensor. (#7429) 
* Add `Tensor` class.
* Add elementwise kernel for CPU and GPU.
* Add unravel index.
* Move some computation to compile time.
 2021-11-14 18:53:13 +08:00
Jiaming Yuan 8cc75f1576 Cleanup Python tests. (#7426) 2021-11-14 15:47:05 +08:00
Jiaming Yuan 38ca96c9fc [CI] Install igraph as binary. (#7417) 2021-11-12 19:04:28 +08:00
Jiaming Yuan 46726ec176 Expose build info (#7399) 2021-11-12 18:22:46 +08:00
Jiaming Yuan 937fa282b5 Extract string view. (#7416) 
* Add equality operators.
* Return a view in substr.
* Add proper iterator types.
 2021-11-12 18:22:30 +08:00
Jiaming Yuan ca6f980932 Check number of trees in inplace predict. (#7409) 2021-11-12 18:20:23 +08:00
Jiaming Yuan 97d7582457 Delay breaking changes to 1.6. (#7420) 
The patch is too big to be backported.
 2021-11-12 16:46:03 +08:00
Jiaming Yuan 8df0a252b7 [doc] Update document for GPU. [skip ci] (#7403) 
* Remove outdated workaround and description.
 2021-11-09 02:05:55 +08:00
Jiaming Yuan d7d1b6e3a6 CPU evaluation for cat data. (#7393) 
* Implementation for one hot based.
* Implementation for partition based. (LightGBM)
 2021-11-06 14:41:35 +08:00
Jiaming Yuan 6ede12412c Update dmlc-core and use data iter for GPU sampling tests. (#7398) 
* Update dmlc-core.
* New parquet parser in dmlc-core.
* Use data iter for GPU sampling tests.
 2021-11-06 05:12:49 +08:00
Jiaming Yuan c968217ca8 [R] Fix global feature importance and predict with 1 sample. (#7394) 
* [R] Fix global feature importance.

* Add implementation for tree index.  The parameter is not documented in C API since we
should work on porting the model slicing to R instead of supporting more use of tree
index.

* Fix the difference between "gain" and "total_gain".

* debug.

* Fix prediction.
 2021-11-05 10:07:00 +08:00
Jiaming Yuan 48aff0eabd [doc][jvm-packages] Update information about Python tracker. [skip ci] (#7396) 2021-11-05 05:55:13 +08:00
Jiaming Yuan b06040b6d0 Implement a general array view. (#7365) 
* Replace existing matrix and vector view.

This is to prepare for handling higher dimension data and prediction when we support multi-target models.
 2021-11-05 04:16:11 +08:00
Jiaming Yuan 232144ca09 Add note about CRAN release [skip ci] (#7395) 2021-11-05 00:34:14 +08:00
Jiaming Yuan 4100827971 Pass infomation about objective to tree methods. (#7385) 
* Define the `ObjInfo` and pass it down to every tree updater.
 2021-11-04 01:52:44 +08:00
Jiaming Yuan ccdabe4512 Support building gradient index with cat data. (#7371) 2021-11-03 22:37:37 +08:00
Jiaming Yuan 57a4b4ff64 Handle OMP_THREAD_LIMIT. (#7390) 2021-11-03 15:44:38 +08:00
Jiaming Yuan e6ab594e14 Change shebang used in CLI demo. (#7389) 
Change from system Python to environment python3.  For Ubuntu 20.04, only `python3` is
available and there's no `python`.  So at least `python3` is consistent with Python
virtual env, Ubuntu and anaconda.
 2021-11-02 22:11:19 +08:00
Jiaming Yuan a55d43ccfd Add test for invalid categorical data values. (#7380) 
* Add test for invalid categorical data values.

* Add check during sketching.
 2021-11-02 18:00:52 +08:00
Jiaming Yuan c74df31bf9 Cleanup the train function. (#7377) 
* Move attribute setter to callback.
* Remove the internal train function.
* Remove unnecessary initialization.
 2021-11-02 18:00:26 +08:00
Jiaming Yuan 154b15060e Move callbacks from fit to __init__. (#7375) 2021-11-02 17:51:42 +08:00
Jiaming Yuan 32e673d8c4 Support building with CTK11.5. (#7379) 
* Support building with CTK11.5.

* Require system cub installation for CTK11.4+.
* Check thrust version for segmented sort.
 2021-11-02 16:22:26 +08:00
Jiaming Yuan a13321148a Support multi-class with base margin. (#7381) 
This is already partially supported but never properly tested. So the only possible way to use it is calling `numpy.ndarray.flatten` with `base_margin` before passing it into XGBoost. This PR adds proper support
for most of the data types along with tests.
 2021-11-02 13:38:00 +08:00
Jiaming Yuan 6295dc3b67 Fix span reverse iterator. (#7387) 
* Fix span reverse iterator.

* Disable `rbegin` on device code to avoid calling host function.
* Add `trbegin` and friends.
 2021-11-02 13:35:59 +08:00
Jiaming Yuan 8211e5f341 Add clang-format config. (#7383) 
Generated using `clang-format -style=google -dump-config > .clang-format`, with column
width changed from 80 to 100 to be consistent with existing cpplint check.
 2021-11-02 13:34:38 +08:00
Jiaming Yuan 0f7a9b42f1 Use double precision in metric calculation. (#7364) 2021-11-02 12:00:32 +08:00
Jiaming YuanandHyunsu Cho 239dbb3c0a Move macos test to github action. (#7382) 
Co-authored-by: Hyunsu Cho <chohyu01@cs.washington.edu>
 2021-10-30 14:40:32 +08:00
Jiaming Yuan c6769488b3 Typehint for subset of core API. (#7348) 2021-10-28 20:47:04 +08:00
Jiaming Yuan 45aef75cca Move skl eval_metric and early_stopping rounds to model params. (#6751) 
A new parameter `custom_metric` is added to `train` and `cv` to distinguish the behaviour from the old `feval`.  And `feval` is deprecated.  The new `custom_metric` receives transformed prediction when the built-in objective is used.  This enables XGBoost to use cost functions from other libraries like scikit-learn directly without going through the definition of the link function.

`eval_metric` and `early_stopping_rounds` in sklearn interface are moved from `fit` to `__init__` and is now saved as part of the scikit-learn model.  The old ones in `fit` function are now deprecated. The new `eval_metric` in `__init__` has the same new behaviour as `custom_metric`.

Added more detailed documents for the behaviour of custom objective and metric.
 2021-10-28 17:20:20 +08:00
Jiaming Yuan 6b074add66 Update setup.py. (#7360) 
* Add new classifiers.
* Typehint.
 2021-10-28 14:58:31 +08:00
Jiaming Yuan 3c4aa9b2ea [breaking] Remove label encoder deprecated in 1.3. (#7357) 2021-10-28 13:24:29 +08:00
Jiaming Yuan d05754f558 Avoid OMP reduction in AUC. (#7362) 2021-10-28 05:03:52 +08:00
Jiaming Yuan ac9bfaa4f2 Handle missing values in dataframe with category dtype. (#7331) 
* Replace -1 in pandas initializer.
* Unify `IsValid` functor.
* Mimic pandas data handling in cuDF glue code.
* Check invalid categories.
* Fix DDM sketching.
 2021-10-28 03:33:54 +08:00
Jiaming Yuan 2eee87423c Remove old custom objective demo. (#7369) 
We have 2 new custom objective demos covering both regression and classification with
accompanying tutorials in documents.
 2021-10-27 16:31:48 +08:00
Jiaming Yuan b9414b6477 Update GPU doc for PR-AUC. [skip ci] (#7368) 2021-10-27 16:31:07 +08:00
Jiaming Yuan d4349426d8 Re-implement PR-AUC. (#7297) 
* Support binary/multi-class classification, ranking.
* Add documents.
* Handle missing data.
 2021-10-26 13:07:50 +08:00
Jiaming Yuan fd61c61071 Avoid omp reduction in rank metric. (#7349) 2021-10-22 14:13:34 +08:00
Jiaming Yuan e36b066344 [doc] Document the status of RTD hosting. [skip ci] (#7353) 2021-10-22 14:12:55 +08:00
Jiaming Yuan 864d236a82 [doc] Remove num_pbuffer. [skip ci] (#7356) 2021-10-22 14:12:32 +08:00
Jiaming YuanandPhilip Hyunsu Cho 7593fa9982 1.5 release note. [skip ci] (#7271) 
Co-authored-by: Philip Hyunsu Cho <chohyu01@cs.washington.edu>
 2021-10-21 13:43:31 +08:00
Jiaming Yuan d1f00fb0b7 Stricter validation for group. (#7345) 2021-10-21 12:13:33 +08:00
Jiaming Yuan 8d7c6366d7 Accept histogram cut instead gradient index in evaluation. (#7336) 2021-10-20 18:04:46 +08:00
Jiaming Yuan 15685996fc [doc] Small improvements for categorical data document. (#7330) 2021-10-20 18:04:32 +08:00
Jiaming Yuan f999897615 [dask] Use nthread in DMatrix construction. (#7337) 
This is consistent with the thread overriding behavior.
 2021-10-20 15:16:40 +08:00
Jiaming Yuan 3b0b74fa94 [doc] Use RTD theme. (#7346) 2021-10-19 23:49:19 -07:00
Jiaming Yuan 376b448015 [doc] Fix broken links. (#7341) 
* Fix most of the link checks from sphinx.
* Remove duplicate explicit target name.
 2021-10-20 14:45:30 +08:00
Jiaming Yuan f53da412aa Add typehint to tracker. (#7338) 2021-10-20 12:49:36 +08:00
Jiaming Yuan 5ff210ed75 Small fix for the release doc and script. [skip ci] (#7332) 
Add Philip as co-maintainer of maven packages.
 2021-10-20 12:49:12 +08:00
Jiaming Yuan c42e3fbcf3 [doc] Fix early stopping document. (#7334) 2021-10-18 11:21:16 -07:00