Hendrik/xgboost
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
5,702 Commits 32 Branches 59 Tags
4d81c741e91c7660648f02d77b61ede33cef8c8d
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Jiaming Yuan 4d81c741e9 External memory support for hist (#7531) 
* Generate column matrix from gHistIndex.
* Avoid synchronization with the sparse page once the cache is written.
* Cleanups: Remove member variables/functions, change the update routine to look like approx and gpu_hist.
* Remove pruner.
 2022-03-22 00:13:20 +08:00
Jiaming Yuan cd55823112 Demo for using custom objective with multi-target regression. (#7736) 2022-03-20 17:44:25 +08:00
Jiaming Yuan 996cc705af Small cleanup to hist tree method. (#7735) 
* Remove special optimization using number of bins.
* Remove 1-based index for column sampling.
* Remove data layout.
* Unify update prediction cache.
 2022-03-20 03:44:55 +08:00
Jiaming Yuan 718472dbe2 [CI] Upgrade GitHub action Windows workers. (#7739) 2022-03-20 01:44:33 +08:00
Jiaming Yuan 9a400731d9 Replace device sync with stream sync. (#7737) 2022-03-19 23:22:23 +08:00
Jiaming Yuan da351621a1 [R] Fix parsing decision stump. (#7689) 2022-03-17 01:08:22 +08:00
Jiaming Yuan e78a38b837 Sort sparse page index when constructing DMatrix. (#7731) 2022-03-16 18:01:05 +08:00
Jiaming Yuan 98d6faefd6 Implement slope for Pseduo-Huber. (#7727) 
* Add objective and metric.
* Some refactoring for CPU/GPU dispatching using linalg module.
 2022-03-14 21:42:38 +08:00
Jiaming Yuan a62a3d991d [dask] prediction with categorical data. (#7708) 2022-03-10 00:21:48 +08:00
Jiaming Yuan 1d468e20a4 Optimize GPU evaluation function for categorical data. (#7705) 
* Use transform and cache.
 2022-02-28 17:46:29 +08:00
Jiaming Yuan 18a4af63aa Update documents and tests. (#7659) 
* Revise documents after recent refactoring and cat support.
* Add tests for behavior of max_depth and max_leaves.
 2022-02-26 03:57:47 +08:00
Jiaming Yuan 5eed2990ad Fix file descriptor leak. (#7704) 2022-02-25 17:49:33 +08:00
Jiaming Yuan 83a66b4994 Support categorical data for hist. (#7695) 
* Extract partitioner from hist.
* Implement categorical data support by passing the gradient index directly into the partitioner.
* Organize/update document.
* Remove code for negative hessian.
 2022-02-25 03:47:14 +08:00
Jiaming Yuan f60d95b0ba [R] Construct booster object in load.raw. (#7686) 2022-02-24 10:06:18 +08:00
Jiaming Yuan 6762c45494 Small cleanup to gradient index and hist. (#7668) 
* Code comments.
* Const accessor to index.
* Remove some weird variables in the `Index` class.
* Simplify the `MemStackAllocator`.
 2022-02-23 11:37:21 +08:00
Jiaming Yuan 49c74a5369 Update R package description. (#7691) 
* Change role.
* Remove cmake file when building the package.
 2022-02-23 08:36:37 +08:00
Jiaming Yuan c859764d29 [doc] Clarify that states in callbacks are mutated. (#7685) 
* Fix copy for cv.  This prevents inserting default callbacks into the input list.
* Clarify the behavior of callbacks in training/cv.
* Fix typos in doc.
 2022-02-22 11:45:00 +08:00
Jiaming Yuan 584bae1fc6 Fix document build with scikit-learn (#7684) 
* Require sphinx >= 4.4 for RTD.

* Install sklearn.
 2022-02-22 08:58:54 +08:00
Jiaming Yuan e56d1779e1 Require Python 3.7. (#7682) 
* Update setup.py.
 2022-02-21 05:46:48 +08:00
Jiaming Yuan 549f3bd781 Honor CPU counts from CFS. (#7654) 2022-02-21 03:13:26 +08:00
Jiaming Yuan 671b3c8d8e Fix typo. (#7680) 2022-02-20 03:42:47 +08:00
Jiaming Yuan b2341eab0c [R] Fix broken links. (#7670) 2022-02-20 00:55:48 +08:00
Jiaming Yuan f08c5dcb06 Cleanup some pylint errors. (#7667) 
* Cleanup some pylint errors.

* Cleanup pylint errors in rabit modules.
* Make data iter an abstract class and cleanup private access.
* Cleanup no-self-use for booster.
 2022-02-19 18:53:12 +08:00
Jiaming Yuan b76c5d54bf Define export symbols in callback module. (#7665) 2022-02-19 18:52:41 +08:00
Jiaming Yuan 7366d3b20c Ensure models with categorical splits don't use old binary format. (#7666) 2022-02-19 08:05:28 +08:00
Jiaming Yuan 14d61b0141 [doc] Update document for building from source. (#7664) 
- Mention standard install command for R package.
- Remove repeated "get source" step.
- Remove troubleshooting on Windows.  It's outdated considering VS 2022 is already out.
 2022-02-19 04:57:03 +08:00
Jiaming Yuan d625dc2047 Work around nvcc error. (#7673) 2022-02-19 01:41:46 +08:00
Jiaming Yuan 3877043d41 Avoid print for R package. (#7672) 2022-02-18 08:06:24 +08:00
Jiaming Yuan 711f7f3851 Avoid std::terminate for R package. (#7661) 
This is part of CRAN policies.
 2022-02-17 01:27:20 +08:00
Jiaming Yuan 12949c6b31 [R] Implement feature weights. (#7660) 2022-02-16 22:20:52 +08:00
Jiaming Yuan 93eebe8664 [doc] Fix broken link. [skip ci] (#7655) 2022-02-15 14:07:34 +08:00
Jiaming Yuan 0da7d872ef [doc] Update for prediction. (#7648) 2022-02-15 05:01:55 +08:00
Jiaming Yuan 0d0abe1845 Support optimal partitioning for GPU hist. (#7652) 
* Implement `MaxCategory` in quantile.
* Implement partition-based split for GPU evaluation.  Currently, it's based on the existing evaluation function.
* Extract an evaluator from GPU Hist to store the needed states.
* Added some CUDA stream/event utilities.
* Update document with references.
* Fixed a bug in approx evaluator where the number of data points is less than the number of categories.
 2022-02-15 03:03:12 +08:00
Jiaming Yuan 2369d55e9a Add tests for prediction cache. (#7650) 
* Extract the test from approx for other tree methods.
* Add note on how it works.
 2022-02-15 00:28:00 +08:00
Jiaming Yuan 5cd1f71b51 [dask] Improve configuration for port. (#7645) 
- Try port 0 to let the OS return the available port.
- Add port configuration.
 2022-02-14 21:34:34 +08:00
Jiaming Yuan b52c4e13b0 [dask] Fix empty partition with pandas input. (#7644) 
Empty partition is different from empty dataset.  For the former case, each worker has
non-empty dask collections, but each collection might contain empty partition.
 2022-02-14 19:35:51 +08:00
Jiaming Yuan 1f020a6097 Add maintainer for R package. (#7649) 2022-02-12 23:45:30 +08:00
Jiaming Yuan 1441a6cd27 [CI] Update R cache. (#7646) 2022-02-11 19:50:11 +08:00
Jiaming Yuan 2775c2a1ab Prepare external memory support for hist. (#7638) 
This PR prepares the GHistIndexMatrix to host the column matrix which is used by the hist tree method by accepting sparse_threshold parameter.

Some cleanups are made to ensure the correct batch param is being passed into DMatrix along with some additional tests for correctness of SimpleDMatrix.
 2022-02-10 16:58:02 +08:00
Jiaming Yuan fe4ce920b2 [dask] Cleanup dask module. (#7634) 
* Add a new utility for mapping function onto workers.
* Unify the type for feature names.
* Clean up the iterator.
* Fix prediction with DaskDMatrix worker specification.
* Fix base margin with DeviceQuantileDMatrix.
* Support vs 2022 in setup.py.
 2022-02-08 20:41:46 +08:00
Jiaming Yuan 926af9951e Add missing train parameter for sklearn interface. (#7629) 
Some other parameters are still missing and rely on **kwargs, for instance parameters from
dart.
 2022-02-08 13:20:19 +08:00
Jiaming Yuan 3e693e4f97 [dask] Fix nthread config with dask sklearn wrapper. (#7633) 2022-02-08 06:38:32 +08:00
Jiaming Yuan 81210420c6 Remove omp_get_max_threads (#7608) 
This is the one last PR for removing omp global variable.

* Add context object to the `DMatrix`.  This bridges `DMatrix` with https://github.com/dmlc/xgboost/issues/7308 .
* Require context to be available at the construction time of booster.
* Add `n_threads` support for R csc DMatrix constructor.
* Remove `omp_get_max_threads` in R glue code.
* Remove threading utilities that rely on omp global variable.
 2022-01-28 16:09:22 +08:00
Jiaming Yuan e060519d4f Avoid regenerating the gradient index for approx. (#7591) 2022-01-26 21:41:30 +08:00
Jiaming Yuan 5d7818e75d Remove omp_get_max_threads in tree updaters. (#7590) 2022-01-26 19:55:47 +08:00
Jiaming Yuan 24789429fd Support latest pandas Index type. (#7595) 2022-01-26 18:20:10 +08:00
Jiaming Yuan 6967ef7267 Remove omp_get_max_threads in objective. (#7589) 2022-01-24 04:35:49 +08:00
Jiaming Yuan 5817840858 Remove omp_get_max_threads in data. (#7588) 2022-01-24 02:44:07 +08:00
Jiaming Yuan f84291c1e1 Fix max_cat_to_onehot doc annotation [skip ci] (#7592) 2022-01-23 16:33:23 +08:00
Jiaming Yuan d262503781 [R] Implement new save raw in R. (#7571) 2022-01-22 20:55:47 +08:00
Jiaming Yuan ef4dae4c0e [dask] Add scheduler address to dask config. (#7581) 
- Add user configuration.
- Bring back to the logic of using scheduler address from dask.  This was removed when we were trying to support GKE, now we bring it back and let xgboost try it if direct guess or host IP from user config failed.
 2022-01-22 01:56:32 +08:00
Jiaming Yuan 5ddd4a9d06 Small cleanup to tests. (#7585) 
* Use random port in dask tests to avoid warnings for occupied port.
* Increase the difficulty of AUC tests.
 2022-01-21 06:26:57 +00:00
Jiaming Yuan 529cf8a54a Configure cub version automatically. (#7579) 
Note that when cub inside CUDA is being used, XGBoost performs checks on input size
instead of using internal cub function to accept inputs larger than maximum integer.
 2022-01-20 19:49:26 +08:00
Jiaming Yuan ac7a36367c [jvm-packages] Implement new save_raw in jvm-packages. (#7570) 
* New `toByteArray` that accepts a parameter for format.
 2022-01-19 16:00:14 +08:00
Jiaming Yuan b4ec1682c6 Update document for multi output and categorical. (#7574) 
* Group together categorical related parameters.
* Update documents about multioutput and categorical.
 2022-01-19 04:35:17 +08:00
Jiaming Yuan dac9eb13bd Implement new save_raw in Python. (#7572) 
* Expose the new C API function to Python.
* Remove old document and helper script.
* Small optimization to the `save_raw` and Json ctors.
 2022-01-19 02:27:51 +08:00
Jiaming Yuan 9f20a3315e Test with latest numpy. (#7573) 2022-01-19 00:46:23 +08:00
Jiaming Yuan bb56bb9a13 Fix merge conflict. (#7577) 2022-01-18 23:01:34 +08:00
Jiaming Yuan cc06fab9a7 Support distributed CPU env for categorical data. (#7575) 
* Add support for cat data in sketch allreduce.
* Share tests between CPU and GPU.
 2022-01-18 21:56:07 +08:00
Jiaming Yuan deab0e32ba Validate out of range categorical value. (#7576) 
* Use float in CPU categorical set to preserve the input value.
* Check out of range values.
 2022-01-18 20:16:19 +08:00
Jiaming Yuan d6ea5cc1ed Cover approx tree method for categorical data tests. (#7569) 
* Add tree to df tests.
* Add plotting tests.
* Add histogram tests.
 2022-01-16 11:31:40 +08:00
Jiaming Yuan 465dc63833 Fix tree param feature type. (#7565) 2022-01-16 04:46:29 +08:00
Jiaming Yuan a1bcd33a3b [breaking] Change internal model serialization to UBJSON. (#7556) 
* Use typed array for models.
* Change the memory snapshot format.
* Add new C API for saving to raw format.
 2022-01-16 02:11:53 +08:00
Jiaming Yuan 13b0fa4b97 Implement get_group. (#7564) 2022-01-16 02:07:42 +08:00
Jiaming Yuan 52277cc3da Rename build info function to be consistent with rest of the API. (#7553) 2022-01-14 00:39:28 +08:00
Jiaming Yuan e94b766310 Fix early stopping with linear model. (#7554) 2022-01-13 21:53:06 +08:00
Jiaming Yuan e5e47c3c99 Clarify the behavior of invalid categorical value handling. (#7529) 2022-01-13 16:11:52 +08:00
Jiaming Yuan 2db808021d Silent some warnings for unused variable. (#7548) 2022-01-11 01:16:26 +08:00
Jiaming Yuan c635d4c46a Implement ubjson. (#7549) 
* Implement ubjson.

This is a partial implementation of UBJSON with support for typed arrays.  Some missing
features are `f64`, typed object, and the no-op.
 2022-01-10 23:24:23 +08:00
Jiaming Yuan 001503186c Rewrite approx (#7214) 
This PR rewrites the approx tree method to use codebase from hist for better performance and code sharing.

The rewrite has many benefits:
- Support for both `max_leaves` and `max_depth`.
- Support for `grow_policy`.
- Support for mono constraint.
- Support for feature weights.
- Support for easier bin configuration (`max_bin`).
- Support for categorical data.
- Faster performance for most of the datasets. (many times faster)
- Support for prediction cache.
- Significantly better performance for external memory.
- Unites the code base between approx and hist.
 2022-01-10 21:15:05 +08:00
Jiaming Yuan ed95e77752 [jvm-packages] Update JNI header. (#7550) 2022-01-10 14:59:40 +08:00
Jiaming Yuan 91c1a1c52f Fix index type for bitfield. (#7541) 2022-01-05 19:23:29 +08:00
Jiaming Yuan 0df2ae63c7 Fix num_boosted_rounds for linear model. (#7538) 
* Add note.

* Fix n boosted rounds.
 2022-01-05 03:29:33 +08:00
Jiaming Yuan 28af6f9abb Remove omp_get_max_threads in gbm and linear. (#7537) 
* Use ctx in gbm.

* Use ctx threads in gbm and linear.
 2022-01-05 03:28:52 +08:00
Jiaming Yuan eea094e1bc Remove some warnings from clang. (#7533) 
* Unused variable.
* Unnecessary virtual function.
 2022-01-05 03:28:21 +08:00
Jiaming Yuan ec56d5869b [doc] Include dask examples into doc. (#7530) 2022-01-05 03:27:22 +08:00
Jiaming Yuan 54582f641a [doc] Use cross references in sphinx doc. (#7522) 
* Use cross references instead of URL.
* Fix auto doc for callback.
 2022-01-05 03:21:25 +08:00
Jiaming Yuan eb1efb54b5 Define feature_names_in_. (#7526) 
* Define `feature_names_in_`.
* Raise attribute error if it's not defined.
 2022-01-05 01:35:34 +08:00
Jiaming Yuan 8f0a42a266 Initial support for multi-label classification. (#7521) 
* Add support in sklearn classifier.
 2022-01-04 23:58:21 +08:00
Jiaming Yuan 68cdbc9c16 Remove omp_get_max_threads in CPU predictor. (#7519) 
This is part of the on going effort to remove the dependency on global omp variables.
 2022-01-04 22:12:15 +08:00
Jiaming Yuan 7f399eac8b Use double for GPU Hist node sum. (#7507) 2021-12-22 08:41:35 +08:00
Jiaming Yuan eabec370e4 [R] Fix single sample prediction. (#7524) 2021-12-21 14:11:07 +08:00
Jiaming Yuan 58a6723eb1 Initial support for multioutput regression. (#7514) 
* Add num target model parameter, which is configured from input labels.
* Change elementwise metric and indexing for weights.
* Add demo.
* Add tests.
 2021-12-18 09:28:38 +08:00
Jiaming Yuan 9ab73f737e Extract Sketch Entry from hist maker. (#7503) 
* Extract Sketch Entry from hist maker.

* Add a new sketch container for sorted inputs.
* Optimize bin search.
 2021-12-18 05:36:56 +08:00
Jiaming Yuan 5b1161bb64 Convert labels into tensor. (#7456) 
* Add a new ctor to tensor for `initilizer_list`.
* Change labels from host device vector to tensor.
* Rename the field from `labels_` to `labels` since it's a public member.
 2021-12-17 00:58:35 +08:00
Jiaming Yuan 6f8a4633b7 Fix Python typehint with upgraded mypy. (#7513) 2021-12-16 23:08:08 +08:00
Jiaming Yuan 70b12d898a [dask] Fix ddqdm with empty partition. (#7510) 
* Fix empty partition.

* war.
 2021-12-16 20:37:29 +08:00
Jiaming Yuan a512b4b394 [doc] Promote dask from experimental. [skip ci] (#7509) 2021-12-16 14:17:06 +08:00
Jiaming Yuan 05497a9141 [dask] Fix asyncio. (#7508) 2021-12-13 01:48:25 +08:00
Jiaming Yuan 01152f89ee Remove unused parameters. (#7499) 2021-12-09 14:24:51 +08:00
Jiaming Yuan 021f8bf28b Fix pylint. (#7498) 2021-12-07 13:23:30 +08:00
Jiaming Yuan eee527d264 Add approx partitioner. (#7467) 2021-11-27 15:22:06 +08:00
Jiaming Yuan 85cbd32c5a Add range-based slicing to tensor view. (#7453) 2021-11-27 13:42:36 +08:00
Jiaming Yuan 557ffc4bf5 Reduce base margin to 2 dim for now. (#7455) 2021-11-27 00:46:13 +08:00
Jiaming Yuan bf7bb575b4 Test CPU histogram with cat data. (#7465) 2021-11-27 00:43:28 +08:00
Jiaming Yuan 820e1c01ef Fix macos package upload. (#7475) 
* Split up the tests.
 2021-11-24 03:43:49 +08:00
Jiaming Yuan 488f12a996 Fix github macos package upload. (#7474) 2021-11-24 00:29:11 +08:00
Jiaming Yuan c024c42dce Modernize XGBoost Python document. (#7468) 
* Use sphinx gallery to integrate examples.
* Remove mock objects.
* Add dask doc inventory.
 2021-11-23 23:24:52 +08:00
Jiaming Yuan b124a27f57 Support scipy sparse in dask. (#7457) 2021-11-23 16:45:36 +08:00
Jiaming Yuan 5262e933f7 Remove unnecessary constexpr. (#7466) 2021-11-23 16:42:08 +08:00