Hendrik/xgboost
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
6,444 Commits 32 Branches 59 Tags
65d83e288f94765f6638d92ae4c82f0e1abbfe09
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Jiaming Yuan fe9dff339c Convert federated learner test into test suite. (#9018) 
* Convert federated learner test into test suite.

- Add specialization to learning to rank.
 2023-04-11 09:52:55 +08:00
Jiaming Yuan 2c8d735cb3 Fix tests with pandas 2.0. (#9014) 
* Fix tests with pandas 2.0.

- `is_categorical` is replaced by `is_categorical_dtype`.
- one hot encoding returns boolean type instead of integer type.
 2023-04-11 00:17:34 +08:00
Jiaming Yuan 1cf4d93246 Convert federated tests into test suite. (#9006) 
- Add specialization for learning to rank.
 2023-04-04 01:29:47 +08:00
Jiaming Yuan 720a8c3273 [doc] Remove parameter type in Python doc strings. (#9005) 2023-04-01 04:04:30 +08:00
Jiaming Yuan 4caca2947d Improve helper script for making release. [skip ci] (#9004) 
* Merge source tarball generation script.
* Generate Python source wheel.
* Generate hashes and release note.
 2023-03-31 23:14:58 +08:00
Jiaming Yuan bcb55d3b6a Portable macro definition. (#8999) 2023-03-31 20:48:59 +08:00
Jiaming Yuan bac22734fb Remove ntree limit in python package. (#8345) 
- Remove `ntree_limit`. The parameter has been deprecated since 1.4.0.
- The SHAP package compatibility is broken.
 2023-03-31 19:01:55 +08:00
Jiaming Yuan b647403baa Update release news. [skip ci] (#9000) 2023-03-31 03:52:09 +08:00
Jiaming Yuan cd05e38533 [doc][R] Update link. (#8998) 2023-03-30 19:09:07 +08:00
Jiaming Yuan d062a9e009 Define pair generation strategies for LTR. (#8984) 2023-03-30 12:00:35 +08:00
Jiaming Yuan a58055075b [dask] Return the first valid booster instead of all valid ones. (#8993) 
* [dask] Return the first valid booster instead of all valid ones.

- Reduce memory footprint of the returned model.

* mypy error.

* lint.

* duplicated.
 2023-03-30 03:16:18 +08:00
Jiaming Yuan 401ce5cf5e Run linters with the multi output demo. (#8966) 2023-03-28 00:47:28 +08:00
Jiaming Yuan acc110c251 [MT-TREE] Support prediction cache and model slicing. (#8968) 
- Fix prediction range.
- Support prediction cache in mt-hist.
- Support model slicing.
- Make the booster a Python iterable by defining `__iter__`.
- Cleanup removed/deprecated parameters.
- A new field in the output model `iteration_indptr` for pointing to the ranges of trees for each iteration.
 2023-03-27 23:10:54 +08:00
Jiaming Yuan c2b3a13e70 [breaking][skl] Remove parameter serialization. (#8963) 
- Remove parameter serialization in the scikit-learn interface.

The scikit-lear interface `save_model` will save only the model and discard all
hyper-parameters. This is to align with the native XGBoost interface, which distinguishes
the hyper-parameter and model parameters.

With the scikit-learn interface, model parameters are attributes of the estimator. For
instance, `n_features_in_`, `n_classes_` are always accessible with
`estimator.n_features_in_` and `estimator.n_classes_`, but not with the
`estimator.get_params`.

- Define a `load_model` method for classifier to load its own attributes.

- Set n_estimators to None by default.
 2023-03-27 21:34:10 +08:00
Jiaming Yuan 21a52c7f98 [doc] Add introduction and notes for the sklearn interface. (#8948) 2023-03-23 13:30:42 +08:00
Jiaming Yuan bf88dadb61 [doc] Fix callback example. (#8944) 2023-03-23 03:27:04 +08:00
Jiaming Yuan 15a2724ff7 Removed outdated configuration serialization logic. (#8942) 
- `saved_params` is empty.
- `saved_configs_` contains `num_round`, which is not used anywhere inside xgboost.
 2023-03-23 01:31:46 +08:00
Jiaming Yuan 151882dd26 Initial support for multi-target tree. (#8616) 
* Implement multi-target for hist.

- Add new hist tree builder.
- Move data fetchers for tests.
- Dispatch function calls in gbm base on the tree type.
 2023-03-22 23:49:56 +08:00
Jiaming Yuan ea04d4c46c [doc] [dask] Troubleshooting NCCL errors. (#8943) 2023-03-22 22:17:26 +08:00
Jiaming Yuan a551bed803 Remove duplicated learning rate parameter. (#8941) 2023-03-22 20:51:14 +08:00
Jiaming Yuan a05799ed39 Specify char type in JSON. (#8949) 
char is defined as signed on x86 but unsigned on arm64

- Use `std::int8_t` instead of char.
- Fix include when clang is pretending to be gcc.
 2023-03-22 19:13:44 +08:00
Jiaming Yuan 5891f752c8 Rework the MAP metric. (#8931) 
- The new implementation is more strict as only binary labels are accepted. The previous implementation converts values greater than 1 to 1.
- Deterministic GPU. (no atomic add).
- Fix top-k handling.
- Precise definition of MAP. (There are other variants on how to handle top-k).
- Refactor GPU ranking tests.
 2023-03-22 17:45:20 +08:00
Jiaming Yuan 9b6cc0ed07 Refactor hist to prepare for multi-target builder. (#8928) 
- Extract the builder from the updater class. We need a new builder for multi-target.
- Extract `UpdateTree`, it can be reused for different builders. Eventually, other tree
  updaters can use it as well.
 2023-03-17 17:21:04 +08:00
Jiaming Yuan a093770f36 Partitioner for multi-target tree. (#8922) 2023-03-16 18:49:34 +08:00
Jiaming Yuan 26209a42a5 Define git attributes for renormalization. (#8921) 2023-03-16 02:43:11 +08:00
Jiaming Yuan f186c87cf9 Check inf in data for all types of DMatrix. (#8911) 2023-03-15 11:24:35 +08:00
Jiaming Yuan 72e8331eab Reimplement the NDCG metric. (#8906) 
- Add support for non-exp gain.
- Cache the DMatrix object to avoid re-calculating the IDCG.
- Make GPU implementation deterministic. (no atomic add)
 2023-03-15 03:26:17 +08:00
Jiaming Yuan 8685556af2 Implement hist evaluator for multi-target tree. (#8908) 2023-03-15 01:42:51 +08:00
Jiaming Yuan 95e2baf7c2 [doc] Fix typo [skip ci] (#8907) 2023-03-15 00:55:17 +08:00
Jiaming Yuan 910ce580c8 Clear all cache after model load. (#8904) 2023-03-14 22:09:36 +08:00
Jiaming Yuan c400fa1e8d Predictor for vector leaf. (#8898) 2023-03-14 19:07:10 +08:00
Jiaming Yuan 8be6095ece Implement NDCG cache. (#8893) 2023-03-13 22:16:31 +08:00
Jiaming Yuan 9bade7203a Remove public access to tree model param. (#8902) 
* Make tree model param a private member.
* Number of features and targets are immutable after construction.

This is to reduce the number of places where we can run configuration.
 2023-03-13 20:55:10 +08:00
Jiaming Yuan 5ba3509dd3 Define multi expand entry. (#8895) 2023-03-13 19:31:05 +08:00
Jiaming Yuan bbee355b45 [doc][dask] Note on reproducible result. [skip ci] (#8903) 2023-03-13 19:30:35 +08:00
Jiaming YuanandPhilip Hyunsu Cho 3689695d16 [CI] Run RMM gtests. (#8900) 
* [CI] Run RMM gtests.

* Update test-cpp-gpu.sh

---------

Co-authored-by: Philip Hyunsu Cho <chohyu01@cs.washington.edu>
 2023-03-12 03:14:31 +08:00
Jiaming Yuan 36a7396658 Replace dmlc any with std any. (#8892) 2023-03-11 06:11:04 +08:00
Jiaming Yuan 2aa838c75e Define multi-strategy parameter. (#8890) 2023-03-11 02:58:01 +08:00
Jiaming Yuan 6deaec8027 Pass obj info by reference instead of by value. (#8889) 
- Pass obj info into tree updater as const pointer.

This way we don't have to initialize the learner model param before configuring gbm, hence
breaking up the dependency of configurations.
 2023-03-11 01:38:28 +08:00
Jiaming Yuan 54e001bbf4 [doc][dask] Reference examples from coiled. [skip ci] (#8891) 2023-03-09 20:03:24 -08:00
Jiaming Yuan c5c8f643f2 Remove the cub submodule. (#8888) 
XGBoost now uses CTK-11.8 for binary packages, there's no need to maintain a cub
submodule anymore.
 2023-03-09 19:43:02 -08:00
Jiaming Yuan 5feee8d4a9 Define core multi-target regression tree structure. (#8884) 
- Define a new tree struct embedded in the `RegTree`.
- Provide dispatching functions in `RegTree`.
- Fix some c++-17 warnings about the use of nodiscard (currently we disable the warning on
  the CI).
- Use uint32_t instead of size_t for `bst_target_t` as it has a defined size and can be used
  as part of dmlc parameter.
- Hide the `Segment` struct inside the categorical split matrix.
 2023-03-09 19:03:06 +08:00
Jiaming Yuan 46dfcc7d22 Define a new ranking parameter. (#8887) 2023-03-09 17:46:24 +08:00
Jiaming Yuan 8c16da8863 [doc] Add note for rabit port. [skip ci] (#8879) 2023-03-08 19:00:10 +08:00
Jiaming Yuan f236640427 Support F order for the tensor type. (#8872) 
- Add F order support for tensor and view.
- Use parameter pack for automatic type cast. (avoid excessive static cast for shape).
 2023-03-08 03:27:49 +08:00
Jiaming Yuan f7ce0ec0df Upgrade gcc toolchain to 9.x. (#8878) 
* Use new tool chain.

* Use gcc-9.

* Use cmake from system.

* DOn't link leak.
 2023-03-07 08:25:23 -08:00
Jiaming Yuan 7eba285a1e Support sklearn cross validation for ranker. (#8859) 
* Support sklearn cross validation for ranker.

- Add a convention for X to include a special `qid` column.

sklearn utilities consider only `X`, `y` and `sample_weight` for supervised learning
algorithms, but we need an additional qid array for ranking.

It's important to be able to support the cross validation function in sklearn since all
other tuning functions like grid search are based on cross validation.
 2023-03-07 00:22:08 +08:00
Jiaming Yuan cad7401783 Disable gcc parallel extension if openmp is not available. (#8871) 
`<parallel/algorithm>` internally includes the <omp.h> header, which leads to an error
when openmp is not available.
 2023-03-06 22:51:06 +08:00
Jiaming Yuan 228a46e8ad Support learning rate for zero-hessian objectives. (#8866) 2023-03-06 20:33:28 +08:00
Jiaming Yuan 173096a6a7 Discover libasan.so.6. (#8864) 2023-03-06 18:56:54 +08:00
Jiaming Yuan 6a892ce281 Specify src path for isort. (#8867) 2023-03-06 17:30:27 +08:00
Jiaming Yuan 4d665b3fb0 Restore clang tidy test. (#8861) 2023-03-03 13:47:04 -08:00
Jiaming Yuan 803d5e3c4c Update c++ requirement to 17 for the R package. (#8860) 2023-03-01 14:49:39 -08:00
Jiaming Yuan d54ef56f6f Fix cache with gc (#8851) 
- Make DMatrixCache thread-safe.
- Remove the use of thread-local memory.
 2023-03-01 00:39:06 +08:00
Jiaming Yuan c0afdb6786 Fix CPU bin compression with categorical data. (#8809) 
* Fix CPU bin compression with categorical data.

* The bug causes the maximum category to be lesser than 256 or the maximum number of bins when
the input data is dense.
 2023-02-16 04:20:34 +08:00
Jiaming Yuan cce4af4acf Initial support for quantile loss. (#8750) 
- Add support for Python.
- Add objective.
 2023-02-16 02:30:18 +08:00
Jiaming Yuan 282b1729da Specify the number of threads for parallel sort. (#8735) 
* Specify the number of threads for parallel sort.

- Pass context object into argsort.
- Replace macros with inline functions.
 2023-02-16 00:20:19 +08:00
Jiaming Yuan c7c485d052 Extract fit intercept. (#8793) 2023-02-15 22:41:31 +08:00
Jiaming Yuan 594371e35b Fix CPP lint. (#8807) 2023-02-15 20:16:35 +08:00
Jiaming Yuan e62167937b [CI] Update action cache for jvm tests. (#8806) 2023-02-15 18:43:48 +08:00
Jiaming Yuan 52d0230b58 Fix merge conflict. (#8791) 2023-02-13 23:43:42 +08:00
Jiaming Yuan 81b2ee1153 Pass DMatrix into metric for caching. (#8790) 2023-02-13 22:15:05 +08:00
Jiaming Yuan 31d3ec07af Extract device algorithms. (#8789) 2023-02-13 20:53:53 +08:00
Jiaming Yuan 457f704e3d Add quantile metric. (#8761) 2023-02-13 19:07:40 +08:00
Jiaming YuanandPhilip Hyunsu Cho d11a0044cf Generalize prediction cache. (#8783) 
* Extract most of the functionality into `DMatrixCache`.
* Move API entry to independent file to reduce dependency on `predictor.h` file.
* Add test.

---------

Co-authored-by: Philip Hyunsu Cho <chohyu01@cs.washington.edu>
 2023-02-13 12:36:43 +08:00
Jiaming Yuan 225b3158f6 Support custom metric in sklearn ranker. (#8786) 2023-02-12 13:14:07 +08:00
Jiaming Yuan 17b709acb9 Rename ranking utils to threading utils. (#8785) 2023-02-12 05:41:18 +08:00
Jiaming Yuan 70c9b885ef Extract floating point rounding routines. (#8771) 2023-02-12 04:26:41 +08:00
Jiaming Yuan e9c178f402 [doc] Document update [skip ci] (#8784) 
- Remove version specifics in cat demo.
- Remove aws yarn.
- Update faq.
- Stop mentioning MPI.
- Update sphinx inventory links.
- Fix typo.
 2023-02-12 04:25:22 +08:00
Jiaming Yuan 8a16944664 Fix ranking with quantile dmatrix and group weight. (#8762) 2023-02-10 20:32:35 +08:00
Jiaming Yuan 199c421d60 Send default configuration from metric to objective. (#8760) 2023-02-09 20:18:07 +08:00
Jiaming Yuan 5f76edd296 Extract make metric name from ranking metric. (#8768) 
- Extract the metric parsing routine from ranking.
- Add a test.
- Accept null for string view.
 2023-02-09 18:30:21 +08:00
Jiaming Yuan 4ead65a28c Increase timeout limit for linear. (#8767) 2023-02-09 18:20:12 +08:00
Jiaming Yuan 48cefa012e Support multiple alphas for segmented quantile. (#8758) 2023-02-07 17:17:59 +08:00
Jiaming Yuan c4802bfcd0 Cleanup booster param types. (#8756) 2023-02-07 15:52:19 +08:00
Jiaming Yuan 7b3d473593 [doc] Add demo for inference using individual tree. (#8752) 2023-02-07 04:40:18 +08:00
Jiaming Yuan 28bb01aa22 Extract optional weight. (#8747) 
- Extract optional weight from coommon.h to reduce dependency on this header.
- Add test.
 2023-02-07 03:11:53 +08:00
Jiaming Yuan 0f37a01dd9 Require black formatter for the python package. (#8748) 2023-02-07 01:53:33 +08:00
Jiaming Yuan a2e433a089 Fix empty DMatrix with categorical features. (#8739) 2023-02-07 00:40:11 +08:00
Jiaming Yuan c1786849e3 Use array interface for CSC matrix. (#8672) 
* Use array interface for CSC matrix.

Use array interface for CSC matrix and align the interface with CSR and dense.

- Fix nthread issue in the R package DMatrix.
- Unify the behavior of handling `missing` with other inputs.
- Unify the behavior of handling `missing` around R, Python, Java, and Scala DMatrix.
- Expose `num_non_missing` to the JVM interface.
- Deprecate old CSR and CSC constructors.
 2023-02-05 01:59:46 +08:00
Jiaming Yuan 0e61ba57d6 Fix GPU L1 error. (#8749) 2023-02-04 03:02:00 +08:00
Jiaming Yuan 1325ba9251 Support primitive types of pyarrow-backed pandas dataframe. (#8653) 
Categorical data (dictionary) is not supported at the moment.
 2023-01-30 17:53:29 +08:00
Jiaming Yuan 3760cede0f Consistent use of context to specify number of threads. (#8733) 
- Use context in all tests.
- Use context in R.
- Use context in C API DMatrix initialization. (0 threads is used as dft).
 2023-01-30 15:25:31 +08:00
Jiaming Yuan 21a28f2cc5 Small refactor for hist builder. (#8698) 
- Use span instead of vector as parameter. No perf change as the builder work on pointer.
- Use const pointer for reg tree.
 2023-01-30 14:06:41 +08:00
Jiaming Yuan 9fb12b20a4 Cleanup the callback module. (#8702) 
- Cleanup pylint markers.
- run formatter.
- Update examples of using callback.
 2023-01-22 00:13:49 +08:00
Jiaming Yuan 34eee56256 Fix compiler warnings. (#8703) 
Fix warnings about signed/unsigned comparisons.
 2023-01-21 15:16:23 +08:00
Jiaming Yuan e49e0998c0 Extract CPU sampling routines. (#8697) 2023-01-19 23:28:18 +08:00
Jiaming Yuan 7a068af1a3 Workaround CUDA warning. (#8696) 2023-01-19 09:16:08 +08:00
Jiaming Yuan 4416452f94 Return single thread from context when called inside omp region. (#8693) 2023-01-18 09:23:37 +08:00
Jiaming Yuan 31b9cbab3d Make sure input numpy array is aligned. (#8690) 
- use `np.require` to specify that the alignment is required.
- scipy csr as well.
- validate input pointer in `ArrayInterface`.
 2023-01-18 08:12:13 +08:00
Jiaming Yuan 175986b739 [doc] Add missing document for pyspark ranker. [skip ci] (#8692) 2023-01-18 07:52:18 +08:00
Jiaming Yuan 247946a875 Cache transformed in QuantileDMatrix for efficiency. (#8666) 2023-01-17 06:02:40 +08:00
Jiaming Yuan 43152657d4 Extract JSON type check. (#8677) 
- Reuse it in `GetMissing`.
- Add test.
 2023-01-17 03:11:07 +08:00
Jiaming Yuan 9f598efc3e Rename context in Metric. (#8686) 2023-01-17 01:10:13 +08:00
Jiaming Yuan d6018eb4b9 Remove all use of DeviceQuantileDMatrix. (#8665) 2023-01-17 00:04:10 +08:00
Jiaming Yuan 0ae8df9a65 Define default ctors for gpair. (#8660) 
* Define default ctors for gpair.

Fix clang warning:

Definition of implicit copy assignment operator for 'GradientPairInternal<float>' is
deprecated because it has a user-declared copy constructor
 2023-01-16 22:52:13 +08:00
Jiaming Yuan e7d612d22c [R] Fix threads used to create DMatrix in predict. (#8681) 2023-01-15 03:09:08 +08:00
Jiaming Yuan f7a2f52136 [R] Get CXX flags from R CMD config. (#8669) 2023-01-14 16:48:21 +08:00
Jiaming Yuan 07cf3d3e53 Fix threads in DMatrix slice. (#8667) 2023-01-14 07:16:57 +08:00
Jiaming Yuan e27cda7626 [CI] Skip pyspark sparse tests. (#8675) 2023-01-14 05:37:00 +08:00