Hendrik/xgboost
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
5,930 Commits 32 Branches 59 Tags
fffb1fca522da1524350b1a563580f2e4e115ef4
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Jiaming Yuan fffb1fca52 Calculate base_score based on input labels for mae. (#8107) 
Fit an intercept as base score for abs loss.
 2022-09-20 20:53:54 +08:00
Jiaming Yuan 2e63af6117 Mitigate flaky data iter test. (#8244) 
- Reduce the number of batches.
- Verify labels.
 2022-09-14 17:54:14 +08:00
Jiaming Yuan bdf265076d Make QuantileDMatrix default to sklearn esitmators. (#8220) 2022-09-13 13:52:19 +08:00
Jiaming Yuan bc818316f2 Prepare for improving Windows networking compatibility. (#8234) 
* Prepare for improving Windows networking compatibility.

* Include dmlc filesystem indirectly as dmlc/filesystem.h includes windows.h, which
  conflicts with winsock2.h
* Define `NOMINMAX` conditionally.
* Link the winsock library when mysys32 is used.
* Add config file for read the doc.
 2022-09-10 15:16:49 +08:00
Jiaming Yuan dd44ac91b8 [CI] Use binary R dependencies on Windows. (#8241) 2022-09-09 19:51:15 -07:00
Jiaming Yuan b5eb36f1af Add max_cat_threshold to GPU and handle missing cat values. (#8212) 2022-09-07 00:57:51 +08:00
Jiaming Yuan 441ffc017a Copy data from Ellpack to GHist. (#8215) 2022-09-06 23:05:49 +08:00
Jiaming Yuan ada4a86d1c Fix dask interface with latest cupy. (#8210) 2022-09-03 03:10:43 +08:00
Jiaming Yuan 8dac90a593 Mark parameter validation non-experimental. (#8206) 2022-08-30 15:49:43 +08:00
Jiaming Yuan 9b32e6e2dc Fix release script. (#8187) (#8195) 2022-08-23 15:08:30 +08:00
Jiaming Yuan 16bca5d4a1 Support CPU input for device QuantileDMatrix. (#8136) 
- Copy `GHistIndexMatrix` to `Ellpack` when needed.
 2022-08-11 21:21:26 +08:00
Jiaming Yuan 36e7c5364d [dask] Deterministic rank assignment. (#8018) 2022-08-11 19:17:58 +08:00
Jiaming Yuan d868126c39 [CI] Fix R build on Jenkins. (#8154) 2022-08-11 14:50:03 +08:00
Jiaming Yuan 570f8ae4ba Use black on more Python files. (#8137) 2022-08-11 01:38:11 +08:00
Jiaming Yuan bdb291f1c2 [doc] Clarification for feature importance. (#8151) 2022-08-11 00:30:42 +08:00
Jiaming Yuan 446d536c23 Fix loading DMatrix binary in distributed env. (#8149) 
- Try to load DMatrix binary before trying to parse text input.
- Remove some unmaintained code.
 2022-08-10 22:53:16 +08:00
Jiaming Yuan 8fc60b31bc Update PyPi wheel size limit. (#8150) 2022-08-10 18:49:57 +08:00
Jiaming Yuan 9ae547f994 Use config_context in sklearn interface. (#8141) 2022-08-09 14:48:54 +08:00
Jiaming Yuan bcc8679a05 Update CUDA docker image and NCCL. (#8139) 2022-08-07 16:32:41 +08:00
Jiaming Yuan d87f69215e Quantile DMatrix for CPU. (#8130) 
- Add a new `QuantileDMatrix` that works for both CPU and GPU.
- Deprecate `DeviceQuantileDMatrix`.
 2022-08-02 15:51:23 +08:00
Jiaming Yuan 2cba1d9fcc Fix compatibility with latest cupy. (#8129) 
* Fix compatibility with latest cupy.

* Freeze mypy.
 2022-08-01 15:24:42 +08:00
Jiaming Yuan 2c70751d1e Implement iterative DMatrix for CPU. (#8116) 2022-07-26 22:34:21 +08:00
Jiaming Yuan 546de5efd2 [pyspark] Cleanup data processing. (#8088) 
- Use numpy stack for handling list of arrays.
- Reuse concat function from dask.
- Prepare for `QuantileDMatrix`.
- Remove unused code.
- Use iterator for prediction to avoid initializing xgboost model
 2022-07-26 15:00:52 +08:00
Jiaming Yuan 3970e4e6bb Move pylint helper from dmlc-core. (#8101) 
* Move pylint helper from dmlc-core.

- Move the helper into the XGBoost ci_build.
- Run it with multiprocessing.

* Fix original test.
 2022-07-23 08:12:37 +08:00
Jiaming Yuan 7785d65c8a Fix feature weights with multiple column sampling. (#8100) 2022-07-22 20:23:05 +08:00
Jiaming Yuan 4a4e5c7c18 Prepare gradient index for Quantile DMatrix. (#8103) 
* Prepare gradient index for Quantile DMatrix.

- Implement push batch with adapter batch.
- Implement `GetFvalue` for prediction.
 2022-07-22 17:26:33 +08:00
Jiaming Yuan ef11b024e8 Cleanup data generator. (#8094) 
- Avoid duplicated definition of data shape.
- Explicitly define numpy iterator for CPU data.
 2022-07-20 13:48:52 +08:00
Jiaming Yuan 5156be0f49 Limit max_depth to 30 for GPU. (#8098) 2022-07-20 12:28:49 +08:00
Jiaming Yuan 8bdea72688 [Python] Require black and isort for new Python files. (#8096) 
* [Python] Require black and isort for new Python files.

- Require black and isort for spark and dask module.

These files are relatively new and are more conform to the black formatter. We will
convert the rest of the library as we move forward.

Other libraries including dask/distributed and optuna use the same formatting style and
have a more strict standard. The black formatter is indeed quite nice, automating it can
help us unify the code style.

- Gather Python checks into a single script.
 2022-07-20 10:25:24 +08:00
Jiaming Yuan 2365f82750 [dask] Mitigate non-deterministic test. (#8077) 2022-07-19 16:55:59 +08:00
Jiaming Yuan 4083440690 Small cleanups to various data types. (#8086) 
- Use `bst_bin_t` in batch param constructor.
- Use `StringView` to avoid `std::string` when appropriate.
- Avoid using `MetaInfo` in quantile constructor to limit the scope of parameter.
 2022-07-18 22:39:36 +08:00
Jiaming Yuan e28f6f6657 [doc] Integrate pyspark module into sphinx doc [skip ci] (#8066) 2022-07-17 10:46:09 +08:00
Jiaming Yuan 0ce80b7bcf Mitigate flaky GPU test. (#8078) 
The flakiness is caused by the global random engine, which will take some time to fix.
 2022-07-16 13:45:32 +08:00
Jiaming Yuan 7a5586f3db Fix GPU quantile distributed test. (#8076) 2022-07-16 11:40:53 +08:00
Jiaming Yuan 8fccc3c4ad [dask] Fix potential error in demo. (#8079) 
* Use dask_cudf instead.
 2022-07-15 18:42:29 +08:00
Jiaming Yuan 647d3844dd Make test for categorical data deterministic. (#8080) 2022-07-15 14:48:39 +08:00
Jiaming Yuan dae7a41baa Update Python requirement to >=3.8. (#8071) 
Additional changes:
- Use mamba for CPU test on Jenkins.
- Cleanup CPU test dependencies.
- Restore some of the modin tests
 2022-07-14 18:01:47 +08:00
Jiaming Yuan 8dd96013f1 Split up column matrix initialization. (#8060) 
* Split up column matrix initialization.

This PR splits the column matrix initialization into 2 steps, the first one initializes
the storage while the second one does the transpose. By doing so, we can reuse the code
for Quantile DMatrix.
 2022-07-14 10:34:47 +08:00
Jiaming Yuan abaa593aa0 Fix compiler warnings. (#8059) 
- Remove unused parameters.
- Avoid comparison of different signedness.
 2022-07-14 05:29:56 +08:00
Jiaming Yuan 937352c78f Fix R package Windows build. (#8065) 2022-07-14 05:27:38 +08:00
Jiaming Yuan 8959622836 [dask] Use an invalid port for test. (#8064) 2022-07-13 11:59:02 +08:00
Jiaming Yuan a5bc8e2c6a Fix mypy error with the latest dask. (#8052) 
* Fix mypy error with latest dask.

Dask is adding type hints to its codebase and as the result, checks in XGBoost can be
performed more rigorously.

- Remove compatibility with old dask version where multi lock was missing.
- Restrict input of `X` to be non-series.
- Adopt latest definition of `Delayed`.
- Avoid passing optional `host_ip`.
- Avoid deprecated `worker.nthreads`.
 2022-07-09 08:02:42 +08:00
Jiaming Yuan 210eb471e9 [R] Implement feature info for DMatrix. (#8048) 2022-07-09 05:57:39 +08:00
Jiaming Yuan 701f32b227 [py-sckl] Raise import error if skl is not installed. (#8049) 2022-07-09 05:56:46 +08:00
Jiaming Yuan ff1c559084 Remove unused variable. (#8046) 2022-07-05 01:59:22 +08:00
Jiaming Yuan 8746f9cddf Rename IterativeDMatrix. (#8045) 2022-07-04 18:52:31 +08:00
Jiaming Yuan f24bfc7684 Bump R cache version. (#8044) 2022-07-03 03:53:05 +08:00
Jiaming YuanandHyunsu Cho e88d6e071d Fix compiler warning in JSON IO. (#8031) 
Co-authored-by: Hyunsu Cho <chohyu01@cs.washington.edu>
 2022-06-30 01:13:22 +08:00
Jiaming Yuan dcaf580476 Fix Python package source install. (#8036) 
* Copy gputreeshap.
 2022-06-29 21:45:09 +08:00
Jiaming Yuan f0c1b842bf Implement sketching with adapter. (#8019) 2022-06-23 00:03:02 +08:00
Jiaming Yuan 142a208a90 Fix compiler warnings. (#8022) 
- Remove/fix unused parameters
- Remove deprecated code in rabit.
- Update dmlc-core.
 2022-06-22 21:29:10 +08:00
Jiaming Yuan 4a87ea49b8 Reduce regularization for CPU gblinear. (#8013) 2022-06-21 01:05:27 +08:00
Jiaming Yuan d285d6ba2a Reduce regularization in GPU gblinear test. (#8010) 2022-06-20 23:55:12 +08:00
Jiaming Yuan e58e417603 [CI] Fix lintr error. (#8011) 2022-06-20 22:17:14 +08:00
Jiaming Yuan 9b0eb66b78 Fix GPU driver test. (#8008) 
* Initialize the training parameter.
 2022-06-20 19:37:31 +08:00
Jiaming Yuan 637e42a0c0 Use 22.04 for RMM. (#8001) 
22.06 is not released yet.
 2022-06-17 04:07:31 +08:00
Jiaming Yuan bb47fd8c49 [jvm-packages] Change log level for tracker message. (#7968) 2022-06-09 18:15:08 +08:00
Jiaming Yuan 8f8bd8147a Fix LTR with weighted Quantile DMatrix. (#7975) 
* Fix LTR with weighted Quantile DMatrix.

* Better tests.
 2022-06-09 01:33:41 +08:00
Jiaming Yuan 1a33b50a0d Fix compiler warnings. (#7974) 
- Remove unused parameters. There are still many warnings that are not yet
addressed. Currently, the warnings in dmlc-core dominate the error log.
- Remove `distributed` parameter from metric.
- Fixes some warnings about signed comparison.
 2022-06-06 22:56:25 +08:00
Jiaming Yuanandjakirkham d48123d23b Fix rmm build (#7973) 
- Optionally switch to c++17
- Use rmm CMake target.
- Workaround compiler errors.
- Fix GPUMetric inheritance.
- Run death tests even if it's built with RMM support.

Co-authored-by: jakirkham <jakirkham@gmail.com>
 2022-06-06 20:18:32 +08:00
Jiaming Yuan b90c6d25e8 Implement max_cat_threshold for CPU. (#7957) 2022-06-04 11:02:46 +08:00
Jiaming Yuan 6b55150e80 Fix pylint errors. (#7967) 2022-06-02 18:04:46 +08:00
Jiaming Yuan 13b15e07e8 Handle formatted JSON input. (#7953) 2022-06-01 16:20:58 +08:00
Jiaming Yuan bde4f25794 Handle missing categorical value in CPU evaluator. (#7948) 2022-05-27 14:15:47 +08:00
Jiaming Yuan 18cbebaeb9 Unify the cat split storage for CPU. (#7937) 
* Unify the cat split storage for CPU.

* Cleanup.

* Workaround.
 2022-05-26 04:14:40 -07:00
Jiaming Yuan 606be9e663 Handle missing values in one hot splits. (#7934) 2022-05-24 20:48:41 +08:00
Jiaming Yuan 18a38f7ca0 Refactor for GHistIndex. (#7923) 
* Pass sparse page as adapter, which prepares for quantile dmatrix.
* Remove old external memory code like `rbegin` and extra `Init` function.
* Simplify type dispatch.
 2022-05-23 23:04:53 +08:00
Jiaming Yuan d314680a15 Verify shared object version at load. (#7928) 2022-05-23 20:53:30 +08:00
Jiaming Yuan 474366c020 Add convergence test for sparse datasets. (#7922) 2022-05-23 18:07:26 +08:00
Jiaming Yuan f93a727869 Address remaining mypy errors in python package. (#7914) 2022-05-18 22:46:15 +08:00
Jiaming Yuan edf9a9608e Fix type conversion warning. (#7916) 2022-05-18 20:14:14 +08:00
Jiaming Yuan 765097d514 Simplify inplace-predict. (#7910) 
Pass the `X` as part of Proxy DMatrix instead of an independent `dmlc::any`.
 2022-05-18 17:52:00 +08:00
Jiaming Yuan 19775ffe15 Use adapter to initialize column matrix. (#7912) 2022-05-18 16:15:12 +08:00
Jiaming Yuan 4fcfd9c96e Fix and cleanup for column matrix. (#7901) 
* Fix missed type dispatching for dense columns with missing values.
* Code cleanup to reduce special cases.
* Reduce memory usage.
 2022-05-16 21:11:50 +08:00
Jiaming Yuan 1baad8650c Small cleanup to Column. (#7898) 
* Define forward iterator to hide the internal state.
 2022-05-15 12:39:10 +08:00
Jiaming Yuan ee382c4153 Update news for 1.6.1 (#7877) 2022-05-14 15:38:18 -07:00
Jiaming YuanandPhilip Hyunsu Cho 1b6538b4e5 [breaking] Drop single precision histogram (#7892) 
Co-authored-by: Philip Hyunsu Cho <chohyu01@cs.washington.edu>
 2022-05-13 19:54:55 +08:00
Jiaming Yuan c8f9d4b6e6 Show libxgboost.so path in build info. (#7893) 2022-05-13 18:08:56 +08:00
Jiaming Yuan 11d65fcb21 Extract partial sum into an independent function. (#7889) 2022-05-13 14:30:35 +08:00
Jiaming Yuan db80671d6b Fix monotone constraint with tuple input. (#7891) 2022-05-13 04:00:03 +08:00
Jiaming Yuan 94ca52b7b7 Fix overflow in prediction size. (#7885) 2022-05-12 02:44:03 +08:00
Jiaming Yuan 8ba4722d04 Remove pyarrow workaround. (#7884) 2022-05-11 20:54:48 +08:00
Jiaming Yuan 16ba74d008 Update CUDA version requirement in CMake script. (#7876) 2022-05-09 04:16:22 +08:00
Jiaming Yuan 46e0bce212 Use maximum category in sketch. (#7853) 2022-05-05 19:56:49 +08:00
Jiaming Yuan 8ab5e13b5d Fix typo [skip ci] (#7861) 2022-05-04 18:34:45 +08:00
Jiaming Yuan 317d7be6ee Always use partition based categorical splits. (#7857) 2022-05-03 22:30:32 +08:00
Jiaming Yuan 50d854e02e [CI] Test with latest RAPIDS. (#7816) 2022-04-30 11:55:10 -07:00
Jiaming Yuan 288c52596c Define bin type. (#7850) 2022-04-29 19:41:39 +08:00
Jiaming Yuan fdf533f2b9 [POC] Experimental support for l1 error. (#7812) 
Support adaptive tree, a feature supported by both sklearn and lightgbm.  The tree leaf is recomputed based on residue of labels and predictions after construction.

For l1 error, the optimal value is the median (50 percentile).

This is marked as experimental support for the following reasons:
- The value is not well defined for distributed training, where we might have empty leaves for local workers. Right now I just use the original leaf value for computing the average with other workers, which might cause significant errors.
- Some follow-ups are required, for exact, pruner, and optimization for quantile function. Also, we need to calculate the initial estimation.
 2022-04-26 21:41:55 +08:00
Jiaming Yuan ad06172c6b Refactor pandas dataframe handling. (#7843) 2022-04-26 18:53:43 +08:00
Jiaming Yuan 332380479b Avoid warning in np primitive type tests. (#7833) 2022-04-23 02:07:01 +08:00
Jiaming Yuan f0f76259c9 Remove STRING_TYPES. (#7827) 2022-04-22 19:07:51 +08:00
Jiaming Yuan c70fa502a5 Expose feature_types to sklearn interface. (#7821) 2022-04-21 20:23:35 +08:00
Jiaming Yuan 401d451569 Clear configuration cache. (#7826) 2022-04-21 19:09:54 +08:00
Jiaming Yuan 52d4eda786 Deprecate use_label_encoder in XGBClassifier. (#7822) 
* Deprecate `use_label_encoder` in XGBClassifier.

* We have removed the encoder, now prepare to remove the indicator.
 2022-04-21 13:14:02 +08:00
Jiaming Yuan 5815df4c46 Remove warning in 1.4. (#7815) 2022-04-20 01:19:09 +08:00
Jiaming YuanandPhilip Hyunsu Cho d0de954af2 v1.6.0 release note. [skip ci] (#7746) 
Co-authored-by: Philip Hyunsu Cho <chohyu01@cs.washington.edu>
 2022-04-16 16:27:54 +08:00
Jiaming Yuan 5dea21273a Fix training continuation with categorical model. (#7810) 
* Make sure the task is initialized before construction of tree updater.

This is a quick fix meant to be backported to 1.6, for a full fix we should pass the model
param into tree updater by reference instead.
 2022-04-15 18:21:02 +08:00
Jiaming Yuan bcce17e688 Remove text loading in basic walk through demo. (#7753) 2022-04-01 00:59:42 +08:00
Jiaming YuanandPhilip Hyunsu Cho fd78af404b Drop support for deprecated CUDA architectures. (#7774) 
Co-authored-by: Philip Hyunsu Cho <chohyu01@cs.washington.edu>
 2022-03-31 21:42:23 +08:00