154 Commits

Author SHA1 Message Date
Jiaming Yuan
3976455af9
[jvm-packages] Use UBJ for checkpoints. (#9954) 2024-01-08 13:26:12 +08:00
Jiaming Yuan
38dd91f491
Save model in ubj as the default. (#9947) 2024-01-05 17:53:36 +08:00
Jiaming Yuan
c75a3bc0a9
[breaking] [jvm-packages] Remove rabit check point. (#9599)
- Add `numBoostedRound` to jvm packages
- Remove rabit checkpoint version.
- Change the starting version of training continuation in JVM [breaking].
- Redefine the checkpoint version policy in jvm package. [breaking]
- Rename the Python check point callback parameter. [breaking]
- Unifies the checkpoint policy between Python and JVM.
2023-09-26 18:06:34 +08:00
Jon Yoquinto
d05ea589fb
Allow JVM-Package to access inplace predict method (#9167)
---------

Co-authored-by: Stephan T. Lavavej <stl@nuwen.net>
Co-authored-by: Jiaming Yuan <jm.yuan@outlook.com>
Co-authored-by: Joe <25804777+ByteSizedJoe@users.noreply.github.com>
2023-09-12 07:29:51 +08:00
Jiaming Yuan
be6a552956
[R] Support multi-class custom objective. (#9526) 2023-08-29 08:27:13 +08:00
Jiaming Yuan
972730cde0
Use matrix for gradient. (#9508)
- Use the `linalg::Matrix` for storing gradients.
- New API for the custom objective.
- Custom objective for multi-class/multi-target is now required to return the correct shape.
- Custom objective for Python can accept arrays with any strides. (row-major, column-major)
2023-08-24 05:29:52 +08:00
Bobby Wang
8f0efb4ab3
[jvm-packages] automatically set the max/min direction for best score (#9404) 2023-07-27 11:09:55 +08:00
Jiaming Yuan
f4fb2be101
[jvm-packages] Add the new device parameter. (#9385) 2023-07-17 18:40:39 +08:00
jinmfeng001
a1367ea1f8
Set feature_names and feature_types in jvm-packages (#9364)
* 1. Add parameters to set feature names and feature types
2. Save feature names and feature types to native json model

* Change serialization and deserialization format to ubj.
2023-07-12 15:18:46 +08:00
Boris
a01df102c9
Scala 2.13 support. (#9099)
1. Updated the test logic
2. Added smoke tests for Spark examples.
3. Added integration tests for Spark with Scala 2.13
2023-05-27 19:34:02 +08:00
Jiaming Yuan
1f9a57d17b
[Breaking] Require format to be specified in input URI. (#9077)
Previously, we use `libsvm` as default when format is not specified. However, the dmlc
data parser is not particularly robust against errors, and the most common type of error
is undefined format.

Along with which, we will recommend users to use other data loader instead. We will
continue the maintenance of the parsers as it's currently used for many internal tests
including federated learning.
2023-04-28 19:45:15 +08:00
Emil Ejbyfeldt
a84a1fde02
[jvm-packages] Update scalatest to 3.2.15 (#8925)
---------

Co-authored-by: Jiaming Yuan <jm.yuan@outlook.com>
2023-04-20 22:16:56 +08:00
Jiaming Yuan
564df59204
[breaking] [jvm-packages] Remove scala-implemented tracker. (#9045) 2023-04-20 16:29:35 +08:00
Jiaming Yuan
c1786849e3
Use array interface for CSC matrix. (#8672)
* Use array interface for CSC matrix.

Use array interface for CSC matrix and align the interface with CSR and dense.

- Fix nthread issue in the R package DMatrix.
- Unify the behavior of handling `missing` with other inputs.
- Unify the behavior of handling `missing` around R, Python, Java, and Scala DMatrix.
- Expose `num_non_missing` to the JVM interface.
- Deprecate old CSR and CSC constructors.
2023-02-05 01:59:46 +08:00
Bobby Wang
f1e9bbcee5
[breakinig] [jvm-packages] change DeviceQuantileDmatrix into QuantileDMatrix (#8461) 2022-12-05 12:23:21 +08:00
Rong Ou
668b8a0ea4
[Breaking] Switch from rabit to the collective communicator (#8257)
* Switch from rabit to the collective communicator

* fix size_t specialization

* really fix size_t

* try again

* add include

* more include

* fix lint errors

* remove rabit includes

* fix pylint error

* return dict from communicator context

* fix communicator shutdown

* fix dask test

* reset communicator mocklist

* fix distributed tests

* do not save device communicator

* fix jvm gpu tests

* add python test for federated communicator

* Update gputreeshap submodule

Co-authored-by: Hyunsu Philip Cho <chohyu01@cs.washington.edu>
2022-10-05 14:39:01 -08:00
Rong Ou
7d43e74e71
JNI wrapper for the collective communicator (#8242) 2022-09-21 04:20:25 +08:00
Jiaming Yuan
bb47fd8c49
[jvm-packages] Change log level for tracker message. (#7968) 2022-06-09 18:15:08 +08:00
Bobby Wang
78694405a6
[jvm-packages] add jni for setting feature name and type (#7966) 2022-06-03 11:09:48 +08:00
Yang Jiandan
27c66f12d1
set log level as ERROR for trackerProcess has some stderr output (#7952) 2022-05-31 22:54:38 +08:00
Bobby Wang
6275cdc486
[jvm-packages] add format option when saving a model (#7940) 2022-05-30 15:49:59 +08:00
Daniel Clausen
755d9d4609
[JVM-Packages] Auto-detection of MUSL is replaced by system properties (#7921)
This PR removes auto-detection of MUSL-based Linux systems in favor of system properties the user can set to configure a specific path for a native library.
2022-05-26 10:53:15 +08:00
Michael Allman
f7db16add1
Ignore all Java exceptions when looking for Linux musl support (#7844) 2022-04-28 15:44:30 +08:00
Bobby Wang
2d83b2ad8f
[jvm-packages] add hostIp and python exec for rabit tracker (#7808) 2022-04-15 16:28:43 +08:00
Daniel Clausen
4dafb5fac8
[JVM-Packages] Add support for detecting musl-based Linux (#7624)
Co-authored-by: Marc Philipp <marc@gradle.com>
2022-03-14 00:37:27 +08:00
Jiaming Yuan
ac7a36367c
[jvm-packages] Implement new save_raw in jvm-packages. (#7570)
* New `toByteArray` that accepts a parameter for format.
2022-01-19 16:00:14 +08:00
Jiaming Yuan
ed95e77752
[jvm-packages] Update JNI header. (#7550) 2022-01-10 14:59:40 +08:00
Bobby Wang
e8c1eb99e4
[jvm-package] Clean up the legacy gpu support tests (#7523) 2021-12-21 09:15:51 +08:00
Bobby Wang
24be04e848
[jvm-packages] Add DeviceQuantileDMatrix to Scala binding (#7459) 2021-11-24 20:23:18 +08:00
nicovdijk
74bab6e504
Control logging for early stopping using shouldPrint() (#7326) 2021-10-21 12:12:06 +08:00
Bobby Wang
4fd149b3a2
[jvm-packages] update checkstyle (#7335)
* [jvm-packages] update scalastyle

1. bump scalastyle-maven-plugin and maven-checkstyle-plugin to latest
2. remove unused imports

* fix code style check
2021-10-18 18:42:01 +08:00
Jiaming Yuan
fbd58bf190
[jvm-packages] Create demo and test for xgboost4j early stopping. (#7252) 2021-09-25 03:29:27 +08:00
Bobby Wang
0ee11dac77
[jvm-packages][xgboost4j-gpu] Support GPU dataframe and DeviceQuantileDMatrix (#7195)
Following classes are added to support dataframe in java binding:

- `Column` is an abstract type for a single column in tabular data.
- `ColumnBatch` is an abstract type for dataframe.

- `CuDFColumn` is an implementaiton of `Column` that consume cuDF column
- `CudfColumnBatch` is an implementation of `ColumnBatch` that consumes cuDF dataframe.

- `DeviceQuantileDMatrix` is the interface for quantized data.

The Java implementation mimics the Python interface and uses `__cuda_array_interface__` protocol for memory indexing.  One difference is on JVM package, the data batch is staged on the host as java iterators cannot be reset.

Co-authored-by: jiamingy <jm.yuan@outlook.com>
2021-09-24 14:25:00 +08:00
Jiaming Yuan
9f63d6fead
[jvm-packages] Deprecate constructors with implicit missing value. (#7225) 2021-09-17 04:35:04 +08:00
Martin Petříček
46c46829ce
Fix model loading from stream (#7067)
Fix bug introduced in 17913713b554d820a8ce94226d854b4a5f1d8bbc (allow loading from byte array)

When loading model from stream, only last buffer read from the input stream is used to construct the model.

This may work for models smaller than 1 MiB (if you are lucky enough to read the whole model at once), but will always fail if the model is larger.
2021-08-15 21:04:33 +08:00
naveenkb
9f7f8b976d
[XGBoost4J-Spark] bestIteration and bestScore for early stopping (#7095) 2021-07-19 18:46:49 +08:00
Jiaming Yuan
663136aa08
Implement feature score for linear model. (#7048)
* Add feature score support for linear model.
* Port R interface to the new implementation.
* Add linear model support in Python.

Co-authored-by: Philip Hyunsu Cho <chohyu01@cs.washington.edu>
2021-06-25 14:34:02 +08:00
ShvetsKS
57c732655e
Merge lossgude and depthwise strategies for CPU hist (#7007)
* fix java/scala test: max depth is also valid parameter for lossguide

Co-authored-by: Kirill Shvets <kirill.shvets@intel.com>
2021-06-03 01:49:43 +08:00
Adam Pocock
2320aa0da2
Making the Java library loader emit helpful error messages on missing dependencies. (#6926) 2021-05-19 14:53:56 +08:00
Jiaming Yuan
74b41637de
Revert "[jvm-packages] Add XGBOOST_RABIT_TRACKER_IP_FOR_TEST to set rabit tracker IP. (#6869)" (#6886)
This reverts commit 2828da3c4c951baa45d1bb6f85c7b3a6657cd607.
2021-04-21 11:20:10 -07:00
Bobby Wang
2828da3c4c
[jvm-packages] Add XGBOOST_RABIT_TRACKER_IP_FOR_TEST to set rabit tracker IP. (#6869)
* Add `XGBOOST_RABIT_TRACKER_IP_FOR_TEST` to set rabit tracker IP

* change spark and rabit tracker IP to 127.0.0.1on GitHub Action.

Co-authored-by: fis <jm.yuan@outlook.com>
2021-04-22 02:00:22 +08:00
Honza Sterba
17913713b5
[jvm] Add ability to load booster direct from byte array (#6655)
* Add ability to load booster direct from byte array

* fix compiler error

* move InputStream to byte-buffer conversion

- move it from Booster to XGBoost facade class
2021-02-23 11:28:27 -08:00
Adam Pocock
fec66d033a
[jvm-packages] JVM library loader extensions (#6630)
* [java] extending the library loader to use both OS and CPU architecture.

* Simplifying create_jni.py's architecture detection.

* Tidying up the architecture detection in create_jni.py
2021-01-25 15:51:39 +08:00
Jiaming Yuan
b180223d18
Cleanup RABIT. (#6290)
* Remove recovery and MPI speed tests.
* Remove readme.
* Remove Python binding.
* Add checks in C API.
2020-10-27 08:48:22 +08:00
Jiaming Yuan
d61b628bf5
Remove RABIT CMake targets. (#6275)
* Now it's built as part of libxgboost.
* Set correct C API error in RABIT initialization and finalization.
* Remove redundant message.
* Guard the tracker print C API.
2020-10-27 01:30:20 +08:00
Hristo Iliev
da61d9460b
[jvm-packages] Add getNumFeature method (#6075)
* Add getNumFeature to the Java API
* Add getNumFeature to the Scala API
* Add unit tests for getNumFeature

Co-authored-by: Philip Hyunsu Cho <chohyu01@cs.washington.edu>
2020-09-07 20:57:46 -07:00
Bobby Wang
8943eb4314
[BLOCKING] [jvm-packages] add gpu_hist and enable gpu scheduling (#5171)
* [jvm-packages] add gpu_hist tree method

* change updater hist to grow_quantile_histmaker

* add gpu scheduling

* pass correct parameters to xgboost library

* remove debug info

* add use.cuda for pom

* add CI for gpu_hist for jvm

* add gpu unit tests

* use gpu node to build jvm

* use nvidia-docker

* Add CLI interface to create_jni.py using argparse

Co-authored-by: Hyunsu Cho <chohyu01@cs.washington.edu>
2020-07-26 21:53:24 -07:00
Jiaming Yuan
dd9aeb60ae
[JVM Packages] Catch dmlc error by ref. (#5678) 2020-05-19 13:00:12 +08:00
Liang-Chi Hsieh
397d8f0ee7
[jvm-packages] XGBoost Spark should deal with NaN when parsing evaluation output (#5546) 2020-04-19 23:10:30 -07:00
Bobby Wang
ad826e913f
[jvm-packages]add feature size for LabelPoint and DataBatch (#5303)
* fix type error

* Validate number of features.

* resolve comments

* add feature size for LabelPoint and DataBatch

* pass the feature size to native

* move feature size validating tests into a separate suite

* resolve comments

Co-authored-by: fis <jm.yuan@outlook.com>
2020-04-07 16:49:52 -07:00