564 Commits

Author SHA1 Message Date
dependabot[bot]
d66b5570f4
Bump commons-logging:commons-logging in /jvm-packages/xgboost4j (#10294)
Bumps commons-logging:commons-logging from 1.3.1 to 1.3.2.

---
updated-dependencies:
- dependency-name: commons-logging:commons-logging
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Jiaming Yuan <jm.yuan@outlook.com>
2024-05-21 15:49:26 +08:00
Jiaming Yuan
a5a58102e5
Revamp the rabit implementation. (#10112)
This PR replaces the original RABIT implementation with a new one, which has already been partially merged into XGBoost. The new one features:
- Federated learning for both CPU and GPU.
- NCCL.
- More data types.
- A unified interface for all the underlying implementations.
- Improved timeout handling for both tracker and workers.
- Exhausted tests with metrics (fixed a couple of bugs along the way).
- A reusable tracker for Python and JVM packages.
2024-05-20 11:56:23 +08:00
Bobby Wang
9b465052ce
[jvm-packages] fix group col for gpu packages (#10254) 2024-05-09 07:44:07 +08:00
Jiaming Yuan
8237920c48
[jvm-packagaes] Freeze spark to 3.4.1 for now. (#10253)
The newer spark version for CPU conflicts with the more conservative version used by
rapids.
2024-05-07 09:00:59 -07:00
dependabot[bot]
f8c3d22587
Bump org.apache.spark:spark-mllib_2.12 (#10070)
Bumps org.apache.spark:spark-mllib_2.12 from 3.4.1 to 3.5.1.

---
updated-dependencies:
- dependency-name: org.apache.spark:spark-mllib_2.12
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-04-28 15:02:14 -07:00
dependabot[bot]
f355418186
Bump org.apache.maven.plugins:maven-gpg-plugin (#10211)
Bumps [org.apache.maven.plugins:maven-gpg-plugin](https://github.com/apache/maven-gpg-plugin) from 3.2.3 to 3.2.4.
- [Release notes](https://github.com/apache/maven-gpg-plugin/releases)
- [Commits](https://github.com/apache/maven-gpg-plugin/compare/maven-gpg-plugin-3.2.3...maven-gpg-plugin-3.2.4)

---
updated-dependencies:
- dependency-name: org.apache.maven.plugins:maven-gpg-plugin
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-04-26 12:35:48 -07:00
dependabot[bot]
4d69ce96b3
Bump org.apache.maven.plugins:maven-jar-plugin (#10210)
Bumps [org.apache.maven.plugins:maven-jar-plugin](https://github.com/apache/maven-jar-plugin) from 3.4.0 to 3.4.1.
- [Release notes](https://github.com/apache/maven-jar-plugin/releases)
- [Commits](https://github.com/apache/maven-jar-plugin/compare/maven-jar-plugin-3.4.0...maven-jar-plugin-3.4.1)

---
updated-dependencies:
- dependency-name: org.apache.maven.plugins:maven-jar-plugin
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-04-26 12:35:34 -07:00
dependabot[bot]
a5003fc8ce
Bump net.alchim31.maven:scala-maven-plugin in /jvm-packages/xgboost4j (#10217)
Bumps net.alchim31.maven:scala-maven-plugin from 4.8.1 to 4.9.0.

---
updated-dependencies:
- dependency-name: net.alchim31.maven:scala-maven-plugin
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-04-26 12:35:20 -07:00
dependabot[bot]
8ed85b8ce7
Bump hadoop.version from 3.3.6 to 3.4.0 in /jvm-packages/xgboost4j (#10156)
Bumps `hadoop.version` from 3.3.6 to 3.4.0.

Updates `org.apache.hadoop:hadoop-hdfs` from 3.3.6 to 3.4.0

Updates `org.apache.hadoop:hadoop-common` from 3.3.6 to 3.4.0

---
updated-dependencies:
- dependency-name: org.apache.hadoop:hadoop-hdfs
  dependency-type: direct:production
  update-type: version-update:semver-minor
- dependency-name: org.apache.hadoop:hadoop-common
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-04-26 11:15:03 -07:00
dependabot[bot]
bb212bf33c
Bump org.apache.flink:flink-clients in /jvm-packages (#10197)
Bumps [org.apache.flink:flink-clients](https://github.com/apache/flink) from 1.18.0 to 1.19.0.
- [Commits](https://github.com/apache/flink/compare/release-1.18.0...release-1.19.0)

---
updated-dependencies:
- dependency-name: org.apache.flink:flink-clients
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-04-18 15:20:31 -07:00
dependabot[bot]
551fa6e25e
Bump scalatest.version from 3.2.17 to 3.2.18 in /jvm-packages/xgboost4j (#10196)
Bumps `scalatest.version` from 3.2.17 to 3.2.18.

Updates `org.scalatest:scalatest_2.12` from 3.2.17 to 3.2.18
- [Release notes](https://github.com/scalatest/scalatest/releases)
- [Commits](https://github.com/scalatest/scalatest/compare/release-3.2.17...release-3.2.18)

Updates `org.scalactic:scalactic_2.12` from 3.2.17 to 3.2.18
- [Release notes](https://github.com/scalatest/scalatest/releases)
- [Commits](https://github.com/scalatest/scalatest/compare/release-3.2.17...release-3.2.18)

---
updated-dependencies:
- dependency-name: org.scalatest:scalatest_2.12
  dependency-type: direct:development
  update-type: version-update:semver-patch
- dependency-name: org.scalactic:scalactic_2.12
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-04-18 11:46:28 -07:00
dependabot[bot]
531ff21b20
Bump org.scala-lang.modules:scala-collection-compat_2.12 (#10193)
Bumps [org.scala-lang.modules:scala-collection-compat_2.12](https://github.com/scala/scala-collection-compat) from 2.11.0 to 2.12.0.
- [Release notes](https://github.com/scala/scala-collection-compat/releases)
- [Commits](https://github.com/scala/scala-collection-compat/compare/v2.11.0...v2.12.0)

---
updated-dependencies:
- dependency-name: org.scala-lang.modules:scala-collection-compat_2.12
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-04-18 07:18:18 -07:00
dependabot[bot]
0aa2600399
Bump org.apache.maven.plugins:maven-jar-plugin (#10202)
Bumps [org.apache.maven.plugins:maven-jar-plugin](https://github.com/apache/maven-jar-plugin) from 3.3.0 to 3.4.0.
- [Release notes](https://github.com/apache/maven-jar-plugin/releases)
- [Commits](https://github.com/apache/maven-jar-plugin/compare/maven-jar-plugin-3.3.0...maven-jar-plugin-3.4.0)

---
updated-dependencies:
- dependency-name: org.apache.maven.plugins:maven-jar-plugin
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-04-17 23:04:41 -07:00
dependabot[bot]
7c0c9677a9
Bump org.apache.maven.plugins:maven-jar-plugin (#10191)
Bumps [org.apache.maven.plugins:maven-jar-plugin](https://github.com/apache/maven-jar-plugin) from 3.3.0 to 3.4.0.
- [Release notes](https://github.com/apache/maven-jar-plugin/releases)
- [Commits](https://github.com/apache/maven-jar-plugin/compare/maven-jar-plugin-3.3.0...maven-jar-plugin-3.4.0)

---
updated-dependencies:
- dependency-name: org.apache.maven.plugins:maven-jar-plugin
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-04-16 19:52:33 -07:00
Philip Hyunsu Cho
32be4669fb
[jvm-packages] Ombinus patch to update all minor dependencies (#10188)
* Fold in #10184

* Fold in #10176

* Fold in #10168

* Fold in #10165

* Fold in #10164

* Fold in #10155

* Fold in #10062

* Fold in #9984

* Fold in #9843

* Upgrade to Maven 3.6.3
2024-04-16 19:43:04 -07:00
dependabot[bot]
a99bb38bd2
Bump org.apache.maven.plugins:maven-gpg-plugin from 3.1.0 to 3.2.2 in /jvm-packages/xgboost4j-spark (#10151) 2024-04-03 16:45:54 -07:00
Jiaming Yuan
230010d9a0
Cleanup set info. (#10139)
- Use the array interface internally.
- Deprecate `XGDMatrixSetDenseInfo`.
- Deprecate `XGDMatrixSetUIntInfo`.
- Move the handling of `DataType` into the deprecated C function.

---------

Co-authored-by: Philip Hyunsu Cho <chohyu01@cs.washington.edu>
2024-03-26 23:26:24 +08:00
Jiaming Yuan
eb281ff9b4
[CI] Fix JVM tests on GH Action (#10064)
---------

Co-authored-by: Hyunsu Cho <chohyu01@cs.washington.edu>
2024-02-22 14:21:32 -08:00
Philip Hyunsu Cho
1168a68872
[jvm-packages] Update release scripts (#9983)
* [jvm-packages] Add Scala version suffix to xgboost-jvm package (#9776)

* Update JVM script (#9714)

* Revamp pom.xml

* Update instructions in prepare_jvm_release.py

* Fix formatting

* [jvm-packages] Fix POM for xgboost-jvm metapackage (#9893)

* [jvm-packages] Fix POM for xgboost-jvm metapackage

* Add script for updating the Scala version

* Update change_scala_version.py to also change scala.version property (#9897)

* Remove 'release-cpu-only' profile

* Remove scala-2.13 profile; enable gpu package for Scala 2.13
2024-01-12 10:37:55 -08:00
Bobby Wang
f88c43801f
[jvm-packages] update rapids dep to 23.12.1 (#9951)
With this PR, XGBoost GPU can support scala 2.13
2024-01-11 14:04:32 -08:00
Bobby Wang
3ff3a5f1ed
[jvm-packages] support jdk 17 for test (#9959) 2024-01-08 17:30:49 +08:00
Jiaming Yuan
3976455af9
[jvm-packages] Use UBJ for checkpoints. (#9954) 2024-01-08 13:26:12 +08:00
Jiaming Yuan
38dd91f491
Save model in ubj as the default. (#9947) 2024-01-05 17:53:36 +08:00
Jiaming Yuan
2d8c67d6dc
[jvm-packages] Bump dependencies. (#9827)
- #9811
- #9814
- #9826
- #9830
- #9833
- #9832
- #9831
- #9834
2023-12-02 07:34:56 +08:00
Jiaming Yuan
e2e089ce12
[jvm-packages] Bump rapids version. (#9820) 2023-11-29 15:51:07 +08:00
Jiaming Yuan
34a2616696
[jvm-packages] Update dependencies. (#9809)
- scalatest: 3.2.17
- maven-checkstyle-plugin: 3.3.1
- maven-surefire-plugin: 3.2.2
- maven-project-info-reports-plugin: 3.5.0
- maven-javadoc-plugin: 3.6.2
2023-11-27 20:09:25 +08:00
Jiaming Yuan
0715ab3c10
Use dlopen to load NCCL. (#9796)
This PR adds optional support for loading nccl with `dlopen` as an alternative of compile time linking. This is to address the size bloat issue with the PyPI binary release.
- Add CMake option to load `nccl` at runtime.
- Add an NCCL stub.

After this, `nccl` will be fetched from PyPI when using pip to install XGBoost, either by a user or by `pyproject.toml`. Others who want to link the nccl at compile time can continue to do so without any change.

At the moment, this is Linux only since we only support MNMG on Linux.
2023-11-22 19:27:31 +08:00
Bobby Wang
36a552ac98
[jvm-packages] support stage-level scheduling (#9775) 2023-11-14 08:59:45 +08:00
Philip Hyunsu Cho
791de7789b
[jvm-packages] Remove hard dependency on libjvm (#9698) 2023-10-21 23:14:38 -07:00
James Lamb
eb562d3829
[CI] address cmakelint warnings about whitespace (#9674) 2023-10-14 12:46:07 +08:00
James Lamb
db8d117f7e
[CI] standardize endif() calls in CMake scripts (#9637) 2023-10-08 11:45:20 +08:00
Jiaming Yuan
c75a3bc0a9
[breaking] [jvm-packages] Remove rabit check point. (#9599)
- Add `numBoostedRound` to jvm packages
- Remove rabit checkpoint version.
- Change the starting version of training continuation in JVM [breaking].
- Redefine the checkpoint version policy in jvm package. [breaking]
- Rename the Python check point callback parameter. [breaking]
- Unifies the checkpoint policy between Python and JVM.
2023-09-26 18:06:34 +08:00
Jon Yoquinto
d05ea589fb
Allow JVM-Package to access inplace predict method (#9167)
---------

Co-authored-by: Stephan T. Lavavej <stl@nuwen.net>
Co-authored-by: Jiaming Yuan <jm.yuan@outlook.com>
Co-authored-by: Joe <25804777+ByteSizedJoe@users.noreply.github.com>
2023-09-12 07:29:51 +08:00
dependabot[bot]
2462e22cd4
Bump com.nvidia:rapids-4-spark_2.12 in /jvm-packages (#9517)
Bumps com.nvidia:rapids-4-spark_2.12 from 23.08.0 to 23.08.1.

---
updated-dependencies:
- dependency-name: com.nvidia:rapids-4-spark_2.12
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-08-29 15:10:16 +08:00
Jiaming Yuan
be6a552956
[R] Support multi-class custom objective. (#9526) 2023-08-29 08:27:13 +08:00
Jiaming Yuan
972730cde0
Use matrix for gradient. (#9508)
- Use the `linalg::Matrix` for storing gradients.
- New API for the custom objective.
- Custom objective for multi-class/multi-target is now required to return the correct shape.
- Custom objective for Python can accept arrays with any strides. (row-major, column-major)
2023-08-24 05:29:52 +08:00
dependabot[bot]
d779a11af9
Bump scala-collection-compat_2.12 from 2.10.0 to 2.11.0 in /jvm-packages (#9311)
Bumps [scala-collection-compat_2.12](https://github.com/scala/scala-collection-compat) from 2.10.0 to 2.11.0.
- [Release notes](https://github.com/scala/scala-collection-compat/releases)
- [Commits](https://github.com/scala/scala-collection-compat/compare/v2.10.0...v2.11.0)

---
updated-dependencies:
- dependency-name: org.scala-lang.modules:scala-collection-compat_2.12
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-08-21 10:27:35 +08:00
dependabot[bot]
74d5056c61
Bump spark.version.gpu in /jvm-packages/xgboost4j-spark-gpu (#9328)
Bumps `spark.version.gpu` from 3.3.2 to 3.4.1.

Updates `spark-core_2.12` from 3.3.2 to 3.4.1

Updates `spark-sql_2.12` from 3.3.2 to 3.4.1

Updates `spark-mllib_2.12` from 3.3.2 to 3.4.1

---
updated-dependencies:
- dependency-name: org.apache.spark:spark-core_2.12
  dependency-type: direct:production
  update-type: version-update:semver-minor
- dependency-name: org.apache.spark:spark-sql_2.12
  dependency-type: direct:production
  update-type: version-update:semver-minor
- dependency-name: org.apache.spark:spark-mllib_2.12
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-08-20 04:20:07 +08:00
dependabot[bot]
5358e1ebf0
Bump org.apache.commons:commons-lang3 in /jvm-packages (#9489)
Bumps org.apache.commons:commons-lang3 from 3.12.0 to 3.13.0.

---
updated-dependencies:
- dependency-name: org.apache.commons:commons-lang3
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-08-20 00:37:15 +08:00
dependabot[bot]
d016309a15
Bump spark.version from 3.4.0 to 3.4.1 in /jvm-packages/xgboost4j-spark (#9326)
Bumps `spark.version` from 3.4.0 to 3.4.1.

Updates `spark-core_2.12` from 3.4.0 to 3.4.1

Updates `spark-sql_2.12` from 3.4.0 to 3.4.1

Updates `spark-mllib_2.12` from 3.4.0 to 3.4.1

---
updated-dependencies:
- dependency-name: org.apache.spark:spark-core_2.12
  dependency-type: direct:production
  update-type: version-update:semver-patch
- dependency-name: org.apache.spark:spark-sql_2.12
  dependency-type: direct:production
  update-type: version-update:semver-patch
- dependency-name: org.apache.spark:spark-mllib_2.12
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-08-19 18:14:35 +08:00
dependabot[bot]
0bb87b5b35
Bump hadoop.version from 3.3.5 to 3.3.6 in /jvm-packages (#9331)
Bumps `hadoop.version` from 3.3.5 to 3.3.6.

Updates `hadoop-hdfs` from 3.3.5 to 3.3.6

Updates `hadoop-common` from 3.3.5 to 3.3.6

---
updated-dependencies:
- dependency-name: org.apache.hadoop:hadoop-hdfs
  dependency-type: direct:production
  update-type: version-update:semver-patch
- dependency-name: org.apache.hadoop:hadoop-common
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-08-18 20:59:04 +08:00
Jiaming Yuan
58530b1bc4
Bump version to 2.1. (#9498) 2023-08-18 01:04:04 +08:00
Jiaming Yuan
c061e3ae50
[jvm-packages] Bump rapids version. (#9482) 2023-08-15 16:26:42 -07:00
Boris
8463107013
Updated versions. Reorganised dependencies. (#9479) 2023-08-14 14:28:28 -07:00
Bobby Wang
344f90b67b
[jvm-packages] throw exception when tree_method=approx and device=cuda (#9478)
---------

Co-authored-by: Jiaming Yuan <jm.yuan@outlook.com>
2023-08-14 17:52:14 +08:00
jinmfeng001
04c99683c3
Change training stage from ResultStage to ShuffleMapStage (#9423) 2023-08-03 23:40:04 +08:00
Bobby Wang
8f0efb4ab3
[jvm-packages] automatically set the max/min direction for best score (#9404) 2023-07-27 11:09:55 +08:00
Bobby Wang
1b657a5513
[jvm-packages] set device to cuda when tree method is "gpu_hist" (#9412) 2023-07-24 18:32:25 +08:00
Jiaming Yuan
0897477af0
Remove unmaintained jvm readme and dev scripts. (#9395) 2023-07-18 18:23:43 +08:00
Jiaming Yuan
f4fb2be101
[jvm-packages] Add the new device parameter. (#9385) 2023-07-17 18:40:39 +08:00