Hendrik/xgboost
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
3,513 Commits 32 Branches 59 Tags
19ee0a357964f4c0a7cef6c00da2e8478e360974
Commit Graph
96 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Nan Zhu 5fbe230636 [jvm-packages] documenting tracker (#3831) 
* add back train method but mark as deprecated

* add back train method but mark as deprecated

* add back train method but mark as deprecated

* add back train method but mark as deprecated

* fix scalastyle error

* fix scalastyle error

* fix scalastyle error

* fix scalastyle error

* documenting tracker

* Make it a separate note
 2018-10-25 18:53:46 -07:00
Nan Zhu 4ae225a08d [Blocking][jvm-packages] fix the early stopping feature (#3808) 
* add back train method but mark as deprecated

* add back train method but mark as deprecated

* add back train method but mark as deprecated

* add back train method but mark as deprecated

* fix scalastyle error

* fix scalastyle error

* fix scalastyle error

* fix scalastyle error

* temp

* add method for classifier and regressor

* update tutorial

* address the comments

* update
 2018-10-23 14:53:13 -07:00
Nan Zhu 785094db53 [jvm-packages] fix issue when spark job execution thread cannot return before we execute first() (#3758) 
* add back train method but mark as deprecated

* add back train method but mark as deprecated

* add back train method but mark as deprecated

* add back train method but mark as deprecated

* fix scalastyle error

* fix scalastyle error

* fix scalastyle error

* fix scalastyle error

* sparjJobThread

* update

* fix issue when spark job execution thread cannot return before we execute first()
 2018-10-05 22:20:50 -07:00
Nan Zhu 79d854c695 [jvm-packages] fix errors in example (#3719) 
* add back train method but mark as deprecated

* fix scalastyle error

* add back train method but mark as deprecated

* add back train method but mark as deprecated

* add back train method but mark as deprecated

* fix scalastyle error

* fix scalastyle error

* fix scalastyle error

* instrumentation

* use log console

* better measurement

* fix erros in example

* update histmaker
 2018-09-22 16:39:38 -07:00
Nan Zhu aa53e9fc8d [jvm-packages] bump spark version (#3709) 2018-09-19 11:18:01 -07:00
Nan Zhu d1e75d615e [jvm-packages] Remove copy paste error in test suite (#3692) 
* add back train method but mark as deprecated

* add back train method but mark as deprecated

* fix scalastyle error

* fix scalastyle error

* remove copy paste error
 2018-09-11 13:08:36 -07:00
Nan Zhu 3261002099 [jvm-packages] throw ControlThrowable instead of InterruptedException (#3632) 
* add back train method but mark as deprecated

* add back train method but mark as deprecated

* fix scalastyle error

* fix scalastyle error

* interrupted exception is not rethrown
 2018-08-25 20:30:21 -07:00
Nan Zhu 4912c1f9c6 [jvm-packages] fix checkpoint save/load (#3614) 
* add back train method but mark as deprecated

* add back train method but mark as deprecated

* fix scalastyle error

* fix scalastyle error

* fix update checkpoint func
 2018-08-21 12:34:24 -07:00
Nan Zhu 73bd590a1d [jvm-packages] add the missing scm urls (#3589) 
for some reason this part was missing in master branch????
 2018-08-14 15:05:23 -07:00
Nan Zhu 1c08b3b2ea [jvm-packages] enable predictLeaf/predictContrib/treeLimit in 0.8 (#3532) 
* add back train method but mark as deprecated

* add back train method but mark as deprecated

* fix scalastyle error

* fix scalastyle error

* partial finish

* no test

* add test cases

* add test cases

* address comments

* add test for regressor

* fix typo
 2018-08-07 14:01:18 -07:00
Nan Zhu 31d1baba3d [jvm-packages] Tutorial of XGBoost4J-Spark (#3534) 
* add back train method but mark as deprecated

* add back train method but mark as deprecated

* fix scalastyle error

* fix scalastyle error

* add new

* update doc

* finish Gang Scheduling

* more

* intro

* Add sections: Prediction, Model persistence and ML pipeline.

* Add XGBoost4j-Spark MLlib pipeline example

* partial finished version

* finish the doc

* adjust code

* fix the doc

* use rst

* Convert XGBoost4J-Spark tutorial to reST

* Bring XGBoost4J up to date

* add note about using hdfs

* remove duplicate file

* fix descriptions

* update doc

* Wrap HDFS/S3 export support as a note

* update

* wrap indexing_mode example in code block
 2018-08-03 21:17:50 -07:00
Nan Zhu 6cf97b4eae [jvm-packages] consider spark.task.cpus when controlling parallelism (#3530) 
* add back train method but mark as deprecated

* add back train method but mark as deprecated

* fix scalastyle error

* fix scalastyle error

* consider spark.task.cpus when controlling parallelism

* fix bug

* fix conf setup

* calculate requestedCores within ParallelismController

* enforce spark.task.cpus = 1

* unify unit test case framework

* enable spark ui
 2018-07-31 06:19:45 -07:00
Nan Zhu b546321c83 [jvm-packages] the current version of xgboost does not consider missing value in prediction (#3529) 
* add back train method but mark as deprecated

* add back train method but mark as deprecated

* fix scalastyle error

* fix scalastyle error

* consider missing value in prediction

* handle single prediction instance

* fix type conversion
 2018-07-30 14:16:24 -07:00
Nan Zhu b5178d3d99 [jvm-packages] a better explanation about the inconsistent issue (#3524) 2018-07-28 17:34:39 -07:00
Nan Zhu e2f09db77a [jvm-packages] minor fix for parameter name in example (#3507) 2018-07-25 19:57:40 -07:00
Nan Zhu aa90e5c6ce [jvm-packages] disable booster setup for xgboost4j-spark (#3456) 
* add back train method but mark as deprecated

* add back train method but mark as deprecated

* fix scalastyle error

* fix scalastyle error

* disable booster setup in spark

* check in parameter conversion

* fix compilation issue

* update exception type
 2018-07-07 21:57:24 -07:00
Nan Zhu d062c6f61b [jvm-packages] Maven central release stuffs (#3401) 
* add back train method but mark as deprecated

* add back train method but mark as deprecated

* fix scalastyle error

* fix scalastyle error

* maven central release
 2018-06-22 06:41:28 -07:00
Nan Zhu f66731181f Update 0.8 version num (#3358) 
* add back train method but mark as deprecated

* add back train method but mark as deprecated

* fix scalastyle error

* fix scalastyle error

* update 0.80
 2018-06-02 07:06:01 -07:00
Nan Zhu 49b9f39818 [jvm-packages] update xgboost4j cross build script to be compatible with older glibc (#3307) 
* add back train method but mark as deprecated

* add back train method but mark as deprecated

* fix scalastyle error

* fix scalastyle error

* static glibc glibc++

* update to build with glib 2.12

* remove unsupported flags

* update version number

* remove properties

* remove unnecessary command

* update poms
 2018-05-10 06:39:44 -07:00
Nan Zhu e1f57b4417 [jvm-packages] scripts to cross-build and deploy artifacts to github (#3276) 
* add back train method but mark as deprecated

* add back train method but mark as deprecated

* fix scalastyle error

* fix scalastyle error

* cross building files

* update

* build with docker

* remove

* temp

* update build script

* update pom

* update

* update version

* upload build

* fix path

* update README.md

* fix compiler version to 4.8.5
 2018-04-28 07:41:30 -07:00
Nan Zhu 25b2919c44 [jvm-packages] change version of jvm to keep consistent with other pkgs (#3253) 
* add back train method but mark as deprecated

* add back train method but mark as deprecated

* fix scalastyle error

* fix scalastyle error

* change version of jvm to keep consistent with other pkgs
 2018-04-19 20:48:50 -07:00
Nan Zhu d9dd485313 [jvm-packages] upgrade spark version to 2.3 (#3254) 
* add back train method but mark as deprecated

* add back train method but mark as deprecated

* fix scalastyle error

* fix scalastyle error

* update default spark version to 2.3
 2018-04-19 20:15:19 -07:00
Nan Zhu 4109818b32 [jvm-packages] add back libsvm notes (#3232) 
* add back train method but mark as deprecated

* add back train method but mark as deprecated

* fix scalastyle error

* fix scalastyle error

* add back libsvm notes
 2018-04-10 09:00:58 -07:00
Nan Zhu a187ed6c8f [jvm-packages] tiny fix for empty partition in predict (#3014) 
* add back train method but mark as deprecated

* add back train method but mark as deprecated

* fix scalastyle error

* fix scalastyle error

* tiny fix for empty partition in predict

* further fix
 2018-01-07 08:34:18 -08:00
Nan Zhu 9747ea2acb [jvm-packages] fix the pattern in dev script and version mismatch (#3009) 
* add back train method but mark as deprecated

* add back train method but mark as deprecated

* fix scalastyle error

* fix scalastyle error

* fix the pattern in dev script and version mismatch
 2018-01-06 06:59:38 -08:00
Nan Zhu 14c6392381 [jvm-packages] add dev script to update version and update versions (#2998) 
* add back train method but mark as deprecated

* add back train method but mark as deprecated

* fix scalastyle error

* fix scalastyle error

* add dev script to update version and update versions
 2018-01-01 21:28:53 -08:00
Nan Zhu 005a4a5e47 [jvm-packages] fix numAliveCores in SparkParallelismTracker when WebUI is disabled (#2990) 
* add back train method but mark as deprecated

* add back train method but mark as deprecated

* fix scalastyle error

* fix scalastyle error

* update resource files

* Update SparkParallelismTracker.scala

* remove xgboost-tracker.properties
 2017-12-29 19:22:58 -08:00
Nan Zhu 4fa917b19f Update .travis.yml (#2951) 2017-12-15 13:09:36 -08:00
Nan Zhu 1190dc62a7 Update CONTRIBUTORS.md (#2719) 2017-09-17 15:07:57 -07:00
Nan Zhu e24f25e0c6 add Qubole example (#2401) 2017-06-09 20:33:26 -07:00
Nan Zhu 96f9776ab0 Update ISSUE_TEMPLATE.md (#2308) 
* Update ISSUE_TEMPLATE.md

* Update ISSUE_TEMPLATE.md
 2017-05-18 08:49:07 -07:00
Nan Zhu a607f697e3 [jvm-packages] Disable fast histo for spark (#2296) 
* add back train method but mark as deprecated

* fix scalastyle error

* disable fast histogram in xgboost4j-spark temporarily
 2017-05-15 20:43:16 -07:00
Nan Zhu 428453f7d6 [jvm-packages] fix the persistence of XGBoostEstimator (#2265) 
* add back train method but mark as deprecated

* fix scalastyle error

* fix the persistence of XGBoostEstimator

* test persistence of a complete pipeline

* fix compilation issue

* do not allow persist custom_eval and custom_obj

* fix the failed tesl
 2017-05-08 21:58:06 -07:00
Nan Zhu 392aa6d1d3 [jvm-packages] make XGBoostModel hold BoosterParams as well (#2214) 
* add back train method but mark as deprecated

* fix scalastyle error

* make XGBoostModel hold BoosterParams as well
 2017-04-21 08:12:50 -07:00
Nan Zhu a837fa9620 [jvm-packages] rdds containing boosters should be cleaned once we got boosters to driver (#2183) 2017-04-11 06:12:49 -07:00
Nan Zhu f08077606c [jvm-packages] Clean external cache (#2181) 
* add back train method but mark as deprecated

* fix scalastyle error

* change class to object in examples

* fix compilation error

* small fix for cleanExternalCache
 2017-04-10 07:49:58 -07:00
Nan Zhu 8d8cbcc6db [jvm-packages] fixed several issues in unit tests (#2173) 
* add back train method but mark as deprecated

* fix scalastyle error

* change class to object in examples

* fix compilation error

* fix several issues in tests
 2017-04-06 06:25:23 -07:00
Nan Zhu ab13fd72bd [jvm-packages] Scala/Java interface for Fast Histogram Algorithm (#1966) 
* add back train method but mark as deprecated

* fix scalastyle error

* first commit in scala binding for fast histo

* java test

* add missed scala tests

* spark training

* add back train method but mark as deprecated

* fix scalastyle error

* local change

* first commit in scala binding for fast histo

* local change

* fix df frame test
 2017-03-04 15:37:24 -08:00
Nan Zhu ac30a0aff5 [jvm-packages][spark]Preserve num classes (#2068) 
* add back train method but mark as deprecated

* fix scalastyle error

* change class to object in examples

* fix compilation error

* bump spark version to 2.1

* preserve num_class issues

* fix failed test cases

* rivising

* add multi class test
 2017-03-04 14:14:31 -08:00
Nan Zhu 63aec12a13 [jvm-packages] Bump spark to 2.1 (#2046) 2017-02-19 08:29:35 -08:00
Nan Zhu 185fe1d645 [jvm-packages] use ML's para system to build the passed-in params to XGBoost (#2043) 
* add back train method but mark as deprecated

* fix scalastyle error

* use ML's para system to build the passed-in params to XGBoost

* clean
 2017-02-18 11:56:27 -08:00
Nan Zhu 965091c4bb [jvm-packages] update methods in test cases to be consistent (#1780) 
* add back train method but mark as deprecated

* fix scalastyle error

* change class to object in examples

* fix compilation error

* update methods in test cases to be consistent

* add blank lines

* fix
 2016-11-20 22:49:18 -05:00
Nan Zhu 5217e53156 stylistic fix (#1789) 
* stylistic fix

* try multiple repos

* fix

* fix
 2016-11-19 22:03:10 -05:00
Nan Zhu 6082184cd1 [jvm-packages] update API docs (#1713) 
* add back train method but mark as deprecated

* fix scalastyle error

* update java doc

* update
 2016-10-27 18:53:22 -07:00
Nan Zhu d321375df5 [jvm-packages] Fix mis configure of nthread (#1709) 
* add back train method but mark as deprecated

* fix scalastyle error

* change class to object in examples

* fix compilation error

* fix mis configuration
 2016-10-27 12:10:35 -04:00
Nan Zhu f12074d355 [jvm-packages] release blog (#1706) 2016-10-26 21:35:42 -04:00
Nan Zhu f801c22710 [jvm-packages] change class to object in examples (#1703) 
* change class to object in examples

* fix compilation error
 2016-10-26 14:54:56 -04:00
Nan Zhu 016ab89484 [jvm-packages] Parameter tuning tool for XGBoost (#1664) 2016-10-23 16:58:18 -04:00
Nan Zhu f5c776f64f [jvm-packages] add apache maven repo url and bump up default spark version to 2.0.1 (#1650) 
* add apache maven repo url and bump up default spark version to 2.0.1
 2016-10-13 08:55:03 -04:00
Nan Zhu 813a53882a [jvm-packages] deprecate Flaky test (#1662) 
* deprecate flaky test
 2016-10-13 07:21:24 -04:00
Nan Zhu b56c6097d9 [jvm-packages] add Spark and XGBoost tutorial (#1649) 
* add back train method but mark as deprecated

* add Spark and XGBoost tutorial

* fix scalastyle error
 2016-10-11 09:41:24 -07:00
Nan Zhu 1673bcbe7e [jvm-packages] separate classification and regression model and integrate with ML package (#1608) 2016-09-30 11:49:03 -04:00
Nan Zhu 37bc122c90 [jvm-packages] Robust dmatrix creation (#1613) 
* add back train method but mark as deprecated

* robust matrix creation in jvm
 2016-09-26 13:35:04 -04:00
Nan Zhu 4ad648e856 [jvm-packages] predictLeaf with Dataframe (#1576) 
* add back train method but mark as deprecated

* predictLeaf with Dataset

* fix

* fix
 2016-09-15 06:15:47 -04:00
Nan Zhu bb388cbb31 default eval func (#1574) 2016-09-14 13:26:16 -04:00
Nan Zhu fb02797e2a [jvm-packages] Integration with Spark Dataframe/Dataset (#1559) 
* bump up to scala 2.11

* framework of data frame integration

* test consistency between RDD and DataFrame

* order preservation

* test order preservation

* example code and fix makefile

* improve type checking

* improve APIs

* user docs

* work around travis CI's limitation on log length

* adjust test structure

* integrate with Spark -1 .x

* spark 2.x integration

* remove spark 1.x implementation but provide instructions on how to downgrade
 2016-09-11 15:02:58 -04:00
Nan Zhu 6dabdd33e3 [jvm-packages] bump to next version (#1535) 
* bump to next version

* fix

* fix
 2016-09-01 12:18:21 -04:00
Nan Zhu 7fb3fbf577 impose shuffle when creating training RDD (#1531) 2016-08-31 07:34:10 -04:00
Nan Zhu 3f198b9fef [jvm-packages] allow training with missing values in xgboost-spark (#1525) 
* allow training with missing values in xgboost-spark

* fix compilation error

* fix bug
 2016-08-29 21:45:49 -04:00
Nan Zhu 74db1e8867 [jvm-packages] remove APIs with DMatrix from xgboost-spark (#1519) 
* test consistency of prediction functions between DMatrix and RDD

* remove APIs with DMatrix from xgboost-spark

* fix compilation error in xgboost4j-example

* fix test cases
 2016-08-28 21:25:49 -04:00
Nan Zhu 6d65aae091 [jvm-packages] test consistency of prediction functions with DMatrix and RDD (#1518) 
* test consistency of prediction functions between DMatrix and RDD

* fix the failed test cases
 2016-08-28 20:27:03 -04:00
Nan Zhu d7f79255ec improve test of save/load model (#1515) 2016-08-27 17:16:22 -04:00
Nan Zhu dc1125eb56 evaluation with RDD data (#1492) 2016-08-20 18:31:10 -04:00
Nan Zhu 582ee63e34 enable train multiple models by distinguishing stage IDs (#1493) 2016-08-20 16:37:07 -04:00
Nan Zhu 70432cac5b make IEvaluation serializable (#1487) 2016-08-19 13:12:39 -04:00
Nan Zhu bd5b07873e [jvm-packages] create dmatrix with specified missing value (#1272) 
* create dmatrix with specified missing value

* update dmlc-core

* support for predict method in spark package

repartitioning

work around

* add more elements to work around training set empty partition issue
 2016-06-21 17:35:17 -04:00
Nan Zhu c9a73fe2a9 explicitly throw exception when detecting empty partition in training dataset (#1281) 2016-06-15 16:03:37 -04:00
Nan Zhu c6631ad2ed specify spark version (#1224) 2016-05-24 18:19:32 -04:00
Nan Zhu c85b9012c6 [jvm-packages] xgboost4j-spark external memory (#1219) 
* implement external memory support for XGBoost4J

* remove extra space

* enable external memory for prediction

* update doc
 2016-05-22 14:01:28 -04:00
Nan Zhu 060350f64c Merge pull request #1103 from dmlc/revert-1100-master 
Revert "updating JVM docs"
 2016-04-11 22:36:37 -04:00
Nan Zhu e6de01baaf Revert "updating JVM docs" 2016-04-11 22:00:45 -04:00
Nan Zhu 6eda06256e Merge pull request #1057 from CodingCat/master 
update doc
 2016-03-28 20:09:53 -04:00
Nan Zhu e27977d416 Merge branch 'master' into master 2016-03-28 19:03:04 -04:00
Nan Zhu 605c23e0dc Merge pull request #1037 from CodingCat/allow_empty_partitions 
[jvm-packages] allow empty partitions
 2016-03-23 15:04:51 -04:00
Nan Zhu dfafce4cfd Merge branch 'master' into allow_empty_partitions 2016-03-23 12:30:33 -04:00
Nan Zhu c135703655 Merge pull request #1012 from CodingCat/master 
update installation doc
 2016-03-19 08:58:41 -04:00
Nan Zhu 8842176888 Merge branch 'master' into master 2016-03-19 08:16:18 -04:00
Nan Zhu 11415f228f Merge pull request #1010 from CodingCat/master 
[jvm-packages] adjust numWorkers for test
 2016-03-18 11:30:31 -04:00
Nan Zhu 14a031a1ab Merge branch 'master' into master 2016-03-18 10:36:20 -04:00
Nan Zhu a2146708bd Merge pull request #1008 from CodingCat/master 
typo fix
 2016-03-18 07:12:14 -04:00
Nan Zhu 0b998d5249 Merge branch 'master' into master 2016-03-18 06:56:06 -04:00
Nan Zhu ffe7af572c Merge branch 'master' into master 2016-03-17 23:00:50 -04:00
Nan Zhu 00caf2c956 Merge pull request #988 from CodingCat/master 
[jvm-packages] getter of XGBoostModel
 2016-03-14 07:54:03 -04:00
Nan Zhu 362eb4bd02 Merge pull request #983 from CodingCat/master 
[jvm-packages] upgrade spark version to 1.6.1
 2016-03-13 23:04:16 -04:00
Nan Zhu e3fa7753f5 Merge branch 'master' into master 2016-03-13 22:46:38 -04:00
Nan Zhu a382c1698a Merge branch 'master' into worker_num 2016-03-13 11:57:28 -04:00
Nan Zhu cc1e608a3d Merge branch 'master' into kryo 2016-03-13 11:57:17 -04:00
Nan Zhu 3ce33563f2 Merge pull request #975 from CodingCat/worker_num 
force the user to set number of workers
 2016-03-12 13:47:27 -05:00
Nan Zhu 980898f3fb Merge pull request #971 from CodingCat/set_nthread 
set nthread to spark.task.cpus by default
 2016-03-11 20:19:56 -05:00
Nan Zhu 61db5aa575 Merge branch 'master' into master 2016-03-11 13:58:24 -05:00
Nan Zhu acdd23e789 Merge pull request #967 from CodingCat/master 
[jvm-packages] change the API name
 2016-03-11 10:59:16 -05:00
Nan Zhu 8e3ce908fe Merge pull request #965 from shaform/fs 
Support the cases when user load dataset and save model to two different file systems
 2016-03-11 09:25:02 -05:00
Nan Zhu 00e7e4eef0 Merge pull request #964 from CodingCat/master 
fix create_jni.sh
 2016-03-11 09:00:20 -05:00
Nan Zhu e0555d2ddc Merge branch 'master' into scala_examples 2016-03-09 17:23:22 -05:00
Nan Zhu b398c145a9 Merge branch 'master' into scala_examples 2016-03-09 12:57:41 -05:00
Nan Zhu 8e0c3b08c7 Merge branch 'master' into jvm_package 2016-03-02 15:26:39 -05:00