Hendrik/xgboost
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
6,277 Commits 32 Branches 59 Tags
Commit Graph

254 Commits

Author SHA1 Message Date
reg.zhuce
3ee145b8dc [jvm-packages] IndexOutOfBoundsException (#1589) 
ml.dmlc.xgboost4j.scala.spark.XGBoost.scala:51

values is empty when we meet it at first time, so values(0) throw an IndexOutOfBoundsException.
It should be  dVector.values(i) instead of values(i).
 2016-09-20 09:13:47 -04:00
Xin Yin
7245145712 [jvm-packages] Fixed the sanity check for parameter 'nthread' against 'spark.task.cpus'. (#1582) 2016-09-16 11:31:35 -04:00
Nan Zhu
4ad648e856 [jvm-packages] predictLeaf with Dataframe (#1576) 
* add back train method but mark as deprecated

* predictLeaf with Dataset

* fix

* fix
 2016-09-15 06:15:47 -04:00
Nan Zhu
bb388cbb31 default eval func (#1574) 2016-09-14 13:26:16 -04:00
Nan Zhu
fb02797e2a [jvm-packages] Integration with Spark Dataframe/Dataset (#1559) 
* bump up to scala 2.11

* framework of data frame integration

* test consistency between RDD and DataFrame

* order preservation

* test order preservation

* example code and fix makefile

* improve type checking

* improve APIs

* user docs

* work around travis CI's limitation on log length

* adjust test structure

* integrate with Spark -1 .x

* spark 2.x integration

* remove spark 1.x implementation but provide instructions on how to downgrade
 2016-09-11 15:02:58 -04:00
Nan Zhu
6dabdd33e3 [jvm-packages] bump to next version (#1535) 
* bump to next version

* fix

* fix
 2016-09-01 12:18:21 -04:00
Nan Zhu
7fb3fbf577 impose shuffle when creating training RDD (#1531) 2016-08-31 07:34:10 -04:00
Nan Zhu
3f198b9fef [jvm-packages] allow training with missing values in xgboost-spark (#1525) 
* allow training with missing values in xgboost-spark

* fix compilation error

* fix bug
 2016-08-29 21:45:49 -04:00
Nan Zhu
74db1e8867 [jvm-packages] remove APIs with DMatrix from xgboost-spark (#1519) 
* test consistency of prediction functions between DMatrix and RDD

* remove APIs with DMatrix from xgboost-spark

* fix compilation error in xgboost4j-example

* fix test cases
 2016-08-28 21:25:49 -04:00
Nan Zhu
6d65aae091 [jvm-packages] test consistency of prediction functions with DMatrix and RDD (#1518) 
* test consistency of prediction functions between DMatrix and RDD

* fix the failed test cases
 2016-08-28 20:27:03 -04:00
Nan Zhu
d7f79255ec improve test of save/load model (#1515) 2016-08-27 17:16:22 -04:00
Nan Zhu
dc1125eb56 evaluation with RDD data (#1492) 2016-08-20 18:31:10 -04:00
Nan Zhu
582ee63e34 enable train multiple models by distinguishing stage IDs (#1493) 2016-08-20 16:37:07 -04:00
Fangzhou
a8adf16228 fix bug: doing rabit call after finalize in spark prediction phase (#1420) 2016-07-28 23:11:20 -05:00
Earthson Lu
d29edc677c fix #1377 spark-mllib scope: default => provided (#1381) 2016-07-20 23:10:49 -04:00
convexquad
313764b3be Expose predictLeaf functionality in Scala XGBoostModel (#1351) 2016-07-12 06:55:24 -04:00
Rahul
f14c160f4f [jvm-packages][xgboost4j-spark][Minor] Move sparkContext dependency from the XGBoostModel (#1335) 
* Move sparkContext dependency from the XGBoostModel

* Update Spark example to declare SparkContext as implict
 2016-07-08 06:43:33 -04:00
Nan Zhu
bd5b07873e [jvm-packages] create dmatrix with specified missing value (#1272) 
* create dmatrix with specified missing value

* update dmlc-core

* support for predict method in spark package

repartitioning

work around

* add more elements to work around training set empty partition issue
 2016-06-21 17:35:17 -04:00
Nan Zhu
c9a73fe2a9 explicitly throw exception when detecting empty partition in training dataset (#1281) 2016-06-15 16:03:37 -04:00
Nan Zhu
c6631ad2ed specify spark version (#1224) 2016-05-24 18:19:32 -04:00
Nan Zhu
c85b9012c6 [jvm-packages] xgboost4j-spark external memory (#1219) 
* implement external memory support for XGBoost4J

* remove extra space

* enable external memory for prediction

* update doc
 2016-05-22 14:01:28 -04:00
CodingCat
d8535313eb allow empty partitions 2016-03-23 12:30:06 -04:00
CodingCat
55ab1c6a22 adjust numWorkers for test 2016-03-18 10:34:36 -04:00
CodingCat
3a951d0ab8 getter of XGBoostModel 2016-03-14 07:26:51 -04:00
Nan Zhu
e3fa7753f5 Merge branch 'master' into master 2016-03-13 22:46:38 -04:00
CodingCat
6f92f1c117 update spark version to 1.6.1 2016-03-13 22:46:06 -04:00
CodingCat
f2ef958ebb support kryo serialization 2016-03-13 11:55:14 -04:00
CodingCat
16b9e92328 force the user to set number of workers 2016-03-12 13:33:57 -05:00
CodingCat
5f441a29a8 set nthread to spark.task.cpus by default 2016-03-11 20:07:09 -05:00
CodingCat
400b1faecc adjust the API signature as well as the docs 2016-03-11 15:22:44 -05:00
CodingCat
aca0096b33 more updates for Flink 
more fix
 2016-03-11 10:15:49 -05:00
CodingCat
43d7a85bc9 change the API name since we support not only HDFS and local file system 2016-03-11 10:05:32 -05:00
Shaform
6558ef3273 support different types of filesystems 2016-03-11 22:06:40 +08:00
CodingCat
d47df5c1d8 allow the user to specify the worker number and avoid unnecessary shuffle 2016-03-10 06:58:30 -05:00
CodingCat
e0a3f1c000 nthread no larger than spark.task.cpus 2016-03-10 05:51:07 -05:00
CodingCat
005b1276d0 remove duplicate in stream close 2016-03-09 12:33:49 -05:00
CodingCat
852c5a4b32 code formatting in XGBoostModel 2016-03-09 12:31:35 -05:00
CodingCat
909c6af330 add test resources manually 2016-03-08 18:43:30 -05:00
CodingCat
fa03aaeb63 revise current API 2016-03-08 17:18:55 -05:00
tqchen
435a0425b9 [Spark] Refactor train, predict, add save 2016-03-06 21:51:08 -08:00
tqchen
c05c5bc7bc [DOC-JVM] Refactor JVM docs 2016-03-06 20:42:01 -08:00
CodingCat
718a9d8c96 use another thread to control spark job 2016-03-06 15:46:27 -05:00
CodingCat
6499422e90 fix the merge 2016-03-06 15:22:05 -05:00
CodingCat
50337d1906 fix rabitEnv 2016-03-06 14:56:49 -05:00
CodingCat
808e30f9fc example of DistTrainWithSpark and trigger job with foreachPartition 2016-03-06 14:34:23 -05:00
CodingCat
f768edfede adjust the return values of RabitTracker.waitFor(), remove typesafe.Config 2016-03-06 08:44:04 -05:00
CodingCat
130ca7b00c test case for XGBoostSpark 2016-03-05 19:41:26 -05:00
CodingCat
f0647ec76d test resources 2016-03-05 18:18:07 -05:00
CodingCat
5c1af13f84 distributed in RDD 2016-03-05 17:50:40 -05:00
CodingCat
fb41e4e673 spark with new labeledpoint 
fix import order
 2016-03-05 17:22:34 -05:00