Hendrik/xgboost
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
5,057 Commits 32 Branches 59 Tags
4e9c4f2d73460e58137dc68d9fd8571844da4970
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Rory MitchellandPhilip Hyunsu Cho f0c3ff313f Update GPUTreeShap, add docs (#6281) 
* Update GPUTreeShap, add docs

* Fix test

Co-authored-by: Philip Hyunsu Cho <chohyu01@cs.washington.edu>
 2020-10-27 18:22:12 +13:00
Rory Mitchellandfis 734a911a26 Loop over copy_if (#6201) 
* Loop over copy_if

* Catch OOM.

Co-authored-by: fis <jm.yuan@outlook.com>
 2020-10-14 10:23:16 +13:00
Rory Mitchell dda9e1e487 Update GPUTreeshap (#6163) 
* Reduce shap test duration

* Test interoperability with shap package

* Add feature interactions

* Update GPUTreeShap
 2020-09-28 09:43:47 +13:00
Rory Mitchell 47350f6acb Allow kwargs in dask predict (#6117) 2020-09-15 13:04:03 +12:00
Rory Mitchell 2e907abdb8 Updates to GPUTreeShap (#6087) 
* Extract paths on device

* Update GPUTreeShap
 2020-09-06 13:39:08 +12:00
Rory MitchellandPhilip Hyunsu Cho 9bddecee05 Update GPUTreeShap (#6064) 
* Update GPUTreeShap

* Update src/CMakeLists.txt

Co-authored-by: Philip Hyunsu Cho <chohyu01@cs.washington.edu>

Co-authored-by: Philip Hyunsu Cho <chohyu01@cs.washington.edu>
 2020-08-27 12:01:53 -07:00
Rory Mitchell 9a4e8b1d81 GPUTreeShap (#6038) 2020-08-25 12:47:41 +12:00
Rory MitchellandPhilip Hyunsu Cho abdf894fcf Add cupy to Windows CI (#5797) 
* Add cupy to Windows CI

* Update Jenkinsfile-win64

Co-authored-by: Philip Hyunsu Cho <chohyu01@cs.washington.edu>

* Update Jenkinsfile-win64

Co-authored-by: Philip Hyunsu Cho <chohyu01@cs.washington.edu>

* Update tests/python-gpu/test_gpu_prediction.py

Co-authored-by: Philip Hyunsu Cho <chohyu01@cs.washington.edu>

Co-authored-by: Philip Hyunsu Cho <chohyu01@cs.washington.edu>
 2020-06-17 21:55:09 -07:00
Rory Mitchell b47b5ac771 Use hypothesis (#5759) 
* Use hypothesis

* Allow int64 array interface for groups

* Add packages to Windows CI

* Add to travis

* Make sure device index is set correctly

* Fix dask-cudf test

* appveyor
 2020-06-16 12:45:59 +12:00
Rory Mitchell 359023c0fa Speed up python test (#5752) 
* Speed up tests

* Prevent DeviceQuantileDMatrix initialisation with numpy

* Use joblib.memory

* Use RandomState
 2020-06-05 11:39:24 +12:00
Rory Mitchell f779980f7e gpu_hist performance tweaks (#5707) 
* Remove device vectors

* Remove allreduce synchronize

* Remove double buffer
 2020-05-29 16:48:53 +12:00
Rory Mitchell fcf57823b6 Reduce device synchronisation (#5631) 
* Reduce device synchronisation

* Initialise pinned memory
 2020-05-07 21:19:46 +12:00
Rory Mitchell 9910265064 Resolve vector<bool>::iterator crash (#5642) 2020-05-07 21:18:01 +12:00
Rory Mitchell b9649e7b8e Refactor gpu_hist split evaluation (#5610) 
* Refactor

* Rewrite evaluate splits

* Add more tests
 2020-04-30 08:58:12 +12:00
Rory Mitchell 660be66207 Avoid rabit calls in learner configuration (#5581) 2020-04-24 14:59:29 +12:00
Rory Mitchell a734f52807 Use cudaDeviceGetAttribute instead of cudaGetDeviceProperties (#5570) 2020-04-21 14:58:29 +12:00
Rory Mitchell b2827a80e1 Use non-synchronising scan (#5560) 2020-04-20 15:51:34 +12:00
Rory Mitchell d6d1035950 gpu_hist performance fixes (#5558) 
* Remove unnecessary cuda API calls

* Fix histogram memory growth
 2020-04-19 12:21:13 +12:00
Rory Mitchell e268fb0093 Use thrust functions instead of custom functions (#5544) 2020-04-16 21:41:16 +12:00
Rory Mitchell ca4e05660e Purge device_helpers.cuh (#5534) 
* Simplifications with caching_device_vector

* Purge device helpers
 2020-04-15 21:51:56 +12:00
Rory Mitchell 093e2227e3 Serialise booster after training to reset state (#5484) 
* Serialise booster after training to reset state

* Prevent process_type being set on load

* Check for correct updater sequence
 2020-04-11 16:27:12 +12:00
Rory Mitchell 15800107ad Small updates to GPU documentation (#5483) 2020-04-04 13:02:27 -07:00
Rory Mitchell 15f40e51e9 Add support for dlpack, expose python docs for DeviceQuantileDMatrix (#5465) 2020-04-01 23:34:32 +13:00
Rory Mitchell 13b10a6370 Device dmatrix (#5420) 2020-03-28 14:42:21 +13:00
Rory Mitchell 1de36cdf1e Add link to GPU documentation (#5437) 2020-03-24 09:29:29 +13:00
Rory Mitchell b745b7acce Fix memory usage of device sketching (#5407) 2020-03-14 13:43:24 +13:00
Rory Mitchell 3ad4333b0e Partial rewrite EllpackPage (#5352) 2020-03-11 10:15:53 +13:00
Rory Mitchell a38e7bd19c Sketching from adapters (#5365) 
* Sketching from adapters

* Add weights test
 2020-03-07 21:07:58 +13:00
Rory Mitchell b0ed3f0a66 Remove unnecessary DMatrix methods (#5324) 2020-02-25 12:40:39 +13:00
Rory Mitchell bc96ceb8b2 Refactor SparsePageSource, delete cache files after use (#5321) 
* Refactor sparse page source

* Delete temporary cache files

* Log fatal if cache exists

* Log fatal if multiple threads used with prefetcher
 2020-02-19 16:43:41 +13:00
Rory Mitchell b2b2c4e231 Remove SimpleCSRSource (#5315) 2020-02-18 16:49:17 +13:00
Rory Mitchell 7e32af5c21 Wide dataset quantile performance improvement (#5306) 2020-02-16 10:24:42 +13:00
Rory Mitchell 24ad9dec0b Testing hist_util (#5251) 
* Rank tests

* Remove categorical split specialisation

* Extend tests to multiple features, switch to WQSketch

* Add tests for SparseCuts

* Add external memory quantile tests, fix some existing tests
 2020-02-14 14:36:43 +13:00
Rory Mitchell 1b3947d929 Make some GPU tests deterministic (#5229) 2020-01-26 11:53:07 +13:00
Rory Mitchell aa9a68010b uint not supported in cudf (#5225) 2020-01-23 16:59:18 +13:00
Rory Mitchell 5d4c24a1fc Fix cupy without cudf import (#5219) 2020-01-22 18:02:39 +13:00
Rory Mitchell 9c56480c61 Support dmatrix construction from cupy array (#5206) 2020-01-22 13:15:27 +13:00
Rory Mitchell a73e25e15f Implement slice via adapters (#5198) 2020-01-14 12:55:41 +13:00
Rory Mitchell 8cbcc53ccb Remove old cudf constructor code (#5194) 2020-01-10 16:35:23 +13:00
Rory Mitchell 87ebfc1315 Implement cudf construction with adapters. (#5189) 2020-01-09 20:23:06 +13:00
Rory Mitchell 9559f81377 Move SimpleDMatrix constructor to .cc file (#5188) 2020-01-09 14:20:13 +13:00
Rory Mitchell 3d04a8cc97 Use dynamic types for array interface columns instead of templates (#5108) 2019-12-21 16:08:10 +13:00
Rory Mitchell 5aa007d7b2 Fix visual studio output library directories (#5119) 2019-12-15 15:08:20 +13:00
Rory Mitchell c7cc657a4d Use adapters for SparsePageDMatrix (#5092) 2019-12-11 15:59:23 +13:00
Rory Mitchell 979f74d51a Group builder modified for incremental building (#5098) 2019-12-10 14:33:56 +13:00
Rory Mitchell e3c34c79be External data adapters (#5044) 
* Use external data adapters as lightweight intermediate layer between external data and DMatrix
 2019-12-04 10:56:17 +13:00
Rory Mitchell e67388fb8f Some guidelines on device memory usage (#5038) 
* Add memory usage demo

* Update documentation
 2019-11-17 07:48:24 +13:00
Rory Mitchell 60748b2071 Use heuristic to select histogram node, avoid rabit call (#4951) 2019-10-18 11:33:54 +13:00
Rory Mitchell aefb1e5c2f Resolve dask performance issues (#4914) 
* Set dask client.map as impure function

* Remove nrows

* Remove slow check in verbose mode
 2019-10-10 16:01:30 +13:00
Rory Mitchell 221e163185 Refactor out row partitioning logic from gpu_hist, introduce caching device vectors (#4554) 2019-06-20 18:24:09 +12:00
Rory Mitchell 9683fd433e Overload device memory allocation (#4532) 
* Group source files, include headers in source files

* Overload device memory allocation
 2019-06-10 11:35:13 +12:00
Rory Mitchell 23a10c8339 Refactor histogram building code for gpu_hist (#4528) 2019-06-03 09:50:10 +12:00
Rory Mitchell 399fabed49 Deprecate gpu_exact, bump required cuda version in docs (#4527) 2019-06-03 09:49:05 +12:00
Rory Mitchell fbbae3386a Smarter choice of histogram construction for distributed gpu_hist (#4519) 
* Smarter choice of histogram construction for distributed gpu_hist

* Limit omp team size in ExecuteShards
 2019-05-31 14:11:34 +12:00
Rory Mitchell 972f693eaf Fix dask API sphinx docstrings (#4507) 
* Fix dask API sphinx docstrings

* Update GPU docs page
 2019-05-28 16:39:26 +12:00
Rory Mitchell 09b90d9329 Add native support for Dask (#4473) 
* Add native support for Dask

* Add multi-GPU demo

* Add sklearn example
 2019-05-27 13:29:28 +12:00
Rory Mitchell 8ddd2715ee Add python RF documentation (#4500) 2019-05-24 23:30:24 -07:00
Rory Mitchell d16d9a9988 Correctly determine cuda version (#4453) 2019-05-10 19:46:57 +12:00
Rory Mitchell 5e582b0fa7 Combine thread launches into single launch per tree for gpu_hist (#4343) 
* Combine thread launches into single launch per tree for gpu_hist
algorithm.

* Address deprecation warning

* Add manual column sampler constructor

* Turn off omp dynamic to get a guaranteed number of threads

* Enable openmp in cuda code
 2019-04-29 09:58:34 +12:00
Rory Mitchell 3f312e30db Retire DVec class in favour of c++20 style span for device memory. (#4293) 2019-03-28 13:59:58 +13:00
Rory Mitchell 6d5b34d824 Further optimisations for gpu_hist. (#4283) 
- Fuse final update position functions into a single more efficient kernel

- Refactor gpu_hist with a more explicit ellpack  matrix representation
 2019-03-24 17:17:22 +13:00
Rory Mitchell 8eab966998 Allow unique prediction vector for each input matrix (#4275) 2019-03-21 11:38:16 +13:00
Rory Mitchell 00465d243d Optimisations for gpu_hist. (#4248) 
* Optimisations for gpu_hist.

* Use streams to overlap operations.

* ColumnSampler now uses HostDeviceVector to prevent repeatedly copying feature vectors to the device.
 2019-03-20 13:30:06 +13:00
Rory Mitchell 7814183199 Fix travis R tests (#4277) 2019-03-20 12:56:04 +13:00
Rory Mitchell 5465b73e7c Fix multi-GPU test failures (#4259) 2019-03-15 14:40:43 +13:00
Rory Mitchell 4eeeded7d1 Remove various synchronisations from cuda API calls, instrument monitor (#4205) 
* Remove various synchronisations from cuda API calls, instrument monitor
with nvtx profiler ranges.
 2019-03-10 15:01:23 +13:00
Rory Mitchell c8c472f39a Fix incorrect device in multi-GPU algorithm (#4161) 2019-02-20 09:23:15 +13:00
Rory Mitchell 71a604fae3 Fix for windows compilation (#4139) 2019-02-17 19:42:32 +13:00
Rory Mitchell 1fc37e4749 Require leaf statistics when expanding tree (#4015) 
* Cache left and right gradient sums

* Require leaf statistics when expanding tree
 2019-01-17 21:12:20 -08:00
Rory Mitchell f75a21af25 Reduce tree expand boilerplate code (#4008) 2018-12-20 15:52:28 +13:00
Rory Mitchell 84c99f86f4 Combine TreeModel and RegTree (#3995) 2018-12-19 12:16:40 +13:00
Rory Mitchell 3d81c48d3f Remove leaf vector, add tree serialisation test, fix Windows tests (#3989) 2018-12-13 10:28:38 +13:00
Rory Mitchell 93f9ce9ef9 Single precision histograms on GPU (#3965) 
* Allow single precision histogram summation in gpu_hist

* Add python test, reduce run-time of gpu_hist tests

* Update documentation
 2018-12-10 10:55:30 +13:00
Rory Mitchell a9d684db18 GPU performance logging/improvements (#3945) 
- Improved GPU performance logging

- Only use one execute shards function

- Revert performance regression on multi-GPU

- Use threads to launch NCCL AllReduce
 2018-11-29 14:36:51 +13:00
Rory Mitchell 7af0946ac1 Improve update position function for gpu_hist (#3895) 2018-11-14 19:33:29 +13:00
Rory Mitchell 926eb651fe Minor refactor of split evaluation in gpu_hist (#3889) 
* Refactor evaluate split into shard

* Use span in evaluate split

* Update google tests
 2018-11-14 00:11:20 +13:00
Rory Mitchell 42200ec03e Allow XGBRanker sklearn interface to use other xgboost ranking objectives (#3848) 2018-11-01 13:34:25 +13:00
Rory Mitchell f00fd87b36 Address #2754, accuracy issues with gpu_hist (#3793) 
* Address windows compilation error

* Do not allow divide by zero in weight calculation

* Update tests
 2018-10-15 17:50:31 +13:00
Rory Mitchell 5d6baed998 Allow sklearn grid search over parameters specified as kwargs (#3791) 2018-10-14 12:44:53 +13:00
Rory Mitchell 34522d56f0 Allow plug-ins to be built by cmake (#3752) 
* Remove references to AVX code.

* Allow plugins to be built by cmake
 2018-10-04 22:03:52 +13:00
Rory Mitchell 70d208d68c Dmatrix refactor stage 2 (#3395) 
* DMatrix refactor 2

* Remove buffered rowset usage where possible

* Transition to c++11 style iterators for row access

* Transition column iterators to C++ 11
 2018-10-01 01:29:03 +13:00
Rory Mitchell 78bea0d204 Add google test for a column sampling, restore metainfo tests (#3637) 
* Add google test for a column sampling, restore metainfo tests

* Update metainfo test for visual studio

* Fix multi-GPU bug introduced in #3635
 2018-08-28 16:10:26 +12:00
Rory Mitchell 686e990ffc GPU memory usage fixes + column sampling refactor (#3635) 
* Remove thrust copy calls

* Fix  histogram memory usage

* Cap extreme histogram memory usage

* More efficient column sampling

* Use column sampler across updaters

* More efficient split evaluation on GPU with column sampling
 2018-08-27 16:26:46 +12:00
Rory Mitchell 645996b12f Remove accidental SparsePage copies (#3583) 2018-08-12 17:49:38 -07:00
Rory Mitchell bbb771f32e Refactor parts of fast histogram utilities (#3564) 
* Refactor parts of fast histogram utilities

* Removed byte packing from column matrix
 2018-08-09 17:59:57 +12:00
Rory Mitchell 07ff52d54c Dynamically allocate GPU histogram memory (#3519) 
* Expand histogram memory dynamically to prevent large allocations for large tree depths (e.g. > 15)

* Remove GPU memory allocation messages. These are misleading as a large number of allocations are now dynamic.

* Fix appveyor R test
 2018-07-28 21:22:41 +12:00
Rory Mitchell a725272e19 Correct mistake from dmatrix refactor (#3408) 2018-07-24 15:03:36 +12:00
Rory Mitchell 0f145a0365 Resolve GPU bug on large files (#3472) 
Remove calls to thrust copy, fix indexing bug
 2018-07-16 20:43:45 +12:00
Rory Mitchell 1b59316444 Updates for GPU CI tests (#3467) 
* Fail GPU CI after test failure

* Fix GPU linear tests

* Reduced number of GPU tests to speed up CI

* Remove static allocations of device memory

* Resolve illegal memory access for updater_fast_hist.cc

* Fix broken r tests dependency

* Update python install documentation for GPU
 2018-07-16 18:05:53 +12:00
Rory Mitchell 24fde92660 Build universal wheels using GPU CI (#3424) 2018-06-29 13:45:24 +00:00
Rory Mitchell a0a1df1aba Refactor python tests (#3410) 
* Add unit test utility

* Refactor updater tests. Add coverage for histmaker.
 2018-06-27 11:20:27 +12:00
Rory Mitchell a96039141a Dmatrix refactor stage 1 (#3301) 
* Use sparse page as singular CSR matrix representation

* Simplify dmatrix methods

* Reduce statefullness of batch iterators

* BREAKING CHANGE: Remove prob_buffer_row parameter. Users are instead recommended to sample their dataset as a preprocessing step before using XGBoost.
 2018-06-07 10:25:58 +12:00
Rory Mitchell 3f7696ff53 Cleanup old artefacts in Jenkins (#3361) 2018-06-05 15:16:37 +12:00
Rory Mitchell 3ee725e3bb Add cuda forwards compatibility (#3316) 2018-05-17 10:59:22 +12:00
Rory Mitchell f8b7686719 Add cuda 8/9.1 centos 6 builds, test GPU wheel on CPU only container. (#3309) 
* Add cuda 8/9.1 centos 6 builds, test GPU wheel on CPU only container.

* Add Google test
 2018-05-17 10:57:01 +12:00
Rory Mitchell 088bb4b27c Prevent multiclass Hessian approaching 0 (#3304) 
* Prevent Hessian in multiclass objective becoming zero

* Set default learning rate to 0.5 for "coord_descent" linear updater
 2018-05-09 20:25:51 +12:00
Rory Mitchell 90a5c4db9d Update Jenkins CI for GPU (#3294) 2018-05-04 16:50:59 +12:00
Rory Mitchell a185ddfe03 Implement GPU accelerated coordinate descent algorithm (#3178) 
* Implement GPU accelerated coordinate descent algorithm. 

* Exclude external memory tests for GPU
 2018-04-20 14:56:35 +12:00
Rory Mitchell ccf80703ef Clang-tidy static analysis (#3222) 
* Clang-tidy static analysis

* Modernise checks

* Google coding standard checks

* Identifier renaming according to Google style
 2018-04-19 18:57:13 +12:00
Rory Mitchell 443ff746e9 Fix logic in GPU predictor cache lookup (#3217) 
* Fix logic in GPU predictor cache lookup

* Add sklearn test for GPU prediction
 2018-04-04 15:08:22 +12:00