Hendrik/xgboost
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
4,125 Commits 32 Branches 59 Tags
c362125d7b00a4ee30f074197801e9ab3ee8f8c0
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Rory Mitchell 15800107ad Small updates to GPU documentation (#5483) 2020-04-04 13:02:27 -07:00
Rory Mitchell 15f40e51e9 Add support for dlpack, expose python docs for DeviceQuantileDMatrix (#5465) 2020-04-01 23:34:32 +13:00
Rory Mitchell 13b10a6370 Device dmatrix (#5420) 2020-03-28 14:42:21 +13:00
Rory Mitchell 1de36cdf1e Add link to GPU documentation (#5437) 2020-03-24 09:29:29 +13:00
Rory Mitchell b745b7acce Fix memory usage of device sketching (#5407) 2020-03-14 13:43:24 +13:00
Rory Mitchell 3ad4333b0e Partial rewrite EllpackPage (#5352) 2020-03-11 10:15:53 +13:00
Rory Mitchell a38e7bd19c Sketching from adapters (#5365) 
* Sketching from adapters

* Add weights test
 2020-03-07 21:07:58 +13:00
Rory Mitchell b0ed3f0a66 Remove unnecessary DMatrix methods (#5324) 2020-02-25 12:40:39 +13:00
Rory Mitchell bc96ceb8b2 Refactor SparsePageSource, delete cache files after use (#5321) 
* Refactor sparse page source

* Delete temporary cache files

* Log fatal if cache exists

* Log fatal if multiple threads used with prefetcher
 2020-02-19 16:43:41 +13:00
Rory Mitchell b2b2c4e231 Remove SimpleCSRSource (#5315) 2020-02-18 16:49:17 +13:00
Rory Mitchell 7e32af5c21 Wide dataset quantile performance improvement (#5306) 2020-02-16 10:24:42 +13:00
Rory Mitchell 24ad9dec0b Testing hist_util (#5251) 
* Rank tests

* Remove categorical split specialisation

* Extend tests to multiple features, switch to WQSketch

* Add tests for SparseCuts

* Add external memory quantile tests, fix some existing tests
 2020-02-14 14:36:43 +13:00
Rory Mitchell 1b3947d929 Make some GPU tests deterministic (#5229) 2020-01-26 11:53:07 +13:00
Rory Mitchell aa9a68010b uint not supported in cudf (#5225) 2020-01-23 16:59:18 +13:00
Rory Mitchell 5d4c24a1fc Fix cupy without cudf import (#5219) 2020-01-22 18:02:39 +13:00
Rory Mitchell 9c56480c61 Support dmatrix construction from cupy array (#5206) 2020-01-22 13:15:27 +13:00
Rory Mitchell a73e25e15f Implement slice via adapters (#5198) 2020-01-14 12:55:41 +13:00
Rory Mitchell 8cbcc53ccb Remove old cudf constructor code (#5194) 2020-01-10 16:35:23 +13:00
Rory Mitchell 87ebfc1315 Implement cudf construction with adapters. (#5189) 2020-01-09 20:23:06 +13:00
Rory Mitchell 9559f81377 Move SimpleDMatrix constructor to .cc file (#5188) 2020-01-09 14:20:13 +13:00
Rory Mitchell 3d04a8cc97 Use dynamic types for array interface columns instead of templates (#5108) 2019-12-21 16:08:10 +13:00
Rory Mitchell 5aa007d7b2 Fix visual studio output library directories (#5119) 2019-12-15 15:08:20 +13:00
Rory Mitchell c7cc657a4d Use adapters for SparsePageDMatrix (#5092) 2019-12-11 15:59:23 +13:00
Rory Mitchell 979f74d51a Group builder modified for incremental building (#5098) 2019-12-10 14:33:56 +13:00
Rory Mitchell e3c34c79be External data adapters (#5044) 
* Use external data adapters as lightweight intermediate layer between external data and DMatrix
 2019-12-04 10:56:17 +13:00
Rory Mitchell e67388fb8f Some guidelines on device memory usage (#5038) 
* Add memory usage demo

* Update documentation
 2019-11-17 07:48:24 +13:00
Rory Mitchell 60748b2071 Use heuristic to select histogram node, avoid rabit call (#4951) 2019-10-18 11:33:54 +13:00
Rory Mitchell aefb1e5c2f Resolve dask performance issues (#4914) 
* Set dask client.map as impure function

* Remove nrows

* Remove slow check in verbose mode
 2019-10-10 16:01:30 +13:00
Rory Mitchell 221e163185 Refactor out row partitioning logic from gpu_hist, introduce caching device vectors (#4554) 2019-06-20 18:24:09 +12:00
Rory Mitchell 9683fd433e Overload device memory allocation (#4532) 
* Group source files, include headers in source files

* Overload device memory allocation
 2019-06-10 11:35:13 +12:00
Rory Mitchell 23a10c8339 Refactor histogram building code for gpu_hist (#4528) 2019-06-03 09:50:10 +12:00
Rory Mitchell 399fabed49 Deprecate gpu_exact, bump required cuda version in docs (#4527) 2019-06-03 09:49:05 +12:00
Rory Mitchell fbbae3386a Smarter choice of histogram construction for distributed gpu_hist (#4519) 
* Smarter choice of histogram construction for distributed gpu_hist

* Limit omp team size in ExecuteShards
 2019-05-31 14:11:34 +12:00
Rory Mitchell 972f693eaf Fix dask API sphinx docstrings (#4507) 
* Fix dask API sphinx docstrings

* Update GPU docs page
 2019-05-28 16:39:26 +12:00
Rory Mitchell 09b90d9329 Add native support for Dask (#4473) 
* Add native support for Dask

* Add multi-GPU demo

* Add sklearn example
 2019-05-27 13:29:28 +12:00
Rory Mitchell 8ddd2715ee Add python RF documentation (#4500) 2019-05-24 23:30:24 -07:00
Rory Mitchell d16d9a9988 Correctly determine cuda version (#4453) 2019-05-10 19:46:57 +12:00
Rory Mitchell 5e582b0fa7 Combine thread launches into single launch per tree for gpu_hist (#4343) 
* Combine thread launches into single launch per tree for gpu_hist
algorithm.

* Address deprecation warning

* Add manual column sampler constructor

* Turn off omp dynamic to get a guaranteed number of threads

* Enable openmp in cuda code
 2019-04-29 09:58:34 +12:00
Rory Mitchell 3f312e30db Retire DVec class in favour of c++20 style span for device memory. (#4293) 2019-03-28 13:59:58 +13:00
Rory Mitchell 6d5b34d824 Further optimisations for gpu_hist. (#4283) 
- Fuse final update position functions into a single more efficient kernel

- Refactor gpu_hist with a more explicit ellpack  matrix representation
 2019-03-24 17:17:22 +13:00
Rory Mitchell 8eab966998 Allow unique prediction vector for each input matrix (#4275) 2019-03-21 11:38:16 +13:00
Rory Mitchell 00465d243d Optimisations for gpu_hist. (#4248) 
* Optimisations for gpu_hist.

* Use streams to overlap operations.

* ColumnSampler now uses HostDeviceVector to prevent repeatedly copying feature vectors to the device.
 2019-03-20 13:30:06 +13:00
Rory Mitchell 7814183199 Fix travis R tests (#4277) 2019-03-20 12:56:04 +13:00
Rory Mitchell 5465b73e7c Fix multi-GPU test failures (#4259) 2019-03-15 14:40:43 +13:00
Rory Mitchell 4eeeded7d1 Remove various synchronisations from cuda API calls, instrument monitor (#4205) 
* Remove various synchronisations from cuda API calls, instrument monitor
with nvtx profiler ranges.
 2019-03-10 15:01:23 +13:00
Rory Mitchell c8c472f39a Fix incorrect device in multi-GPU algorithm (#4161) 2019-02-20 09:23:15 +13:00
Rory Mitchell 71a604fae3 Fix for windows compilation (#4139) 2019-02-17 19:42:32 +13:00
Rory Mitchell 1fc37e4749 Require leaf statistics when expanding tree (#4015) 
* Cache left and right gradient sums

* Require leaf statistics when expanding tree
 2019-01-17 21:12:20 -08:00
Rory Mitchell f75a21af25 Reduce tree expand boilerplate code (#4008) 2018-12-20 15:52:28 +13:00
Rory Mitchell 84c99f86f4 Combine TreeModel and RegTree (#3995) 2018-12-19 12:16:40 +13:00
Rory Mitchell 3d81c48d3f Remove leaf vector, add tree serialisation test, fix Windows tests (#3989) 2018-12-13 10:28:38 +13:00
Rory Mitchell 93f9ce9ef9 Single precision histograms on GPU (#3965) 
* Allow single precision histogram summation in gpu_hist

* Add python test, reduce run-time of gpu_hist tests

* Update documentation
 2018-12-10 10:55:30 +13:00
Rory Mitchell a9d684db18 GPU performance logging/improvements (#3945) 
- Improved GPU performance logging

- Only use one execute shards function

- Revert performance regression on multi-GPU

- Use threads to launch NCCL AllReduce
 2018-11-29 14:36:51 +13:00
Rory Mitchell 7af0946ac1 Improve update position function for gpu_hist (#3895) 2018-11-14 19:33:29 +13:00
Rory Mitchell 926eb651fe Minor refactor of split evaluation in gpu_hist (#3889) 
* Refactor evaluate split into shard

* Use span in evaluate split

* Update google tests
 2018-11-14 00:11:20 +13:00
Rory Mitchell 42200ec03e Allow XGBRanker sklearn interface to use other xgboost ranking objectives (#3848) 2018-11-01 13:34:25 +13:00
Rory Mitchell f00fd87b36 Address #2754, accuracy issues with gpu_hist (#3793) 
* Address windows compilation error

* Do not allow divide by zero in weight calculation

* Update tests
 2018-10-15 17:50:31 +13:00
Rory Mitchell 5d6baed998 Allow sklearn grid search over parameters specified as kwargs (#3791) 2018-10-14 12:44:53 +13:00
Rory Mitchell 34522d56f0 Allow plug-ins to be built by cmake (#3752) 
* Remove references to AVX code.

* Allow plugins to be built by cmake
 2018-10-04 22:03:52 +13:00
Rory Mitchell 70d208d68c Dmatrix refactor stage 2 (#3395) 
* DMatrix refactor 2

* Remove buffered rowset usage where possible

* Transition to c++11 style iterators for row access

* Transition column iterators to C++ 11
 2018-10-01 01:29:03 +13:00
Rory Mitchell 78bea0d204 Add google test for a column sampling, restore metainfo tests (#3637) 
* Add google test for a column sampling, restore metainfo tests

* Update metainfo test for visual studio

* Fix multi-GPU bug introduced in #3635
 2018-08-28 16:10:26 +12:00
Rory Mitchell 686e990ffc GPU memory usage fixes + column sampling refactor (#3635) 
* Remove thrust copy calls

* Fix  histogram memory usage

* Cap extreme histogram memory usage

* More efficient column sampling

* Use column sampler across updaters

* More efficient split evaluation on GPU with column sampling
 2018-08-27 16:26:46 +12:00
Rory Mitchell 645996b12f Remove accidental SparsePage copies (#3583) 2018-08-12 17:49:38 -07:00
Rory Mitchell bbb771f32e Refactor parts of fast histogram utilities (#3564) 
* Refactor parts of fast histogram utilities

* Removed byte packing from column matrix
 2018-08-09 17:59:57 +12:00
Rory Mitchell 07ff52d54c Dynamically allocate GPU histogram memory (#3519) 
* Expand histogram memory dynamically to prevent large allocations for large tree depths (e.g. > 15)

* Remove GPU memory allocation messages. These are misleading as a large number of allocations are now dynamic.

* Fix appveyor R test
 2018-07-28 21:22:41 +12:00
Rory Mitchell a725272e19 Correct mistake from dmatrix refactor (#3408) 2018-07-24 15:03:36 +12:00
Rory Mitchell 0f145a0365 Resolve GPU bug on large files (#3472) 
Remove calls to thrust copy, fix indexing bug
 2018-07-16 20:43:45 +12:00
Rory Mitchell 1b59316444 Updates for GPU CI tests (#3467) 
* Fail GPU CI after test failure

* Fix GPU linear tests

* Reduced number of GPU tests to speed up CI

* Remove static allocations of device memory

* Resolve illegal memory access for updater_fast_hist.cc

* Fix broken r tests dependency

* Update python install documentation for GPU
 2018-07-16 18:05:53 +12:00
Rory Mitchell 24fde92660 Build universal wheels using GPU CI (#3424) 2018-06-29 13:45:24 +00:00
Rory Mitchell a0a1df1aba Refactor python tests (#3410) 
* Add unit test utility

* Refactor updater tests. Add coverage for histmaker.
 2018-06-27 11:20:27 +12:00
Rory Mitchell a96039141a Dmatrix refactor stage 1 (#3301) 
* Use sparse page as singular CSR matrix representation

* Simplify dmatrix methods

* Reduce statefullness of batch iterators

* BREAKING CHANGE: Remove prob_buffer_row parameter. Users are instead recommended to sample their dataset as a preprocessing step before using XGBoost.
 2018-06-07 10:25:58 +12:00
Rory Mitchell 3f7696ff53 Cleanup old artefacts in Jenkins (#3361) 2018-06-05 15:16:37 +12:00
Rory Mitchell 3ee725e3bb Add cuda forwards compatibility (#3316) 2018-05-17 10:59:22 +12:00
Rory Mitchell f8b7686719 Add cuda 8/9.1 centos 6 builds, test GPU wheel on CPU only container. (#3309) 
* Add cuda 8/9.1 centos 6 builds, test GPU wheel on CPU only container.

* Add Google test
 2018-05-17 10:57:01 +12:00
Rory Mitchell 088bb4b27c Prevent multiclass Hessian approaching 0 (#3304) 
* Prevent Hessian in multiclass objective becoming zero

* Set default learning rate to 0.5 for "coord_descent" linear updater
 2018-05-09 20:25:51 +12:00
Rory Mitchell 90a5c4db9d Update Jenkins CI for GPU (#3294) 2018-05-04 16:50:59 +12:00
Rory Mitchell a185ddfe03 Implement GPU accelerated coordinate descent algorithm (#3178) 
* Implement GPU accelerated coordinate descent algorithm. 

* Exclude external memory tests for GPU
 2018-04-20 14:56:35 +12:00
Rory Mitchell ccf80703ef Clang-tidy static analysis (#3222) 
* Clang-tidy static analysis

* Modernise checks

* Google coding standard checks

* Identifier renaming according to Google style
 2018-04-19 18:57:13 +12:00
Rory Mitchell 443ff746e9 Fix logic in GPU predictor cache lookup (#3217) 
* Fix logic in GPU predictor cache lookup

* Add sklearn test for GPU prediction
 2018-04-04 15:08:22 +12:00
Rory Mitchell a1ec7b1716 Change reduce operation from thrust to cub. Fix for cuda 9.1 error (#3218) 
* Change reduce operation from thrust to cub. Fix for cuda 9.1 runtime error

* Unit test sum reduce
 2018-04-04 14:21:48 +12:00
Rory Mitchell 9fa45d3a9c Fix bug with gpu_predictor caching behaviour (#3177) 
* Fixes #3162
 2018-03-18 10:35:10 +13:00
Rory Mitchell 7a81c87dfa Fix incorrect minimum value in quantile generation (#3167) 2018-03-14 08:21:18 -07:00
Rory Mitchell dd82b28e20 Update GPU code with dmatrix changes (#3117) 2018-02-17 12:11:48 +13:00
Rory Mitchell 10eb05a63a Refactor linear modelling and add new coordinate descent updater (#3103) 
* Refactor linear modelling and add new coordinate descent updater

* Allow unsorted column iterator

* Add prediction cacheing to gblinear
 2018-02-17 09:17:01 +13:00
Rory Mitchell f87802f00c Fix GPU bugs (#3051) 
* Change uint to unsigned int

* Fix no root predictions bug

* Remove redundant splitting due to numerical instability
 2018-01-23 13:14:15 +13:00
Rory Mitchell 7759ab99ee Fix Google test warnings and error (#2957) 2017-12-20 00:13:56 +13:00
Rory Mitchell 1b77903eeb Fix several GPU bugs (#2916) 
* Fix #2905

* Fix gpu_exact test failures

* Fix bug in GPU prediction where multiple calls to batch prediction can produce incorrect results

* Fix GPU documentation formatting
 2017-12-04 08:27:49 +13:00
Rory Mitchell c51adb49b6 Monotone constraints for gpu_hist (#2904) 2017-11-30 10:26:19 +13:00
Rory Mitchell c55f14668e Update gpu_hist algorithm (#2901) 2017-11-27 13:44:24 +13:00
Rory Mitchell 24f527a1c0 AVX gradients (#2878) 
* AVX gradients

* Add google test for AVX

* Create fallback implementation, remove fma instruction

* Improved accuracy of AVX exp function
 2017-11-27 08:56:01 +13:00
Rory Mitchell 40c6e2f0c8 Improved gpu_hist_experimental algorithm (#2866) 
- Implement colsampling, subsampling for gpu_hist_experimental

 - Optimised multi-GPU implementation for gpu_hist_experimental

 - Make nccl optional

 - Add Volta architecture flag

 - Optimise RegLossObj

 - Add timing utilities for debug verbose mode

 - Bump required cuda version to 8.0
 2017-11-11 13:58:40 +13:00
Rory Mitchell 16c63f30d0 Fix MultiIndex detection (breaks for latest pandas==0.21.0). (#2872) 2017-11-11 11:12:23 +13:00
Rory Mitchell d9d5293cdb Add warnings for large labels when using GPU histogram algorithms (#2834) 2017-10-26 17:31:10 +13:00
Rory Mitchell 13e7a2cff0 Various bug fixes (#2825) 
* Fatal error if GPU algorithm selected without GPU support compiled

* Resolve type conversion warnings

* Fix gpu unit test failure

* Fix compressed iterator edge case

* Fix python unit test failures due to flake8 update on pip
 2017-10-25 14:45:01 +13:00
Rory Mitchell 4cb2f7598b -Add experimental GPU algorithm for lossguided mode (#2755) 
-Improved GPU algorithm unit tests
-Removed some thrust code to improve compile times
 2017-10-01 00:18:35 +13:00
Rory Mitchell 55ba362154 Fix cuda 9.0 compilation (#2718) 2017-09-17 17:13:11 +12:00
Rory Mitchell 9c85903f0b Add GPU documentation (#2695) 
* Add GPU documentation

* Update Python GPU tests
 2017-09-10 19:42:46 +12:00
Rory Mitchell e6a9063344 Integer gradient summation for GPU histogram algorithm. (#2681) 2017-09-08 15:07:29 +12:00
Rory Mitchell 15267eedf2 [GPU-Plugin] Major refactor 2 (#2664) 
* Change cmake option

* Move source files

* Move google tests

* Move python tests

* Move benchmarks

* Move documentation

* Remove makefile support

* Fix test run

* Move GPU tests
 2017-09-08 09:57:16 +12:00
Rory Mitchell 19a53814ce [GPU-Plugin] Major refactor (#2644) 
* Removal of redundant code/files.
* Removal of exact namespace in GPU plugin
* Revert double precision histograms to single precision for performance on Maxwell/Kepler
 2017-08-30 10:53:52 +12:00