Hendrik/xgboost
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
3,951 Commits 32 Branches 59 Tags
f2277e71062bfe0944e0c9f55f58dd54afd5bfca
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Rory Mitchell e67388fb8f Some guidelines on device memory usage (#5038) 
* Add memory usage demo

* Update documentation
 2019-11-17 07:48:24 +13:00
Rory Mitchell 60748b2071 Use heuristic to select histogram node, avoid rabit call (#4951) 2019-10-18 11:33:54 +13:00
Rory Mitchell aefb1e5c2f Resolve dask performance issues (#4914) 
* Set dask client.map as impure function

* Remove nrows

* Remove slow check in verbose mode
 2019-10-10 16:01:30 +13:00
Rory Mitchell 221e163185 Refactor out row partitioning logic from gpu_hist, introduce caching device vectors (#4554) 2019-06-20 18:24:09 +12:00
Rory Mitchell 9683fd433e Overload device memory allocation (#4532) 
* Group source files, include headers in source files

* Overload device memory allocation
 2019-06-10 11:35:13 +12:00
Rory Mitchell 23a10c8339 Refactor histogram building code for gpu_hist (#4528) 2019-06-03 09:50:10 +12:00
Rory Mitchell 399fabed49 Deprecate gpu_exact, bump required cuda version in docs (#4527) 2019-06-03 09:49:05 +12:00
Rory Mitchell fbbae3386a Smarter choice of histogram construction for distributed gpu_hist (#4519) 
* Smarter choice of histogram construction for distributed gpu_hist

* Limit omp team size in ExecuteShards
 2019-05-31 14:11:34 +12:00
Rory Mitchell 972f693eaf Fix dask API sphinx docstrings (#4507) 
* Fix dask API sphinx docstrings

* Update GPU docs page
 2019-05-28 16:39:26 +12:00
Rory Mitchell 09b90d9329 Add native support for Dask (#4473) 
* Add native support for Dask

* Add multi-GPU demo

* Add sklearn example
 2019-05-27 13:29:28 +12:00
Rory Mitchell 8ddd2715ee Add python RF documentation (#4500) 2019-05-24 23:30:24 -07:00
Rory Mitchell d16d9a9988 Correctly determine cuda version (#4453) 2019-05-10 19:46:57 +12:00
Rory Mitchell 5e582b0fa7 Combine thread launches into single launch per tree for gpu_hist (#4343) 
* Combine thread launches into single launch per tree for gpu_hist
algorithm.

* Address deprecation warning

* Add manual column sampler constructor

* Turn off omp dynamic to get a guaranteed number of threads

* Enable openmp in cuda code
 2019-04-29 09:58:34 +12:00
Rory Mitchell 3f312e30db Retire DVec class in favour of c++20 style span for device memory. (#4293) 2019-03-28 13:59:58 +13:00
Rory Mitchell 6d5b34d824 Further optimisations for gpu_hist. (#4283) 
- Fuse final update position functions into a single more efficient kernel

- Refactor gpu_hist with a more explicit ellpack  matrix representation
 2019-03-24 17:17:22 +13:00
Rory Mitchell 8eab966998 Allow unique prediction vector for each input matrix (#4275) 2019-03-21 11:38:16 +13:00
Rory Mitchell 00465d243d Optimisations for gpu_hist. (#4248) 
* Optimisations for gpu_hist.

* Use streams to overlap operations.

* ColumnSampler now uses HostDeviceVector to prevent repeatedly copying feature vectors to the device.
 2019-03-20 13:30:06 +13:00
Rory Mitchell 7814183199 Fix travis R tests (#4277) 2019-03-20 12:56:04 +13:00
Rory Mitchell 5465b73e7c Fix multi-GPU test failures (#4259) 2019-03-15 14:40:43 +13:00
Rory Mitchell 4eeeded7d1 Remove various synchronisations from cuda API calls, instrument monitor (#4205) 
* Remove various synchronisations from cuda API calls, instrument monitor
with nvtx profiler ranges.
 2019-03-10 15:01:23 +13:00
Rory Mitchell c8c472f39a Fix incorrect device in multi-GPU algorithm (#4161) 2019-02-20 09:23:15 +13:00
Rory Mitchell 71a604fae3 Fix for windows compilation (#4139) 2019-02-17 19:42:32 +13:00
Rory Mitchell 1fc37e4749 Require leaf statistics when expanding tree (#4015) 
* Cache left and right gradient sums

* Require leaf statistics when expanding tree
 2019-01-17 21:12:20 -08:00
Rory Mitchell f75a21af25 Reduce tree expand boilerplate code (#4008) 2018-12-20 15:52:28 +13:00
Rory Mitchell 84c99f86f4 Combine TreeModel and RegTree (#3995) 2018-12-19 12:16:40 +13:00
Rory Mitchell 3d81c48d3f Remove leaf vector, add tree serialisation test, fix Windows tests (#3989) 2018-12-13 10:28:38 +13:00
Rory Mitchell 93f9ce9ef9 Single precision histograms on GPU (#3965) 
* Allow single precision histogram summation in gpu_hist

* Add python test, reduce run-time of gpu_hist tests

* Update documentation
 2018-12-10 10:55:30 +13:00
Rory Mitchell a9d684db18 GPU performance logging/improvements (#3945) 
- Improved GPU performance logging

- Only use one execute shards function

- Revert performance regression on multi-GPU

- Use threads to launch NCCL AllReduce
 2018-11-29 14:36:51 +13:00
Rory Mitchell 7af0946ac1 Improve update position function for gpu_hist (#3895) 2018-11-14 19:33:29 +13:00
Rory Mitchell 926eb651fe Minor refactor of split evaluation in gpu_hist (#3889) 
* Refactor evaluate split into shard

* Use span in evaluate split

* Update google tests
 2018-11-14 00:11:20 +13:00
Rory Mitchell 42200ec03e Allow XGBRanker sklearn interface to use other xgboost ranking objectives (#3848) 2018-11-01 13:34:25 +13:00
Rory Mitchell f00fd87b36 Address #2754, accuracy issues with gpu_hist (#3793) 
* Address windows compilation error

* Do not allow divide by zero in weight calculation

* Update tests
 2018-10-15 17:50:31 +13:00
Rory Mitchell 5d6baed998 Allow sklearn grid search over parameters specified as kwargs (#3791) 2018-10-14 12:44:53 +13:00
Rory Mitchell 34522d56f0 Allow plug-ins to be built by cmake (#3752) 
* Remove references to AVX code.

* Allow plugins to be built by cmake
 2018-10-04 22:03:52 +13:00
Rory Mitchell 70d208d68c Dmatrix refactor stage 2 (#3395) 
* DMatrix refactor 2

* Remove buffered rowset usage where possible

* Transition to c++11 style iterators for row access

* Transition column iterators to C++ 11
 2018-10-01 01:29:03 +13:00
Rory Mitchell 78bea0d204 Add google test for a column sampling, restore metainfo tests (#3637) 
* Add google test for a column sampling, restore metainfo tests

* Update metainfo test for visual studio

* Fix multi-GPU bug introduced in #3635
 2018-08-28 16:10:26 +12:00
Rory Mitchell 686e990ffc GPU memory usage fixes + column sampling refactor (#3635) 
* Remove thrust copy calls

* Fix  histogram memory usage

* Cap extreme histogram memory usage

* More efficient column sampling

* Use column sampler across updaters

* More efficient split evaluation on GPU with column sampling
 2018-08-27 16:26:46 +12:00
Rory Mitchell 645996b12f Remove accidental SparsePage copies (#3583) 2018-08-12 17:49:38 -07:00
Rory Mitchell bbb771f32e Refactor parts of fast histogram utilities (#3564) 
* Refactor parts of fast histogram utilities

* Removed byte packing from column matrix
 2018-08-09 17:59:57 +12:00
Rory Mitchell 07ff52d54c Dynamically allocate GPU histogram memory (#3519) 
* Expand histogram memory dynamically to prevent large allocations for large tree depths (e.g. > 15)

* Remove GPU memory allocation messages. These are misleading as a large number of allocations are now dynamic.

* Fix appveyor R test
 2018-07-28 21:22:41 +12:00
Rory Mitchell a725272e19 Correct mistake from dmatrix refactor (#3408) 2018-07-24 15:03:36 +12:00
Rory Mitchell 0f145a0365 Resolve GPU bug on large files (#3472) 
Remove calls to thrust copy, fix indexing bug
 2018-07-16 20:43:45 +12:00
Rory Mitchell 1b59316444 Updates for GPU CI tests (#3467) 
* Fail GPU CI after test failure

* Fix GPU linear tests

* Reduced number of GPU tests to speed up CI

* Remove static allocations of device memory

* Resolve illegal memory access for updater_fast_hist.cc

* Fix broken r tests dependency

* Update python install documentation for GPU
 2018-07-16 18:05:53 +12:00
Rory Mitchell 24fde92660 Build universal wheels using GPU CI (#3424) 2018-06-29 13:45:24 +00:00
Rory Mitchell a0a1df1aba Refactor python tests (#3410) 
* Add unit test utility

* Refactor updater tests. Add coverage for histmaker.
 2018-06-27 11:20:27 +12:00
Rory Mitchell a96039141a Dmatrix refactor stage 1 (#3301) 
* Use sparse page as singular CSR matrix representation

* Simplify dmatrix methods

* Reduce statefullness of batch iterators

* BREAKING CHANGE: Remove prob_buffer_row parameter. Users are instead recommended to sample their dataset as a preprocessing step before using XGBoost.
 2018-06-07 10:25:58 +12:00
Rory Mitchell 3f7696ff53 Cleanup old artefacts in Jenkins (#3361) 2018-06-05 15:16:37 +12:00
Rory Mitchell 3ee725e3bb Add cuda forwards compatibility (#3316) 2018-05-17 10:59:22 +12:00
Rory Mitchell f8b7686719 Add cuda 8/9.1 centos 6 builds, test GPU wheel on CPU only container. (#3309) 
* Add cuda 8/9.1 centos 6 builds, test GPU wheel on CPU only container.

* Add Google test
 2018-05-17 10:57:01 +12:00
Rory Mitchell 088bb4b27c Prevent multiclass Hessian approaching 0 (#3304) 
* Prevent Hessian in multiclass objective becoming zero

* Set default learning rate to 0.5 for "coord_descent" linear updater
 2018-05-09 20:25:51 +12:00
Rory Mitchell 90a5c4db9d Update Jenkins CI for GPU (#3294) 2018-05-04 16:50:59 +12:00
Rory Mitchell a185ddfe03 Implement GPU accelerated coordinate descent algorithm (#3178) 
* Implement GPU accelerated coordinate descent algorithm. 

* Exclude external memory tests for GPU
 2018-04-20 14:56:35 +12:00
Rory Mitchell ccf80703ef Clang-tidy static analysis (#3222) 
* Clang-tidy static analysis

* Modernise checks

* Google coding standard checks

* Identifier renaming according to Google style
 2018-04-19 18:57:13 +12:00
Rory Mitchell 443ff746e9 Fix logic in GPU predictor cache lookup (#3217) 
* Fix logic in GPU predictor cache lookup

* Add sklearn test for GPU prediction
 2018-04-04 15:08:22 +12:00
Rory Mitchell a1ec7b1716 Change reduce operation from thrust to cub. Fix for cuda 9.1 error (#3218) 
* Change reduce operation from thrust to cub. Fix for cuda 9.1 runtime error

* Unit test sum reduce
 2018-04-04 14:21:48 +12:00
Rory Mitchell 9fa45d3a9c Fix bug with gpu_predictor caching behaviour (#3177) 
* Fixes #3162
 2018-03-18 10:35:10 +13:00
Rory Mitchell 7a81c87dfa Fix incorrect minimum value in quantile generation (#3167) 2018-03-14 08:21:18 -07:00
Rory Mitchell dd82b28e20 Update GPU code with dmatrix changes (#3117) 2018-02-17 12:11:48 +13:00
Rory Mitchell 10eb05a63a Refactor linear modelling and add new coordinate descent updater (#3103) 
* Refactor linear modelling and add new coordinate descent updater

* Allow unsorted column iterator

* Add prediction cacheing to gblinear
 2018-02-17 09:17:01 +13:00
Rory Mitchell f87802f00c Fix GPU bugs (#3051) 
* Change uint to unsigned int

* Fix no root predictions bug

* Remove redundant splitting due to numerical instability
 2018-01-23 13:14:15 +13:00
Rory Mitchell 7759ab99ee Fix Google test warnings and error (#2957) 2017-12-20 00:13:56 +13:00
Rory Mitchell 1b77903eeb Fix several GPU bugs (#2916) 
* Fix #2905

* Fix gpu_exact test failures

* Fix bug in GPU prediction where multiple calls to batch prediction can produce incorrect results

* Fix GPU documentation formatting
 2017-12-04 08:27:49 +13:00
Rory Mitchell c51adb49b6 Monotone constraints for gpu_hist (#2904) 2017-11-30 10:26:19 +13:00
Rory Mitchell c55f14668e Update gpu_hist algorithm (#2901) 2017-11-27 13:44:24 +13:00
Rory Mitchell 24f527a1c0 AVX gradients (#2878) 
* AVX gradients

* Add google test for AVX

* Create fallback implementation, remove fma instruction

* Improved accuracy of AVX exp function
 2017-11-27 08:56:01 +13:00
Rory Mitchell 40c6e2f0c8 Improved gpu_hist_experimental algorithm (#2866) 
- Implement colsampling, subsampling for gpu_hist_experimental

 - Optimised multi-GPU implementation for gpu_hist_experimental

 - Make nccl optional

 - Add Volta architecture flag

 - Optimise RegLossObj

 - Add timing utilities for debug verbose mode

 - Bump required cuda version to 8.0
 2017-11-11 13:58:40 +13:00
Rory Mitchell 16c63f30d0 Fix MultiIndex detection (breaks for latest pandas==0.21.0). (#2872) 2017-11-11 11:12:23 +13:00
Rory Mitchell d9d5293cdb Add warnings for large labels when using GPU histogram algorithms (#2834) 2017-10-26 17:31:10 +13:00
Rory Mitchell 13e7a2cff0 Various bug fixes (#2825) 
* Fatal error if GPU algorithm selected without GPU support compiled

* Resolve type conversion warnings

* Fix gpu unit test failure

* Fix compressed iterator edge case

* Fix python unit test failures due to flake8 update on pip
 2017-10-25 14:45:01 +13:00
Rory Mitchell 4cb2f7598b -Add experimental GPU algorithm for lossguided mode (#2755) 
-Improved GPU algorithm unit tests
-Removed some thrust code to improve compile times
 2017-10-01 00:18:35 +13:00
Rory Mitchell 55ba362154 Fix cuda 9.0 compilation (#2718) 2017-09-17 17:13:11 +12:00
Rory Mitchell 9c85903f0b Add GPU documentation (#2695) 
* Add GPU documentation

* Update Python GPU tests
 2017-09-10 19:42:46 +12:00
Rory Mitchell e6a9063344 Integer gradient summation for GPU histogram algorithm. (#2681) 2017-09-08 15:07:29 +12:00
Rory Mitchell 15267eedf2 [GPU-Plugin] Major refactor 2 (#2664) 
* Change cmake option

* Move source files

* Move google tests

* Move python tests

* Move benchmarks

* Move documentation

* Remove makefile support

* Fix test run

* Move GPU tests
 2017-09-08 09:57:16 +12:00
Rory Mitchell 19a53814ce [GPU-Plugin] Major refactor (#2644) 
* Removal of redundant code/files.
* Removal of exact namespace in GPU plugin
* Revert double precision histograms to single precision for performance on Maxwell/Kepler
 2017-08-30 10:53:52 +12:00
Rory Mitchell 70071fc38c Fix demo typo (#2632) 2017-08-23 17:21:51 +02:00
Rory Mitchell 332b26df95 Update GPU acceleration demo (#2617) 
* Update GPU acceleration demo

* Fix parameter formatting
 2017-08-19 21:27:48 +12:00
Rory Mitchell 5661a67d20 Add parallel sort for MSVC (#2609) 2017-08-17 17:14:39 +12:00
Rory Mitchell ef23e424f1 [GPU-Plugin] Add GPU accelerated prediction (#2593) 
* [GPU-Plugin] Add GPU accelerated prediction

* Improve allocation message

* Update documentation

* Resolve linker error for predictor

* Add unit tests
 2017-08-16 12:31:59 +12:00
Rory Mitchell 71e5e622b1 Update cub submodule again (fixes GPU build) (#2599) 2017-08-13 22:14:40 +12:00
Rory Mitchell ac2d0d0ac5 Updated cub submodule reference (#2597) 2017-08-12 23:00:56 -07:00
Rory Mitchell eda9e180f0 [GPU-Plugin] Various fixes (#2579) 
* Fix test large

* Add check for max_depth 0

* Update readme

* Add LBS specialisation for dense data

* Add bst_gpair_precise

* Temporarily disable accuracy tests on test_large.py

* Solve unused variable compiler warning

* Fix max_bin > 1024 error
 2017-08-05 22:16:23 +12:00
Rory Mitchell 0e06d1805d [WIP] Extract prediction into separate interface (#2531) 
* [WIP] Extract prediction into separate interface

* Add copyright, fix linter errors

* Add predictor to amalgamation

* Fix documentation

* Move prediction cache into predictor, add GBTreeModel

* Updated predictor doc comments
 2017-07-28 17:01:03 -07:00
Rory Mitchell 56550ff3f1 Fix pylint (#2537) 2017-07-21 11:41:56 +12:00
Rory Mitchell c85bf9859e [GPU-Plugin] Improved load balancing search (#2521) 2017-07-17 11:50:57 +12:00
Rory Mitchell 530f01e21c [GPU-Plugin] Add load balancing search to gpu_hist. Add compressed iterator. (#2504) 2017-07-11 22:36:39 +12:00
Rory Mitchell 6bfc472bec Update nccl (#2494) 2017-07-07 12:36:26 +12:00
Rory Mitchell e939192978 Cmake improvements (#2487) 
* Cmake improvements
* Add google test to cmake
 2017-07-06 18:05:11 +12:00
Rory Mitchell ed8bc4521e [GPU-Plugin] Resolve double compilation issue (#2479) 2017-07-03 13:29:10 +12:00
Rory Mitchell 5f1b0bb386 [GPU-Plugin] Unify gpu_gpair/bst_gpair. Refactor. (#2477) 2017-07-01 17:31:13 +12:00
Rory Mitchell 48f3003302 [GPU-Plugin] Change GPU plugin to use tree_method parameter, bump cmake version to 3.5 for GPU plugin, add compute architecture 3.5, remove unused cmake files (#2455) 2017-06-29 16:19:45 +12:00
Rory Mitchell 0e48f87529 [GPU-Plugin] Make node_idx type 32 bit for hist algo. Set default n_gpus to 1. (#2445) 2017-06-23 18:26:45 +12:00
Rory Mitchell 1899f9e744 [GPU-Plugin] Add basic continuous integration for GPU plugin. (#2431) 2017-06-22 10:15:28 -04:00
Rory Mitchell 6bf968efe6 [GPU Plugin] Fast histogram speed improvements. Updated benchmarks. (#2258) 2017-05-08 09:21:38 -07:00
Rory Mitchell 8ab5d4611c [GPU-Plugin] (#2227) 
* Add fast histogram algorithm
* Fix Linux build
* Add 'gpu_id' parameter
 2017-04-25 16:37:10 -07:00
Rory Mitchell a33fa05bda GPU Plugin: Bug fix #2048 (#2155) 2017-03-29 10:10:57 -07:00
Rory Mitchell 1957e6fb4d Fix cmake build for linux. Update GPU benchmarks. (#1904) 2016-12-23 09:18:56 +01:00
Rory Mitchell b49b339183 GPU Plugin: Add subsample, colsample_bytree, colsample_bylevel (#1895) 2016-12-22 16:30:36 +01:00
Rory Mitchell d943720883 GPU Plugin: Add bosch demo, update build instructions (#1872) 2016-12-15 07:57:27 +01:00
RAMitchell 2b6aa7736f Add benchmarks, fix GCC build (#1848) 2016-12-08 18:59:10 +01:00