1b77903eeb
* Fix #2905 * Fix gpu_exact test failures * Fix bug in GPU prediction where multiple calls to batch prediction can produce incorrect results * Fix GPU documentation formatting
106 lines
5.1 KiB
Markdown
106 lines
5.1 KiB
Markdown
XGBoost GPU Support
|
|
===================
|
|
|
|
This page contains information about GPU algorithms supported in XGBoost.
|
|
To install GPU support, checkout the [build and installation instructions](../build.md).
|
|
|
|
# CUDA Accelerated Tree Construction Algorithms
|
|
This plugin adds GPU accelerated tree construction and prediction algorithms to XGBoost.
|
|
## Usage
|
|
Specify the 'tree_method' parameter as one of the following algorithms.
|
|
|
|
### Algorithms
|
|
|
|
```eval_rst
|
|
+--------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|
|
| tree_method | Description |
|
|
+==============+=======================================================================================================================================================================+
|
|
| gpu_exact | The standard XGBoost tree construction algorithm. Performs exact search for splits. Slower and uses considerably more memory than 'gpu_hist' |
|
|
+--------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|
|
| gpu_hist | Equivalent to the XGBoost fast histogram algorithm. Much faster and uses considerably less memory. NOTE: Will run very slowly on GPUs older than Pascal architecture. |
|
|
+--------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|
|
```
|
|
|
|
### Supported parameters
|
|
|
|
```eval_rst
|
|
.. |tick| unicode:: U+2714
|
|
.. |cross| unicode:: U+2718
|
|
|
|
+----------------------+------------+-----------+
|
|
| parameter | gpu_exact | gpu_hist |
|
|
+======================+============+===========+
|
|
| subsample | |cross| | |tick| |
|
|
+----------------------+------------+-----------+
|
|
| colsample_bytree | |cross| | |tick| |
|
|
+----------------------+------------+-----------+
|
|
| colsample_bylevel | |cross| | |tick| |
|
|
+----------------------+------------+-----------+
|
|
| max_bin | |cross| | |tick| |
|
|
+----------------------+------------+-----------+
|
|
| gpu_id | |tick| | |tick| |
|
|
+----------------------+------------+-----------+
|
|
| n_gpus | |cross| | |tick| |
|
|
+----------------------+------------+-----------+
|
|
| predictor | |tick| | |tick| |
|
|
+----------------------+------------+-----------+
|
|
| grow_policy | |cross| | |tick| |
|
|
+----------------------+------------+-----------+
|
|
| monotone_constraints | |cross| | |tick| |
|
|
+----------------------+------------+-----------+
|
|
```
|
|
|
|
GPU accelerated prediction is enabled by default for the above mentioned 'tree_method' parameters but can be switched to CPU prediction by setting 'predictor':'cpu_predictor'. This could be useful if you want to conserve GPU memory. Likewise when using CPU algorithms, GPU accelerated prediction can be enabled by setting 'predictor':'gpu_predictor'.
|
|
|
|
The device ordinal can be selected using the 'gpu_id' parameter, which defaults to 0.
|
|
|
|
Multiple GPUs can be used with the grow_gpu_hist parameter using the n_gpus parameter. which defaults to 1. If this is set to -1 all available GPUs will be used. If gpu_id is specified as non-zero, the gpu device order is mod(gpu_id + i) % n_visible_devices for i=0 to n_gpus-1. As with GPU vs. CPU, multi-GPU will not always be faster than a single GPU due to PCI bus bandwidth that can limit performance.
|
|
|
|
This plugin currently works with the CLI, python and R - see installation guide for details.
|
|
|
|
Python example:
|
|
```python
|
|
param['gpu_id'] = 0
|
|
param['max_bin'] = 16
|
|
param['tree_method'] = 'gpu_hist'
|
|
```
|
|
## Benchmarks
|
|
To run benchmarks on synthetic data for binary classification:
|
|
```bash
|
|
$ python tests/benchmark/benchmark.py
|
|
```
|
|
|
|
Training time time on 1,000,000 rows x 50 columns with 500 boosting iterations and 0.25/0.75 test/train split on i7-6700K CPU @ 4.00GHz and Pascal Titan X.
|
|
|
|
```eval_rst
|
|
+--------------+----------+
|
|
| tree_method | Time (s) |
|
|
+==============+==========+
|
|
| gpu_hist | 13.87 |
|
|
+--------------+----------+
|
|
| hist | 63.55 |
|
|
+--------------+----------+
|
|
| gpu_exact | 161.08 |
|
|
+--------------+----------+
|
|
| exact | 1082.20 |
|
|
+--------------+----------+
|
|
|
|
```
|
|
|
|
[See here](http://dmlc.ml/2016/12/14/GPU-accelerated-xgboost.html) for additional performance benchmarks of the 'gpu_exact' tree_method.
|
|
|
|
## References
|
|
[Mitchell R, Frank E. (2017) Accelerating the XGBoost algorithm using GPU computing. PeerJ Computer Science 3:e127 https://doi.org/10.7717/peerj-cs.127](https://peerj.com/articles/cs-127/)
|
|
|
|
[Nvidia Parallel Forall: Gradient Boosting, Decision Trees and XGBoost with CUDA](https://devblogs.nvidia.com/parallelforall/gradient-boosting-decision-trees-xgboost-cuda/)
|
|
|
|
## Author
|
|
Rory Mitchell
|
|
Jonathan C. McKinney
|
|
Shankara Rao Thejaswi Nanditale
|
|
Vinay Deshpande
|
|
... and the rest of the H2O.ai and NVIDIA team.
|
|
|
|
Please report bugs to the xgboost/issues page.
|
|
|