Hendrik/xgboost
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Update Python demos with tests. (#5651)

Browse Source 
* Remove GPU memory usage demo.
* Add tests for demos.
* Remove `silent`.
* Remove shebang as it's not portable.
This commit is contained in:
Jiaming Yuan
2020-05-12 12:04:42 +08:00
committed by GitHub
parent 4e64e2ef8e
commit 2c1a439869
25 changed files with 158 additions and 158 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
demo/gpu_acceleration/README.md
View File

@@ -1,5 +1,3 @@
# GPU Acceleration Demo
`cover_type.py` shows how to train a model on the [forest cover type](https://archive.ics.uci.edu/ml/datasets/covertype) dataset using GPU acceleration. The forest cover type dataset has 581,012 rows and 54 features, making it time consuming to process. We compare the run-time and accuracy of the GPU and CPU histogram algorithms.
`memory.py` shows how to repeatedly train xgboost models while freeing memory between iterations.
`cover_type.py` shows how to train a model on the [forest cover type](https://archive.ics.uci.edu/ml/datasets/covertype) dataset using GPU acceleration. The forest cover type dataset has 581,012 rows and 54 features, making it time consuming to process. We compare the run-time and accuracy of the GPU and CPU histogram algorithms.

1
demo/gpu_acceleration/cover_type.py
View File

@@ -1,5 +1,4 @@
import xgboost as xgb
import numpy as np
from sklearn.datasets import fetch_covtype
from sklearn.model_selection import train_test_split
import time

51
demo/gpu_acceleration/memory.py
View File

@@ -1,51 +0,0 @@
import xgboost as xgb
import numpy as np
import time
import pickle
import GPUtil
n = 10000
m = 1000
X = np.random.random((n, m))
y = np.random.random(n)
param = {'objective': 'binary:logistic',
'tree_method': 'gpu_hist'
}
iterations = 5
dtrain = xgb.DMatrix(X, label=y)
# High memory usage
# active bst objects with device memory persist across iterations
boosters = []
for i in range(iterations):
bst = xgb.train(param, dtrain)
boosters.append(bst)
print("Example 1")
GPUtil.showUtilization()
del boosters
# Better memory usage
# The bst object can be destroyed by the python gc, freeing device memory
# The gc may not immediately free the object, so more than one booster can be allocated at a time
boosters = []
for i in range(iterations):
bst = xgb.train(param, dtrain)
boosters.append(pickle.dumps(bst))
print("Example 2")
GPUtil.showUtilization()
del boosters
# Best memory usage
# The gc explicitly frees the booster before starting the next iteration
boosters = []
for i in range(iterations):
bst = xgb.train(param, dtrain)
boosters.append(pickle.dumps(bst))
del bst
print("Example 3")
GPUtil.showUtilization()
del boosters