Doc modernization (#3474)
* Change doc build to reST exclusively * Rewrite Intro doc in reST; create toctree * Update parameter and contribute * Convert tutorials to reST * Convert Python tutorials to reST * Convert CLI and Julia docs to reST * Enable markdown for R vignettes * Done migrating to reST * Add guzzle_sphinx_theme to requirements * Add breathe to requirements * Fix search bar * Add link to user forum
This commit is contained in:
Philip Hyunsu Cho
committed by
GitHub
GitHub
parent
c004cea788
commit
05b089405d
@@ -1,10 +0,0 @@
|
||||
XGBoost Python Package
|
||||
======================
|
||||
This page contains links to all the python related documents on python package.
|
||||
To install the package package, checkout [Build and Installation Instruction](../build.md).
|
||||
|
||||
Contents
|
||||
--------
|
||||
* [Python Overview Tutorial](python_intro.md)
|
||||
* [Learning to use XGBoost by Example](../../demo)
|
||||
* [Python API Reference](python_api.rst)
|
||||
14
doc/python/index.rst
Normal file
14
doc/python/index.rst
Normal file
@@ -0,0 +1,14 @@
|
||||
######################
|
||||
XGBoost Python Package
|
||||
######################
|
||||
This page contains links to all the python related documents on python package.
|
||||
To install the package package, checkout :doc:`Installation Guide </build>`.
|
||||
|
||||
********
|
||||
Contents
|
||||
********
|
||||
|
||||
.. toctree::
|
||||
python_intro
|
||||
python_api
|
||||
Python examples <https://github.com/dmlc/xgboost/tree/master/demo/guide-python>
|
||||
@@ -2,8 +2,6 @@ Python API Reference
|
||||
====================
|
||||
This page gives the Python API reference of xgboost, please also refer to Python Package Introduction for more information about python package.
|
||||
|
||||
The document in this page is automatically generated by sphinx. The content do not render at github, you can view it at http://xgboost.readthedocs.org/en/latest/python/python_api.html
|
||||
|
||||
Core Data Structure
|
||||
-------------------
|
||||
.. automodule:: xgboost.core
|
||||
|
||||
@@ -1,173 +0,0 @@
|
||||
Python Package Introduction
|
||||
===========================
|
||||
This document gives a basic walkthrough of xgboost python package.
|
||||
|
||||
***List of other Helpful Links***
|
||||
* [Python walkthrough code collections](https://github.com/tqchen/xgboost/blob/master/demo/guide-python)
|
||||
* [Python API Reference](python_api.rst)
|
||||
|
||||
Install XGBoost
|
||||
---------------
|
||||
To install XGBoost, do the following:
|
||||
|
||||
* Run `make` in the root directory of the project
|
||||
* In the `python-package` directory, run
|
||||
```shell
|
||||
python setup.py install
|
||||
```
|
||||
|
||||
To verify your installation, try to `import xgboost` in Python.
|
||||
```python
|
||||
import xgboost as xgb
|
||||
```
|
||||
|
||||
Data Interface
|
||||
--------------
|
||||
The XGBoost python module is able to load data from:
|
||||
- libsvm txt format file
|
||||
- comma-separated values (CSV) file
|
||||
- Numpy 2D array
|
||||
- Scipy 2D sparse array, and
|
||||
- xgboost binary buffer file.
|
||||
|
||||
The data is stored in a ```DMatrix``` object.
|
||||
|
||||
* To load a libsvm text file or a XGBoost binary file into ```DMatrix```:
|
||||
```python
|
||||
dtrain = xgb.DMatrix('train.svm.txt')
|
||||
dtest = xgb.DMatrix('test.svm.buffer')
|
||||
```
|
||||
* To load a CSV file into ```DMatrix```:
|
||||
```python
|
||||
# label_column specifies the index of the column containing the true label
|
||||
dtrain = xgb.DMatrix('train.csv?format=csv&label_column=0')
|
||||
dtest = xgb.DMatrix('test.csv?format=csv&label_column=0')
|
||||
```
|
||||
(Note that XGBoost does not support categorical features; if your data contains
|
||||
categorical features, load it as a numpy array first and then perform
|
||||
[one-hot encoding](http://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.OneHotEncoder.html).)
|
||||
|
||||
* To load a numpy array into ```DMatrix```:
|
||||
```python
|
||||
data = np.random.rand(5, 10) # 5 entities, each contains 10 features
|
||||
label = np.random.randint(2, size=5) # binary target
|
||||
dtrain = xgb.DMatrix(data, label=label)
|
||||
```
|
||||
* To load a scpiy.sparse array into ```DMatrix```:
|
||||
```python
|
||||
csr = scipy.sparse.csr_matrix((dat, (row, col)))
|
||||
dtrain = xgb.DMatrix(csr)
|
||||
```
|
||||
* Saving ```DMatrix``` into a XGBoost binary file will make loading faster:
|
||||
```python
|
||||
dtrain = xgb.DMatrix('train.svm.txt')
|
||||
dtrain.save_binary('train.buffer')
|
||||
```
|
||||
* Missing values can be replaced by a default value in the ```DMatrix``` constructor:
|
||||
```python
|
||||
dtrain = xgb.DMatrix(data, label=label, missing=-999.0)
|
||||
```
|
||||
* Weights can be set when needed:
|
||||
```python
|
||||
w = np.random.rand(5, 1)
|
||||
dtrain = xgb.DMatrix(data, label=label, missing=-999.0, weight=w)
|
||||
```
|
||||
|
||||
Setting Parameters
|
||||
------------------
|
||||
XGBoost can use either a list of pairs or a dictionary to set [parameters](../parameter.md). For instance:
|
||||
* Booster parameters
|
||||
```python
|
||||
param = {'max_depth': 2, 'eta': 1, 'silent': 1, 'objective': 'binary:logistic'}
|
||||
param['nthread'] = 4
|
||||
param['eval_metric'] = 'auc'
|
||||
```
|
||||
* You can also specify multiple eval metrics:
|
||||
```python
|
||||
param['eval_metric'] = ['auc', 'ams@0']
|
||||
|
||||
# alternatively:
|
||||
# plst = param.items()
|
||||
# plst += [('eval_metric', 'ams@0')]
|
||||
```
|
||||
|
||||
* Specify validations set to watch performance
|
||||
```python
|
||||
evallist = [(dtest, 'eval'), (dtrain, 'train')]
|
||||
```
|
||||
|
||||
Training
|
||||
--------
|
||||
|
||||
Training a model requires a parameter list and data set.
|
||||
```python
|
||||
num_round = 10
|
||||
bst = xgb.train(plst, dtrain, num_round, evallist)
|
||||
```
|
||||
After training, the model can be saved.
|
||||
```python
|
||||
bst.save_model('0001.model')
|
||||
```
|
||||
The model and its feature map can also be dumped to a text file.
|
||||
```python
|
||||
# dump model
|
||||
bst.dump_model('dump.raw.txt')
|
||||
# dump model with feature map
|
||||
bst.dump_model('dump.raw.txt', 'featmap.txt')
|
||||
```
|
||||
A saved model can be loaded as follows:
|
||||
```python
|
||||
bst = xgb.Booster({'nthread': 4}) # init model
|
||||
bst.load_model('model.bin') # load data
|
||||
```
|
||||
|
||||
Early Stopping
|
||||
--------------
|
||||
If you have a validation set, you can use early stopping to find the optimal number of boosting rounds.
|
||||
Early stopping requires at least one set in `evals`. If there's more than one, it will use the last.
|
||||
|
||||
`train(..., evals=evals, early_stopping_rounds=10)`
|
||||
|
||||
The model will train until the validation score stops improving. Validation error needs to decrease at least every `early_stopping_rounds` to continue training.
|
||||
|
||||
If early stopping occurs, the model will have three additional fields: `bst.best_score`, `bst.best_iteration` and `bst.best_ntree_limit`. Note that `train()` will return a model from the last iteration, not the best one.
|
||||
|
||||
This works with both metrics to minimize (RMSE, log loss, etc.) and to maximize (MAP, NDCG, AUC). Note that if you specify more than one evaluation metric the last one in `param['eval_metric']` is used for early stopping.
|
||||
|
||||
Prediction
|
||||
----------
|
||||
A model that has been trained or loaded can perform predictions on data sets.
|
||||
```python
|
||||
# 7 entities, each contains 10 features
|
||||
data = np.random.rand(7, 10)
|
||||
dtest = xgb.DMatrix(data)
|
||||
ypred = bst.predict(dtest)
|
||||
```
|
||||
|
||||
If early stopping is enabled during training, you can get predictions from the best iteration with `bst.best_ntree_limit`:
|
||||
```python
|
||||
ypred = bst.predict(dtest, ntree_limit=bst.best_ntree_limit)
|
||||
```
|
||||
|
||||
Plotting
|
||||
--------
|
||||
|
||||
You can use plotting module to plot importance and output tree.
|
||||
|
||||
To plot importance, use ``plot_importance``. This function requires ``matplotlib`` to be installed.
|
||||
|
||||
```python
|
||||
xgb.plot_importance(bst)
|
||||
```
|
||||
|
||||
To plot the output tree via ``matplotlib``, use ``plot_tree``, specifying the ordinal number of the target tree. This function requires ``graphviz`` and ``matplotlib``.
|
||||
|
||||
```python
|
||||
xgb.plot_tree(bst, num_trees=2)
|
||||
```
|
||||
|
||||
When you use ``IPython``, you can use the ``to_graphviz`` function, which converts the target tree to a ``graphviz`` instance. The ``graphviz`` instance is automatically rendered in ``IPython``.
|
||||
|
||||
```python
|
||||
xgb.to_graphviz(bst, num_trees=2)
|
||||
```
|
||||
203
doc/python/python_intro.rst
Normal file
203
doc/python/python_intro.rst
Normal file
@@ -0,0 +1,203 @@
|
||||
###########################
|
||||
Python Package Introduction
|
||||
###########################
|
||||
This document gives a basic walkthrough of xgboost python package.
|
||||
|
||||
**List of other Helpful Links**
|
||||
|
||||
* `Python walkthrough code collections <https://github.com/tqchen/xgboost/blob/master/demo/guide-python>`_
|
||||
* :doc:`Python API Reference <python_api>`
|
||||
|
||||
Install XGBoost
|
||||
---------------
|
||||
To install XGBoost, follow instructions in :doc:`/build`.
|
||||
|
||||
To verify your installation, run the following in Python:
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
import xgboost as xgb
|
||||
|
||||
Data Interface
|
||||
--------------
|
||||
The XGBoost python module is able to load data from:
|
||||
|
||||
- LibSVM text format file
|
||||
- Comma-separated values (CSV) file
|
||||
- NumPy 2D array
|
||||
- SciPy 2D sparse array, and
|
||||
- XGBoost binary buffer file.
|
||||
|
||||
(See :doc:`/tutorials/input_format` for detailed description of text input format.)
|
||||
|
||||
The data is stored in a :py:class:`DMatrix <xgboost.DMatrix>` object.
|
||||
|
||||
* To load a libsvm text file or a XGBoost binary file into :py:class:`DMatrix <xgboost.DMatrix>`:
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
dtrain = xgb.DMatrix('train.svm.txt')
|
||||
dtest = xgb.DMatrix('test.svm.buffer')
|
||||
|
||||
* To load a CSV file into :py:class:`DMatrix <xgboost.DMatrix>`:
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
# label_column specifies the index of the column containing the true label
|
||||
dtrain = xgb.DMatrix('train.csv?format=csv&label_column=0')
|
||||
dtest = xgb.DMatrix('test.csv?format=csv&label_column=0')
|
||||
|
||||
(Note that XGBoost does not support categorical features; if your data contains
|
||||
categorical features, load it as a NumPy array first and then perform
|
||||
`one-hot encoding <http://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.OneHotEncoder.html>`_.)
|
||||
|
||||
* To load a NumPy array into :py:class:`DMatrix <xgboost.DMatrix>`:
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
data = np.random.rand(5, 10) # 5 entities, each contains 10 features
|
||||
label = np.random.randint(2, size=5) # binary target
|
||||
dtrain = xgb.DMatrix(data, label=label)
|
||||
|
||||
* To load a :py:mod:`scipy.sparse` array into :py:class:`DMatrix <xgboost.DMatrix>`:
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
csr = scipy.sparse.csr_matrix((dat, (row, col)))
|
||||
dtrain = xgb.DMatrix(csr)
|
||||
|
||||
* Saving :py:class:`DMatrix <xgboost.DMatrix>` into a XGBoost binary file will make loading faster:
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
dtrain = xgb.DMatrix('train.svm.txt')
|
||||
dtrain.save_binary('train.buffer')
|
||||
|
||||
* Missing values can be replaced by a default value in the :py:class:`DMatrix <xgboost.DMatrix>` constructor:
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
dtrain = xgb.DMatrix(data, label=label, missing=-999.0)
|
||||
|
||||
* Weights can be set when needed:
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
w = np.random.rand(5, 1)
|
||||
dtrain = xgb.DMatrix(data, label=label, missing=-999.0, weight=w)
|
||||
|
||||
Setting Parameters
|
||||
------------------
|
||||
XGBoost can use either a list of pairs or a dictionary to set :doc:`parameters </parameter>`. For instance:
|
||||
|
||||
* Booster parameters
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
param = {'max_depth': 2, 'eta': 1, 'silent': 1, 'objective': 'binary:logistic'}
|
||||
param['nthread'] = 4
|
||||
param['eval_metric'] = 'auc'
|
||||
|
||||
* You can also specify multiple eval metrics:
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
param['eval_metric'] = ['auc', 'ams@0']
|
||||
|
||||
# alternatively:
|
||||
# plst = param.items()
|
||||
# plst += [('eval_metric', 'ams@0')]
|
||||
|
||||
* Specify validations set to watch performance
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
evallist = [(dtest, 'eval'), (dtrain, 'train')]
|
||||
|
||||
Training
|
||||
--------
|
||||
|
||||
Training a model requires a parameter list and data set.
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
num_round = 10
|
||||
bst = xgb.train(param, dtrain, num_round, evallist)
|
||||
|
||||
After training, the model can be saved.
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
bst.save_model('0001.model')
|
||||
|
||||
The model and its feature map can also be dumped to a text file.
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
# dump model
|
||||
bst.dump_model('dump.raw.txt')
|
||||
# dump model with feature map
|
||||
bst.dump_model('dump.raw.txt', 'featmap.txt')
|
||||
|
||||
A saved model can be loaded as follows:
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
bst = xgb.Booster({'nthread': 4}) # init model
|
||||
bst.load_model('model.bin') # load data
|
||||
|
||||
Early Stopping
|
||||
--------------
|
||||
If you have a validation set, you can use early stopping to find the optimal number of boosting rounds.
|
||||
Early stopping requires at least one set in ``evals``. If there's more than one, it will use the last.
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
train(..., evals=evals, early_stopping_rounds=10)
|
||||
|
||||
The model will train until the validation score stops improving. Validation error needs to decrease at least every ``early_stopping_rounds`` to continue training.
|
||||
|
||||
If early stopping occurs, the model will have three additional fields: ``bst.best_score``, ``bst.best_iteration`` and ``bst.best_ntree_limit``. Note that :py:meth:`xgboost.train` will return a model from the last iteration, not the best one.
|
||||
|
||||
This works with both metrics to minimize (RMSE, log loss, etc.) and to maximize (MAP, NDCG, AUC). Note that if you specify more than one evaluation metric the last one in ``param['eval_metric']`` is used for early stopping.
|
||||
|
||||
Prediction
|
||||
----------
|
||||
A model that has been trained or loaded can perform predictions on data sets.
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
# 7 entities, each contains 10 features
|
||||
data = np.random.rand(7, 10)
|
||||
dtest = xgb.DMatrix(data)
|
||||
ypred = bst.predict(dtest)
|
||||
|
||||
If early stopping is enabled during training, you can get predictions from the best iteration with ``bst.best_ntree_limit``:
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
ypred = bst.predict(dtest, ntree_limit=bst.best_ntree_limit)
|
||||
|
||||
Plotting
|
||||
--------
|
||||
|
||||
You can use plotting module to plot importance and output tree.
|
||||
|
||||
To plot importance, use :py:meth:`xgboost.plot_importance`. This function requires ``matplotlib`` to be installed.
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
xgb.plot_importance(bst)
|
||||
|
||||
To plot the output tree via ``matplotlib``, use :py:meth:`xgboost.plot_tree`, specifying the ordinal number of the target tree. This function requires ``graphviz`` and ``matplotlib``.
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
xgb.plot_tree(bst, num_trees=2)
|
||||
|
||||
When you use ``IPython``, you can use the :py:meth:`xgboost.to_graphviz` function, which converts the target tree to a ``graphviz`` instance. The ``graphviz`` instance is automatically rendered in ``IPython``.
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
xgb.to_graphviz(bst, num_trees=2)
|
||||
|
||||
Reference in New Issue
Block a user