Co-authored-by: Bobby Wang <wbo4958@gmail.com>
This commit is contained in:
parent
21d95f3d8f
commit
03f3879b71
@ -107,8 +107,8 @@ virtualenv and pip:
|
|||||||
python -m venv xgboost_env
|
python -m venv xgboost_env
|
||||||
source xgboost_env/bin/activate
|
source xgboost_env/bin/activate
|
||||||
pip install pyarrow pandas venv-pack xgboost
|
pip install pyarrow pandas venv-pack xgboost
|
||||||
# https://rapids.ai/pip.html#install
|
# https://docs.rapids.ai/install#pip-install
|
||||||
pip install cudf-cu11 --extra-index-url=https://pypi.ngc.nvidia.com
|
pip install cudf-cu11 --extra-index-url=https://pypi.nvidia.com
|
||||||
venv-pack -o xgboost_env.tar.gz
|
venv-pack -o xgboost_env.tar.gz
|
||||||
|
|
||||||
With Conda:
|
With Conda:
|
||||||
@ -240,7 +240,7 @@ additional spark configurations and dependencies:
|
|||||||
--master spark://<master-ip>:7077 \
|
--master spark://<master-ip>:7077 \
|
||||||
--conf spark.executor.resource.gpu.amount=1 \
|
--conf spark.executor.resource.gpu.amount=1 \
|
||||||
--conf spark.task.resource.gpu.amount=1 \
|
--conf spark.task.resource.gpu.amount=1 \
|
||||||
--packages com.nvidia:rapids-4-spark_2.12:22.08.0 \
|
--packages com.nvidia:rapids-4-spark_2.12:23.04.0 \
|
||||||
--conf spark.plugins=com.nvidia.spark.SQLPlugin \
|
--conf spark.plugins=com.nvidia.spark.SQLPlugin \
|
||||||
--conf spark.sql.execution.arrow.maxRecordsPerBatch=1000000 \
|
--conf spark.sql.execution.arrow.maxRecordsPerBatch=1000000 \
|
||||||
--archives xgboost_env.tar.gz#environment \
|
--archives xgboost_env.tar.gz#environment \
|
||||||
|
|||||||
Loading…
x
Reference in New Issue
Block a user