diff --git a/tests/README.md b/tests/README.md
index ee7c8fcc6..2dca93048 100644
--- a/tests/README.md
+++ b/tests/README.md
@@ -1 +1,20 @@
-This folder contains testcases for xgboost.
+This folder contains testcases for XGBoost c++ core, Python package and some other CI
+facilities.
+
+# Directories
+  * ci_build:  Test facilities for Jenkins CI and GitHub action.
+  * cli: Basic test for command line executable `xgboost`.  Most of the other command line
+    specific tests are in Python test `test_cli.py`
+  * cpp: Tests for C++ core, using Google test framework.
+  * python: Tests for Python package, demonstrations and CLI.  For how to setup the
+    dependencies for tests, see conda files in `ci_build`.
+  * python-gpu: Similar to python tests, but for GPU.
+  * travis: CI facilities for travis.
+  * distributed: Legacy tests for distributed system.  Most of the distributed tests are
+    in Python tests using `dask` and jvm package using `spark`.
+  * benchmark: Legacy benchmark code.  There are a number of benchmark projects for
+    XGBoost with much better configurations.
+
+# Others
+  * pytest.ini: Describes the `pytest` marker for python tests, some markers are generated
+    by `conftest.py` file.
\ No newline at end of file