[EM] Compress dense ellpack. (#10821)

This helps reduce the memory copying needed for dense data. In addition, it helps reduce memory usage even if external memory is not used.

- Decouple the number of symbols needed in the compressor with the number of features when the data is dense.
- Remove the fetch call in the `at_end_` iteration.
- Reduce synchronization and kernel launches by using the `uvector` and ctx.
This commit is contained in:
Jiaming Yuan
2024-09-20 18:20:56 +08:00
committed by GitHub
parent d5e1c41b69
commit 24241ed6e3
28 changed files with 485 additions and 285 deletions

View File

@@ -0,0 +1,19 @@
/**
* Copyright 2024, XGBoost Contributors
*/
#include <gtest/gtest.h>
#include "../../../src/common/common.h"
namespace xgboost::common {
TEST(Common, HumanMemUnit) {
auto name = HumanMemUnit(1024 * 1024 * 1024ul);
ASSERT_EQ(name, "1GB");
name = HumanMemUnit(1024 * 1024ul);
ASSERT_EQ(name, "1MB");
name = HumanMemUnit(1024);
ASSERT_EQ(name, "1KB");
name = HumanMemUnit(1);
ASSERT_EQ(name, "1B");
}
} // namespace xgboost::common