Open
Description
Starting version 3.0, XGBoost computes a smart default for base_score. For example, for the binary:logistic objective, XGBoost assigns the mean label to base_score.
This logic works most of the time, but fails if the training data contains all identical labels.
Reproducer:
import numpy as np
import xgboost as xgb
dtrain = xgb.DMatrix(np.asarray([[1.0], [1.0]]), label=[1.0, 1.0])
dtest = xgb.DMatrix(np.asarray([[1.0]]), label=[1.0])
bst = xgb.train(
{"objective": "binary:logistic"},
dtrain,
1,
evals=[(dtest, "validation")],
)Error:
Traceback (most recent call last):
File "/home/phcho/test.py", line 7, in <module>
bst = xgb.train(
^^^^^^^^^^
File "/home/phcho/miniforge3/lib/python3.12/site-packages/xgboost/core.py", line 729, in inner_f
return func(**kwargs)
^^^^^^^^^^^^^^
File "/home/phcho/miniforge3/lib/python3.12/site-packages/xgboost/training.py", line 183, in train
bst.update(dtrain, iteration=i, fobj=obj)
File "/home/phcho/miniforge3/lib/python3.12/site-packages/xgboost/core.py", line 2246, in update
_check_call(
File "/home/phcho/miniforge3/lib/python3.12/site-packages/xgboost/core.py", line 310, in _check_call
raise XGBoostError(py_str(_LIB.XGBGetLastError()))
xgboost.core.XGBoostError: [10:03:12] /workspace/src/objective/./regression_loss.h:69: Check failed: base_score > 0.0f && base_score < 1.0f: base_score must be in (0,1) for logistic loss, got: 1
Stack trace:
[bt] (0) /home/phcho/miniforge3/lib/python3.12/site-packages/xgboost/lib/libxgboost.so(+0x2a6ecc) [0x7f8df94a6ecc]
[bt] (1) /home/phcho/miniforge3/lib/python3.12/site-packages/xgboost/lib/libxgboost.so(+0xeda139) [0x7f8dfa0da139]
[bt] (2) /home/phcho/miniforge3/lib/python3.12/site-packages/xgboost/lib/libxgboost.so(+0x682723) [0x7f8df9882723]
[bt] (3) /home/phcho/miniforge3/lib/python3.12/site-packages/xgboost/lib/libxgboost.so(+0x682aec) [0x7f8df9882aec]
[bt] (4) /home/phcho/miniforge3/lib/python3.12/site-packages/xgboost/lib/libxgboost.so(+0x68d03b) [0x7f8df988d03b]
[bt] (5) /home/phcho/miniforge3/lib/python3.12/site-packages/xgboost/lib/libxgboost.so(XGBoosterUpdateOneIter+0x77) [0x7f8df93b6fa7]
[bt] (6) /home/phcho/miniforge3/lib/python3.12/lib-dynload/../../libffi.so.8(+0x6a4a) [0x7f8e7f843a4a]
[bt] (7) /home/phcho/miniforge3/lib/python3.12/lib-dynload/../../libffi.so.8(+0x5fea) [0x7f8e7f842fea]
[bt] (8) /home/phcho/miniforge3/lib/python3.12/lib-dynload/_ctypes.cpython-312-x86_64-linux-gnu.so(+0x984d) [0x7f8e7c61a84d]
This error was first reported in optuna/optuna-integration#216
Proposed fix: Do not initialize base_score if the computed initial value is not valid.
Metadata
Metadata
Assignees
Labels
No labels