Update Xgboost Benchmarks - #220
Conversation
…save preprocessing args to results
| "tree_method": "hist", | ||
| "n_jobs": "[SPECIAL_VALUE]physical_cpus" | ||
| "n_jobs": "[SPECIAL_VALUE]physical_cpus", | ||
| "enable_categorical": true |
There was a problem hiding this comment.
This is enabled by default in the latest versions. Which xgboost version are you using where it makes an effect?
There was a problem hiding this comment.
We need to specify it to ensure comparison with previous xgboost version is correct
| # Read with the same engine used for writing (fastparquet). Reading a | ||
| # fastparquet-written file with the default "auto"/pyarrow engine drops | ||
| # pandas "category" dtype (string categories come back as object), | ||
| # which would prevent XGBoost's native categorical handling from ever | ||
| # seeing category columns loaded from cache. | ||
| data = pd.read_parquet(filepath, engine="fastparquet") |
There was a problem hiding this comment.
Wouldn't a better fix be to switch both to pyarrow, or to use polars to read it?
There was a problem hiding this comment.
If the changes from the current PR are applied benchmark caches are not invalidated, we only change the code that is responsible for reading. If we switch to another engine then existing caches would be invalidated that might silently affect some CI jobs etc. Probably it worth doing later, I just don't want to deal with these changes in the current PR
| Source: OpenML dataset id 46939 (kddcup09_appetency, small training set). | ||
|
|
||
| Classification task. n_classes = 2. |
There was a problem hiding this comment.
These comments are redundant given the rest of the code below.
There was a problem hiding this comment.
I would say that they are unnecessary, this brief description is really useful when you look at the configs and benchmarking results
There was a problem hiding this comment.
But you have the exact same info in a more succinct form right below it.
| # Remember which columns are categorical (and their exact CategoricalDtype) | ||
| # before the numpy round-trip below flattens the DataFrame and drops this | ||
| # metadata. The categoricals are restored after converting back to pandas, | ||
| # so estimators with native categorical support (e.g. XGBoost with | ||
| # enable_categorical=True) still see "category" columns. | ||
| categorical_dtypes = None | ||
| if ( | ||
| dformat == "pandas" | ||
| and isinstance(data, pd.DataFrame) | ||
| and any(str(t) == "category" for t in data.dtypes) | ||
| ): |
There was a problem hiding this comment.
2 participants
Description
In this PR the new configs for xgboost benchmarks are introduced. Motivation is to make the use cases closer to real-world usage scenarios and cover some features from the recent xgboost versions.
Checklist:
Completeness and readability
Testing