Skip to content

Migrate TargetMeanRegressor to narwhals, add polars support - #1069

Open
solegalli wants to merge 2 commits into
narwhals-migrationfrom
narwhals-target-mean-regressor
Open

solegalli wants to merge 2 commits into
narwhals-migrationfrom
narwhals-target-mean-regressor

Conversation

@solegalli

Copy link
Copy Markdown
Collaborator

Stacked on #1066 (BaseTargetMeanEstimator migration), which is not merged yet: until it is, this diff also shows #1066's changes. The only commit of this PR is the last one.

Summary

  • TargetMeanRegressor no longer imports pandas. fit and predict are typed for any narwhals-supported dataframe, and fit takes the target as a series, numpy array or list. All the computation lives in the base class (Migrate BaseTargetMeanEstimator to narwhals, add polars support #1066); the regressor only adds the binary-target check (type_of_target) and predict/score.
  • Docstring: mention of pandas and polars support, a polars example (run, real output pasted), a typo fix ("should of" -> "should be of").
  • There is no user guide page for this class (the _prediction module is private), so no docs page changed.
  • Tests rewritten to the conventions (make_df, make_series, full-message match=, explicit expected values).

Benchmarks

The regressor's own work is type_of_target(y). I compared it on the native target with type_of_target(np.asarray(y)), and against the whole fit (median of 7 / 5 runs, ms, float target, half numerical and half categorical columns):

backend rows cols type_of_target(native) type_of_target(numpy) fit predict score
pandas 10k 4 0.3 0.1 10.5 6.5 2.1
pandas 100k 20 0.2 0.1 138.3 48.8 47.7
pandas 500k 4 0.6 0.5 129.6 46.8 48.5
pandas 500k 20 0.7 0.6 689.2 214.2 203.5
polars 500k 4 1.5 0.6 49.2 25.7 28.2
polars 1M 20 1.0 1.0 382.0 212.8 220.3
polars 2M 4 3.4 3.2 144.5 82.4 98.1
polars 2M 20 2.3 2.1 714.4 475.3 557.8

The check is under 2% of fit on both backends and converting to numpy first saves at most ~1 ms, so I kept type_of_target on the target as passed (no branch needed).

Behaviour

  • pandas: identical to Migrate BaseTargetMeanEstimator to narwhals, add polars support #1066's head. I recorded predictions, score, encoder_dict_, binner_dict_ and error messages over 160 cases (both strategies, bins 1/3/10, 4 variable selections, series/list/numpy targets, binary target, NaN target, short target, unseen category, NaN in X, not fitted, reordered columns, integer column names, non-dataframe input) before and after: all equal.
  • polars: same values as pandas in every one of those cases, with polars, numpy and list targets.

Tests

  • tests/test_prediction: 122 passed before, 160 passed after, 0 failures.
  • tests/test_selection (imports TargetMeanRegressor): 177 failed / 251 passed before and after, same failing tests (the selection module is not migrated yet).
  • flake8 feature_engine tests clean; mypy feature_engine has the same 2 errors as the base ref (datetime_subtraction.py, log.py).
  • The old test_classifier_numerical_variables checked num_var_B with np.array_equal(...) without assert; the new test asserts it.

Needs decision

  • Mixing backends between X and y fails in fit, in add_target_to_X (shared helper, used by the base class): pandas X with a polars y raises AttributeError: PolarsSeries has not attribute '_broadcast', and polars X with a pandas y raises a narwhals ComputeError about a duplicate 'literal' column. Lists and numpy arrays work with both. Should mixed backends be supported (convert y to numpy when its backend differs from X's), or rejected with a clear error? I did not change the helper or the base here.
  • The df_regression fixture in tests/test_prediction/conftest.py is no longer used (the data now lives in the regressor's test file, as it is used by one file only). I left it to avoid a conflict with the parallel TargetMeanClassifier PR, which edits the same fixture file; it can be removed after that one merges.

solegalli and others added 2 commits September 19, 2026 11:28
Compute the target mean per bin and per category directly instead of
through a Pipeline of discretiser and MeanEncoders, reusing the fitted
discretiser's bin edges and labels. Fit and _predict accept pandas and
polars dataframes, and pandas integer column names now work.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Drop the pandas import and type fit and predict for any narwhals-supported
dataframe, with the target as a series, numpy array or list. Rewrite the
tests to run on pandas and polars, and add a polars example to the docstring.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant