I think the distinction becomes clearer when you ask **why** a transformation exists.

If the transformation is creating new information or exposing a signal that wasn’t previously available, I consider that feature engineering. Examples include customer tenure buckets, rolling averages, interaction terms, or extracting meaningful patterns from timestamps.

If the transformation is preparing existing information so a model can consume it, scaling, encoding, imputing, normalizing, I’d classify that as preprocessing.

That said, I don’t think the implementation boundary has to match the conceptual boundary.

In production, I prefer keeping both inside the pipeline whenever possible. A custom transformer can handle feature engineering, while a `ColumnTransformer` manages preprocessing. This keeps training and inference consistent and reduces the risk of data leakage or training-serving skew.

The trade-off is maintainability. I’ve seen pipelines become so deeply nested that understanding the business logic was harder than building the model itself.

My rule of thumb is:

> Separate responsibilities logically, but keep execution reproducible.

Whether a transformation is “feature engineering” or “preprocessing” matters less than ensuring the entire workflow is transparent, testable, and produces the same result every time it’s run.

I’m curious how others handle highly domain-specific features. Do you build them directly into custom pipeline transformers, or do you generate them upstream and treat them as part of the dataset before modeling begins?

I tend to draw the line based on ownership and reusability rather than the complexity of the transformation.

If a transformation requires domain knowledge and creates new business meaning, I consider it feature engineering. Examples would be customer lifetime value calculations, rolling averages, interaction features, or extracting signals from timestamps.

If the transformation is preparing data for a model—scaling, encoding, imputing missing values, etc.—I treat it as preprocessing.

That said, I still prefer keeping both inside the pipeline whenever possible. Not because it’s cleaner, but because it prevents one of the most common production issues: training-serving skew.

A pattern I’ve found useful is:

1. Custom transformers for feature engineering.

2. ColumnTransformer for preprocessing those engineered features.

3. Model as the final pipeline step.

This keeps the entire workflow reproducible while maintaining a logical separation between “creating information” and “preparing information.”

The bigger challenge isn’t where the line is drawn—it’s keeping the pipeline understandable six months later when someone else has to debug it. I’ve seen technically elegant pipelines become nearly impossible to maintain because every business rule was buried inside nested transformers.

Curious how others handle this trade-off. Do you prioritize keeping everything inside a single pipeline, or do you intentionally separate feature generation from model preprocessing for readability and maintenance?