Feature Engineering
Feature Engineering
What is feature engineering?
- Turning raw data into useful inputs
- 'Good ingredient prep'
- Drives traditional ML performance
What feature engineering really changes is not the model's performance figure but the difficulty of the problem itself. Feed the same algorithm the same table, and the problem the model must solve is entirely different when a column reads 'signed up 2021-03-14' versus '412 days since signup, 0 orders in the last 30 days.' Because it shortens the distance between the answer and the input, building one derived variable well often pays off more than tuning hyperparameters to squeeze out 0.5 percentage points. It also comes first in the sequence: it is work that must be finished before you choose which model to use.
In practice, the work usually falls into four groups: transformations that fix scale (log, normalization); derived features, which combine raw columns into new ones (90-day totals, month-over-month change); encoding, which turns categories into numbers (one-hot, target encoding); and selection, which removes useless columns. For encoding, the number of unique values is the key issue: one-hot encoding a column with thousands of values, such as product codes, makes the column count explode, slows training, and leads to overfitting. For tabular data pulled from internal systems, the claim that 'deep learning learns the features by itself' rarely holds, and boosting models combined with hand-built derived variables are still the default. This is different from domains where the raw data is itself a dense signal, such as pixels or waveforms.
The most common failure in practice is data leakage. Add a 'cancellation reason code' or 'penalty charge amount' to a churn model and validation accuracy jumps to 0.95, but those columns are only filled in after the customer has already canceled, so in production they are always empty. Normalizing with the mean of the entire training data before splitting into training and validation sets is the same kind of contamination. Without a step that checks, column by column, whether the value is actually available at prediction time, a model that looked fine in the pilot falls apart in the first week after deployment. Another trap is believing more is better and pouring in 500 features. Once preprocessing code that can't be reproduced piles up, it becomes a burden no one dares touch the moment the owner changes.
Sungil KimKAIST and Samsung Electronics alumnus and global pitching expert who coached the winning teams at the UN Tourism and Select USA IR pitch competitions
Ara LimExpert in global leadership, AI, and business communicationA core practice of traditional machine learning, rooted in the importance of how data is prepared and represented.
Consider a hypothetical case: Company A, a food subscription business with 210 employees and annual revenue in the KRW 90 billion (≈ US$67,335,000) range. Its monthly cancellation rate had risen from 6.8% to 9.2% and had not come down in six months. The data team ran a churn prediction model twice, but accuracy stayed in the 60% range, and the CEO told them to “try a different model.” The team kept the model as it was and spent eight weeks rebuilding the input data instead.
- Manager Kim laid out all 42 columns of the raw table and began by asking, 'Which of these get filled in after the day of cancellation?' Seven columns, including refund processing date and cancellation reason code, were removed as leakage candidates.
- Marketing Team Lead Lee counted CS support-log inquiries mentioning 'delivery delay' over two windows, the last 30 and 90 days, and designed four new cumulative-complaint features per customer.
- Instead of one-hot encoding the 1,800+ subscription product codes, they grouped them into 12 categories and 5 price bands, cutting the number of input columns from 1,800 to 17.
- After redefining order cycle as 'the current interval as a multiple of the average reorder interval,' the customer group beyond 1.8 times their usual interval showed a cancellation rate three times that of everyone else.
- Just before deployment, Manager Kim checked one by one whether each column is actually populated in the early-morning batch, and removed two more. The team's rule was: 'no columns that only work in the pilot.'
- With the model untouched and only the features changed, churn-prediction accuracy rose from 61% to 78%, and the share of at-risk customers saved by coupons sent to the top 10% rose from 12% to 27%.