IEEE Escalations in Customer Support

The Problem

The hardest part of predicting escalation isn't the text — it's the temporal structure. Given a customer support case with metadata, status history, milestone logs, and comment threads, I needed to predict whether and when the case would escalate next. The catch: the dataset gives you the full timeline of each case, but you can only use information available before the escalation moment. You have to construct your own training labels.

This is a subtle but critical design challenge. If you naively use all available features, you're leaking future information — the model sees events that happened after the escalation decision and learns patterns that don't exist at prediction time. The entire pipeline had to be built around a temporal cut-point: for each case, determine the moment when a prediction would be made, then use only information from before that moment.

An additional complexity: the text was anonymized with ID tokens, requiring a lemma mapping file to decode back to real words. Raw TF-IDF on anonymized IDs would be meaningless. And 13 categorical variables each had hundreds of levels — site, company, product, user group — making one-hot encoding impractical and label encoding uninformative.

My Approach

I built the entire pipeline around a temporal cut-point sampling strategy. For cases that actually escalated, the cut-point was natural: just before the escalation event. For non-escalated cases, I needed to simulate realistic observation windows. The solution was to sample random cut-points using the same distribution as real escalation timing. This way, the model sees realistic observation windows for both classes — not artificially early or late ones that would bias predictions.

For the anonymized text, I took a two-stage approach rather than dumping thousands of sparse TF-IDF columns into the main model. First, I decoded the text using the lemma mapping. Then I built a separate NLP sub-model (TF-IDF fed into XGBoost) that predicted escalation from text alone. The key innovation: I used the probability output of this NLP model as a single feature in the final model. This is model stacking at the feature level — the NLP model's confidence becomes a structured signal that the main model can weigh against other evidence.

For the 13 high-cardinality categoricals, I used target encoding: each category becomes its historical non-escalation rate, computed only on training data to avoid leakage. This collapses hundreds of levels into a single informative numeric feature per variable while preserving the signal about which categories are escalation-prone.

The final ensemble blended CatBoost (55%) and XGBoost (45%), both trained on the same feature set augmented with the NLP probability score and POS/NER dictionary features that captured whether case discussions focused on entities versus actions.

Key Decisions

Key Takeaway

This project taught me that in temporal prediction problems, the label construction strategy matters more than the model architecture. How you define "when would a prediction be made?" determines what the model learns. Get this wrong, and you build a model that performs perfectly on historical data but fails in production because it relied on information that wouldn't exist at prediction time. The temporal cut-point sampling was the single highest-impact decision in the entire pipeline.