0
$\begingroup$

I am building a model to predict customer churn using survival analysis, COX regression in particular. While it is pretty straight forward to collect churn events, it is a little tricky to sample for the not churned events. I think it is safe to include the services that were renewed at the end of term as non-event. Due to the nature of our services (subscritpion based), the contract terms are somewhere between 3 and 10 years. So it is not surprising the churn events and renewals usually happen at the end of term.

However there are many more active services with different months in their terms (tenure). Should I consider them as non-event? One of the potential problem is that it will make the dataset extremely imbalanced. The month-in-term of current services is a wide range. Some of them just started a month ago. Maybe I need to selectively chose some of the active services with longer tenure. What would be a good approach of sampling from these current non-event samples?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
2
$\begingroup$

What you have to do for all your different offerings/products(contracts) is decide how long the follow-up period is. You can have dynamic enrollment where customers are always signing up, but in any case, the length of time over which you determine failure (churned) or renewed (not a failure) has to be the same. So maybe the Cox model for the long contract will set fail=1 if customer churned before 10 years after initial signup. The survival time is first contract date to the date churned (since you know they churned within 10 years). The failure date for non-churned is the first contract date to the last known renewal date, not today. But you could assume today for the non-churns, as long as it's less than 10 years.

You're also supposed to have one or more grouping variables for predictors, so maybe it's one price vs. discount or coupon customers (vs. legacy customers).

When done, each customer will have fail=0 or fail=1 (within 10 years) and a survival time. For churned, survival time is time between first contract and when churned. For non-churned, survival time is first contract date to last renewal or today if they are active. Customers that extended their contract for more than 10 years will have their survival time set to 10 years, since you are specifying that the follow-up length for each customer is, for example, limited to 10 years.

When done, you basically have 2 groups, churned, non-churned. Churned get fail=1, and their time is date churned minus first contract date. For the "censored," who didn't churn, their fail=0 and their time is today minus their first contract date. Any times longer than 10 years is truncated (set to) to 10 years. If the program was decades ago, instead of using today's date, you would have to ensure no customer's survival time is greater than 10 years.

Improve this answer
$\endgroup$
6
  • $\begingroup$ question is how to decide how long the follow-up period is, why 10 years? I discovered that different product indeed has different survival rate indicted by K-M curves. Should I use different period window then, e.g. max term length of that particular product? $\endgroup$ Commented Jul 2, 2021 at 1:04
  • $\begingroup$ Yes, could could simply use varying window lengths of time. So the answer would be for a 2-year study (window), we found this, and for a 5-year study (window) we found this. I was pretty rigid about the 10 years, for example, because in medical follow-up studies and clinical trials, you can't be all over the map on windows, as there is only one window. For example, both the Pfizer and Moderna mrna covid19 vaccine trials were 120 days long, and almost everyone was enrolled in the first month. But the point is the study investigators were not all over the map with windows of varying size. $\endgroup$ Commented Jul 2, 2021 at 1:12
  • $\begingroup$ Reviewers who make decisions about funding medical clinical trials like Pfizer and Moderna also expect one time window. You are only doing this since you are data dredging ("fishing expedition"),trying to find something among data for which there was no a priori research design with a fixed time period for each product. This happens all the time, and the data are called "administrative data," i.e. data are just found in a database which wasn't generated for research. $\endgroup$ Commented Jul 2, 2021 at 1:12
  • $\begingroup$ There two scenarios for non-churn events. The ones that got renewal at the end of turn and active services. For latter as you pointed out their time is today minus the first contract date. But the time can be as little as 1 month. If I consider every active services no matter how far they are in term, it will be a lot more non-churn events than churn events probably 20:1 which makes the dataset very imbalanced. Would that be a problem? $\endgroup$ Commented Jul 2, 2021 at 1:32
  • $\begingroup$ So have 2 Cox PH regression runs, one with churn that 20 times greater, and the other with churn that 20 times lower. Either that, or introduce a dummy (0,1) binary variable into the model that will adjust the model results by the two groups. That is, you can control for the two scenarios. Get rid of large heterogeneities in outcome results by breaking data up. A regression model looking at risk can only include records for objects (customers) for which everything is fair, or like-balanced. $\endgroup$ Commented Jul 2, 2021 at 1:59

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.