Is following this approach a good idea for my situation?

For this 50.000 feet view, I would say yes. Splitting up a complex process into several small, independent steps is an excellent, praxis-proven approach for keeping such kind of processes manageable. You can (and should!) write unit tests for each of the individual transformation, and it should be possible for you to run them in isolation.

The main downside I can see is that the code for assembling the pipeline will grow to be very complex

Not if you stay away from putting business logic and flow-of-control into the assembling code. If the only kind of logic there is the feature-flag evaluation, the assembling might become longish, but not very complex.

Don't forget the most important rule for such refactorings: make sure you have enough automated regression tests in place before you start to change anything. Then run these tests frequently, whenever you factored a new transformation out, otherwise you run into trouble! Writing such tests may require some effort up-front, but it is definitely worth it.

Is following this approach a good idea for my situation?

Only having this 50.000 feet view presented, I would say yes. Splitting up a complex process into several small, independent steps with a clear dataflow is an excellent, praxis-proven approach for keeping such kind of processes manageable. You can (and should!) write unit tests for each of the individual transformation, and it should be possible for you to run them in isolation.

The main downside I can see is that the code for assembling the pipeline will grow to be very complex

Not if you stay away from putting business logic and flow-of-control into the assembling code. If the only kind of logic there is the feature-flag evaluation, the assembling might become longish, but not very complex.

Don't forget the most important rule for such refactorings: make sure you have enough automated regression tests in place before you start to change anything. Then run these tests frequently, whenever you factored a new transformation out, otherwise you run into trouble! Writing such tests may require some effort up-front, but it is definitely worth it.

Is following this approach a good idea for my situation?

For this 50.000 feet view, I would say yes. Splitting up a complex process into several small, independent steps is an excellent, praxis-proven approach for keeping such kind of processes manageable.

The main downside I can see is that the code for assembling the pipeline will grow to be very complex

Not if you stay away from putting business logic and flow-of-control the assembling. If the only kind of logic there is the feature-flag evaluation, the assembling might become longish, but not very complex.

Is following this approach a good idea for my situation?

For this 50.000 feet view, I would say yes. Splitting up a complex process into several small, independent steps is an excellent, praxis-proven approach for keeping such kind of processes manageable. You can (and should!) write unit tests for each of the individual transformation, and it should be possible for you to run them in isolation.

The main downside I can see is that the code for assembling the pipeline will grow to be very complex

Not if you stay away from putting business logic and flow-of-control into the assembling code. If the only kind of logic there is the feature-flag evaluation, the assembling might become longish, but not very complex.

Don't forget the most important rule for such refactorings: make sure you have enough automated regression tests in place before you start to change anything. Then run these tests frequently, whenever you factored a new transformation out, otherwise you run into trouble! Writing such tests may require some effort up-front, but it is definitely worth it.