It's known that the RLS filter converges faster than the LMS filter in general, but that if you're tracking time varying parameters the LMS algorithm can perform better.
My question is under what conditions does this hold? I understand that the LMS filter is like a point estimate, but the RLS uses more data - when would using less data be helpful?
Using less data is helpful when, as you said, the parameters are time varying, and in particular when they change a lot.
The key difference is that LMS is a Markov process. It has its current state, but other than that it does not remember data from the past. For time-varying signals this is a feature because past data will give you erroneous information about the current parameters.
The RLS algorithm uses all of the information, past and present, but that can be a problem if the past data is misleading for the current parameters.
If you are looking for a quantitative rule for when to use one or the other, I don't have one. RLS is more computationally intensive than LMS, so if LMS is good enough then that is the safe one to go with. RLS converges faster, but is more computationally intensive and has the time-varying weakness, so I would only use it if the parameters don't vary much and you really needed the fast convergence.
