A paper "Accurately computing running variance" at http://www.johndcook.com/standard_deviation.html shows how to compute running mean, variance and standard deviations.

Are there algorithms where the parameters of a linear or logistic regression model can be similarly "dynamically" updated as each new training record is provided?