Equal weighting of low- and high-confidence observations occurs for Huber, Talwar, and
Barya weighting functions when Newton’s method is used to solve robust linear regression
problems. This leads to easy updates and/or downdates of existing matrix factorizations
or easy computation of coefficient matrices in linear systems from previous ones. Thus
Newton’s method based on these functions has been shown to be computationally cheap.
In this paper we show that a combination of Newton’s method and an iterative method
is a promising approach for solving robust linear regression problems. We show that
Newton’s method based on the Talwar function is an active set method. Further we
show that it is possible to obtain improved estimates of the solution vector by combining
a line search method like Newton’s method with an active set method.