Questions tagged [least-squares]
Questions about (linear or nonlinear) least-squares, an estimation method used in statistics, signal processing and elsewhere.
1,899 questions
1 vote
0 answers
38 views
How can I smooth noisy experimental data while preserving the overall shape and monotonic trends of the curve?
I have several sets of experimental data representing borehole diametrical closure versus distance along a borehole. Each dataset shows a general smooth trend (for example, a small increase followed ...
- 11
8 votes
4 answers
2k views
Why is the geometric solution of minimizing error via orthogonality called a "least squares" solution?
I'm trying to understand the connection between the geometric interpretation of solving an inconsistent system $A\mathbf{x} = \mathbf{b}$ and the name "least squares." I understand the ...
- 149
0 votes
1 answer
171 views
How do you compute the solution to least-squares problem if neither $A^TA$ nor $AA^T$ nor $A$ are invertible?
For a least-squares problem find $x$ such that $\|Ax - b\|_2, A \in \mathbb{R}^{m \times n}$ is minimized, the solution of is captured by the pseudo-inverse, $$x = A^\dagger b$$ There exists three ...
- 8,711
6 votes
2 answers
180 views
Best-fit cuboid to 8 points
Given 8 target points $p_0,...,p_7$ in $\mathbb{R}^3$, what is the cuboid (as defined on that wikipedia page-- not necessarily rectangular) whose ordered vertices are as close as possible to the ...
- 1,363
1 vote
1 answer
94 views
Weighted least-squares that minimizes the operator norm
I have a question that is related to weighted least-squares and operator norm. Assume we have a model $y(x) = a_1 \psi_1 + \dots + a_n \psi_n(x)$ and its noisy $N (>n)$meaurements $$ \{ (x_1, y(x_1)...
- 141
2 votes
2 answers
216 views
Robust Method to Fit an Ellipse in $\mathbb{R}^{2}$
Summary I am looking for a convex and robust formulation to fit an ellipse to a set of points. Specifically, can handle an extreme condition number of the Scattering Matrix. Full Question The ...
- 10.6k
0 votes
0 answers
79 views
Is this a legitimate interpretation of OLS
The purpose of this post is to investigate the most natural way to visualize what is happening in the well-known formula: $$ w = (X^T X)^{-1} X^T y $$ given the context of ordinary least squares (OLS) ...
- 131
1 vote
0 answers
103 views
Local minimizers of norm of a linear operator
I'm struggling to understand something about what I saw referred to as the lower norm function and its possible relation to singular values. Let $A\in \mathbb{R}_{m\times n}$ be a matrix with $m\geq n$...
- 8,566
0 votes
1 answer
59 views
When applying least squares to a regression problem, what is the geometric intuition of multiplying each basis vector by every other basis vector?
I am trying to better understand Finite Element methods for PDEs, and hence I am reading Hans Peter Langtangen's book Introduction to Numerical Method for Variational Problems. The book is pretty good,...
- 2,705
0 votes
0 answers
44 views
Covariance function and positive-definiteness of covariance matrix
A.M. Yaglom, in his "Correlation Theory of Stationary and Related Random Functions. Volume I: Basic Results", ISBN: 0-387-96268-9, presents the damped cosine function (equation (2.116) of ...
- 135
1 vote
1 answer
89 views
Relation of singular values of restriction to the spectrum
I've not used singular before, so I hope this question is not silly or trivial. I assume I have a finite nonempty real set $\mathbb{V}\subseteq \mathbb{R}$ and a potential function $V:\mathbb{Z}^2\to \...
- 8,566
1 vote
0 answers
100 views
Minimization of $\|\mathbf{Ax}\|_2$ for constant $\mathbf{x}$
I know that $\|\mathbf{Ax}\|^2_2 \leq \|\mathbf{A}\|^2_{\text{F}} \|\mathbf{x}\|^2_2$. For $\mathbf{x}$ constant, is the problem of minimizing $\|\mathbf{Ax}\|_2$ equivalent to finding the matrix $\...
- 135
0 votes
1 answer
104 views
Fitting general but "smooth" convex shape to points
I am aware that there are algorithms to fit, say, an ellipse to a bunch of given points on a plane. For instance, this SO question has answers which feature both literature on the algorithms and ...
1 vote
1 answer
117 views
Derivation for least squares regression
This is a graph showcasing the lemonade sales as per temperature. Multiple data points: $(30, 90), (35, 100), (37, 110), (42, 125), (50, 140)$, etc. Now, as I have been studying the equation of a line....
- 836
0 votes
1 answer
49 views
Ordered Least Square Estimation problem
Let $X_{1}, X_{2}, \ldots, X_{n}$ be a sample from distribution $F(x; \mu, \sigma)$ where $\mu$ and $\sigma$ are location and scale parameters respectively and let $X_{1:n}, X_{2:n}, \ldots, X_{n:n}$ ...
- 17