1
$\begingroup$

I want to use mathematica to symbolically solve the minimization problem in simple linear regression:

$${\text{Find }}\text{arg}\min _{\alpha ,\,\beta }Q(\alpha ,\beta ),\qquad {\text{for }}Q(\alpha ,\beta ) =\sum _{i=1}^{n}(y_{i}-\alpha -\beta x_{i})^{2},$$

where $y_i$, $x_i$, and $n$ are symbolic but not specific numbers.The expected answer would be something like

$$\begin{align} \hat {\beta }&={\frac {\sum _{i=1}^{n}(x_{i}-{\bar {x}})(y_{i}-{\bar {y}})}{\sum _{i=1}^{n}(x_{i}-{\bar {x}})^{2}}},\\ {\hat {\alpha }}&={\bar {y}}-{\hat {\beta }}\,{\bar {x}} \end{align}$$

Can someone show some code example for doing this? Thanks in advance!

$\endgroup$
4
  • $\begingroup$ Thanks Michael for your suggestions! $\endgroup$ Commented Jul 9, 2016 at 21:49
  • $\begingroup$ See (107426). $\endgroup$ Commented Jul 9, 2016 at 21:54
  • $\begingroup$ What have you done so far? People are more likely to help if you show effort. $\endgroup$ Commented Jul 9, 2016 at 21:56
  • 1
    $\begingroup$ You're welcome. Maybe someone else can help with your question. My experience has been that Mathematica's facility with algebraic/symbolic sums is limited. $\endgroup$ Commented Jul 9, 2016 at 21:56

1 Answer 1

4
$\begingroup$

The following is an adaptation of my answer to this question, which focused on the 3D linear least-squares problem.

(* Rules to get constants out of sums (or integrals etc) *) outrules = { Sum[f_ + g_, it : {x_Symbol, __}] :> Sum[f, it] + Sum[g, it], Sum[c_ f_, it : {x_Symbol, __}] :> c Sum[f, it] /; FreeQ[c, x], Sum[c_, it : {x_Symbol, __}] :> c Sum[1, it] /; FreeQ[c, x] }; (*Generate the sum of squares*) Sum[Expand[(y[i] - a x[i] - b)^2], {i, 1, n}]; (*Calculate the derivatives*) Grad[%, {a, b}]; (*Use the linearity property of sums*) Distribute /@ %; (*Pull out any constants from summations*) % //. outrules; (*Set the derivatives equal to zero to generate a system of equations*) Simplify[Thread[% == 0]]; (*Solve for the a, b parameters*) Solve[%, {a, b}] // FullSimplify 

Mathematica graphics

These sums are not yet expressed as a function of $(\bar{x},\bar{y})$, i.e. the average values of the $(x_i,y_i)$, respectively, as you have them in your question, but that should be a question of some algebraic transformations. Some such transformations are reported e.g. in this MathWorld document of least-squares fitting.

$\endgroup$

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.