Skip to main content
small fix
Source Link
Sven
  • 1.1k
  • 9
  • 10

Yes you can ("asas long as your weights are integers"integers (fractional to be pedantic)), though it's obviously not very efficient.

To see this, note that most loss functions can be written as $$\text{loss}(y, p) = \sum_{i=1}^n l(y_i, p_i)$$ where $p_i$ is the predicted value of $y_i$ for a suitable function $l$.

We can easily transform this to a weighted loss function by introducing weights: $$\text{weighted loss}(y, p) = \sum_{i=1}^n w_i l(y_i, p_i)$$

Now we see that if we duplicate each observation $i$ $w_i$ times, and minimize the (unweighted) loss, that this is equivalent to minimizing the weighted loss with weights $w_i$. Of course duplicating something $\pi$ times is difficult so make sure your weights are integers.

Note that adding a regularization penalty to the loss function does not have any effects on this reasoning.

Yes you can ("as long as your weights are integers"), though it's obviously not very efficient.

To see this, note that most loss functions can be written as $$\text{loss}(y, p) = \sum_{i=1}^n l(y_i, p_i)$$ where $p_i$ is the predicted value of $y_i$ for a suitable function $l$.

We can easily transform this to a weighted loss function by introducing weights: $$\text{weighted loss}(y, p) = \sum_{i=1}^n w_i l(y_i, p_i)$$

Now we see that if we duplicate each observation $i$ $w_i$ times, and minimize the (unweighted) loss, that this is equivalent to minimizing the weighted loss with weights $w_i$. Of course duplicating something $\pi$ times is difficult so make sure your weights are integers.

Note that adding a regularization penalty to the loss function does not have any effects on this reasoning.

Yes you can (as long as your weights are integers (fractional to be pedantic)), though it's obviously not very efficient.

To see this, note that most loss functions can be written as $$\text{loss}(y, p) = \sum_{i=1}^n l(y_i, p_i)$$ where $p_i$ is the predicted value of $y_i$ for a suitable function $l$.

We can easily transform this to a weighted loss function by introducing weights: $$\text{weighted loss}(y, p) = \sum_{i=1}^n w_i l(y_i, p_i)$$

Now we see that if we duplicate each observation $i$ $w_i$ times, and minimize the (unweighted) loss, that this is equivalent to minimizing the weighted loss with weights $w_i$. Of course duplicating something $\pi$ times is difficult so make sure your weights are integers.

Note that adding a regularization penalty to the loss function does not have any effects on this reasoning.

Source Link
Sven
  • 1.1k
  • 9
  • 10

Yes you can ("as long as your weights are integers"), though it's obviously not very efficient.

To see this, note that most loss functions can be written as $$\text{loss}(y, p) = \sum_{i=1}^n l(y_i, p_i)$$ where $p_i$ is the predicted value of $y_i$ for a suitable function $l$.

We can easily transform this to a weighted loss function by introducing weights: $$\text{weighted loss}(y, p) = \sum_{i=1}^n w_i l(y_i, p_i)$$

Now we see that if we duplicate each observation $i$ $w_i$ times, and minimize the (unweighted) loss, that this is equivalent to minimizing the weighted loss with weights $w_i$. Of course duplicating something $\pi$ times is difficult so make sure your weights are integers.

Note that adding a regularization penalty to the loss function does not have any effects on this reasoning.