4
$\begingroup$

I'm not sure which word to use to differentiate a self-organizing map (SOM) training procedure in which updates for the entire data set are aggregated before they are applied to the network from a training procedure in which the network is updated with each data point individually.

In the case of other algorithms I'd say stochastic gradient descent, but I'm not sure gradient descent is correct for SOM learning. To my knowledge, SOM learning does not follow any energy function exactly, so that should mean it doesn't do gradient descent exactly, right?

Another term would be online learning, but `online' sort of implies I train my SOM in the real world with data points streaming in, eg. from a set of sensors. It may also imply that I use every data point only once, which is not what I want to say.

Improve this question
$\endgroup$
0

1 Answer 1

Reset to default
1
$\begingroup$

SOMs

Self-organizing maps (SOMs) are one specific type of neural networks. In contrast to multilayer perceptrons (MLPs; they are used much more often) the SOMs neurons have a position on a regular grid.

Training of SOMs

SOMs are usually trained in a stochastic way (source: [1]). This means one training example is used at a time. After every single training example the network learns something.

In contrast, batch learning means the network is presented the complete batch of all training-examples (or a big part of usually 128 examples; then you call it mini-batch). The parameters are only updated after all of the examples were presented to the network.

In contrast to MLPs, you don't apply gradient descent (as far as I know; I've never used them, but only read about SOMs). I wouldn't even know to which function you would apply the gradient descent algorithm, because there is no error function in SOMs. (At least no standard error function).

For MLPs, I know that the advantage of batch training compared to stochastic training is that the weights are not jumping as much. So you rather go in the right direction. However, the advantage of stochastic training is that you make updates much more often. You want to get the best compromise, so you make mini-batch training.

Online Learning

"Online" has quite a lot of different meanings:

  1. "online" as in "in the internet".
  2. "online" as in "online algorithms" which get input piece-by-piece and have to make a decision before knowing all of the input (e.g. when you have severe time constraints or when you can't store all the data like in CERN where you have to throw most of the data away right after you get it)
  3. "online" in handwriting recognition in contrast to the offline OCR methods (see On-line Recognition of Handwritten Mathematical Symbols if you're interested). Online methods use the way how symbols were written, offline methods only use the image.

According to Wikipedia:

Online machine learning is used when data becomes available in a sequential order to determine a mapping from data set corresponding labels. The difference between online learning and batch learning (or "offline" learning) techniques, is that in online learning the mapping is updated after the arrival of every new data point in a scale fashion, whereas batch techniques are used when one has access to the entire training data set at once.

Sources

[1] Andries P. Engelbrecht: Computational Intelligence: An Introduction. Page 62.

Improve this answer
$\endgroup$
1
  • $\begingroup$ The ambiguity you mention is exactly why I was looking for a different word. I was trying to describe a setting in which I had all my robotic data (so, offline), but was updating the network with every data point separately (ie. online). As a side note, I wouldn't say online training of SOMs is more common---it's just conceptually purer or simpler. In fact, according to "Mr. SOM", Teuvo Kohonen himself, batch training in SOMs is faster and safer than online learning[1]. [1] Essentials of the Self-Organizing Map. Neural Networks, Vol. 37 (January 2013), pp. 52-65, by Teuvo K. Kohonen $\endgroup$ Commented Dec 10, 2015 at 12:08

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.