5
$\begingroup$

This is my question about a very common problem faced while training several data science and AI algorithms, and most importantly while backpropogating errors in neural networks, which is getting trapped in a local minima while descending gradient.

So, according to the discussion under the qn, it is claimed to be off-topic

However, in the defence of my post, I think it is perfectly on-topic in this site, as it asks about a legit problem faced while training neural nets and several other AI algorithms.

So, I am looking forward to what the community thinks regarding this.

$\endgroup$

3 Answers 3

Reset to default
3
$\begingroup$

The question is off-topic, as it's about how to the use of machine learning algorithms. (the other questions on neural nets, their architectures, backpropogation, are also off-topic).

Programming, algorithm, modeling, math, philosophy, and history questions should the off-topic, as they are already on-topic in other SE, such as Stats and Data Science.

Data science and the Stats SE already have a huge overlap (>~80%), and I am worried to have a third SE that also significantly overlaps with them. Personally, it would further demotivate me to write any answer, as it gets tiring to copy-paste content, and updating duplicated answers is a pain.

$\endgroup$
3
  • $\begingroup$ Thanks for clarifying, Frank. So, if questions regarding Gradient Descent and Monte Carlo search which are one of the most imp. algos. of neural nets and in understanding the Alpha Go bot respectively, I don't really get the scope of the site :( $\endgroup$ Commented Aug 6, 2016 at 2:44
  • $\begingroup$ I personally believe that the (entirely healthy) difference in perspectives on this is indicative of a deeper issue in AI: many areas that were historically AI but are now claimed to be 'solved problems' are in fact not. See my answer to the question about OCR on the main site for more on this. One way of clarifying the scope of this site might usefully be any area where heuristics are still required - this would avoid artificial demarcations by technique. $\endgroup$ Commented Aug 6, 2016 at 8:29
  • $\begingroup$ "Programming, algorithm, modeling, math, philosophy, and history questions should the off-topic, as they are already on-topic in other SE" So... what's left for us? I'm yet to to see a question about AI that is NOT on-topic on ANY other SE site. $\endgroup$ Commented Aug 6, 2016 at 19:42
1
$\begingroup$

Escaping local optima is an extremely ubiquitous problem (in case it's unclear - there are vastly more applications than backprop), leading to many open questions (a great deal of metaheuristics research, indisputably part of AI, is concerned with this).

So, it is much more open-ended (and therefore subject to heuristic/AI solutions) than the more pedestrian questions (with procedural anwers) about e.g. backprop that appear to be within the AI SE remit.

Hence, I'd say it is definitely on topic ;-)

$\endgroup$
5
  • 1
    $\begingroup$ I'd say that those are exactly the reasons why it should NOT be here, but rather on Data Science. It's really more of a mathematical/optimization topic than AI. You wouldn't talk about differential equations on Physics.SE either, even though a lot of the subject involves them. $\endgroup$ Commented Aug 5, 2016 at 12:50
  • 1
    $\begingroup$ @InquisitiveLurker - state space search has always been one of the pillars of AI. Optimization needn't be "of a numerical value" nor is it terribly strongly connected to data science as a discipline. $\endgroup$ Commented Aug 5, 2016 at 12:53
  • 1
    $\begingroup$ @InquisitiveLurker please make your opinion an answer, so people can vote. Lets see which way the rest of the community leans $\endgroup$ Commented Aug 5, 2016 at 14:27
  • $\begingroup$ I feel I am still not getting sufficient clarity on what is on-topic here vs. Data Science and Cross Validated SE sites. $\endgroup$ Commented Aug 5, 2016 at 15:08
  • $\begingroup$ @Harsh I've written it now. $\endgroup$ Commented Aug 6, 2016 at 6:30
0
$\begingroup$

Personally, I consider gradient descent something akin to what something like differential equations is to physics - a useful piece of mathematics that has a large array of applications, but not really an AI topic by itself.

When we talk about AI, there are different levels of detail and "technicalness" we can go into and I believe it's necessary to draw the line somewhere.

To illustrate what I mean, let me use the example of self-driving cars:

  • There's the concept of the self-driving car itself
  • The car has some sort of computer-vision system
  • That system might involve a neural network
  • That network needs to be trained somehow - there are different algorithms for that
  • One of the most common ones is backpropagation
  • Backpropagation often uses gradient descent
  • Gradient descent is an optimization algorithm
  • and so on...

We could look at an AI problem at any of those levels. But at some point, it becomes no longer really about AI but rather about mathematics or statistics. And those topics are already covered well by other sites.

Basically, what I believe is that this site should mainly concentrate on the top few lines of that list, and leave the rest to more appropriate venues.

$\endgroup$
2
  • 1
    $\begingroup$ A nice decomposition, but AI is arguably not so 'top down' in practice. At the lower levels, we often lack good heuristics. This then is an AI problem in it's own right. By all means, let's direct people to other sites when the best solution method is well understood, but let's not prevent discussion of a gamut of AI alternatives. Restricting the site to a top level 'systems' view would preclude discussion of some open research issues which might best not be pigeonholed as 'about mathematics or statistics'. $\endgroup$ Commented Aug 6, 2016 at 6:51
  • $\begingroup$ @InquisitiveLurker Since you've explained technologies behind self-driving cars, you may want to answer this question: What technologies are needed for a self-driving car? $\endgroup$ Commented Aug 15, 2016 at 19:34

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.