Question:
Is anyone aware of a publicly accessible python package for large scale SVMs? Thanks!
Why the Question Ought to be Answerable:
As has been noted (e.g., here), the SVM problem can be computationally intensive with a large dataset. However, researches have offered plenty of work-arounds. For example,
- The Wikipedia page on SVM's links the paper A Dual Coordinate Descent Method for Large-scale Linear SVM which looks like it should be very effective.
- The paper Using the Nyström Method to Speed Up Kernel Machines offers a way to compute the kernel matrix in less than cubic time (though, by my estimate, the cost of their method is still quadratic, not linear in $n$ as the paper claims...)
- A simple Google search for "large scale SVM" returns the more recent paper Recipe for Fast Large-scale SVM Training: Polishing, Parallelism, and more RAM! along with many others (another good Google query is "large scale quadratic program").
As these papers allude to, SVM's can certainly be made to work on large data sets. In my opinion, saying otherwise would be like saying "neural networks don't work on large datasets because full batch gradient descent isn't computationally feasible." Rather, plenty of work-arounds have been proposed for both large scale NN's and large scale SVM's.
It appears that open-source ML libraries implement many such work-arounds for NN's (e.g., you can do mini-batch SGD using pytorch), but lack such implementations for SVM's (e.g., I'm not aware of any pytorch implementation of the coordinate descent algorithm described in the 2008 paper linked above). This really feels to me like a "missing feature" in the major machine learning libraries. Hence my question. Thanks, again!