6
$\begingroup$

When I was in a university, I didn't major in math but took some math classes. However, I dropped out of math classes pretty quick.

Some person recommended that I learn some set theory because it'll help me with Machine Learning . He recommended a book named Naive Set Theory by Halmos

Other people say it's not going to help much because set theory lives on a far higher abstraction level than mathematics used in machine learning do.

Since I don't know math well, I can't judge who's right.

Can anyone tell me how set theory is going to help me with machine learning ?

$\endgroup$
1
  • $\begingroup$ Others have given good answers to your question but one thing I would like to point out is that Naive Set Theory is a super short book. You could reasonably get through it in a weekend and I am sure that you would gain quite a bit of helpful knowledge. $\endgroup$ Commented Apr 14, 2016 at 1:34

3 Answers 3

Reset to default
5
$\begingroup$

Machine learning, if you really want to understand it rather than just believe the algorithms do what they are supposed to do, requires quite a heavy dose of mathematics. That mathematics is written in a language and that language is set theory a la Halmos (i.e., the naive kind). If you wish to properly understand the mathematics involved in ML, then it is a good idea to have a good grasp of the underlying universal language which is set theory. Halmos' book is quite short and does not go to the very high level of abstraction that set theory proper goes to. If ML is your goal, I'm would not say Halmos is a necessity, but it is certainly a worthwhile reading. Just don't expect it to translate into any algorithm.

$\endgroup$
2
  • 2
    $\begingroup$ If I read halmos' book, will I be able to understand mathematics used in ML and make new ML algorithms? Can you refer me to the list of math fields that I need to learn for understanding ML? $\endgroup$ Commented Apr 15, 2015 at 14:00
  • 1
    $\begingroup$ @Minsky: This is pretty unlikely. At best you will be able to be able to read ML algorithms; but you will need more to understand them, and considerably more to be able to crate new ones. $\endgroup$ Commented Apr 24, 2015 at 12:16
2
$\begingroup$

TL;DR : Set theory is a good thing to know , but there is more math out there which is actually critical for Machine Learning

You should probably learn Math in this order :

1) Pick up this OR this and flirt with the sections on proofs , set theory and its notations , functions , combinatorics , graphs ; till the time you feel comfortable solving a simple non-trivial problem . Do not pick up books like Halmos , please , -- not discrediting the book in any way - just not required here .

2) Learn Single Variable Calculus from here and here . Once you understand the intuition behind it , pick up a standard textbook and solve a lot of problems .

3) Learn a bit of Multi-Variable Calculus from here

4) Learn Linear Algebra from Strang . Ah ! the joys of learning from Strang . Seriously . You can do yourself no further good . Hop around to other MOOCs like this one .

5) Learn Probability and Statistics , IMO the most important math field required for ML . There are already pretty good resources and answers on the net , so will not repeat.

6) Learn how to apply statistics , probability theory to real-world problems intuitively by learning Data Mining and doing competitions on Kaggle

7) Take up Ng's Wonderful Course and embark on a hard yet fun journey of ML

Hope this helps :)

Disclaimer : I am not a ML Researcher , only a CS undergrad who has studied ML and is now working on a ML-based paper under a professor ; so it is possible that the order might vary according to your mileage

$\endgroup$
13
  • 1
    $\begingroup$ Hi @Minsky : (1) I do not want you to avoid Set Theory ; get proficient with it , but you really do not need to go deep into it , which Halmos does . (2) The MIT Math Lectures are pretty nice as well the lectures available here . (3) 3 calculus materials because the first one gives you a taste and the intuition , the second and the third are full blown courses . (4) Not heard of "Linear Algebra Done Right" , but Strang is perfect . Any more doubts , will be happy to answer :) If you like my answer - you can accept it by selecting the tick :) $\endgroup$ Commented Apr 24, 2015 at 9:30
  • 1
    $\begingroup$ Thanks for clarification, @pranav . (1) Do you know a better material than Rosen's? Amazon reviews are very very angry toward Rosen. (2) I am not sure if it's a good approach to learn calculus by MIT video lectures instead of a book. How do you compare videos and books? (3) I thought Strang's MIT video lectures are meant to be accompanied by his book. Am I right? $\endgroup$ Commented Apr 24, 2015 at 12:01
  • 1
    $\begingroup$ @Minsky: (1) The difficulty with replacing Rosen is that it's kind of a one-stop shop for a ton of disparate mathematical disciplines, without assuming any background in any of them. However, as a zeroth-order approximation, I would see what you think of Morash's "Bridge to Abstract Mathematics." It only covers proofs-sets-functions; but you might find it a good pace. (2) I think learning from lectures is a very good idea; it took me an extremely long time to learn math from books. Of course, this depends on your style of learning. $\endgroup$ Commented Apr 24, 2015 at 12:13
  • 1
    $\begingroup$ @pranav I don't necessarily want to read 'how to prove it' from cover to cover unless it's a fast read and if Rosen contains similar topics. $\endgroup$ Commented Apr 24, 2015 at 12:31
  • 1
    $\begingroup$ @pranav It seems to solve my problem for now. I'll come back after reading 'how to solve it' and discrete mathematics. $\endgroup$ Commented Apr 25, 2015 at 0:21
0
$\begingroup$

Set theory is a very deep and philosophical field. Much of set theory also fall in the realm of incomputability. Computer languages are first-order and finite, not even countably infinite. There are only countably infinitely many programs/routines/codes in Python, C++, R, Matlab, Wolfram Mathematica, etc. combined. ML algorithms, implemented on a machine, are therefore only countably infinitely many. Set theory, on the other hand, goes beyond the "inaccessible" (context: inaccessible cardinals). Even reading Halmos leads you to realm way beyond what you would ever need for a great ML research career.

Want to know enough of set theory for a career in ML? Study a Discrete Math textbook deeply, chapters related to Set Theory, Relations, and Functions. That's about all you need to know about set theory and how they are normally used in mainstream mathematics as well as in statistics, probability, and ML. Make sure you do the exercises and solve the problems in the textbook. You will have a solid foundation by then.

$\endgroup$

You must log in to answer this question.