8
$\begingroup$

I'm thinking of starting a reading group on category theory. The members (myself included) will likely be people trained in natural sciences rather than mathematics, and will probably all have backgrounds in information theory and statistics. Not necessarily measure theory but graphical models, Markov processes, machine learning, that sort of thing.

Because of this, I'm wondering if there is an introductory book or paper that draws some of its examples from these fields. In all of the introductory texts I found so far, including those aimed at scientists, probability seems to be treated as something of an advanced topic, with the result that we can't easily use it to prime our intuitions early on in the way that I would like.

On the other hand, I know that there are some interesting and useful applications of category theory to probability, both in the form of classic works (the Giry monad etc.), and the more recent stuff from John Baez' group, which is what I really want us to learn about. The issue is just that this stuff isn't very accessible to a beginner, so you have to go on quite a long journey to learn the relevant concepts in some other context, before you can have a chance of understanding it.

Broadly speaking, we'd be aiming towards the topics that fall under "applied category theory" (i.e. monoidal categories and their applications), though we may find we want to spend some time on the basics first.

To illustrate what I mean, here are some of the more applied introductions to categories that I know about:

  • Fong & Spivak - Seven sketches in compositionality: doesn't cover probability at all.

  • Spivak - Category theory for scientists: covers probability only in a short section in chapter 5, and doesn't develop it much further than the definition.

  • Baez & Stay - Physics, Topology, Logic and Computation: A Rosetta Stone: doesn't cover probability at all.

  • Coecke & Paquette - Categories for the practicing physicist: it's concerned largely with Hilbert spaces but spends little time on their relationship to probability, and doesn't mention classical probability at all.

Ideally, I'm looking for something along the lines of these works, but with more of an emphasis on probability, especially from the perspective of Bayesian networks, machine learning, etc., if it exists. Otherwise, any introductory text that has at least some examples from these fields would be very useful!

We may also consider tackling one of the classic mathematical textbooks (e.g. Mac Lane, Lawvere etc.), but these also tend not to mention probability. If there is something along those lines that does, that would be useful too.

$\endgroup$
4
  • $\begingroup$ mathoverflow.net/a/20828/148687 $\endgroup$ Commented Mar 3, 2020 at 16:28
  • $\begingroup$ @user658409 I'm familiar with that question and answer, but it doesn't really address the point. We are not pure mathematicians looking for a structuralist definition of probability, we are applied scientists with probability backgrounds looking to get into category theory. The idea is to use our shared background to motivate topics in category theory, not the other way around. $\endgroup$ Commented Mar 3, 2020 at 16:49
  • 3
    $\begingroup$ Paolo Perrone's "Notes on Category Theory with examples from basic mathematics " (available on ArXiv) contain a few examples from probability theory. $\endgroup$ Commented Mar 3, 2020 at 17:24
  • $\begingroup$ @ArnaudD. thank you, that's very helpful! $\endgroup$ Commented Mar 3, 2020 at 18:17

2 Answers 2

Reset to default
3
$\begingroup$

I'm pretty sure no such introduction exists. Probability theory really didn't get any categorical treatment at all, to the best of my knowledge, until people started working on this stuff about the Giry monad. It is questionable whether anyone has given a really convincing categorical presentation of probabilistic or machine-learning topics, as probability theory is somehow not really about morphisms, for instance of measure spaces.

Your best bet is probably to read an introduction aimed at applied scientists, such as the ones you name, perhaps augmented by a higher-level introduction; for instance Awodey's is at least aimed at computer scientists. With some of the basics introduced you can look at papers on the Giry monad or on categorical approaches to machine learning.

$\endgroup$
3
  • $\begingroup$ There is some pretty good stuff on Bayesian networks and Markov processes from John Baez' group, which I alluded to in the question. The field definitely does exist, it's just a question of the most efficient way to get into it, given that we have the appropriate domain knowledge but not the category theory background. $\endgroup$ Commented Mar 3, 2020 at 17:18
  • $\begingroup$ Thanks for that second link, it looks quite useful. $\endgroup$ Commented Mar 3, 2020 at 17:24
  • $\begingroup$ @Nathaniel Yes, I definitely agree the field exists. It's just an active research area, not something people have used so far to introduce categories. $\endgroup$ Commented Mar 3, 2020 at 18:12
3
$\begingroup$

I'm posting a self-answer because I've found what looks like a really excellent resource:

Tobias Fritz (2019). A synthetic approach to Markov kernels, conditional independence and theorems on sufficient statistics. arXiv:1908.07021

At 98 pages this is kind of book-length. It isn't an introduction to category theory - we will have to be comfortable with symmetric monoidal categories and string diagrams first, but there are plenty of introductory papers and I understand those things well enough to teach them to my colleagues. Fritz' paper is a perfect next step after that because it takes a pretty gentle approach, mostly not relying on heavy category-theoretical machinery where it isn't necessary. For example, he tells us that $\mathbf{Stoch}$ is the Kleisli category of the Giry monad, but then constructs it in detail as a concrete category as well, so that monad theory isn't a prerequisite for understanding it.

It also nicely demonstrates the power of a category-theoretic approach, by proving a theorem early on that shows how all the results about random variables can automatically be converted to results about Markov processes. From a complex systems perspective this is solid gold.

I haven't read much beyond that point, but those two points have convinced me that this is the one to go through in detail.

$\endgroup$
2
  • $\begingroup$ That's exciting! Please update if you find anything on ML/AI. $\endgroup$ Commented Sep 27, 2020 at 6:32
  • $\begingroup$ @user3146 you might enjoy "backprop as functor", arxiv.org/abs/1711.10455 $\endgroup$ Commented Sep 28, 2020 at 0:13

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.