In PyTorch, we have torch.nn.funtional and torch.nn, and the classes/functions within the former are typically referred to as "functions" while the latter as "modules".
There seems to be a lot of overlap between the two, so I'm wondering what are each used for and what their differences are?
Here are the differences:
torch.nn.functional is the base functional interface (in terms of programming paradigm) to apply PyTorch operators on torch.Tensor.
torch.nn contains the wrapper nn.Module that provide a object-oriented interface to those operators.
So indeed there is a complete overlap, modules are a different way of accessing the operators provided by those functions.
Every single tensor operator in PyTorch is available in the form of a function and its wrapper class. For instance F.conv2d, F.relu, F.dropout, F.batch_norm etc... have corresponding modules nn.Conv2d, nn.ReLU, nn.Dropout, nn.BatchNorm2d, 2d, 3d.
torch.nn.functional is lower level than torch.nn? When should we choose one over the other?
nn.Module are classes that wrap this function, by that I mean they exclusively use a function themselves without introducing additional logic. What you can do with the OOP interface you can do the functional interface since the former is built on top of the latter. In practice nn.Module have the advantage of benefiting from an initializer (as with the OOP paradigm in C++)...__call__ to perform the operation... which will effectively call the underlying functional interface implementation itself, with the arguments defined earlier in the class initialization. I hope this gives you a little better outlook on how this works out ;). If you use the operators several times, it's usually good practice to utilize the class over the function as it is more cumbersome to call.