Implementation of Learnable Fourier Features for Multi-Dimensional Spatial Positional Encoding by Li, Si, Li, Hsieh and Bengio.
pip install learnable_fourier_positional_encodingimport torch from learnable_fourier_positional_encoding import LearnableFourierPositionalEncoding G = 3 M = 17 x = torch.randn((97, G, M)) enc = LearnableFourierPositionalEncoding(G, M, 768, 32, 768, 10) pex = enc(x) print(pex.shape)