A standard deep neural network (DNN) is, technically speaking, parametric since it has a fixed number of parameters. However, most DNNs have so many parameters that they could be interpreted as nonparametric; it has been proven that in the limit of infinite width, a deep neural network can be seen as a Gaussian process (GP), which is a nonparametric model (Lee, 2018).

Nevertheless, let's strictly interpret DNNs as parametric for the rest of this answer.

Some examples of parametric deep learning models are:

• Deep autoregressive network (DARN)
• Sigmoid belief network (SBN)
• Recurrent neural network (RNN), Pixel CNN/RNN
• Variational autoencoder (VAE), other deep latent Gaussian models e.g. DRAW

Some examples of nonparametric deep learning models are:

• Deep Gaussian process (GPs)
• Recurrent GP
• State space GP
• Hierarchical Dirichlet process
• Cascaded Indian Buffet process

Image from Shakir Mohamed's tutorial on deep generative models.

References:

• "Deep Neural Networks as Gaussian Processes" (Lee et al., 2018)

A standard deep neural network (DNN) is, technically speaking, parametric since it has a fixed number of parameters. However, most DNNs have so many parameters that they could be interpreted as nonparametric; it has been proven that in the limit of infinite width, a deep neural network can be seen as a Gaussian process (GP), which is a nonparametric model [Lee et al., 2018].

Nevertheless, let's strictly interpret DNNs as parametric for the rest of this answer.

Some examples of parametric deep learning models are:

• Deep autoregressive network (DARN)
• Sigmoid belief network (SBN)
• Recurrent neural network (RNN), Pixel CNN/RNN
• Variational autoencoder (VAE), other deep latent Gaussian models e.g. DRAW

Some examples of nonparametric deep learning models are:

• Deep Gaussian process (GPs)
• Recurrent GP
• State space GP
• Hierarchical Dirichlet process
• Cascaded Indian Buffet process

Image from Shakir Mohamed's tutorial on deep generative models.

References:

• "Deep Neural Networks as Gaussian Processes" (Lee et al., 2018)

A standard deep neural network (DNN) is, technically speaking, parametric since it has a fixed number of parameters. However, most DNNs have so many parameters that they could be interpreted as nonparametric; it has been proven that in the limit of infinite width, a deep neural network can be seen as a Gaussian process (GP), which is a nonparametric model (Lee, 2018).

Nevertheless, let's strictly interpret DNNs as parametric for the rest of this answer.

Some examples of parametric deep learning models are:

• Deep autoregressive network (DARN)
• Sigmoid belief network (SBN)
• Recurrent neural network (RNN), Pixel CNN/RNN
• Variational autoencoder (VAE), other deep latent Gaussian models e.g. DRAW

Some examples of nonparametric deep learning models are:

• Deep Gaussian process (GPs)
• Recurrent GP
• State space GP
• Hierarchical Dirichlet process
• Cascaded Indian Buffet process

Image from [Shakir Mohamed's tutorial on deep generative models

References:

• "Deep Neural Networks as Gaussian Processes" (Lee et al., 2018)

A standard deep neural network (DNN) is, technically speaking, parametric since it has a fixed number of parameters. However, most DNNs have so many parameters that they could be interpreted as nonparametric; it has been proven that in the limit of infinite width, a deep neural network can be seen as a Gaussian process (GP), which is a nonparametric model (Lee, 2018).

Nevertheless, let's strictly interpret DNNs as parametric for the rest of this answer.

Some examples of parametric deep learning models are:

• Deep autoregressive network (DARN)
• Sigmoid belief network (SBN)
• Recurrent neural network (RNN), Pixel CNN/RNN
• Variational autoencoder (VAE), other deep latent Gaussian models e.g. DRAW

Some examples of nonparametric deep learning models are:

• Deep Gaussian process (GPs)
• Recurrent GP
• State space GP
• Hierarchical Dirichlet process
• Cascaded Indian Buffet process

Image from Shakir Mohamed's tutorial on deep generative models.

References:

• "Deep Neural Networks as Gaussian Processes" (Lee et al., 2018)