Transformer related optimization, including BERT, GPT

TensorRT is a C++ library for high performance inference on NVIDIA GPUs and deep learning accelerators.

ONNX Runtime: cross-platform, high performance ML inferencing and training accelerator

Open deep learning compiler stack for cpu, gpu and specialized accelerators

Tensors and Dynamic neural networks in Python with strong GPU acceleration

DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.