Added cudnn_frontend api in caffe to support CUDA11+cuDNN8 #2184
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
- added cudnn-frontend submodule - updated cmake - changed caffe submodule repo target
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit. This suggestion is invalid because no changes were made to the code. Suggestions cannot be applied while the pull request is closed. Suggestions cannot be applied while viewing a subset of changes. Only one suggestion per line can be applied in a batch. Add this suggestion to a batch that can be applied as a single commit. Applying suggestions on deleted lines is not supported. You must change the existing code in this line in order to create a valid suggestion. Outdated suggestions cannot be applied. This suggestion has been applied or marked resolved. Suggestions cannot be applied from pending reviews. Suggestions cannot be applied on multi-line comments. Suggestions cannot be applied while the pull request is queued to merge. Suggestion cannot be applied right now. Please check back later.
I tested this setup with CUDA11.7 + cuDNN8.5 on a GTX1660TI. It runs openpose for human pose extraction normally without the huge GPU memory usage issue. The GPU memory usage is the same as the CUDA10.2+cuDNN7 setup, while the inference speed is about ~1fps faster.
Hope this helps someone who needs to use CUDA11 very badly.
Changelog:
cudnn_frontend.-- added
DUSE_CUDNN_FRONTENDoption. Uses the frontend api instead of the current algorithm wrappercudnnGetConvolutionForwardAlgorithm_v7for cuDNN8.-- added
cudnn_v8_utils.hpp+cudnn_v8_utils.cppfiles for cudnn_frontend api. It currently only supports forwardpass.-- fixed warnings.
-- reduced GPU memory usage by setting CUDNN_STREAMS_PER_GROUP=1
-- added compute capability check in tensor creation to enable tensor core usage in ampere cards.