Is there a way to run NFSv4 with RDMA?
yes, I run rdma over mellanox infiniband in rhel-8.10 for nfs vers=4.2. The first thing is the network hardware needs to support rdma - if I try to do rdma over my 1gbps intel NIC will get that or some similar error.
If you are sure your network hardware should do rdma then:
in /etc/nfs.conf you will see rdma=n
[nfsd] ... # rdma=n # rdma-port=20049 under the [nfsd]nfsd section;section by default it is off. You need to change that to y and then either reboot or restart all the necessary nfs services so it is in affect.
RDMA in rhel-7.x was slightly problematic, it seems to have been considerably cleaned up and streamlined in rhel-8 {and later}. My experience is everything including rdma works best if your /etc/nfs.conf and /etc/nfsmount.conf is the redhat (or rocky) default which is basically everything commented out; and then simply do rdma=y. When things start changing from default is when problems arise... the server's and client's conf files have to be in sync (i.e. port numbers and protocols) for things to work and it's easy to miss one little thing.
RDMA bypasses the firewall so you should not need to open whatever port in the firewall or disable the firewall (when in doubt easiest troubleshooting if need be is to disable the firewall or course)
make sure no explicit ports or happening for lockd, statd, and so on; just let everything happen over 2049/tcp (which needs to be open in the firewall)
A simple mount -o rdma ip_address:/foldername /foldername is all that's needed in rhel/rocky 8 and later.
your rocky 9.5 is the same as rhel-9.5