We use hundreds of pisPis in a production environment (as a replacement for thin clients). We've noticed a similar problem. It happens on either ethernet or wireless. After about 5-10 minutes of inactivity a new request to the server can result in a delay anywhere from 30 seconds to 2 minutes.
This only seems to happen on the 4.14 kernel, does not happen on 4.9. Because of this we can't use any of the new 3B+ pisPis yet.