For RSA, we need two primes p and q to define N = pq. We will only look how long it takes to generate a prime for p because the process is similar for q.
From my lecture slides, my professor states that the Rabin-Miller prime testing "algorithm can be made deterministic assuming the GRH (!), by running it for all a between 1 and 2(log p)2." Here, a is the arbitrary base. I couldn't find any other course material mentioning GRH, but I assume he's referring to the Generalized Riemann Hypothesis. So, assuming GRH (!) (still not quite sure what that means), we can find a prime p in O(2(log p)2). Obviously, there are enough primes where we can simply guess and check, but I wanted to mention the running time of the Rabin-Miller algorithm mentioned in my course slides.
For DH, we need a prime p and a generator g ∈ (Z/pZ)* (i.e., all of the powers gx mod p give every element in (Z/pZ)*). Finding a discrete logarithm using the Pohlig–Hellman algorithm has a worst-case runtime of O($\sqrt{p}$). Does this imply that generating a p and g will take this long in the worst case as well?
The question specifically states that generating arbitrarily large primes for DH typically takes much longer than for RSA, and I am to verify this claim. Using the worst-case scenarios above, yes, DH takes longer, but the runtimes seem similar when graphing them.