Return to Answer

1. The entropy rate will not match the output rate, i.e. not every bit generated will have a bit of entropy.
2. The distribution may not be as close to uniform as expected.
3. The source may have hidden bias; TRNG's are next to impossible to certify for that reason.
4. It may be hard to validate that the TRNG is implemented as expected by the user, especially if certification bodies are not trusted by them.
5. And even if it is, errors within the TRNG may be hard to detect (though they may go through self-tests on startup).
6. A TRNG is a single source, which may require system access or system calls.
7. A TRNG may block or have latency and other performance-related issues if not enough entropy is available when called upon.
8. TRNG's may have other properties that are considered detrimental such as slow startup times (warm up times, self-tests etc.).
9. The TRNG or the communication with the TRNG may leak information to a listener or eavesdropper (more an issue if it is an external device, of course).
10. The TRNG may be malleable so that it doesn't produce valid random values anymore (again, mainly an issue if it is an external device and/or is dependent on external input).

1. The entropy rate will not match the output rate, i.e. not every bit generated will have a bit of entropy.
2. The distribution may not be as close to uniform as expected / the source may have hidden bias.
3. It may be hard to validate that the TRNG is implemented as expected by the user, especially if certification bodies are not trusted by them.
4. And even if it is, errors within the TRNG may be hard to detect (though they may go through self-tests on startup).
5. A TRNG is a single source, which may require system access or system calls.
6. A TRNG may block or have latency and other performance-related issues if not enough entropy is available when called upon.
7. TRNG's may have other properties that are considered detrimental such as slow startup times (warm up times, self-tests etc.).
8. The TRNG or the communication with the TRNG may leak information to a listener or eavesdropper (more an issue if it is an external device, of course).
9. The TRNG may be malleable so that it doesn't produce valid random values anymore (again, mainly an issue if it is an external device and/or is dependent on external input).

TL;DR: It is mainly used as an entropy source for Deterministic Random Bit Generators (DRBG's), used for seeding and – preferably – reseeding.

1. The entropy rate will not match the output rate, i.e. not every bit generated will have a bit of entropy.
2. The distribution may not be as close to uniform as expected.
3. The source may have hidden bias; TRNG's are next to impossible to certify for that reason.
4. It may be hard to validate that the TRNG is implemented as expected by the user, especially if certification bodies are not trusted by them.
5. And even if it is, errors within the TRNG may be hard to detect (though they may go through self-tests on startup).
6. A TRNG is a single source, which may require system access or system calls.
7. A TRNG may block or have latency and other performance-related issues if not enough entropy is available when called upon.
8. TRNG's may have other properties that are considered detrimental such as slow startup times (warm up times, self-tests etc.).
9. The TRNG or the communication with the TRNG may leak information to a listener or eavesdropper (more an issue if it is an external device, of course).
10. The TRNG may be malleable so that it doesn't produce valid random values anymore.

Note that the use of the term DRBG here is deliberate; it's more the definition that NIST uses, which includes aspects such as reseeding, i.e. the mixing in of additional entropy within the state of the algorithm. Those aspects are not always included when the term (Cryptographically Secure) Pseudo-Random Number Generator (CS)PRNG is used. This doesn't mean that they are the only source, nor that only NIST algorithms should be used; the designs of DRBG's in operating systems are generally more complex but of course do include reseeding capabilities.

TL;DR: TNRG's are mainly used as an entropy source for Deterministic Random Bit Generators (DRBG's), i.e. they are used for seeding and – where implemented – reseeding DRBG's. This kind of use is reflected in the design within Intel-based RDRAND (DRBG) and RDSEED (TRNG) instructions.

1. The entropy rate will not match the output rate, i.e. not every bit generated will have a bit of entropy.
2. The distribution may not be as close to uniform as expected.
3. The source may have hidden bias; TRNG's are next to impossible to certify for that reason.
4. It may be hard to validate that the TRNG is implemented as expected by the user, especially if certification bodies are not trusted by them.
5. And even if it is, errors within the TRNG may be hard to detect (though they may go through self-tests on startup).
6. A TRNG is a single source, which may require system access or system calls.
7. A TRNG may block or have latency and other performance-related issues if not enough entropy is available when called upon.
8. TRNG's may have other properties that are considered detrimental such as slow startup times (warm up times, self-tests etc.).
9. The TRNG or the communication with the TRNG may leak information to a listener or eavesdropper (more an issue if it is an external device, of course).
10. The TRNG may be malleable so that it doesn't produce valid random values anymore (again, mainly an issue if it is an external device and/or is dependent on external input).

Note that the use of the term DRBG here is deliberate; it's more the definition that NIST uses, which includes aspects such as reseeding, i.e. the mixing in of additional entropy within the state of the algorithm. Those aspects are not always included when the term (Cryptographically Secure) Pseudo-Random Number Generator (CS)PRNG is used. 

This doesn't mean that only NIST algorithms should be used; the answer just borrows the term. The designs of random number generators in operating systems for instance are often using the same techniques without directly using the NIST specified algorithms. Their design can be complex due to the different entropy sources used for seeding.

1. The entropy rate will not match the output rate, i.e. not every bit generated will have a bit of entropy.
2. The distribution may not be as close to uniform as expected.
3. The source may have hidden bias; TRNG's are next to impossible to certify for that reason.
4. It may be hard to validate that the TRNG is implemented as expected by the user, especially if certification bodies are not trusted by them.
5. And even if it is, errors within the TRNG may be hard to detect (though they may go through self-tests on startup).
6. A TRNG is a single source, which may require system access or system calls.
7. A TRNG may block if not enough entropy is available.
8. TRNG's may have other properties that are considered detrimental such as slow startup times.
9. The TRNG or the communication with the TRNG may leak information to a listener or eavesdropper.
10. The TRNG may be malleable so that it doesn't produce valid random values anymore.

1. The entropy rate will not match the output rate, i.e. not every bit generated will have a bit of entropy.
2. The distribution may not be as close to uniform as expected.
3. The source may have hidden bias; TRNG's are next to impossible to certify for that reason.
4. It may be hard to validate that the TRNG is implemented as expected by the user, especially if certification bodies are not trusted by them.
5. And even if it is, errors within the TRNG may be hard to detect (though they may go through self-tests on startup).
6. A TRNG is a single source, which may require system access or system calls.
7. A TRNG may block or have latency and other performance-related issues if not enough entropy is available when called upon.
8. TRNG's may have other properties that are considered detrimental such as slow startup times (warm up times, self-tests etc.).
9. The TRNG or the communication with the TRNG may leak information to a listener or eavesdropper (more an issue if it is an external device, of course).
10. The TRNG may be malleable so that it doesn't produce valid random values anymore.