Improve z3 trace management

[tjruwase]

• Trace cache invalidation when needed
• Separate nvme prefetch from all-gather prefetch
• Handle back-to-back execution of a submodule

[ghost]

Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.

❌ tjruwase sign now
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[jeffra]

Excellent, thanks @tjruwase this is really great.

[stas00]

please don't merge it yet, as while the functionality is correct, the performance regression hasn't been validated yet.

[tjruwase]

Comparing to v0.6.0, I don't see any performance degradation on 2 nodes of A100-40GB (16 GPUs). I think this is fine to merge.

@stas00, @jeffra, @SeanNaren