What is ControlNet and how is it different?
In terms of “control,” you may recall textual embedding, LoRA, and the image-to-image diffusion pipeline. But what makes ControlNet different and useful?
Unlike other solutions, ControlNet is a model that works on the UNet diffusion process directly. We compare these solutions in Table 13.1:
| Control Method | Functioning Stage | Usage Scenario |
| Textual Embedding | Text encoder | Add a new style, a new concept, or a new face |
| LoRA | Merge LoRA weights to the UNet model (and the CLIP text encoder, optional) | Add a set of styles, concepts, and generate content |
| Image-to-Image | Provide the initial latent image | Fix... |
