本页介绍了如何使用 Firebase AI Logic SDK 通过 Imagen 使用图像修复功能从图片中移除对象。
修图是一种基于蒙版的修改。蒙版是一种数字叠加层,用于定义您要修改的特定区域。
运作方式:您提供一张原始图片和一张相应的蒙版图片(自动生成或由您提供),该图片定义了要移除的对象或主题的蒙版。您还可以选择提供一段文本提示,用于描述要移除的内容,或者让模型智能检测要移除的对象。然后,模型会移除对象,并使用新的、符合上下文的内容填充该区域。
例如,您可以遮盖一个球,并将其替换为空白墙壁或草地。
准备工作
| 仅在将 Vertex AI Gemini API 用作 API 提供方时可用。 |
如果您尚未完成入门指南,请先完成该指南。该指南介绍了如何设置 Firebase 项目、将应用连接到 Firebase、添加 SDK、为所选的 API 提供方初始化后端服务,以及创建 ImagenModel 实例。
支持此功能的模型
Imagen 通过其 capability 模型提供图片编辑功能:
imagen-3.0-capability-001
请注意,对于 Imagen 模型,不支持 global 位置。
使用自动生成的蒙版移除对象
| 在试用此示例之前,请完成本指南的准备工作部分,以设置您的项目和应用。 |
以下示例展示了如何使用修复功能从图片中移除内容,并使用自动蒙版生成功能。 您提供原始图片和文本提示,Imagen 会自动检测并创建蒙版区域来修改原始图片。
Swift
Swift 不支持使用 Imagen 模型进行图片编辑。今年晚些时候再回来查看!
Kotlin
如需移除具有自动生成的遮罩的对象,请指定 ImagenBackgroundMask。使用 editImage() 并将编辑配置设置为使用 ImagenEditMode.INPAINT_REMOVAL。
// Using this SDK to access Imagen models is a Preview release and requires opt-in @OptIn(PublicPreviewAPI::class) suspend fun customizeImage() { // Initialize the Vertex AI Gemini API backend service // Optionally specify the location to access the model (for example, `us-central1`) val ai = Firebase.ai(backend = GenerativeBackend.vertexAI(location = "us-central1")) // Create an `ImagenModel` instance with an Imagen "capability" model val model = ai.imagenModel("imagen-3.0-capability-001") // This example assumes 'originalImage' is a pre-loaded Bitmap. // In a real app, this might come from the user's device or a URL. val originalImage: Bitmap = TODO("Load your original image Bitmap here") // Provide the prompt describing the content to be removed. val prompt = "ball" // Use the editImage API to remove the unwanted content. // Pass the original image, the prompt, and an editing configuration. val editedImage = model.editImage( referenceImages = listOf( ImagenRawImage(originalImage.toImagenInlineImage()), ImagenBackgroundMask(), // Use ImagenBackgroundMask() to auto-generate the mask. ), prompt = prompt, // Define the editing configuration for inpainting and insertion. config = ImagenEditingConfig(ImagenEditMode.INPAINT_REMOVAL) ) // Process the resulting 'editedImage' Bitmap, for example, by displaying it in an ImageView. } Java
如需移除具有自动生成的遮罩的对象,请指定 ImagenBackgroundMask。使用 editImage() 并将编辑配置设置为使用 ImagenEditMode.INPAINT_REMOVAL。
// Initialize the Vertex AI Gemini API backend service // Optionally specify the location to access the model (for example, `us-central1`) // Create an `ImagenModel` instance with an Imagen "capability" model ImagenModel imagenModel = FirebaseAI.getInstance(GenerativeBackend.vertexAI("us-central1")) .imagenModel( /* modelName */ "imagen-3.0-capability-001"); ImagenModelFutures model = ImagenModelFutures.from(imagenModel); // This example assumes 'originalImage' is a pre-loaded Bitmap. // In a real app, this might come from the user's device or a URL. Bitmap originalImage = null; // TODO("Load your image Bitmap here"); // Provide the prompt describing the content to be removed. String prompt = "ball"; // Define the list of sources for the editImage call. // This includes the original image and the auto-generated mask. ImagenRawImage rawOriginalImage = new ImagenRawImage(ImagenInlineImageKt.toImagenInlineImage(originalImage)); // Use ImagenBackgroundMask() to auto-generate the mask. ImagenBackgroundMask rawMaskedImage = new ImagenBackgroundMask(); ImagenEditingConfig config = new ImagenEditingConfig(); // Use the editImage API to remove the unwanted content. // Pass the original image, the auto-generated masked image, the prompt, and an editing configuration. Futures.addCallback(model.editImage(Arrays.asList(rawOriginalImage, rawMaskedImage), prompt, config), new FutureCallback<ImagenGenerationResponse>() { @Override public void onSuccess(ImagenGenerationResponse result) { if (result.getImages().isEmpty()) { Log.d("ImageEditor", "No images generated"); } Bitmap editedImage = ((ImagenInlineImage) result.getImages().get(0)).asBitmap(); // Process and use the bitmap to display the image in your UI } @Override public void onFailure(Throwable t) { // ... } }, Executors.newSingleThreadExecutor()); Web
Web 应用不支持使用 Imagen 模型进行图片编辑。今年晚些时候再回来查看!
Dart
如需移除具有自动生成的遮罩的对象,请指定 ImagenBackgroundMask。使用 editImage() 并将编辑配置设置为使用 ImagenEditMode.inpaintRemoval。
import 'dart:typed_data'; import 'package:firebase_ai/firebase_ai.dart'; import 'package:firebase_core/firebase_core.dart'; import 'firebase_options.dart'; // Initialize FirebaseApp await Firebase.initializeApp( options: DefaultFirebaseOptions.currentPlatform, ); // Initialize the Vertex AI Gemini API backend service // Optionally specify a location to access the model (for example, `us-central1`) final ai = FirebaseAI.vertexAI(location: 'us-central1'); // Create an `ImagenModel` instance with an Imagen "capability" model final model = ai.imagenModel(model: 'imagen-3.0-capability-001'); TODO - FLUTTER// This example assumes 'originalImage' is a pre-loaded Uint8List. // In a real app, this might come from the user's device or a URL. final Uint8List originalImage = Uint8List(0); // TODO: Load your original image data here. // Provide the prompt describing the content to be removed. final prompt = 'ball'; try { // Use the editImage API to remove the unwanted content. // Pass the original image, the prompt, and an editing configuration. final response = await model.editImage( sources: [ ImagenRawImage(originalImage), ImagenBackgroundMask(), // Use ImagenBackgroundMask() to auto-generate the mask. ], prompt, // Define the editing configuration for inpainting and removal. config: const ImagenEditingConfig( editMode: ImagenEditMode.inpaintRemoval, ), ); // Process the result. if (response.images.isNotEmpty) { final editedImage = response.images.first.bytes; // Use the editedImage (a Uint8List) to display the image, save it, etc. print('Image successfully generated!'); } else { // Handle the case where no images were generated. print('Error: No images were generated.'); } } catch (e) { // Handle any potential errors during the API call. print('An error occurred: $e'); } Unity
Unity 不支持使用 Imagen 模型进行图片编辑。今年晚些时候再回来查看!
使用提供的遮罩移除对象
| 在试用此示例之前,请完成本指南的准备工作部分,以设置您的项目和应用。 |
以下示例展示了如何使用修复功能从图片中移除内容,方法是使用您提供的图片中定义的蒙版。 您提供原始图片、文本提示和蒙版图片。
如果您提供遮盖图片,则无需提供文本提示。 Imagen 可以智能检测要从遮罩区域中移除的对象。不过,如果您要移除的对象不明显,或者您只想移除蒙版区域中的特定对象,请提供文本提示,以帮助模型移除正确的对象。
Swift
Swift 不支持使用 Imagen 模型进行图片编辑。今年晚些时候再回来查看!
Kotlin
如需移除对象并提供您自己的蒙版图片,请使用蒙版图片指定 ImagenRawMask。使用 editImage() 并将编辑配置设置为使用 ImagenEditMode.INPAINT_REMOVAL。
// Using this SDK to access Imagen models is a Preview release and requires opt-in @OptIn(PublicPreviewAPI::class) suspend fun customizeImage() { // Initialize the Vertex AI Gemini API backend service // Optionally specify the location to access the model (for example, `us-central1`) val ai = Firebase.ai(backend = GenerativeBackend.vertexAI(location = "us-central1")) // Create an `ImagenModel` instance with an Imagen "capability" model val model = ai.imagenModel("imagen-3.0-capability-001") // This example assumes 'originalImage' is a pre-loaded Bitmap. // In a real app, this might come from the user's device or a URL. val originalImage: Bitmap = TODO("Load your original image Bitmap here") // This example assumes 'maskImage' is a pre-loaded Bitmap that contains the masked area. // In a real app, this might come from the user's device or a URL. val maskImage: Bitmap = TODO("Load your masked image Bitmap here") // Provide the prompt describing the content to be removed. val prompt = "ball" // Use the editImage API to remove the unwanted content. // Pass the original image, the masked image, the prompt, and an editing configuration. val editedImage = model.editImage( referenceImages = listOf( ImagenRawImage(originalImage.toImagenInlineImage()), ImagenRawMask(maskImage.toImagenInlineImage()), // Use ImagenRawMask() to provide your own masked image. ), prompt = prompt, // Define the editing configuration for inpainting and removal. config = ImagenEditingConfig(ImagenEditMode.INPAINT_REMOVAL) ) // Process the resulting 'editedImage' Bitmap, for example, by displaying it in an ImageView. } Java
如需移除对象并提供您自己的蒙版图片,请使用蒙版图片指定 ImagenRawMask。使用 editImage() 并将编辑配置设置为使用 ImagenEditMode.INPAINT_REMOVAL。
// Initialize the Vertex AI Gemini API backend service // Optionally specify the location to access the model (for example, `us-central1`) // Create an `ImagenModel` instance with an Imagen "capability" model ImagenModel imagenModel = FirebaseAI.getInstance(GenerativeBackend.vertexAI("us-central1")) .imagenModel( /* modelName */ "imagen-3.0-capability-001"); ImagenModelFutures model = ImagenModelFutures.from(imagenModel); // This example assumes 'originalImage' is a pre-loaded Bitmap. // In a real app, this might come from the user's device or a URL. Bitmap originalImage = null; // TODO("Load your original image Bitmap here"); // This example assumes 'maskImage' is a pre-loaded Bitmap that contains the masked area. // In a real app, this might come from the user's device or a URL. Bitmap maskImage = null; // TODO("Load your masked image Bitmap here"); // Provide the prompt describing the content to be removed. String prompt = "ball"; // Define the list of source images for the editImage call. ImagenRawImage rawOriginalImage = new ImagenRawImage(ImagenInlineImageKt.toImagenInlineImage(originalImage)); // Use ImagenRawMask() to provide your own masked image. ImagenBackgroundMask rawMaskedImage = new ImagenRawMask(ImagenInlineImageKt.toImagenInlineImage(maskImage)); ImagenEditingConfig config = new ImagenEditingConfig(); // Use the editImage API to remove the unwanted content. // Pass the original image, the masked image, the prompt, and an editing configuration. Futures.addCallback(model.editImage(Arrays.asList(rawOriginalImage, rawMaskedImage), prompt, config), new FutureCallback<ImagenGenerationResponse>() { @Override public void onSuccess(ImagenGenerationResponse result) { if (result.getImages().isEmpty()) { Log.d("ImageEditor", "No images generated"); } Bitmap editedImage = ((ImagenInlineImage) result.getImages().get(0)).asBitmap(); // Process and use the bitmap to display the image in your UI } @Override public void onFailure(Throwable t) { // ... } }, Executors.newSingleThreadExecutor()); Web
Web 应用不支持使用 Imagen 模型进行图片编辑。今年晚些时候再回来查看!
Dart
如需移除对象并提供您自己的蒙版图片,请使用蒙版图片指定 ImagenRawMask。使用 editImage() 并将编辑配置设置为使用 ImagenEditMode.inpaintRemoval。
import 'dart:typed_data'; import 'package:firebase_ai/firebase_ai.dart'; import 'package:firebase_core/firebase_core.dart'; import 'firebase_options.dart'; // Initialize FirebaseApp await Firebase.initializeApp( options: DefaultFirebaseOptions.currentPlatform, ); // Initialize the Vertex AI Gemini API backend service // Optionally specify a location to access the model (for example, `us-central1`) final ai = FirebaseAI.vertexAI(location: 'us-central1'); // Create an `ImagenModel` instance with an Imagen "capability" model final model = ai.imagenModel(model: 'imagen-3.0-capability-001'); // This example assumes 'originalImage' is a pre-loaded Uint8List. // In a real app, this might come from the user's device or a URL. final Uint8List originalImage = Uint8List(0); // TODO: Load your original image data here. // This example assumes 'maskImage' is a pre-loaded Uint8List that contains the masked area. // In a real app, this might come from the user's device or a URL. final Uint8List maskImage = Uint8List(0); // TODO: Load your masked image data here. // Provide the prompt describing the content to be removed. final prompt = 'ball'; try { // Use the editImage API to remove the unwanted content. // Pass the original image, the prompt, and an editing configuration. final response = await model.editImage( sources: [ ImagenRawImage(originalImage), ImagenRawMask(maskImage), // Use ImagenRawMask() to provide your own masked image. ], prompt: prompt, // Define the editing configuration for inpainting and removal. config: const ImagenEditingConfig( editMode: ImagenEditMode.inpaintRemoval, ), ); // Process the result. if (response.images.isNotEmpty) { final editedImage = response.images.first.bytes; // Use the editedImage (a Uint8List) to display the image, save it, etc. print('Image successfully generated!'); } else { // Handle the case where no images were generated. print('Error: No images were generated.'); } } catch (e) { // Handle any potential errors during the API call. print('An error occurred: $e'); } Unity
Unity 不支持使用 Imagen 模型进行图片编辑。今年晚些时候再回来查看!
最佳做法和限制
建议您在编辑图片时扩大遮罩。这有助于平滑编辑的边界,使其看起来更逼真。一般来说,建议将扩张值设为 1% 或 2%(0.01 或 0.02)。
就您使用 Firebase AI Logic 的体验提供反馈