ai transformers.js translation offline web worker
remove background offscreen canvas
maintained with ❤️ by
Volver AI is an Hugging Face Transformers.js wrapper for add AI capabilities to your web applications in a simple way with multithreading support.
Add the library to your project using your favorite package manager.
# pnpm pnpm add @volverjs/ai # yarn yarn add @volverjs/ai # npm npm i -s @volverjs/aiThen you can import and instantiate the AI functionality you need.
import { Translator, RemoveBackground } from '@volverjs/ai' const translator = new Translator() const removeBackground = new RemoveBackground()The library automatically downloads and run the model inside a Web Worker. Using a Web Worker allows the library to run the model in a separate thread, preventing the main thread from blocking.
Warning
Any instance of the AI classes initializes a new Web Worker with the model and manages the communication between the main thread and the worker. If you need to use the same worker in multiple places, you can create a single instance and share it across your application.
The first time you instantiate the model, it will download the model files from Hugging Face Hub. This process can take a few seconds or minutes depending on the model size and your internet connection.
await translator.init()Otherwise the model is initialized automatically when you call any method that requires the model to be loaded.
You can listen to the download progress of the model using the progress event.
translator.on('progress', ({ file, progrees }) => { console.log(`Download progress of ${file} is ${progress}%`) })Or with onProgress initialization option.
translator.init({ onProgress: ({ file, progress }) => { console.log(`Download progress of ${file} is ${progress}%`) } })Translator allows to convert text from one language to another with the AI model of your choice
By default, the library uses the Xenova/nllb-200-distilled-600M model to translate text.
import { Translator } from '@volverjs/ai' const translator = new Translator()You can change the model by passing the model option to the constructor. You can find a list of available models here.
const translator = new Translator({ model: 'Xenova/opus-mt-it-en' })To translate text, use the translate method passing the text to translate and the from and to options with the language codes of the source and target languages.
const text = 'Ciao, come stai?' const translated = await translator.translate(text, { from: 'ita_Latn', to: 'eng_Latn' })For Xenova/nllb-200-distilled-600M model you can find the full list of supported languages here.
During the translation process, you can get the partial translation using the onUpdate option.
const text = 'How are you?' const translated = await translator.translate(text, { from: 'eng_Latn', to: 'ita_Latn', onUpdate: ({ output }) => { console.log(`Partial translation: ${output}`) } })If you are using Vue.js, you can use the Composition API to create a reactive translation.
<script lang="ts" setup> import { useTranslator } from '@volverjs/ai/vue' import { ref } from 'vue' const text = ref('') const { translate } = useTranslator() const { result } = translate(text, { from: 'eng_Latn', to: 'ita_Latn' }) </script> <template> <input v-model="text" type="text" name="from"> <p>{{ result }}</p> </template>Tip
The result reactive property contains the partial translation during process and the final result when the translation is completed.
You can use a custom model by passing the model option to the useTranslator function.
const { translate } = useTranslator({ model: 'Xenova/opus-mt-it-en' })By default, a new worker is created for each useTranslator call. You can share the worker across multiple components by passing global option.
const { translate } = useTranslator({ global: true })The model is initialized automatically when you use the translate function. To monitor the download progress, you can use the progress reactive property.
const { result, progress } = translate(text, { from: 'eng_Latn', to: 'ita_Latn' }) watch(progress, ({ file, progress: value }) => { console.log(`Download of file ${file} with progress: ${value}%`) })To initialize the model manually, you can use the init method.
const { init } = useTranslator() const { isReady, progress } = init()Below is a complete example of a Vue.js component that uses the useTranslator function.
<script lang="ts" setup> import { useTranslator } from '@volverjs/ai/vue' import { ref, watch } from 'vue' const text = ref('') const { translate } = useTranslator() const { // Ref<string> // the partial translation and the final result result, // Ref<{file: string, progress: number}[]> // the download progress of the model progress, // Ref<Error | undefined> // the error object if an error occurs error, // Ref<boolean> // is true when an error occurs isError, // Ref<boolean> // is true when the model is downloading isLoading, // Ref<boolean> // is true when the model is ready isReady, // Promise<string> // the promise that resolves when the translation is completed promise, // (text?: string) => Promise<string | undefined> // manually execute the translation execute } = translate( // Ref<string> | string | undefined // the text to translate text, { // the source language from: 'eng_Latn', // the target language to: 'ita_Latn', // boolean (default: true) // set to false to disable the first translation execution immediate: true, // number (default: 500) // the debounce time in milliseconds to wait before executing the translation debounce: 800 } ) </script> <template> <input v-model="text" type="text" name="from"> <p>{{ result }}</p> </template>Remove the background from an image uses briaai/RMBG-1.4 as background removal AI model, designed to effectively separate foreground from background in a range of categories and image types.
To remove the background from an image, use the predict method passing the image URL.
import { RemoveBackground } from '@volverjs/ai' const sourceImageURL = 'https://example.com/image.jpg' const removeBackground = new RemoveBackground() const result = await removeBackground.predict(sourceImageURL)The URL could be a local file or a remote file. The library automatically downloads the image and processes it.
You can customize the quality and the type of the output image using the quality and type options. The quality option accepts a number from 0 to 1, and the type option accepts the image MIME type.
const resultImageBlob = await removeBackground.predict(sourceImageURL, { quality: 0.5, type: 'image/webp' }) const resultImageURL = URL.createObjectURL(resultImageBlob)You can pass a canvas element in the DOM to the predict method to draw the result directly on the canvas.
const canvas = document.getElementById('canvas') as HTMLCanvasElement removeBackground.predict(sourceImageURL, { canvas })Tip
@volverjs/ai use the OffscreenCanvas API to delegate the image processing in the canvas to the worker thread.
If you are using Vue.js, you can use the Composition API to create a reactive background removal.
<script lang="ts" setup> import { useRemoveBackground } from '@volverjs/ai/vue' import { ref } from 'vue' const sourceImageURL = ref('https://example.com/image.jpg') const { predict } = useRemoveBackground() const { resultUrl } = predict(sourceImageURL) </script> <template> <img :src="sourceImageURL" alt="Source Image"> <img v-if="resultUrl" :src="resultUrl" alt="Result Image"> </template>The resultUrl reactive property contains the URL of the processed image. Change the sourceImageURL to trigger the background removal process.
To use a canvas element, pass the canvas element to the predict method.
<script lang="ts" setup> import { useRemoveBackground } from '@volverjs/ai/vue' import { ref } from 'vue' const sourceImageURL = ref('https://example.com/image.jpg') const canvasEl = ref<HTMLCanvasElement>() const { predict } = useRemoveBackground() predict(sourceImageURL, { canvas: canvasEl }) </script> <template> <img :src="sourceImageURL" alt="Source Image"> <canvas ref="canvasEl" /> </template>By default, a new worker is created for each useRemoveBackground call. You can share the worker across multiple components by passing global option.
const { predict } = useRemoveBackground({ global: true })The model is initialized automatically when you use the translate function. To monitor the download progress, you can use the progress reactive property.
const { resultUrl, progress } = predict(sourceImageURL) watch(progress, ({ file, progress: value }) => { console.log(`Download of file ${file} with progress: ${value}%`) })To initialize the model manually, you can use the init method.
const { init } = useRemoveBackground() const canvasEl = ref<HTMLCanvasElement>() onMounted(() => { init({ canvas: canvasEl }) })Below is a complete example of a Vue.js component that uses the useRemoveBackground function.
<script lang="ts" setup> import { useRemoveBackground } from '@volverjs/ai/vue' import { ref, watch } from 'vue' const sourceImageURL = ref('https://example.com/image.jpg') const canvasEl = ref<HTMLCanvasElement>() const { predict } = useRemoveBackground() const { // Ref<Blob | undefined> // the processed image blob result, // Ref<string | undefined> // the processed image URL resultUrl, // Ref<{file: string, progress: number}[]> // the download progress of the model progress, // Ref<Error | undefined> // the error object if an error occurs error, // Ref<boolean> // is true when an error occurs isError, // Ref<boolean> // is true when the model is downloading isLoading, // Ref<boolean> // is true when the model is ready isReady, // Promise<string> // the promise that resolves when the removal is completed promise, // (sourceUrl?: string) => <Promise<Blob | undefined>> // manually execute the removal execute } = predict( // Ref<string> | string | undefined // the source image URL sourceImageURL, { // the source language canvas: canvasEl, // the target language type: 'image/webp', // number (default: 0.5) // the quality of the output image quality: 0.5, // boolean (default: true) // set to false to disable the first removal execution immediate: true, } ) </script> <template> <img :src="sourceImageURL" alt="Source Image"> <canvas ref="canvasEl" /> </template>To learn more about @volverjs/ai, check its documentation.