Reassembles the frames of a GIF to sync its animation to the beat of an audio file.
Uses the TempoCNN models provided by Essentia.
Credits: Dance Reaction GIF by DOMCAKE x Gypsy Girl by Toman.
Credits: Nodding Yes GIF x Epik High - 낙화 (落花) {The Falling Flower}.
Install Docker Compose and start the Docker daemon.
Build the container with Essentia, Tensorflow, and ffmpeg
docker compose build Start the container and get a shell inside
docker compose up -d && docker compose exec app bash Find an audio file that starts exactly on beat. Find a GIF and label the frames you want to align to the beat. Pass them to the sync.py script as in the example below.
python sync.py \ --audio_filepath "demo/gypsy.m4a" \ --gif_filepath "demo/alien.gif" \ --beat_frames 0 9 17 28 36 45 54 63 72 81 --tempo_multiplier TEMPO_MULTIPLIER A multiplier applied to the extracted tempo. Speeds up or slows down the animation. (default: 1.0) --output_directory OUTPUT_DIRECTORY The directory to which the output will be saved. (default: .) --bpm BPM The BPM of the audio. Will be estimated if not passed. (default: None) --interpolation INTERPOLATION The method of interpolation to use. Options: [linear, cubic, quadratic] (default: linear)