JFrog Uncovers PixelSmash, a 16-Year-Old FFmpeg Flaw Exploitable via a 50 KB Video File

Overview

JFrog Security Research says it built working remote-code-execution exploit chains against both a Jellyfin media server and a Nextcloud instance using nothing more than a single crafted 50 KB AVI file, according to a technical write-up the company published on its blog, JFrog. The underlying bug, which JFrog named PixelSmash, is a heap out-of-bounds write in the MagicYUV decoder inside FFmpeg’s libavcodec library, tracked as CVE-2026-8461 and rated 8.8 out of 10 on the CVSS scale with a CWE-787 out-of-bounds-write classification, according to the National Vulnerability Database. The flaw has persisted in FFmpeg’s codebase for 16 years, according to InfoQ.

FFmpeg’s decoders sit inside a vast range of consumer and enterprise software that processes video, which is why a single decoder bug can cascade so widely once it is weaponized.

What We Know

According to JFrog, the root cause is a rounding mismatch between how FFmpeg’s frame allocator and its MagicYUV decoder compute chroma-plane heights for subsampled pixel formats such as YUV420P. When a crafted file sets an odd slice height, ceiling-rounded right-shift math adds extra rows per slice that accumulate past the buffer FFmpeg actually allocated, producing a one-row heap overflow in the chroma plane, per JFrog.

JFrog’s demonstrated attack chain against Jellyfin works through the media server’s own automation: an attacker uploads a 50 KB AVI file containing a malicious MagicYUV stream, Jellyfin’s file-system monitor detects the new file and triggers an ffprobe metadata scan, and FFmpeg’s decoder processes the stream. The resulting heap overwrite hijacks an internal buffer’s function pointer so that, during frame cleanup, it calls the system’s command-execution function instead of freeing memory — spawning a reverse shell running as Jellyfin’s own service account, according to JFrog.

Against Nextcloud, the same overwrite is triggered through the file-sharing platform’s optional movie-preview provider, which invokes FFmpeg server-side to generate thumbnails whenever a file appears in a user’s Files view. JFrog describes the resulting compromise as running with the privileges of Nextcloud’s web-server user, which has access to other users’ data and database credentials, and notes the attack requires close to no interaction once the preview provider is enabled — simply browsing a folder listing is enough to trigger the vulnerable code path, per JFrog.

JFrog’s own accounting of affected software spans several categories: media servers including Jellyfin (version 10.11.9) and Emby (4.8.11); desktop players such as mpv, Kodi, VLC, and OBS Studio; workplace platforms including Nextcloud, Slack, Discord, and Telegram, which generate video previews server-side; AI and machine-learning tooling such as vLLM, Hugging Face Datasets, and NVIDIA DALI; and consumer network-attached storage devices from Synology and QNAP, according to JFrog. The company says its proof-of-concept demonstrated denial-of-service crashes reliably across every target it tested, even in cases where full code execution was not achieved, per the same write-up.

On disclosure, JFrog reported the flaw to FFmpeg’s security team on May 13, 2026; FFmpeg acknowledged the report on May 19. JFrog then notified Jellyfin’s security team on May 24, and reports followed to mpv, the OBS Project, PhotoPrism, and Immich on May 26 — the same day Jellyfin updated its bundled FFmpeg version. PhotoPrism opened a GitHub issue to exclude the vulnerable format on May 27, and JFrog notified vLLM’s security team on May 31 and Kodi’s on June 7. FFmpeg shipped the fix in version 8.1.2 on June 17, 2026, the CVE was published the following day, and JFrog published its full technical analysis on June 22, according to JFrog. The National Vulnerability Database likewise lists FFmpeg versions before 8.1.2 as affected, corroborating the fixed release, per NVD.

Organizations that cannot immediately upgrade can disable the vulnerable decoder at compile time by rebuilding FFmpeg with --disable-decoder=magicyuv, and can check whether a given FFmpeg build is exposed by running ffmpeg -decoders | grep magicyuv, according to JFrog. InfoQ separately confirms the same workaround command and describes the eventual fix as a minimal seven-line patch to the decoder’s validation logic.

JFrog also flags two important caveats to how damaging the bug is in practice. First, its remote-code-execution demonstrations required address space layout randomization to be disabled on the target; turning the crash into reliable code execution against a hardened, ASLR-enabled production system would require chaining it with a separate information-leak bug, and JFrog says it identified one candidate — an unpatched flaw in FFmpeg’s FlashSV decoder — but did not chain the two in its demonstration, per JFrog. Second, JFrog says the corruption can occur silently: both Jellyfin and Emby continued to report a normal exit code even while a file was actively triggering the vulnerable code path, meaning affected operators would see no obvious error signal, according to the same source.

What We Don’t Know

JFrog’s disclosure timeline documents outreach to FFmpeg, Jellyfin, mpv, the OBS Project, PhotoPrism, Immich, vLLM, and Kodi, and confirms that Jellyfin bumped its bundled FFmpeg version and PhotoPrism opened a GitHub issue to exclude the vulnerable format. Neither JFrog’s write-up nor InfoQ’s coverage specifies whether Emby, Kodi, mpv, OBS Studio, or the chat platforms named in JFrog’s affected-software list have since shipped their own independent fixes beyond whatever FFmpeg version they bundle. Whether the vulnerability was ever exploited in the wild before JFrog’s research is also not addressed in either source.