SoC 2007 Project Dynamic Video Overlays

From VideoLAN Wiki
Jump to navigation Jump to search
This project is part of Google Summer of Code 2007.
Student: Søren Bøg
Mentor: Antoine Cellerier


I (Søren Bøg) will be using this area to describe and track my progress, on implementing dynamic overlays in VLC, for Google Summer of Code 2007.


The goal of this is to create a video filter for VLC, allowing third party applications to overlay dynamic images on top of the VLC video output.


The idea here is to provide VLC with a video filter that allows a third party application (hereafter the controller) to request an overlay on top of the video that VLC is displaying. The controller is then handed a piece of shared memory with a specified pixel format. The controller may then draw all that it desires to this shared memory, Finally when the shared memory has been written to, the controller will notify VLC of the change, and VLC will then composite the overlay into subsequent frames, or at least until instructed to do something else by the controller.

This is somewhat similar the the roles that bomvl, bmovl2 and vf_overlay play for MPlayer.


Well, because I can, why else? More seriously I can see two practical use-cases for this. The first case is my primary motivation for working on this project.

The first use case is a computer gaming website which would like to broadcast computer game tournaments over the internet using VLC. This gaming site would then be able to overlay match statistics and other information onto the streamed video.

Additionally it may be useful for HTPC projects which can then use VLC as a canvas to draw user interfaces on. Such as the MeBox does with MPlayer and bmovl2.


Task Description Status
Make simple no-op video filters Done
Define controller protocol Done
Implement full video filter In progress
Implement bmovl/bmovl2 compatibility Undecided
Implement interface library Undecided
Implement kaa interface Undecided
Speed up VLC blending (Using assembly or GCC intrinsics) In progress


It is assumed that all messages return an indication of success or failure, which is not mentioned in the descriptions below.



Creates an image used for overlay.


  • A numeric ID for referring to this particular image.


Deletes an image, and frees any resources


  • The image ID to be freed



Starts an atomic action. Any image parameter modifying commands sent until the next EndAtomic, will first be executed at the EndAtomic. Any returns will first be available at this point.


Ends an atomic action. Any image parameter modifying commands since the last StartAtomic will be executed at this point, and any returns made available.

Image data


Sends image data to an overlay, using shared memory. The controlling program MAY NOT modify the shared memory, until a return has been received for this command.


  • The image ID to send the data to
  • The width of the image data
  • The height of the image data
  • The format of the data (FOURCC code)
  • The (system-specific) identifier for the shared memory.

Image parameters


Set the position of an overlay image.


  • The ID of the image to reposition
  • The X-coordinate of the new position
  • The Y-coordinate of the new position


Sets an general transparency for a given image.


  • The ID of the image to set transparency for.
  • The new transparency


Sets an images visibility.


  • The ID of the image to set transparency for.
  • The boolean indicating of the image is visible.


As a part of my work on this, I have looked at speeding up vlc's blending routines.


I've done some benchmarking for blending various colorspaces. The format for the tables is <alpha-blending speed> / <blending speed>. Each result is the average of 10 test, with the standard deviation in parenthesis. The result is the number of megapixels blended per second. For refences a SD frame is 0.44 MPix (PAL) or 0.37 MPix (NTSC), 720i/p is 0,92 MPix and 1080i/p is 2,07 MPix.

Test 1

Image Dimensions: 128x128 Loops: 10000

Input\Output I420 RV24
I420 137.3 (10.04) / 299.2 (99.36) 25.06 (1.357) / 28.70 (1.047)
YUVA 81.84 (4.023) / 79.56 (5.392) 38.09 (2.104) / 37.29 (2.832)
RGBA 220.6 (15.53) / 230.3 (10.13) 263.6 (9.678) / 254.5 (21.75)

Test 2

Image Dimensions: 2048x2048 Loops: 200

Input\Output I420 RV24
I420 208.8 (6.933) / 1083. (75.65) 26.81 (.3689) / 31.45 (.3413)
YUVA 99.37 (1.981) / 98.55 (2.984) 39.55 (.7752) / 39.88 (.5518)
RGBA 268.3 (6.393) / 269.3 (4.213) 302.9 (8.844) / 303.0 (6.622)