Difference between revisions of "SoC 2007 Project Dynamic Video Overlays"

From VideoLAN Wiki
Jump to navigation Jump to search
(Added initial benchmarks)
 
(3 intermediate revisions by the same user not shown)
Line 26: Line 26:
 
| Make simple no-op video filters                                                || style="background: #ddffdd" | Done
 
| Make simple no-op video filters                                                || style="background: #ddffdd" | Done
 
|-
 
|-
−
| Define controller protocol                                                    || style="background: #ffffdd" | In progess
+
| Define controller protocol                                                    || style="background: #ddffdd" | Done
 
|-
 
|-
−
| Implement full video filter                                                    || style="background: #ffdddd" | Not complete
+
| Implement full video filter                                                    || style="background: #ffffdd" | In progress
 
|-
 
|-
 
| Implement bmovl/bmovl2 compatibility                                          || Undecided
 
| Implement bmovl/bmovl2 compatibility                                          || Undecided
Line 36: Line 36:
 
| Implement [http://freevo.sourceforge.net/cgi-bin/freevo-2.0/Kaa kaa] interface || Undecided
 
| Implement [http://freevo.sourceforge.net/cgi-bin/freevo-2.0/Kaa kaa] interface || Undecided
 
|-
 
|-
−
| Speed up VLC blending (Using assembly or GCC intrinsics)                      || Undecided
+
| Speed up VLC blending (Using assembly or GCC intrinsics)                      || style="background: #ffffdd" | In progress
 
|-
 
|-
 +
|}
 +
 +
== Protocol ==
 +
It is assumed that all messages return an indication of success or failure, which is not mentioned in the descriptions below.
 +
 +
=== Allocation ===
 +
 +
==== GenImage ====
 +
Creates an image used for overlay.
 +
 +
Returns:
 +
* A numeric ID for referring to this particular image.
 +
 +
==== DeleteImage ====
 +
Deletes an image, and frees any resources
 +
 +
Parameters:
 +
* The image ID to be freed
 +
 +
=== Atomicity ===
 +
 +
==== StartAtomic ====
 +
Starts an atomic action. Any image parameter modifying commands sent until the next EndAtomic, will first be executed at the EndAtomic. Any returns will first be available at this point.
 +
 +
==== EndAtomic ====
 +
Ends an atomic action. Any image parameter modifying commands since the last StartAtomic will be executed at this point, and any returns made available.
 +
 +
=== Image data ===
 +
 +
==== DataSharedMem ====
 +
Sends image data to an overlay, using shared memory. The controlling program MAY NOT modify the shared memory, until a return has been received for this command.
 +
 +
Parameters:
 +
* The image ID to send the data to
 +
* The width of the image data
 +
* The height of the image data
 +
* The format of the data (FOURCC code)
 +
* The (system-specific) identifier for the shared memory.
 +
 +
=== Image parameters ===
 +
 +
==== SetPosition ====
 +
Set the position of an overlay image.
 +
 +
Parameters:
 +
* The ID of the image to reposition
 +
* The X-coordinate of the new position
 +
* The Y-coordinate of the new position
 +
 +
==== SetAlpha ====
 +
Sets an general transparency for a given image.
 +
 +
Parameters:
 +
* The ID of the image to set transparency for.
 +
* The new transparency
 +
 +
==== SetVisibility ====
 +
Sets an images visibility.
 +
 +
Parameters:
 +
* The ID of the image to set transparency for.
 +
* The boolean indicating of the image is visible.
 +
 +
== Blending ==
 +
As a part of my work on this, I have looked at speeding up vlc's blending routines.
 +
 +
=== Benchmarking ===
 +
I've done some benchmarking for blending various colorspaces. The format for the tables is <alpha-blending speed> / <blending speed>. Each result is the average of 10 test, with the standard deviation in parenthesis. The result is the number of megapixels blended per second. For refences a SD frame is 0.44 MPix (PAL) or 0.37 MPix (NTSC), 720i/p is 0,92 MPix and 1080i/p is 2,07 MPix.
 +
 +
==== Test 1 ====
 +
Image Dimensions: 128x128
 +
Loops: 10000
 +
 +
{| class="wikitable"
 +
! Input\Output !! I420 !! RV24
 +
|-
 +
| I420 || 137.3 (10.04) / 299.2 (99.36) || 25.06 (1.357) / 28.70 (1.047)
 +
|-
 +
| YUVA || 81.84 (4.023) / 79.56 (5.392) || 38.09 (2.104) / 37.29 (2.832)
 +
|-
 +
| RGBA || 220.6 (15.53) / 230.3 (10.13) || 263.6 (9.678) / 254.5 (21.75)
 +
|-
 +
|}
 +
 +
==== Test 2 ====
 +
Image Dimensions: 2048x2048
 +
Loops: 200
 +
 +
{| class="wikitable"
 +
! Input\Output !! I420 !! RV24
 +
|-
 +
| I420 || 208.8 (6.933) / 1083. (75.65) || 26.81 (.3689) / 31.45 (.3413)
 +
|-
 +
| YUVA || 99.37 (1.981) / 98.55 (2.984) || 39.55 (.7752) / 39.88 (.5518)
 +
|-
 +
| RGBA || 268.3 (6.393) / 269.3 (4.213) || 302.9 (8.844) / 303.0 (6.622)
 +
|-
 +
|}

Latest revision as of 12:45, 27 August 2007

This project is part of Google Summer of Code 2007.
Student: Søren Bøg
Mentor: Antoine Cellerier

Intro

I (Søren Bøg) will be using this area to describe and track my progress, on implementing dynamic overlays in VLC, for Google Summer of Code 2007.

Goal

The goal of this is to create a video filter for VLC, allowing third party applications to overlay dynamic images on top of the VLC video output.

Details

The idea here is to provide VLC with a video filter that allows a third party application (hereafter the controller) to request an overlay on top of the video that VLC is displaying. The controller is then handed a piece of shared memory with a specified pixel format. The controller may then draw all that it desires to this shared memory, Finally when the shared memory has been written to, the controller will notify VLC of the change, and VLC will then composite the overlay into subsequent frames, or at least until instructed to do something else by the controller.

This is somewhat similar the the roles that bomvl, bmovl2 and vf_overlay play for MPlayer.


Why

Well, because I can, why else? More seriously I can see two practical use-cases for this. The first case is my primary motivation for working on this project.

The first use case is a computer gaming website which would like to broadcast computer game tournaments over the internet using VLC. This gaming site would then be able to overlay match statistics and other information onto the streamed video.

Additionally it may be useful for HTPC projects which can then use VLC as a canvas to draw user interfaces on. Such as the MeBox does with MPlayer and bmovl2.

Tasks

Task Description Status
Make simple no-op video filters Done
Define controller protocol Done
Implement full video filter In progress
Implement bmovl/bmovl2 compatibility Undecided
Implement interface library Undecided
Implement kaa interface Undecided
Speed up VLC blending (Using assembly or GCC intrinsics) In progress

Protocol

It is assumed that all messages return an indication of success or failure, which is not mentioned in the descriptions below.

Allocation

GenImage

Creates an image used for overlay.

Returns:

  • A numeric ID for referring to this particular image.

DeleteImage

Deletes an image, and frees any resources

Parameters:

  • The image ID to be freed

Atomicity

StartAtomic

Starts an atomic action. Any image parameter modifying commands sent until the next EndAtomic, will first be executed at the EndAtomic. Any returns will first be available at this point.

EndAtomic

Ends an atomic action. Any image parameter modifying commands since the last StartAtomic will be executed at this point, and any returns made available.

Image data

DataSharedMem

Sends image data to an overlay, using shared memory. The controlling program MAY NOT modify the shared memory, until a return has been received for this command.

Parameters:

  • The image ID to send the data to
  • The width of the image data
  • The height of the image data
  • The format of the data (FOURCC code)
  • The (system-specific) identifier for the shared memory.

Image parameters

SetPosition

Set the position of an overlay image.

Parameters:

  • The ID of the image to reposition
  • The X-coordinate of the new position
  • The Y-coordinate of the new position

SetAlpha

Sets an general transparency for a given image.

Parameters:

  • The ID of the image to set transparency for.
  • The new transparency

SetVisibility

Sets an images visibility.

Parameters:

  • The ID of the image to set transparency for.
  • The boolean indicating of the image is visible.

Blending

As a part of my work on this, I have looked at speeding up vlc's blending routines.

Benchmarking

I've done some benchmarking for blending various colorspaces. The format for the tables is <alpha-blending speed> / <blending speed>. Each result is the average of 10 test, with the standard deviation in parenthesis. The result is the number of megapixels blended per second. For refences a SD frame is 0.44 MPix (PAL) or 0.37 MPix (NTSC), 720i/p is 0,92 MPix and 1080i/p is 2,07 MPix.

Test 1

Image Dimensions: 128x128 Loops: 10000

Input\Output I420 RV24
I420 137.3 (10.04) / 299.2 (99.36) 25.06 (1.357) / 28.70 (1.047)
YUVA 81.84 (4.023) / 79.56 (5.392) 38.09 (2.104) / 37.29 (2.832)
RGBA 220.6 (15.53) / 230.3 (10.13) 263.6 (9.678) / 254.5 (21.75)

Test 2

Image Dimensions: 2048x2048 Loops: 200

Input\Output I420 RV24
I420 208.8 (6.933) / 1083. (75.65) 26.81 (.3689) / 31.45 (.3413)
YUVA 99.37 (1.981) / 98.55 (2.984) 39.55 (.7752) / 39.88 (.5518)
RGBA 268.3 (6.393) / 269.3 (4.213) 302.9 (8.844) / 303.0 (6.622)
Retrieved from "https://wiki.videolan.org/index.php?title=SoC_2007_Project_Dynamic_Video_Overlays&oldid=9750"