VideoLAN Wiki - User contributions [en]

Hacker Guide/Video Output

2009-08-10T14:30:35Z

Sterops: /* How to write a video output plug-in */

=The video output layer=

==Data structures and main loop==

Important data structures are defined in include/video.h and include/video_output.h. The main data structure is picture_t, which describes everything a video decoder thread needs. Please refer to this file for more information. Typically, p_data will be a pointer to YUV planar picture.

Note also the subpicture_t structure. In fact the VLC SPU decoder only parses the SPU header, and converts the SPU graphical data to an internal format which can be rendered much faster. So a part of the "real" SPU decoder lies in src/video_output/video_spu.c.

The vout_thread_t structure is much more complex, but you needn't understand everything. Basically the video output thread manages a heap of pictures and subpictures (5 by default). Every picture has a status (displayed, destroyed, empty...) and eventually a presentation time. The main job of the video output is an infinite loop to : [this is subject to change in the near future]

#Find the next picture to display in the heap.
#Find the current subpicture to display.
#Render the picture (if the video output plug-in doesn't support YUV overlay). Rendering will call an optimized YUV plug-in, which will also do the scaling, add subtitles and an optional picture information field.
#Sleep until the specified date.
#Display the picture (plug-in function). For outputs which display RGB data, it is often accomplished with a buffer switching. p_vout->p_buffer is an array of two buffers where the YUV transform takes place, and p_vout->i_buffer_index indicates the currently displayed buffer.
#Manage events.

==Methods used by video decoders==

The video output exports a bunch of functions so that decoders can send their decoded data. The most important function is vout_CreatePicture which allocates the picture buffer to the size indicated by the video decoder. It then just needs to feed (void *) p_picture->p_data with the decoded data, and call vout_DisplayPicture and vout_DatePicture upon necessary.

*picture_t * vout_CreatePicture ( vout_thread_t *p_vout, int i_type, int i_width, int i_height ) : Returns an allocated picture buffer. i_type will be for instance YUV_420_PICTURE, and i_width and i_height are in pixels.

=Warning=

If no picture is available in the heap, vout_CreatePicture will return NULL.

*vout_LinkPicture ( vout_thread_t *p_vout, picture_t *p_pic ) : Increases the refcount of the picture, so that it doesn't get accidently freed while the decoder still needs it. For instance, an I or P picture can still be needed after displaying to decode interleaved B pictures.
*vout_UnlinkPicture ( vout_thread_t *p_vout, picture_t *p_pic ) : Decreases the refcount of the picture. An unlink must be done for every link previously made.
*vout_DatePicture ( vout_thread_t *p_vout, picture_t *p_pic ) : Gives the picture a presentation date. You can start working on a picture before knowing precisely at what time it will be displayed. For instance to date an I or P picture, you must wait until you have decoded all previous B pictures (which are indeed placed after - decoding order != presentation order).
*vout_DisplayPicture ( vout_thread_t *p_vout, picture_t *p_pic ) : Tells the video output that a picture has been completely decoded and is ready to be rendered. It can be called before or after vout_DatePicture.
*vout_DestroyPicture ( vout_thread_t *p_vout, picture_t *p_pic ) : Marks the picture as empty (useful in case of a stream parsing error).
*subpicture_t * vout_CreateSubPicture ( vout_thread_t *p_vout, int i_channel, int i_type ) : Returns an allocated subpicture buffer. i_channel is the ID of the subpicture channel, i_type is DVD_SUBPICTURE or TEXT_SUBPICTURE, i_size is the length in bytes of the packet.
*vout_DisplaySubPicture ( vout_thread_t *p_vout, subpicture_t *p_subpic ) : Tells the video output that a subpicture has been completely decoded. It obsoletes the previous subpicture.
*vout_DestroySubPicture ( vout_thread_t *p_vout, subpicture_t *p_subpic ) : Marks the subpicture as empty.

==How to write a video output plug-in==

A video output takes care of the system calls to display the pictures and manage the output window. For the most minimalistic framework, look at vmem.c, which just "renders" to dummy internal memory. From there, go on to directfb.c and svgalib.c, and finally go for x11 and windows.

Video outputs reside in the modules/video_output/ directory. modules/video_output/x11/xcommon.c gives example of functions you must write :
* int Activate ( vlc_object_t *p_this )
** As for any VLC module, this function verifies if the module suits VLC core needs, by interrogating p_this structure.
** In case of success, Activate sets up callback functions needed to run the video output (needed functions explained below), and returns VLC_SUCCESS
** In case of failure, Activate frees memory and returns an error code (VLC_EGENERIC, VLC_ENOMEM, ...)
* void Deactivate ( vlc_object_t *p_this )
** Called to destroy the module: Deactivate needs to free all allocated memory and destroys cleanly the video output thread

The callback functions you need to implement are (callback name, function prototype) :
* (p_vout->pf_init, static int InitVideo ( vout_thread_t * );)
** Called at the beginning of the output thread and each time the window is resized, it creates a buffer of output pictures
* (p_vout->pf_end, static void EndVideo ( vout_thread_t * );)
** Destroys output pictures created by InitVideo
* (p_vout->pf_manage, static int ManageVideo ( vout_thread_t * );)
** Manage X11 events, and handle window resizing
* (p_vout->pf_render, static void RenderVideo ( vout_thread_t *, picture_t * );)
** Optional (set p_vout->pf_render to NULL if not implemented); Used when pictures from the decoder need to be rendered before display
* (p_vout->pf_display, static void DisplayVideo ( vout_thread_t *, picture_t * );)
** Does the actual display of pictures onto screen
* (p_vout->pf_control, static int Control ( vout_thread_t *, int, va_list );)
** Control facility for the video output

==How to write a YUV plug-in==

Look at the C source plugins/yuv/transforms_yuv.c. You need to redefine just the same transformations. Basically, it is a matrix multiply operation. Good luck.

Hacker Guide/Video Output

2009-08-10T14:29:00Z

Sterops: /* How to write a video output plug-in */

=The video output layer=

==Data structures and main loop==

Important data structures are defined in include/video.h and include/video_output.h. The main data structure is picture_t, which describes everything a video decoder thread needs. Please refer to this file for more information. Typically, p_data will be a pointer to YUV planar picture.

Note also the subpicture_t structure. In fact the VLC SPU decoder only parses the SPU header, and converts the SPU graphical data to an internal format which can be rendered much faster. So a part of the "real" SPU decoder lies in src/video_output/video_spu.c.

The vout_thread_t structure is much more complex, but you needn't understand everything. Basically the video output thread manages a heap of pictures and subpictures (5 by default). Every picture has a status (displayed, destroyed, empty...) and eventually a presentation time. The main job of the video output is an infinite loop to : [this is subject to change in the near future]

#Find the next picture to display in the heap.
#Find the current subpicture to display.
#Render the picture (if the video output plug-in doesn't support YUV overlay). Rendering will call an optimized YUV plug-in, which will also do the scaling, add subtitles and an optional picture information field.
#Sleep until the specified date.
#Display the picture (plug-in function). For outputs which display RGB data, it is often accomplished with a buffer switching. p_vout->p_buffer is an array of two buffers where the YUV transform takes place, and p_vout->i_buffer_index indicates the currently displayed buffer.
#Manage events.

==Methods used by video decoders==

The video output exports a bunch of functions so that decoders can send their decoded data. The most important function is vout_CreatePicture which allocates the picture buffer to the size indicated by the video decoder. It then just needs to feed (void *) p_picture->p_data with the decoded data, and call vout_DisplayPicture and vout_DatePicture upon necessary.

*picture_t * vout_CreatePicture ( vout_thread_t *p_vout, int i_type, int i_width, int i_height ) : Returns an allocated picture buffer. i_type will be for instance YUV_420_PICTURE, and i_width and i_height are in pixels.

=Warning=

If no picture is available in the heap, vout_CreatePicture will return NULL.

*vout_LinkPicture ( vout_thread_t *p_vout, picture_t *p_pic ) : Increases the refcount of the picture, so that it doesn't get accidently freed while the decoder still needs it. For instance, an I or P picture can still be needed after displaying to decode interleaved B pictures.
*vout_UnlinkPicture ( vout_thread_t *p_vout, picture_t *p_pic ) : Decreases the refcount of the picture. An unlink must be done for every link previously made.
*vout_DatePicture ( vout_thread_t *p_vout, picture_t *p_pic ) : Gives the picture a presentation date. You can start working on a picture before knowing precisely at what time it will be displayed. For instance to date an I or P picture, you must wait until you have decoded all previous B pictures (which are indeed placed after - decoding order != presentation order).
*vout_DisplayPicture ( vout_thread_t *p_vout, picture_t *p_pic ) : Tells the video output that a picture has been completely decoded and is ready to be rendered. It can be called before or after vout_DatePicture.
*vout_DestroyPicture ( vout_thread_t *p_vout, picture_t *p_pic ) : Marks the picture as empty (useful in case of a stream parsing error).
*subpicture_t * vout_CreateSubPicture ( vout_thread_t *p_vout, int i_channel, int i_type ) : Returns an allocated subpicture buffer. i_channel is the ID of the subpicture channel, i_type is DVD_SUBPICTURE or TEXT_SUBPICTURE, i_size is the length in bytes of the packet.
*vout_DisplaySubPicture ( vout_thread_t *p_vout, subpicture_t *p_subpic ) : Tells the video output that a subpicture has been completely decoded. It obsoletes the previous subpicture.
*vout_DestroySubPicture ( vout_thread_t *p_vout, subpicture_t *p_subpic ) : Marks the subpicture as empty.

==How to write a video output plug-in==

A video output takes care of the system calls to display the pictures and manage the output window. For the most minimalistic framework, look at vmem.c, which just "renders" to dummy internal memory. From there, go on to directfb.c and svgalib.c, and finally go for x11 and windows.

Video outputs reside in the modules/video_output/ directory. modules/video_output/x11/xcommon.c gives example of functions you must write :
# int Activate ( vlc_object_t *p_this )
## As for any VLC module, this function verifies if the module suits VLC core needs, by interrogating p_this structure.
## In case of success, Activate sets up callback functions needed to run the video output (needed functions explained below), and returns VLC_SUCCESS
## In case of failure, Activate frees memory and returns an error code (VLC_EGENERIC, VLC_ENOMEM, ...)
# void Deactivate ( vlc_object_t *p_this )
## Called to destroy the module: Deactivate needs to free all allocated memory and destroys cleanly the video output thread

The callback functions you need to implement are (callback name, function prototype) :
# (p_vout->pf_init, static int InitVideo ( vout_thread_t * );)
## Called at the beginning of the output thread and each time the window is resized, it creates a buffer of output pictures
# (p_vout->pf_end, static void EndVideo ( vout_thread_t * );)
## Destroys output pictures created by InitVideo
# (p_vout->pf_manage, static int ManageVideo ( vout_thread_t * );)
## Manage X11 events, and handle window resizing
# (p_vout->pf_render, static void RenderVideo ( vout_thread_t *, picture_t * );)
## Optional (set p_vout->pf_render to NULL if not implemented); Used when pictures from the decoder need to be rendered before display
# (p_vout->pf_display, static void DisplayVideo ( vout_thread_t *, picture_t * );)
## Does the actual display of pictures onto screen
# (p_vout->pf_control, static int Control ( vout_thread_t *, int, va_list );)
## Control facility for the video output

==How to write a YUV plug-in==

Look at the C source plugins/yuv/transforms_yuv.c. You need to redefine just the same transformations. Basically, it is a matrix multiply operation. Good luck.

SoC 2009 projects

2009-05-31T09:47:10Z

Sterops: GPU decoding

Here is a list of the accepted projects for Google Summer of Code 2OO9.

= List of current VLC projects =

== QuickTimeVR Playback ==

* '''Project page: [[SoC 2009/QuickTimeVR Playback]]'''
* '''Student''': [[User:Leonox|León Moctezuma]]
* '''Mentor''': [[User:Dionoea|Antoine Cellerier]]
* '''Comentor''': Yuval Levy
* '''Abstract''':

: VLC is a cross platform media player widely used and FreePV is a panoramic viewer that was initially developed due to the lack of an implementation of QuickTime and Shockwave for Linux, both work with QuickTime files and the problem arise from that, since video and vr share the mime type, to solve this, the main goal is to integrate the FreePV library into VLC to enable it to play QuickTimeVR contents, the second goal is to explore the use of wiimote as an advance interaction method.

==Media Library==

* '''Project page: [[SoC 2009/Media Library]]'''
* '''Student''': [[User:jetru|Srikanth Raju]]
* '''Mentor''': [[User:J-Peg|Jean-Philippe Andre]]
* '''Abstract''':

: The project is about extending the Media Library(ML) for VLC Player. The media library will allow users to manage all their local and network media. The project will use the basic structure that already exists for the ML and will focus on extending the features. New features include Search, Smart playlists, Annotations and "Just play music".

==Lua Services Discovery==

* '''Project page: [[SoC 2009/Lua Services Discovery]]'''
* '''Student''': [[User:Sephiroth87|Fabio Ritrovato]]
* '''Mentor''': [[User:J-Peg|Jean-Philippe Andre]]
* '''Abstract''':

:The main goal of this project will be giving VideoLAN a Lua framework to load scriptable services discovery, to make them easier to write, if they don't require C complexity.
:Second goal will be giving VideoLAN access to a great number of online music and video services, like Jamendo, Magnatune, youtube, Dailymotion, etc..., using the new Lua modules.

==A Media Center Interface for VLC==

* '''Project page: [[SoC 2009/Media Center Interface for VLC]]'''
* '''Student''': [[User:barrywardell|Barry Wardell]]
* '''Mentor''': [[User:ILEoo|Ilkka Ollakka]]
* '''Abstract''':

:While it is easy to use and functional, the existing default, Qt based GUI for VLC is quite minimal and basic. I propose to develop a new "Media Center" style GUI. This GUI will be based on Qt/OpenGL and will aim to be aesthetically pleasing, while remaining easy to use and functional. It will draw on many nice features of other popular media center software such as Front Row, Windows Media Center and XBMC.

==Qt Interface Improvements==

* '''Project page: [[SoC 2009/Qt Interface Improvements]]'''
* '''Student''': [[User:cyril7|Cyril Nikolaev]]
* '''Mentor''': [[User:J-b|Jean-Baptiste Kempf]]
* '''Abstract''':

:Though existing Qt interface offers the user extensive control over VLC functionality, there is much to do from a usability point of view. The project aims to streamline the interface, deliver it from pointless elements and integrate neatly the uncoordinated dilog boxes (like Adjustments or Playlist) into main interface. Reorganization of the interface would reduce user efforts (and mouse click) needed to access desired feature.

==DXVA integration==

* '''Project page: [[SoC 2009/DXVA integration]]'''
* '''Student''': [[User:Geal|Geoffroy Couprie]]
* '''Mentor''': Jérôme Decoodt
* '''Abstract''':

:The Windows DXVA API gives access to video decoding and processing in the graphic card. Decoding with the GPU will reduce the load on the CPU and (hopefully) speed up the decoding. This project aims at creating a MPEG2 and H.264 decoder for VLC using this API.

==RTMP Flash Streaming==

* '''Project page: [[SoC 2009/RTMP Flash Streaming]]'''
* '''Student''': [[User:zhigang|Zhigang Wang]]
* '''Mentor''': [[User:Courmisch|Remi Denis-Courmont]]
* '''Abstract''':

:This project is about to implement the RTMP Streaming function for VLC media player. It includes RTMP input and output parts(Fix bugs in RTMP input module and total rewrite the RTMP output module). while Adobe is about to publish the RTMP protocol spec in the first half of this year, It's the time to do this job.

== Audio Filters in lua ==

* '''Project page: [[SoC_2009/Audio_Filter_Improvement]]'''
* '''Student''': Xiang Wang
* '''Mentor''': [[User:Dionoea|Antoine Cellerier]]
* '''Abstract''':

:Lua script language has been used successfully to VLC.First I'll try to create scriptable new audio filters in lua and enable users to create whatever audio filtering function they want in a Lua script. The second task is to implement new algorithm for 3D effects of VLC filters.

== GPU Decoding ==

* '''Project page: [[SoC_2009/GPU_Decoding_Support]]'''
* '''Student''': [[User:Sterops|Etienne Membrives]]
* '''Mentor''': [[User:jpsaman|Jean-Paul Saman]]
* '''Abstract''':

:The Video Acceleration API is an API designed to provide hardware acceleration for graphic application using the processing power of graphic cards. Drivers exists for Intel graphic cards. This project aims to implement a decoder and the corresponding video output for VLC, using the power of VA API to reduce CPU consumption while reading h.264, VC-1 and MPEG-4 videos.

= List of current x264 projects =
...

= List of current VLMC projects =
==Enhancing VLMC==

* '''Project page: [[SoC 2009/Enhancing VLMC]]'''
* '''Student''': [[User:chouquette|Hugo Beauzee-Luyssen]]
* '''Mentor''': Ludovic 'etix' Fauvet
* '''Abstract''':

:The aim of this student project is to provide a functionnal video generation workflow for the video editing tool 'VLMC'. It includes an effect and a transition API, in order to allow developpers to create new modules for this application. This project will be developped closely with VLMC team, so they can add required features, such as the timeline, a previsualisation widget, and many others...

[[Category:SoC_2009_Project]][[Category:SoC]]

SoC 2009/GPU Decoding Support

2009-05-18T14:39:20Z

Sterops: Git repository

{{SoCProject|year=2009|student=[[User:Sterops|Etienne Membrives]]|mentor=[[User:jpsaman|Jean-Paul Saman]]}}

== Project Abstract ==
The Video Acceleration API is an API designed to provide hardware acceleration for graphic application using the processing power of graphic cards. Drivers exists for Intel graphic cards. A backend for the VDPAU API (by nVidia) has been released and the same is under way for ATI cards.
Thus, this project aims to implement a decoder and the corresponding video output for VLC, using the power of VA API to reduce CPU consumption while reading h.264, VC-1 and MPEG-4 videos.

== Timeline ==
* '''March, 23''': Official start date
* '''June, 5''': End of classes, I will be fully available for GSoC from this date
* '''June, 8''': (''Report 0'') Report on already existing solutions (other softwares)
* '''June, 15''': (''Report 1'')
* '''June, 22''': (''Report 2'') Functional VAAPI Decoder
* '''June, 29''': (''Report 3'')
* '''July, 5''': (''Report 4'') Functional VAAPI Video Output
* '''July, 12''': (''Report 5'') Mid-term evaluations
* '''July, 19''': (''Report 6'') Integration of video filters
* '''July, 26''': (''Report 7'') Debugging
* '''August, 3''': (''Report 8'') Documentation
* '''August, 10''': «Pencil down» date

== GIT Repository ==
* GitWeb : [http://membrives.fr/cgi-bin/gitweb.cgi?p=vlc/.git]

SoC 2009/VAAPI Support

2009-05-16T20:32:01Z

Sterops: SoC 2009/VAAPI Support moved to SoC 2009/GPU Decoding Support: More general subject for the project

#REDIRECT [[SoC 2009/GPU Decoding Support]]

SoC 2009/GPU Decoding Support

2009-05-16T20:32:01Z

Sterops: SoC 2009/VAAPI Support moved to SoC 2009/GPU Decoding Support: More general subject for the project

SoC 2009/GPU Decoding Support

2009-05-16T20:03:32Z

Sterops: Timeline

SoC 2009/GPU Decoding Support

2009-05-16T19:07:22Z

Sterops: Initial page