Hacker Guide/Video Output

The video output layer

Data structures and main loop

Important data structures are defined in include/video.h and include/video_output.h. The main data structure is picture_t, which describes everything a video decoder thread needs. Please refer to this file for more information. Typically, p_data will be a pointer to YUV planar picture.

Note also the subpicture_t structure. In fact the VLC SPU decoder only parses the SPU header, and converts the SPU graphical data to an internal format which can be rendered much faster. So a part of the "real" SPU decoder lies in src/video_output/video_spu.c.

The vout_thread_t structure is much more complex, but you needn't understand everything. Basically the video output thread manages a heap of pictures and subpictures (5 by default). Every picture has a status (displayed, destroyed, empty...) and eventually a presentation time. The main job of the video output is an infinite loop to : [this is subject to change in the near future]

1. Find the next picture to display in the heap.
2. Find the current subpicture to display.
3. Render the picture (if the video output plug-in doesn't support YUV overlay). Rendering will call an optimized YUV plug-in, which will also do the scaling, add subtitles and an optional picture information field.
4. Sleep until the specified date.
5. Display the picture (plug-in function). For outputs which display RGB data, it is often accomplished with a buffer switching. p_vout->p_buffer is an array of two buffers where the YUV transform takes place, and p_vout->i_buffer_index indicates the currently displayed buffer.
6. Manage events.

Methods used by video decoders

The video output exports a bunch of functions so that decoders can send their decoded data. The most important function is vout_CreatePicture which allocates the picture buffer to the size indicated by the video decoder. It then just needs to feed (void *) p_picture->p_data with the decoded data, and call vout_DisplayPicture and vout_DatePicture upon necessary.

• picture_t * vout_CreatePicture ( vout_thread_t *p_vout, int i_type, int i_width, int i_height ) : Returns an allocated picture buffer. i_type will be for instance YUV_420_PICTURE, and i_width and i_height are in pixels.

Warning

If no picture is available in the heap, vout_CreatePicture will return NULL.

• vout_LinkPicture ( vout_thread_t *p_vout, picture_t *p_pic ) : Increases the refcount of the picture, so that it doesn't get accidently freed while the decoder still needs it. For instance, an I or P picture can still be needed after displaying to decode interleaved B pictures.
• vout_UnlinkPicture ( vout_thread_t *p_vout, picture_t *p_pic ) : Decreases the refcount of the picture. An unlink must be done for every link previously made.
• vout_DatePicture ( vout_thread_t *p_vout, picture_t *p_pic ) : Gives the picture a presentation date. You can start working on a picture before knowing precisely at what time it will be displayed. For instance to date an I or P picture, you must wait until you have decoded all previous B pictures (which are indeed placed after - decoding order != presentation order).
• vout_DisplayPicture ( vout_thread_t *p_vout, picture_t *p_pic ) : Tells the video output that a picture has been completely decoded and is ready to be rendered. It can be called before or after vout_DatePicture.
• vout_DestroyPicture ( vout_thread_t *p_vout, picture_t *p_pic ) : Marks the picture as empty (useful in case of a stream parsing error).
• subpicture_t * vout_CreateSubPicture ( vout_thread_t *p_vout, int i_channel, int i_type ) : Returns an allocated subpicture buffer. i_channel is the ID of the subpicture channel, i_type is DVD_SUBPICTURE or TEXT_SUBPICTURE, i_size is the length in bytes of the packet.
• vout_DisplaySubPicture ( vout_thread_t *p_vout, subpicture_t *p_subpic ) : Tells the video output that a subpicture has been completely decoded. It obsoletes the previous subpicture.
• vout_DestroySubPicture ( vout_thread_t *p_vout, subpicture_t *p_subpic ) : Marks the subpicture as empty.

How to write a video output plug-in

A video output takes care of the system calls to display the pictures and manage the output window. For the most minimalistic framework, look at vmem.c, which just "renders" to dummy internal memory. From there, go on to directfb.c and svgalib.c, and finally go for x11 and windows.

Have a look at plugins/x11/vout_x11.c. [OUTDATED?] You must write the following functions :

1. int vout_Probe ( probedata_t *p_data ) : Returns a score between 0 and 999 to indicate whether it can run on the architecture. 999 is the best. p_data is currently unused.
2. int vout_Create ( vout_thread_t *p_vout ) : Basically, initializes and opens a new window. Returns TRUE if it failed.
3. int vout_Init ( vout_thread_t *p_vout ) : Creates optional picture buffers (for instance ximages or xvimages). Returns TRUE if it failed.
4. vout_End ( vout_thread_t *p_vout ) : Frees optional picture buffers.
5. vout_Destroy ( vout_thread_t *p_vout ) : Unmaps the window and frees all allocated resources.
6. int vout_Manage ( vout_thread_t *p_vout ) : Manages events (including for instance resize events).
7. vout_Display ( vout_thread_t *p_vout ) : Displays a previously rendered buffer.
8. vout_SetPalette ( vout_thread_t *p_vout, u16 *red, u16 *green, u16 *blue, u16 *transp ) : Sets the 8 bpp palette. red, green and blue are arrays of 256 unsigned shorts.

How to write a YUV plug-in

Look at the C source plugins/yuv/transforms_yuv.c. You need to redefine just the same transformations. Basically, it is a matrix multiply operation. Good luck.