This is the [[Hacker_Guide|Hacker's Guide]] for video filters ('''video filter2''', VLC 1.2).

== Location in source tree, compilation ==

The right place for video filters is in the {{VLCSourceFile|name=modules/video_filter/}} folder.

See {{VLCSourceFile|name=modules/video_filter/Modules.am}} for a list of video filters currently in VLC, and which source files belong to which.

See [[Documentation:Hacker's_Guide/How_To_Write_a_Module#Git_and_Repository_integration|new module integration]] for additional help on how to integrate a new module in the build system.

== Additional documentation ==

Reading [[Documentation:Hacker's_Guide/How_To_Write_a_Module|How to write a module]] first is highly advised.

Some ''video filters'' may have their own [[Documentation:Hacker's Guide|Hacker's Guide]] sections.

Currently, the '''deinterlace''' filter has both a [[Documentation:Hacker's Guide/Video Filters/Deinterlace|Hacker's Guide section]] and [[Deinterlace|user documentation]] with some useful additional technical information.

== Programming conventions and tips ==

{{VLC}} [[video filters]] are usually written in C99, like the rest of VLC. Some use C++, like the AtmoLight plugin.

The usual stuff holds here too:
* Use '''assert()''' for checking conditions that, in presence of no bugs, always hold.
** ''Don't'' use '''assert()''' for checking stuff that may fail at runtime. Handle errors gracefully.
* Keep your code readable. Make private functions when it simplifies things.
* Prefer 80 characters per line at maximum, if possible.

'''Particularly''' for video filters: keep in mind that this is '''low-level stuff''' and needs to be '''fast'''.
* How fast? Note that the source may be interlaced, and someone may have thrown a framerate doubler into the filter chain... this means that for a 60i signal, the entire processing chain from decode to render has about 16ms per output frame.
* Keep the required memory bandwidth down.
** Don't copy whole pictures unless you absolutely must. Moving pixels costs time.
** Allocate an output picture and render directly into it.
** If you need more than one pass, do in-place processing in the subsequent ones if possible.
** If your processing cannot avoid multiple passes, consider doing all passes for a picture line (or a few) at a time, instead of iterating over the whole picture separately for each pass. This will help to keep down the pressure on the CPU cache (since a few picture lines may fit in the L2 cache).
* Consider writing the inner processing loops in vectorized inline assembly, such as MMX. This can often gain a factor of 2-8x in speed, depending on how much and what kind of processing your algorithm needs.

== Some notes on video filter API ==

'''Note''': See {{VLCSourceFile|name=modules/video_filter/deinterlace/deinterlace.c}} and {{VLCSourceFile|name=modules/video_filter/deinterlace/deinterlace.h}} for an example of everything discussed in this section. See also the [[Documentation:Hacker's Guide/Video Filters/Deinterlace|Deinterlace Hacker's Guide]] for some further details.

The video filters use an object-based model. Calls to the filter will come with an instance pointer, so each filter instance can keep local data.

Usually, each filter defines its own internal data structure called '''filter_sys_t'''. It is completely private to the filter.

The API for video filters in VLC 1.2 is called '''video filter2'''. This requires the functions '''Open()''' and '''Close()'''.

The '''Open()''' function is expected to allocate private data (if any), and set up the filter structure with pointers to a frame processing function ('''p_filter->pf_video_filter''', which is the actual filter), a flush function ('''p_filter->pf_video_flush'''), and a mouse event function ('''p_filter->pf_video_mouse'''). The relevant types are defined in {{VLCSourceFile|name=include/vlc_filter.h}}.

The '''Close()''' function should deallocate private data.

See [[Documentation:Hacker's_Guide/How_To_Write_a_Module|How to write a module]] for details about these functions in modules in general.

The mouse function should convert mouse events, from filter output format to filter input format (note the direction!). This is needed when doing scaling or geometrical distortions, so that the mouse position will map correctly. See {{VLCSourceFile|name=include/vlc_filter.h}} for the definition and '''Mouse()''' in {{VLCSourceFile|name=modules/video_filter/deinterlace/deinterlace.c}} for an example. If you do not need to remap mouse events, you can leave '''p_filter->pf_video_mouse''' as '''NULL'''.

=== Filter work (frame processing function) ===
The frame processing function takes a frame as input ('''picture_t*''', {{VLCSourceFile|name=include/vlc_picture.h}}), and optionally outputs one or more frames ('''picture_t*''') as a linked list. The usual case is one frame in, one frame out, but this is not a requirement. (See framerate doublers and IVTC in the [[deinterlace]] module for counterexamples.)

If you do not wish to output a frame at a particular call to your processing function, you can return '''NULL'''. If you wish to output several, make a linked list by using the '''p_next''' member of '''picture_t'''.

You don't have to necessarily output the ''same'' frame (after processing) that just came in. You can keep a private input frame history in your '''filter_sys_t''', pick the base frame for your output from that, and adjust the PTSs (presentation timestamps, member '''date''' in '''picture_t''') of your output as you deem necessary. The deinterlacer provides an example of how to do this.

Note however that there may be other filters in the chain that already set up a frame offset; this leaves less time (until the designated PTS) for the rest of the filters to do their processing.

The input picture coming in to the processing function is considered as owned by the processing function, and should be released before returning. To do this, use
<pre>picture_Release( p_inpic );</pre>

=== Frame allocation ===
Be sure to allocate your output pictures using <pre>picture_t *p_outpic = filter_NewPicture( p_filter );</pre>
This function requests a new picture from the private pool (see {{VLCSourceFile|name=src/video_output/vout_wrapper.c}}). This is very important! Be aware that pictures created like
<pre>picture_t *p_temp = picture_NewFromFormat( &p_input_pic->format );</pre>
lack a shared memory context (in Linux; or the equivalent in other OS), and thus cannot be passed to the video output logic. If your filter attempts to do so, VLC will crash.

If you need temporary pictures, you can allocate those using '''picture_NewFromFormat()''', but remember to ''always create output pictures using '''filter_NewPicture()'''''. Here an ''output picture'' is defined as a picture which goes out from the filter to the caller.

There is a limit to the number of simultaneous picture slots available in the private pool. Currently (<nowiki>[71992c5fbc75ee2c94cd2a3ab1aaca70bfc688a9]</nowiki>) this is 3 pictures for the filter layer. See the constant '''private_picture''' in {{VLCSourceFile|name=src/video_output/vout_wrapper.c}}. If you try to allocate more output pictures than this (during a single processing call), the allocation will fail.

=== Notes ===
* In VLC 1.2, the video format (resolution, chroma) never changes on the fly. The whole filter chain is '''Close()'''d and then '''Open()'''ed again if the format changes.

* Note that it ''is'' allowed to flush the filter without closing it. This actually happens under some circumstances, so ''don't deallocate stuff in your flush function''. Rather, allocate dynamic resources in '''Open()''', and deallocate them in '''Close()''' (or in private functions called from those). See [[Documentation:Hacker's_Guide/How_To_Write_a_Module|How to write a module]].

* Supported input and output formats are decided by each filter. The usual thing is to support [[YUV]] formats, e.g. [[I420]], [[J420]], [[YV12]], [[I422]] and [[J422]]. See {{VLCSourceFile|name=include/vlc_fourcc.h}}. The most important ones to support are [[I420]] and [[I422]].

== Pitch, visible pitch, planes et al. ==

=== Pitch and visible pitch ===
Important to know:

* i_pitch = number of (macro)pixels on one line
* i_visible_pitch = number of (macro)pixels on one line, adjusted for memory alignment constraints etc. (see below)

Note that the pitches may be different for each plane, and indeed in YUV formats the luma ('''Y_PLANE''') and chroma ('''U_PLANE''', '''V_PLANE''') have different pitches due to chroma subsampling. (In 4:2:0 formats, the number of lines differs, too.)

Pitches also tend to slightly differ depending on how the picture was allocated, even if the visible size is the same. The input picture to the filter may have one pitch, temporary pictures ('''picture_NewFromFormat()''') another, and output pictures ('''filter_NewPicture()''') yet another.

''Be sure to always use the correct pitch when you handle the pixels of a picture.'' That is, always use the '''i_pitch''' member of the actual '''plane_t''' you are working on.

If you absolutely need matching pitches (e.g. if you are glueing in processing code from another GPL-compatible project which assumes this and you don't want to rewrite it...), consider making temporary copies with '''picture_NewFromFormat()'''. See {{VLCSourceFile|name=modules/video_filter/deinterlace/deinterlace.c}} (the history mechanism <nowiki>pp_history[]</nowiki>) and {{VLCSourceFile|name=modules/video_filter/deinterlace/algo_yadif.c}} (its usage) for an example.

If you are processing pixels from one picture to another, the safe thing to do is to take the smallest '''i_visible_pitch''', and loop from x = 0 until the visible pitch has been reached, but use the individual '''i_pitch'''es for computing the pixel locations. See {{VLCSourceFile|name=modules/video_filter/deinterlace/helpers.c}} for examples.

=== Video_format_t vs. Plane_t ===

In '''video_format_t''', the '''i_visible_width''' and '''i_visible_height''' members go with '''i_x_offset''' and '''i_y_offset''', but they are not related to '''plane_t::i_visible_*'''.

The '''plane_t::i_visible_*''' fields are there to know which part of the memory planes can at most be displayed. Usually these are different from the whole surface due to memory alignment constraints (like mod 16 for SSE2 or to be aligned on a macroblock). Those values are decided when the picture is allocated and they usually don't change for a given picture pointer (but they can change, like in direct3d buffers).

The '''video_format_t:i_visible_width/height''' and '''i_x/y_offset''' are more a 'hint' regarding what will be actually displayed. A filter should update '''filter_t::fmt_out''' if needed to ensure that the allocated pictures get the right value. A filter should, however, not limit its processing to this area, because VLC supports dynamic cropping.

Thus, the correct thing to do in a filter is to process the whole picture area, or at least the first i_visible_* lines and pixels ''according to the '''plane_t'''s''. Filters do not need to care about margin positioning or that there even exists such a thing as margins.

== Disclaimer ==

The information is mostly based on a few months' hacking on the deinterlacer.

This page is still a stub. Please add new relevant stuff.

Also, the stuff that already exists could use better organization and formatting.

{{Documentation}}

Hacker Guide/Video Filters

2012-01-27T10:57:49Z

Technologicat: Hacker's guide link update

This is the [[Hacker_Guide|Hacker's Guide]] for video filters ('''video filter2''', VLC 1.2).

== Location in source tree, compilation ==

The right place for video filter is in the {{VLCSourceFile|name=modules/video_filter/}} folder.

See {{VLCSourceFile|name=modules/video_filter/Modules.am}} for a list of video filters currently in VLC, and which source files belong to which.

See [[Documentation:Hacker's_Guide/How_To_Write_a_Module#Git_and_Repository_integration|new module integration]] for additional help on how to integrate a new module in the build-system.

== Additional documentation ==

Reading [[Documentation:Hacker's_Guide/How_To_Write_a_Module|How to write a module]] before is very advised.

Some ''video filters'' may have their own [[Documentation:Hacker's Guide|Hacker's Guide]] sections.

Currently, the '''deinterlace''' filter has both a [[Documentation:Hacker's Guide/Video Filters/Deinterlace|Hacker's Guide section]] and [[Deinterlace|user documentation]] with some useful additional technical information.

== Programming conventions and tips ==

{{VLC}} [[video filters]] are usually written in C99, like the rest of VLC. Some use C++, like the AtmoLight plugin.

The usual stuff holds here too:
* Use '''assert()''' for checking conditions that, in presence of no bugs, always hold.
** ''Don't'' use '''assert()''' for checking stuff that may fail at runtime. Handle errors gracefully.
* Keep your code readable. Make private functions when it simplifies things.
* Prefer 80 characters per line at maximum, if possible.

'''Particularly''' for video filters: keep in mind that this is '''low-level stuff''' and needs to be '''fast'''.
* How fast? Note that the source may be interlaced, and someone may have thrown a framerate doubler into the filter chain... this means that for a 60i signal, the entire processing chain from decode to render has about 16ms per output frame.
* Keep the required memory bandwidth down.
** Don't copy whole pictures unless you absolutely must. Moving pixels costs time.
** Allocate an output picture and render directly into it.
** If you need more than one pass, do in-place processing in the subsequent ones if possible.
** If your processing cannot avoid multiple passes, consider doing all passes for a picture line (or a few) at a time, instead of iterating over the whole picture separately for each pass. This will help to keep down the pressure on the CPU cache.
* Consider writing the inner processing loops in vectorized inline assembly, such as MMX. This can often gain a factor of 2-8x in speed, depending on how much and what kind of processing your algorithm needs.

== Some notes on video filter API ==

'''Note''': See {{VLCSourceFile|name=modules/video_filter/deinterlace/deinterlace.c}} and {{VLCSourceFile|name=modules/video_filter/deinterlace/deinterlace.h}} for an example of everything discussed in this section. See also the [[Documentation:Hacker's Guide/Video Filters/Deinterlace|Deinterlace Hacker's Guide]] for some further details.

The video filters use an object-based model. Calls to the filter will come with an instance pointer, so each filter instance can keep local data.

Usually, each filter defines its own internal data structure called '''filter_sys_t'''. It is completely private to the filter.

The API for video filters in VLC 1.2 is called '''video filter2'''. This requires the functions '''Open()''' and '''Close()'''.

The '''Open()''' function is expected to allocate private data (if any), and set up the filter structure with pointers to a frame processing function ('''p_filter->pf_video_filter''', which is the actual filter), a flush function ('''p_filter->pf_video_flush'''), and a mouse event function ('''p_filter->pf_video_mouse'''). The relevant types are defined in {{VLCSourceFile|name=include/vlc_filter.h}}.

The '''Close()''' function should deallocate private data.

The mouse function should convert mouse events, from filter output format to filter input format (note the direction!). This is needed when doing scaling or geometrical distortions, so that the mouse position will map correctly. See {{VLCSourceFile|name=include/vlc_filter.h}} for the definition and '''Mouse()''' in {{VLCSourceFile|name=modules/video_filter/deinterlace/deinterlace.c}} for an example. If you do not need to remap mouse events, you can leave '''p_filter->pf_video_mouse''' as '''NULL'''.

=== Filter work (frame processing function) ===
The frame processing function takes a frame as input ('''picture_t*''', {{VLCSourceFile|name=include/vlc_picture.h}}), and optionally outputs one or more frames ('''picture_t*''') as a linked list. The usual case is one frame in, one frame out, but this is not a requirement. (See framerate doublers and IVTC in the [[deinterlace]] module for counterexamples.)

If you do not wish to output a frame at a particular call to your processing function, you can return '''NULL'''. If you wish to output several, make a linked list by using the '''p_next''' member of '''picture_t'''.

You don't have to necessarily output the ''same'' frame (after processing) that just came in. You can keep a private input frame history in your '''filter_sys_t''', pick the base frame for your output from that, and adjust the PTSs (presentation timestamps, member '''date''' in '''picture_t''') of your output as you deem necessary. The deinterlacer provides an example of how to do this.

Note however that there may be other filters in the chain that already set up an offset; this leaves less time (until the designated PTS) for the rest of the filters to do their processing.

The input picture coming in to the processing function is considered as owned by the processing function, and should be released before returning. To do this, use
<pre>picture_Release( p_inpic );</pre>

=== Frame allocation ===
Be sure to allocate your output pictures using <pre>picture_t *p_outpic = filter_NewPicture( p_filter );</pre>
This function requests a new picture from the private pool (see {{VLCSourceFile|name=src/video_output/vout_wrapper.c}}). This is very important! Be aware that pictures created like
<pre>picture_t *p_temp = picture_NewFromFormat( &p_input_pic->format );</pre>
lack a shared memory context (in Linux; or the equivalent in other OS), and thus cannot be passed to the video output logic. If your filter attempts to do so, VLC will crash.

If you need temporary pictures, you can allocate those using '''picture_NewFromFormat()''', but remember to ''always create output pictures using '''filter_NewPicture()'''''. Here an ''output picture'' is defined as a picture which goes out from the filter to the caller.

There is a limit to the number of simultaneous picture slots available in the private pool. Currently (<nowiki>[71992c5fbc75ee2c94cd2a3ab1aaca70bfc688a9]</nowiki>) this is 3 pictures for the filter layer. See constant '''private_picture''' in {{VLCSourceFile|name=src/video_output/vout_wrapper.c}}. If you try to allocate more output pictures than this (during a single processing call), the allocation will fail.

=== Notes ===
* In VLC 1.2, the video format (resolution, chroma) never changes on the fly. The whole filter chain is '''Close()'''d and then '''Open()'''ed again if the format changes.

* Note that it ''is'' allowed to flush the filter without closing it. This actually happens under some circumstances, so ''don't deallocate stuff in your flush function''. Rather, allocate dynamic resources in '''Open()''', and deallocate them in '''Close()''' (or in private functions called from those).

* Supported input and output formats are left up to each filter. The usual thing is to support YUV formats, e.g. I420, J420, YV12, I422 and J422. See {{VLCSourceFile|name=include/vlc_fourcc.h}}. The most important ones to support are I420 and I422.

== Pitch, visible pitch, planes et al. ==

=== Pitch and visible pitch ===
Important to know:

* i_pitch = number of (macro)pixels on one line
* i_visible_pitch = number of (macro)pixels on one line, adjusted for memory alignment constraints etc. (see below)

Note that the pitches may be different for each plane, and indeed in YUV formats the luma ('''Y_PLANE''') and chroma ('''U_PLANE''', '''V_PLANE''') have different pitches due to chroma subsampling. (In 4:2:0 formats, the number of lines differs, too.)

Pitches also tend to slightly differ depending on how the picture was allocated, even if the visible size is the same. The input picture to the filter may have one pitch, temporary pictures ('''picture_NewFromFormat()''') another, and output pictures ('''filter_NewPicture()''') yet another.

''Be sure to always use the correct pitch when you handle the pixels of a picture.'' That is, always use the '''i_pitch''' member of the actual '''plane_t''' you are working on.

If you absolutely need matching pitches (e.g. if you are glueing in processing code from another GPL-compatible project which assumes this and you don't want to rewrite it...), consider making temporary copies with '''picture_NewFromFormat()'''. See {{VLCSourceFile|name=modules/video_filter/deinterlace/deinterlace.c}} (the history mechanism <nowiki>pp_history[]</nowiki>) and {{VLCSourceFile|name=modules/video_filter/deinterlace/algo_yadif.c}} (its usage) for an example.

If you are processing pixels from one picture to another, the safe thing to do is to take the smallest '''i_visible_pitch''', and loop from x = 0 until the visible pitch has been reached, but use the individual '''i_pitch'''es for computing the pixel locations. See {{VLCSourceFile|name=modules/video_filter/deinterlace/helpers.c}} for examples.

=== Video_format_t vs. Plane_t ===

In '''video_format_t''', the '''i_visible_width''' and '''i_visible_height''' members go with '''i_x_offset''' and '''i_y_offset''', but they are not related to '''plane_t::i_visible_*'''.

The '''plane_t::i_visible_*''' fields are there to know which part of the memory planes can at most be displayed. Usually these are different from the whole surface due to memory alignment constraints (like mod 16 for SSE2 or to be aligned on a macroblock). Those values are decided when the picture is allocated and they usually don't change for a given picture pointer (but they can change, like in direct3d buffers).

The '''video_format_t:i_visible_width/height''' and '''i_x/y_offset''' are more a 'hint' regarding what will be actually displayed. A filter should update '''filter_t::fmt_out''' if needed to ensure that the allocated pictures get the right value. A filter should, however, not limit its processing to this area, because VLC supports dynamic cropping.

Thus, the correct thing to do in a filter is to process the whole picture area, or at least the first i_visible_* lines and pixels ''according to the '''plane_t'''s''. Filters do not need to care about margin positioning or that there even exists such a thing as margins.

== Disclaimer ==

The information is mostly based on a few months' hacking on the deinterlacer.

This page is still a stub. Please add new relevant stuff.

Also, the stuff that already exists could use better organization and formatting.

{{Documentation}}

User:Technologicat

2012-01-27T10:56:34Z

Technologicat: Added note of video filter hacker guide

Hacker Guide/How To Write a Module

2012-01-27T10:53:14Z

Technologicat: Minor corrections to the English

VLC is based on many independent modules, like most competing multimedia frameworks. Each module introduces a new functionality.

This article focuses on adding a new module (a.k.a. plug-in) to VLC. You '''will need to read''' [[Documentation:Hacker%27s_Guide/Core#VLC_Pipeline_and_Modularity|VLC Core and Modules ]] and [[Documentation:VLC_Modules_Loading|How VLC loads modules]] first, otherwise you will not be able to flesh out the content of your new module.

== In-tree and out-of-tree modules ==

Most existing VLC modules are provided as source code within the main VLC source code repository (and also the source tarballs). They are compiled at the same time as the VLC core, and usually provided distributed together with VLC binary packages and installers. These modules are called in-tree modules.

However, it is also possible to write and compile VLC modules outside of VLC. That has some pros and cons over developing modules in-tree:
===Pros===
* Compilation is a lot faster (VLC and other modules are not included in the process).
* You can use your own version control system, or even none at all.
* The copyright license does not need to abide by the requirements of the [[VideoLAN]] association for inclusion in VLC.
* The source code does not need to be provided, reviewed and accepted by the VLC developers.
* The release schedule is independent of VLC releases. New versions of the module can be published at any time regardless of VLC release planning.
* Different programming languages can be used at least in theory. (The main VLC code base only uses C, C++ and Lua, and on MacOS Objective C.)
* The module can use software libraries that would be inadequate for VLC to depend on.

===Cons===
* The VLC developers will not review the code, which would be a good opportunity fo improve the code quality.
* VLC translators will not take care of localization for the module(s) where applicable. VLC is translated in many tens of languages.
* The module(s) cannot be distributed through the [http://www.videolan.org/ videolan.org] website and use the VideoLAN infrastructure such as the bug tracker and the build bots.
* The module(s) will only work with the particular VLC (major) version that it has been compiled for. For instance, a module compiled for VLC 1.1.x will not work with VLC 1.0.x or VLC 2.0.x.
* The module(s) will only work on the particular operating systems and architecture that it has been compiled for. For instance, a Windows 32-bit module will only work with Windows 32-bit versions of VLC. VLC supports many different combinations of operating systems and architectures.

== Example stub module ==

Let's start with a small example module in the C language:
<pre>/**
* @file hello.c
* @brief Hello world interface VLC module example
*/
#ifdef HAVE_CONFIG_H
# include "config.h"
#endif

#include <stdlib.h>
/* VLC core API headers */
#include <vlc_common.h>
#include <vlc_plugin.h>
#include <vlc_interface.h>

/* Forward declarations */
static int Open(vlc_object_t *);
static void Close(vlc_object_t *);

/* Module descriptor */
vlc_module_begin()
set_shortname(N_("Hello"))
set_description(N_("Hello interface"))
set_capability("interface", 0)
set_callbacks(Open, Close)
set_category(CAT_INTERFACE)
add_string("hello-who", "world", "Target", "Whom to say hello to.", false)
vlc_module_end ()

/* Internal state for an instance of the module */
struct intf_sys_t
{
char *who;
};

/**
* Starts our example interface.
*/
static int Open(vlc_object_t *obj)
{
intf_thread_t *intf = (intf_thread_t *)obj;

/* Allocate internal state */
intf_sys_t *sys = malloc(sizeof (*sys));
if (unlikely(sys == NULL))
return VLC_ENOMEM;
intf->p_sys = sys;

/* Read settings */
char *who = var_InheritString(intf, "hello-who");
if (who == NULL)
{
msg_Err(intf, "Nobody to say hello to!");
goto error;
}
sys->who = who;

msg_Info(intf, "Hello %s!", who);
return VLC_SUCCESS;

error:
free(sys);
return VLC_EGENERIC;
}

/**
* Stops the interface.
*/
static void Close(vlc_object_t *obj)
{
intf_thread_t *intf = (intf_thread_t *)obj;
intf_sys_t *sys = intf->p_sys;

msg_Info(intf, "Good bye %s!");

/* Free internal state */
free(sys->who);
free(sys);
}</pre>

And now some explanations about the code...

=== Module Descriptor ===

A {{VLC}} module '''must''' include a ''description'' of itself, and the ''parameters'' it accepts.

The module descriptor begins with:
<pre>vlc_module_begin();
</pre>
You should set some basic information about your module. This is for the dvdread module:
<pre> set_shortname(N_("DVD without menus"));
set_description(N_("DVDRead Input"));
set_category(CAT_INPUT);
set_subcategory(SUBCAT_INPUT_ACCESS);
</pre>
''Note'' the use of N_("") to create a string that needs to be translated by gettext.

==== Capability and score ====

Definition '''Example''':
<pre> set_capability("interface", 0)
</pre>
This defines a module of "interface" capability and a score of 0.

The capability determines the type of module we are dealing with. It could be an access, a demux, a decoder, an interface, etc. Now is the time to '''re-read [[Documentation:VLC Modules Loading#How_does_the_loading_of_modules_happens|how VLC loads modules]]'''.

*If VLC needs to load a specific name, it will load it by its name and VLC directly opens this module
*If VLC needs a type of module ("I need a decoder"), VLC will load all modules matching this capability in a decreasing score order until one modules's Open() function (see later) returns VLC_SUCCESS.

See the major types of [[Documentation:VLC Modules Loading#Major_Capabilities_of_Modules|capabilities of VLC]].

'''Score''' should be an integer, and related to other scores in the same category. Score 0 is a [[Documentation:VLC_Modules_Loading#Score_of_0|special case]].

==== Configuration categories and sub-categories ====

You should use '''one''' of the '''predefined categories''' for configuration. The configuration categories and sub-categories specify '''where''' the module will appear in the '''preferences''' UI dialog.

The configuration categories include:
*CAT_INTERFACE
*CAT_AUDIO
*CAT_VIDEO
*CAT_INPUT
*CAT_SOUT
*CAT_ADVANCED
*CAT_PLAYLIST

You should use one of '''predefined sub-categories''' as well. See {{VLCSourceFile|name=include/vlc_configuration.h}} for definition of all configuration categories and sub-categories.

==== Configuration parameters ====

You may need options to configure the run-time behavior of your module. Defining new options is easy.

All option definitions take the following argument list:
<pre> add_integer(name, value, text, longtext, advanced) </pre>
*'''name''' is the string that identifies this parameter in the configuration. This name is used at the command prompt to set the configuration value.
*'''value''' is the default value for this parameter,
*'''text''' A short description of the parameter, use _("") to create a string that needs to be translated,
*'''longtext''' A complete description of the parameter, use _("") to create a string that needs to be translated,
*'''advanced''' Boolean, ADVanced Configuration. If TRUE, this parameter will only be displayed when using the --advanced flag.

 You may add the following options/parameter types to your module:
*add_integer,
*add_string,
*add_float,
*add_bool,
*add_key,
*add_file,
*add_directory,

For complete definitions, see {{VLCSourceFile|name=include/vlc_plugin.h}}

==== Callback ====

The activation and deactivation functions, detailed afterwards, must be defined in the descriptor. This is so that the VLC core knows how to instantiate and run the module.

The set_callbacks() macro allows you to define 2 parameters: the first parameter is the pf_activate callback, and the second one, pf_deactivate. The functions are most often called "Open" and "Close" respectively, though. VLC invokes the pf_activate callback if/when it needs a plugin instance providing the correct interface, as declared with the set_capability() macro.

Conversely, VLC invokes the pf_deactivate callback when the plugin is no longer needed - but only if the pf_activate callback returned VLC_SUCCESS (0) earlier.

=== Open(vlc_object_t *) ===

The most important function of a module is the opening: the usually-named Open() function.
<pre>static int Open ( vlc_object_t * );</pre>
The Open() function is called when the VLC core tries to open the module, and wants to load it.

During Open(), setup of structures, devices or I/O, checks should be done. A successful open should return VLC_SUCCESS. If the module cannot complete its initialization, it can return any other value, usually VLC_EGENERIC or VLC_ENOMEM.

The '''Open()''' function is expected to allocate private data (if any), and set up the private structure.

If the Opening fails, you may need to free any already allocated resources before returning. Otherwise, leaks will occur.

=== Close(vlc_object_t *) ===
The second most important function of a module is the closing: the usually-named Close() function.
<pre>static int Close ( vlc_object_t * );</pre>

The Close() function is called when the VLC core tries to close or unload an '''already-loaded''' module.

'''NB:''' If the Open() function failed, Close() will not get called.

During Close(), closing devices or I/O, and cleaning of structures should be done. Do not leak memory here!

The '''Close()''' function should deallocate private data.

== In-tree module integration ==

=== Git ===
If you plan to submit your work to VLC upstream, be sure to look at [[Git |the git introduction]] and check the [[Git#Submitting_patches_to_the_vlc-devel_or_x264-devel|send patches part]].

=== Compiling your module ===

==== Modules.am ====
First, find the right subdirectory under [[Modules_source_tree| modules/]] to add your new code.

# If the module has only one source code file module, simply add it in the subdirectory (e.g. modules/control/hello.c).
# Larger modules should get a sub-subdirectory of their own (e.g. modules/control/hello/*).

Then you need to declare the module in the build system. For example, the file {{VLCSourceFile|name=modules/control/Modules.am}} tells the build system which source files are needed for each control module. For the example above, we could add these lines:

<pre>
libhello_plugin_la_SOURCES = hello.c
libhello_plugin_la_CFLAGS = $(AM_CFLAGS)
libhello_plugin_la_LIBADD = $(AM_LIBADD)
libhello_plugin_la_DEPENDENCIES =
# Always compile the hello module:
libvlc_LTLIBRARIES += libhello_plugin.la
</pre>

Note that indentation in Modules.am (if needed) uses tabulations (ASCII 0x09), not white spaces.

==== configure.ac ====

If the module depends on some new library, some architecture or some operating system characteristics, you may need to extend configure.ac to detect when and how to build the module. Refer to the configure.ac file and the [http://www.gnu.org/software/autoconf/manual/index.html GNU autoconf documentation] for details.

Once this is done, you should only need to rebuild VLC:
make
(This will probably trigger a re-run of autoconf and automake, so it might take a while.)

=== Loading your module ===

VLC keeps a cache of available modules for performance reasons. It should be updated automatically. But you can use ''./vlc --reset-plugins-cache'' to force a reset.

Then use
./vlc -vvv --color --list
to check that your plugin is seen by {{VLC}}.

You should also see it in the plugins dialog of the [[Qt interface]] (Linux and Windows).

== Out-of-tree module ==

There is a dedicated article. Please read [[OutOfTreeCompile|out of tree compilation]].

== Sub-modules ==

Sub-modules, declared in some module descriptors with

add_submodule()

work exactly the same way as modules. They are useful when different modules (usually but not necessarily of different capability) share common code. All sub-modules will be included in the same run-time library as the main module.

== Module types ==

Depending on the module capability, you will need more information, about the necessary functions to implement.

We will detail those here:
* [[Hacker_Guide/Access|Access]]
* [[Hacker_Guide/Demux|Demux]]
* [[Hacker_Guide/Access_Demux|Access_Demux]]
* [[Hacker_Guide/Decoder|Decoder]]
* Interface
* [[Hacker_Guide/Video_Filters|Video filter]]
* [[Hacker_Guide/Audio_Filters|Audio filter]]
* [[Hacker_Guide/Audio_Output|Audio output]]

== Module load troubleshooting ==

Sometimes when build an in-tree module, stuff doesn't work due to build system problems and other inconsistencies.

You probably need to go to the root of your VLC source tree, and do something akin to the following. The examples here assume the ''bash'' shell.

==='''Mild version'''===

In some cases, automake dependencies break (for instance after some filenames have changed). This might then work:

find . -name .deps -exec rm -rf \{\} \;
./config.status
make

...but not always, so it may save some headaches to always use the "medium version" below.

==='''Medium version''' (try this first)===

This is a more radical but still safe rebuild procedure:

find . -name .deps -exec rm -rf \{\} \;
./bootstrap
./configure
make

==='''Extreme version'''===

If the none of the above helped, you can clean the source tree as a measure of last resort.
Before you proceed, it is highly recommended that you check which files are going to be erased:
git clean -nxd

And then check what source code changes you would lose (if any):
git diff

You can extremely easily lose entire days of hard work with the following commands.
The first command will permanently remove any files not tracked in [[git]], including files that you might have created yourself.
The second command will remove any uncommitted modification to existing files. Consider yourself warned.

'''!!!BEWARE: THIS MAY CAUSE UNRECOVERABLE DATA LOSS!!!'''

git clean -fxd
git reset --hard HEAD
./bootstrap
./configure
make

{{Documentation}}
[[Category:Coding]]

User:Technologicat

2012-01-27T10:39:24Z

Technologicat: User page created

My name is Juha Jeronen. Technically, I'm not an official VLC developer, but I subscribe to vlc-devel.

I designed and programmed the IVTC and Phosphor deinterlacers for VLC 2.0 (which was then VLC 1.2), with help from fenrir for the MMX inline assembly. I also refactored the deinterlacer module at that time. I've updated the [[Deinterlacing|deinterlacer]] documentation on the wiki, and written a [[Hacker_Guide/Video_Filters/Deinterlace|hacker's guide]] for the deinterlacer.

As of 2012-01-27, I maintain the Finnish translation.

Hacker Guide/Video Filters/Deinterlace

2012-01-27T10:13:05Z

Technologicat: Added some more links to YUV page. Mainly the new J420, J422 (which now have a description there), but also some missing I420, I422, YV12.

{{Module|name=deinterlace|type=video filter2|description=Removes interlacing from display}}

This is the [[Documentation:Hacker's_Guide|Hacker's Guide]] for the deinterlace module. See also the technical summary in the [[Deinterlace|user documentation]].

== Interfacing with the deinterlacer ==

This section deals with interfacing other parts of {{VLC}} with the deinterlacer. For a description of the internals, skip ahead.

=== How to prepare input ===

The deinterlacer assumes that the input consists of full frames (picture_t), with i_nb_fields == 2 or 3 (depending of repeat_pict flags in the input stream), b_progressive == false, and full vertical resolution (i_visible_lines) of the input stream. The b_top_field_first flag is also optionally used by the deinterlacer, and must be set correctly to guarantee correct operation in all modes.

Each plane is stored linearly, one line at a time, in an uint8_t array of size i_lines*i_pitch, where i_lines and i_pitch can be read off the plane_t for each plane (e.g. <nowiki>p_pic->p[plane_index].i_lines</nowiki>). In YUV formats, the luma and chroma planes have different resolution.

Note that i_lines may differ from i_visible_lines and i_pitch from i_visible_pitch. The value of i_pitch depends on how the picture_t was created. The filter picture pool may, and usually does, have a pitch different from that produced by picture_NewFromFormat(), even though the visible pitches are the same.

It is assumed that the active picture area starts at the upper left edge. The first i_visible_lines lines (0 through i_visible_lines-1, inclusive), make up the visible lines. The rest of the lines (i_visible_lines through i_lines-1, inclusive) are the (invisible) vertical margin.

Pixels at offsets 0 to i_visible_pitch-1 relative to each visible line start are visible. The rest of the pixels, at offsets i_visible_pitch to i_pitch-1, make up the (invisible) horizontal margin.

It is not necessary to initialize the memory in the invisible parts; writing just the visible part of the picture is enough.

Note that this implies that any processing loops only need to iterate through the part from 0 to i_visible_pitch (and i_visible_lines), but the pixel offsets in the array must be computed using i_pitch.

For any plane that has full vertical resolution, the (macro-)pixels are stored as if the frame was progressive, i.e. the fields are interleaved. Line 0 of the plane comes from the top field, line 1 from the bottom field, line 2 from the top field, and so on. This always holds for the luma plane, and in the 4:2:2 chroma formats ([[I422]] and [[J422]]), also for the chroma planes.

If the chroma format is 4:2:0 ([[I420]], [[J420]] or [[YV12]]), both fields share the same chroma. This kind of chroma is stored as-is, at half resolution (both horizontal and vertical) in the U and V planes of the picture_t. The same linear format is used as above (array of size i_lines*i_pitch, first i_visible_lines used, and first i_visible_pitch pixels on each visible line used), but because there is only one "chroma field" in this case, the fields do not alternate. Instead, chroma line 0 corresponds to luma lines 0 and 1, chroma line 1 to luma lines 2 and 3, and so on.

Additionally, if chroma is [[YV12]], the U plane is stored in the array index V_PLANE, and the V plane is stored in the array index U_PLANE. This is because [[YV12]] uses the plane ordering YVU, while the array index constants (see include/vlc_picture.h) are named according to the more common ordering YUV. For all chroma formats using the more common ordering, U_PLANE refers to the U component and V_PLANE to the V component, as expected.

Example:

<pre>p_pic->p[Y_PLANE].p_pixels[23*w+42]</pre>

where <nowiki>w = p_pic->p[Y_PLANE].i_pitch</nowiki>, refers to the pixel on the luma
plane at x = 42, y = 23 (0-based indexing).

If the chroma is [[I420]] or [[J420]],

<pre>p_pic->p[U_PLANE].p_pixels[11*w+21]</pre>

where now <nowiki>w = p_pic->p[U_PLANE].i_pitch</nowiki>, refers to the chroma macropixel
on the U plane that corresponds to the above example. Obviously, 11 =
floor(23/2).

For 4:2:2 chroma ([[I422]] or [[J422]]), only the horizontal chroma resolution is half that of the luma, and the corresponding macropixel is

<pre>p_pic->p[U_PLANE].p_pixels[23*w+21]</pre>

=== How to interpret output ===

The frames output by the deinterlacer are picture_t's, with i_nb_fields == 2 and b_progressive == true. The output chroma format and number of visible lines are algorithm-dependent.

Resolution and chroma conversions are allowed, and it should be assumed there is no particular relation between input and output. Currently possible output chroma formats are [[I420]], [[YV12]], [[J420]], [[I422]] and [[J422]]. Vertical resolution may be original or half. Thus, chroma format and vertical resolution should be read off the output frame metadata (appropriate picture_t data fields).

From the top-level Deinterlace(), the output pictures go into a linked list (using the "next" pointer in picture_t), which is then given to the caller. What happens to the pictures then, is outside the scope of the module.

Temporally, non-doubling deinterlacers produce exactly one output picture per one input, IVTC produces one or zero (at the dropped frame), and framerate doublers produce two or three (depending on repeat_pict of each input frame).

The timings (presentation timestamp; PTS; picture_t data field "date") may change arbitrarily between input and output. It is not even guaranteed that calling the deinterlacer for an input frame outputs a frame corresponding to that input frame. Yadif uses the frame offset feature, and IVTC effectively does, too. This means that when frame "2" goes in, what comes out is deinterlaced frame "1" - in the case of Yadif, still with its original PTS! (The offset does NOT introduce a delay; thus it keeps A/V sync intact. This is why it is called i_frame_offset and not i_frame_delay in the code.) In the case of IVTC, the PTS is somewhat like original, but corrected for the 29.97 > 23.976 fps conversion... usually.

The [[deinterlace]] filter is not required to actually output anything the first few times it is called. Some algorithms keep a history and use it for temporal filtering. Currently, this is 3 input pictures, and the first call to Deinterlace() always outputs a picture. The second call may drop. From the third call on, the history buffer has filled, and at this point it is guaranteed that normal operation starts as defined by the chosen algorithm.

For generality, it should be assumed that M input pictures map to N output pictures, with arbitrary, different M and N. Note that even though repeat_pict means "repeat first field", the first and third output pictures from a framerate doubler, for any given input frame, are allowed to be different due to temporal filtering. (Phosphor does this.) Thus, it should also be assumed that each output picture is unique.

It may or may not be safe to display-hold still images, depending on the deinterlacing algorithm. For non-doublers, this is safe. For doublers, it is not. This is because framerate doubling algorithms are often designed based on the illusion of increased perceived resolution that can be achieved by rapidly alternating half-resolution images in an appropriate way.

Thus, "progressive" pictures from framerate doublers are not actually progressive (at least not at full resolution), but only appear so while the filter is constantly producing new pictures at a steady framerate. Unfortunately, currently there is no way to determine from the outside which kind of algorithm has been chosen.

== File structure ==

This applies to <nowiki>[c7d289cad5c9f73d1bfc5b136b503fc3646b6e41]</nowiki> (1st May 2011) and later.

The deinterlace filter is located in [http://git.videolan.org/?p=vlc.git;f=modules/video_filter/deinterlace;hb=HEAD modules/video_filter/deinterlace/].

* '''<nowiki>modules/video_filter/deinterlace/deinterlace.[ch]</nowiki>'''
** The main files of the filter.
** {{VLCSourceFile|name=modules/video_filter/deinterlace/deinterlace.c}}
** {{VLCSourceFile|name=modules/video_filter/deinterlace/deinterlace.h}}
* '''{{VLCSourceFile|name=modules/video_filter/deinterlace/common.h}}'''
** Common macros (min, max, etc.).
* '''{{VLCSourceFile|name=modules/video_filter/deinterlace/mmx.h}}'''
** Macros for MMX inline assembly support. Safe to include only #ifdef CAN_COMPILE_MMXEXT.
* '''<nowiki>modules/video_filter/deinterlace/helpers.[ch]</nowiki>'''
** General-use helper functions: interlace detection, motion detection, glueing a field pair to make a frame.
** {{VLCSourceFile|name=modules/video_filter/deinterlace/helpers.c}}
** {{VLCSourceFile|name=modules/video_filter/deinterlace/helpers.h}}
* '''<nowiki>modules/video_filter/deinterlace/merge.[ch]</nowiki>'''
** Line-blending functions and (in header) macros for mixing two picture lines into one. (You'll want to use the Merge() and EndMerge() macros.)
** {{VLCSourceFile|name=modules/video_filter/deinterlace/merge.c}}
** {{VLCSourceFile|name=modules/video_filter/deinterlace/merge.h}}
* '''<nowiki>modules/video_filter/deinterlace/algo_basic.[ch]</nowiki>'''
** Basic algorithms: Discard, Bob, Linear, Mean, Blend.
** {{VLCSourceFile|name=modules/video_filter/deinterlace/algo_basic.c}}
** {{VLCSourceFile|name=modules/video_filter/deinterlace/algo_basic.h}}
* '''<nowiki>modules/video_filter/deinterlace/algo_*.[ch]</nowiki>'''
** Advanced algorithms, one algorithm per set of .c and .h.
** If the same algorithm has framerate-doubling and non-doubling versions, it counts here as the same algorithm. For example, both yadif and yadif2x are implemented in algo_yadif.c (in the same render function, even).
** As of <nowiki>[c7d289cad5c9f73d1bfc5b136b503fc3646b6e41]</nowiki>, the following files exist:
** {{VLCSourceFile|name=modules/video_filter/deinterlace/algo_x.c}}
** {{VLCSourceFile|name=modules/video_filter/deinterlace/algo_x.h}}
** {{VLCSourceFile|name=modules/video_filter/deinterlace/algo_yadif.c}}
** {{VLCSourceFile|name=modules/video_filter/deinterlace/algo_yadif.h}}
** {{VLCSourceFile|name=modules/video_filter/deinterlace/algo_phosphor.c}}
** {{VLCSourceFile|name=modules/video_filter/deinterlace/algo_phosphor.h}}
** {{VLCSourceFile|name=modules/video_filter/deinterlace/algo_ivtc.c}}
** {{VLCSourceFile|name=modules/video_filter/deinterlace/algo_ivtc.h}}

The list of files making up the module (for the build system) is in {{VLCSourceFile|name=modules/video_filter/Modules.am}}.

== Understanding the code ==

It is recommended to look at the existing algorithms:

* The simple algorithms in algo_basic.c make good first examples in getting to know the internals of the deinterlacer. But to avoid becoming confused:
** First, make sure to understand the implications of the flags (b_double_rate, b_half_height) for each mode in SetFilterMethod() (deinterlace.c).
** Then, understand the implications of the output chroma choosing logic in GetOutputFormat() (deinterlace.c).
** Note that these flags are nowhere to be seen inside algo_basic.c itself!
** You can also refer to the technical summary table in [[Deinterlacing]]. After reading that, it should all make (more) sense.
* algo_x.c provides an example of an advanced algorithm with lots of private functions.
* algo_yadif.c provides an example of glueing an existing algorithm to VLC. The original file from MPlayer is yadif.h; algo_yadif.* are glue code for VLC.
* ComposeFrame() in helpers.c provides an example of chroma conversions. See algo_phosphor.c for its usage.
* Soft field repeat, which '''must''' be supported by all framerate doublers, is handled differently in different filters. What is a sensible way depends on how the algorithm works. See Bob, Yadif, and Phosphor for very different examples.

Also, generally:

* See the available macros and helper functions in common.h, helpers.h, merge.h, and also {{VLCSourceFile|name=include/vlc_picture.h}}.
* Make sure you understand 4:2:0 and 4:2:2 chroma (explanation and pictures at [http://en.wikipedia.org/wiki/Chroma_subsampling#Sampling_systems_and_ratios Wikipedia]).
** The filter supports [[I420]], [[J420]], [[YV12]], [[I422]] and [[J422]] chroma. See [[YUV]] on the VideoLAN wiki.
** [[YV12]] is YVU (3 planes), others are YUV (3 planes). Except the plane order, [[YV12]] is identical to [[I420]].
** The Y component of J has full ("digital") scale 0..255, instead of the "analog" 16..240 of I. Usually you don't need to care about this distinction, if you stay within the same chroma, or only convert between 4:2:0 and 4:2:2, preserving I/J.
** Note that if you want to upconvert [[YV12]] into [[I422]], you will need to swap the U and V planes. For an example, see ComposeFrame() in helpers.c.
** See {{VLCSourceFile|name=include/vlc_fourcc.h}}.
** Useful constants: Y_PLANE, U_PLANE, V_PLANE ({{VLCSourceFile|name=include/vlc_picture.h}}). For [[YV12]], U_PLANE actually refers to V, and V_PLANE to U.
* Much of the stuff is thoroughly documented. See the comments (and please keep them up to date when you modify things).

== Adding a new mode ==

So, you have thought up a new algorithm, or maybe you want to add in an existing one from another GPL-compatible project.

Here is a checklist for implementing a new mode:

* Choose a unique internal name (for config system and module internal use), and a user label (for GUI).
** Update the mode lists in {{VLCSourceFile|name=modules/video_filter/deinterlace/deinterlace.h}}
** The internal names must not be localized; use bare strings, e.g. "label"
** The user labels should be localized if they contain English, e.g. N_("My Label")
** In some special cases, user labels might not need localization, e.g. "X"
** If you need unicode characters in the user label, use the \uxxxx feature of C99 (e.g. \u00b2 is the superscript two / math squared symbol, so <nowiki>static const char* const my_label = "X\u00b2";</nowiki> will render as X2 in the GUI).
*** Here unicode = anything that is not 7-bit ASCII
*** This is especially important if you intend it for translation, since gettext does not allow unicode characters in the original C locale strings.
* Write support code in {{VLCSourceFile|name=modules/video_filter/deinterlace/deinterlace.c}}:
** Need configuration options? Search for ''phosphor-chroma'' for an example of making one.
*** If you do add an option, update ppsz_filter_options[]. It is used to validate what options exist.
** Update SetFilterMethod():
*** Setup the flags for your mode. It is up to you to define how you want your algorithm to behave; certain conversions between input and output are allowed. (Keep this in mind when reading existing modes as examples! Refer to this function if necessary.)
**** b_double_rate = framerate doubler/field renderer.
***** If true, it means your mode outputs one frame for each input field.
***** If false, it means your mode outputs one frame for each input frame.
***** These two types behave differently; see e.g. RenderDiscard() (no doubling) and RenderBob() (doubling) for the simplest examples.
***** Framerate doublers '''must''' support soft field repeat (nb_fields == 3). If you're not quite sure how to process the middle field, it's usually safe just to copy the input picture, since a field repeat usually implies no motion during that frame.
***** Soft field repeat is necessary to support some NTSC film sources correctly. These include anything that is soft-telecined, as well as sources where some space has been saved by using field repeat in some frames with no motion. Examples from anime: Sol Bianca (field repeats; this is a 24p/60i hybrid anime), Silent Mobius (field repeats, soft TC), Angel Links (soft TC), Stellvia of the Universe (soft TC).
**** b_half_height = halved vertical resolution in output.
***** If false, the output has as many lines as the input. (This is the usual choice.)
***** If true, the output has only half the number of lines when compared to input. Useful for some special cases, though it's fair to say all useful ones have already been covered in algo_basic.c.
***** Note that this refers to the data resolution; you can still output a picture with doubled (copied) lines even if you use full resolution (RenderBob() does this).
**** b_use_frame_history = use the input frame history buffer.
***** If true, you will have the three (HISTORY_SIZE) latest frames available in the array pp_history[] in filter_sys_t. You can fetch the frames from there; your render function does not need an input picture parameter. This is useful for temporal filtering. See RenderYadif() for an example of using the history buffer.
***** If false, you will need to feed each input picture to your render function directly. This is meant for filters that map the frames in a simple 1 -> 1 (or 1 -> 2) manner, needing no history. See anything in algo_basic.c, or RenderX() (algo_x.c), for an example.
***** It is up to you to decide which frame you want to output at each call; see RenderYadif() (outputs second latest frame) and RenderPhosphor() (outputs latest frame) for examples. See i_frame_offset in filter_sys_t.
** Update GetOutputFormat():
*** Setup which chroma your mode outputs, and under which kind of input.
**** Usually you just want to pass it through. This is the default for 4:2:0 input, but '''do''' add your mode into the first switch statement to support 4:2:2 input correctly.
**** What "correct" means is up to you; see the Phosphor algorithm for an example doing chroma conversions.
*** Half-heighting is already handled by setting the flag in SetFilterMethod(), so there's no need to touch that part.
** Deinterlace() handles some things for you automatically:
*** Allocation of output frames (so you can simply render into them).
*** Setting the correct output timestamps (PTS), also for framerate doublers.
**** You '''can''' use i_frame_offset = CUSTOM_PTS and compute the timestamps yourself, but this is needed only for nontrivial framerate conversions. For an example, see algo_ivtc.c, function IVTCOutputOrDropFrame().
** Update the switch statement in Deinterlace() to handle your new algorithm. See the existing cases for examples.
* Create algo_zzz.c and algo_zzz.h, where zzz is the internal name of your new mode.
** You want to write at least one public function: RenderZzz(), called from Deinterlace(). Most of the parameters of the render functions are self-explanatory, but i_order and i_field may need an explanation.
*** These two parameters are used only for algorithms, which support framerate doubling.
*** i_order is the temporal number, inside the current frame, of the current field being rendered: 0 (first), 1 (second) or 2 (repeated first, used only when nb_fields == 3).
*** i_field indicates which field is being rendered: 0 for top (lines 0, 2, 4, ...), 1 for bottom (lines 1, 3, 5, ...).
**** This needs to match i_order and the b_top_field_first flag ''of the picture you are currently rendering''. (Note that this may not be the latest input picture, if you use the history buffer.)
**** To automatically get it correct, implement your case in Deinterlace() like the existing ones, and use i_frame_offset as instructed in deinterlace.h. Note that if you set i_frame_offset in your render function, it takes effect at the '''next''' input frame (defined as the next time Deinterlace() is called). See the rationale in {{VLCSourceFile|name=modules/video_filter/deinterlace/deinterlace.h}}.
*** If you want to use a framerate doubling algorithm in non-doubling mode, you can pass i_order = 0 and i_field = 0 (like the existing filters, which have both modes).
** If you need to have startup or cleanup code, see the Phosphor and IVTC algorithms for examples.
*** Phosphor configuration options are read in Open().
*** IVTCClearState() is called in Open() and in Flush(). It initializes/resets IVTC state.
*** Note that the filter may be flushed while running under some circumstances. If you allocated something upon startup, don't deallocate from Flush(), but make a separate deallocation function and call that from Close().
*** Note that the filter chain '''never''' changes the input format on the fly; it always Close()s and Open()s again.
** Keep in mind that the deinterlacer is low-level code and needs to be fast.
*** For example, for a 60i stream, a framerate doubler should render one output frame in preferably under 5ms. Note that the field interval in 60i is about 16ms, and some time is needed for decoding, rendering and such. 8ms and above is dangerously slow (will cause skipping).
*** Consider writing the inner processing loops in vectorized inline assembly, such as MMX. This can often gain a factor of 2-8x in speed, depending on how much and what kind of processing your algorithm needs.
* Update {{VLCSourceFile|name=modules/video_filter/Modules.am}}, so that the build system will compile your new files into the module.
See [[Documentation:Hacker's_Guide/How_To_Write_a_Module#Git_and_Repository_integration|new module integration]] for additional help.

Finally, inform the rest of VLC about the existence of your new mode. There are some deinterlacer mode lists and validity checking code that is external to the deinterlacer module itself. Some of these contain the internal name, some the user label. Make sure that these match the ones you put in {{VLCSourceFile|name=modules/video_filter/deinterlace/deinterlace.h}}.

As of this writing, these are located in:

* {{VLCSourceFile|name=src/libvlc-module.c}}
* {{VLCSourceFile|name=src/control/video.c}}
* {{VLCSourceFile|name=src/video_output/interlacing.c}}

Search for the string ''yadif'' to find the relevant parts quickly. The string is unique enough to match even from the whole source tree.

Speaking of which, a pretty good way to keep up to date as to where to find this stuff is to go to the root of the vlc source tree in the terminal, and search for ''yadif'' recursively:

<pre>find . -name "*.[ch]" -exec grep -Hr 'yadif' \{\} \;</pre>

Then just note down anything that matches anywhere else except modules/video_filter/deinterlace/.

{{Documentation}}

Hacker Guide/Video Filters/Deinterlace

2012-01-27T10:06:55Z

Technologicat: /* Understanding the code */ Original statement maybe too strong...

{{Module|name=deinterlace|type=video filter2|description=Removes interlacing from display}}

This is the [[Documentation:Hacker's_Guide|Hacker's Guide]] for the deinterlace module. See also the technical summary in the [[Deinterlace|user documentation]].

== Interfacing with the deinterlacer ==

This section deals with interfacing other parts of {{VLC}} with the deinterlacer. For a description of the internals, skip ahead.

=== How to prepare input ===

The deinterlacer assumes that the input consists of full frames (picture_t), with i_nb_fields == 2 or 3 (depending of repeat_pict flags in the input stream), b_progressive == false, and full vertical resolution (i_visible_lines) of the input stream. The b_top_field_first flag is also optionally used by the deinterlacer, and must be set correctly to guarantee correct operation in all modes.

Each plane is stored linearly, one line at a time, in an uint8_t array of size i_lines*i_pitch, where i_lines and i_pitch can be read off the plane_t for each plane (e.g. <nowiki>p_pic->p[plane_index].i_lines</nowiki>). In YUV formats, the luma and chroma planes have different resolution.

Note that i_lines may differ from i_visible_lines and i_pitch from i_visible_pitch. The value of i_pitch depends on how the picture_t was created. The filter picture pool may, and usually does, have a pitch different from that produced by picture_NewFromFormat(), even though the visible pitches are the same.

It is assumed that the active picture area starts at the upper left edge. The first i_visible_lines lines (0 through i_visible_lines-1, inclusive), make up the visible lines. The rest of the lines (i_visible_lines through i_lines-1, inclusive) are the (invisible) vertical margin.

Pixels at offsets 0 to i_visible_pitch-1 relative to each visible line start are visible. The rest of the pixels, at offsets i_visible_pitch to i_pitch-1, make up the (invisible) horizontal margin.

It is not necessary to initialize the memory in the invisible parts; writing just the visible part of the picture is enough.

Note that this implies that any processing loops only need to iterate through the part from 0 to i_visible_pitch (and i_visible_lines), but the pixel offsets in the array must be computed using i_pitch.

For any plane that has full vertical resolution, the (macro-)pixels are stored as if the frame was progressive, i.e. the fields are interleaved. Line 0 of the plane comes from the top field, line 1 from the bottom field, line 2 from the top field, and so on. This always holds for the luma plane, and in the 4:2:2 chroma formats ([[I422]] and J422), also for the chroma planes.

If the chroma format is 4:2:0 ([[I420]], J420 or [[YV12]]), both fields share the same chroma. This kind of chroma is stored as-is, at half resolution (both horizontal and vertical) in the U and V planes of the picture_t. The same linear format is used as above (array of size i_lines*i_pitch, first i_visible_lines used, and first i_visible_pitch pixels on each visible line used), but because there is only one "chroma field" in this case, the fields do not alternate. Instead, chroma line 0 corresponds to luma lines 0 and 1, chroma line 1 to luma lines 2 and 3, and so on.

Additionally, if chroma is [[YV12]], the U plane is stored in the array index V_PLANE, and the V plane is stored in the array index U_PLANE. This is because YV12 uses the plane ordering YVU, while the array index constants (see include/vlc_picture.h) are named according to the more common ordering YUV. For all chroma formats using the more common ordering, U_PLANE refers to the U component and V_PLANE to the V component, as expected.

Example:

<pre>p_pic->p[Y_PLANE].p_pixels[23*w+42]</pre>

where <nowiki>w = p_pic->p[Y_PLANE].i_pitch</nowiki>, refers to the pixel on the luma
plane at x = 42, y = 23 (0-based indexing).

If the chroma is I420 or J420,

<pre>p_pic->p[U_PLANE].p_pixels[11*w+21]</pre>

where now <nowiki>w = p_pic->p[U_PLANE].i_pitch</nowiki>, refers to the chroma macropixel
on the U plane that corresponds to the above example. Obviously, 11 =
floor(23/2).

For 4:2:2 chroma (I422 or J422), only the horizontal chroma resolution is half that of the luma, and the corresponding macropixel is

<pre>p_pic->p[U_PLANE].p_pixels[23*w+21]</pre>

=== How to interpret output ===

The frames output by the deinterlacer are picture_t's, with i_nb_fields == 2 and b_progressive == true. The output chroma format and number of visible lines are algorithm-dependent.

Resolution and chroma conversions are allowed, and it should be assumed there is no particular relation between input and output. Currently possible output chroma formats are [[I420]], [[YV12]], J420, [[I422]] and J422. Vertical resolution may be original or half. Thus, chroma format and vertical resolution should be read off the output frame metadata (appropriate picture_t data fields).

From the top-level Deinterlace(), the output pictures go into a linked list (using the "next" pointer in picture_t), which is then given to the caller. What happens to the pictures then, is outside the scope of the module.

Temporally, non-doubling deinterlacers produce exactly one output picture per one input, IVTC produces one or zero (at the dropped frame), and framerate doublers produce two or three (depending on repeat_pict of each input frame).

The timings (presentation timestamp; PTS; picture_t data field "date") may change arbitrarily between input and output. It is not even guaranteed that calling the deinterlacer for an input frame outputs a frame corresponding to that input frame. Yadif uses the frame offset feature, and IVTC effectively does, too. This means that when frame "2" goes in, what comes out is deinterlaced frame "1" - in the case of Yadif, still with its original PTS! (The offset does NOT introduce a delay; thus it keeps A/V sync intact. This is why it is called i_frame_offset and not i_frame_delay in the code.) In the case of IVTC, the PTS is somewhat like original, but corrected for the 29.97 > 23.976 fps conversion... usually.

The [[deinterlace]] filter is not required to actually output anything the first few times it is called. Some algorithms keep a history and use it for temporal filtering. Currently, this is 3 input pictures, and the first call to Deinterlace() always outputs a picture. The second call may drop. From the third call on, the history buffer has filled, and at this point it is guaranteed that normal operation starts as defined by the chosen algorithm.

For generality, it should be assumed that M input pictures map to N output pictures, with arbitrary, different M and N. Note that even though repeat_pict means "repeat first field", the first and third output pictures from a framerate doubler, for any given input frame, are allowed to be different due to temporal filtering. (Phosphor does this.) Thus, it should also be assumed that each output picture is unique.

It may or may not be safe to display-hold still images, depending on the deinterlacing algorithm. For non-doublers, this is safe. For doublers, it is not. This is because framerate doubling algorithms are often designed based on the illusion of increased perceived resolution that can be achieved by rapidly alternating half-resolution images in an appropriate way.

Thus, "progressive" pictures from framerate doublers are not actually progressive (at least not at full resolution), but only appear so while the filter is constantly producing new pictures at a steady framerate. Unfortunately, currently there is no way to determine from the outside which kind of algorithm has been chosen.

== File structure ==

This applies to <nowiki>[c7d289cad5c9f73d1bfc5b136b503fc3646b6e41]</nowiki> (1st May 2011) and later.

The deinterlace filter is located in [http://git.videolan.org/?p=vlc.git;f=modules/video_filter/deinterlace;hb=HEAD modules/video_filter/deinterlace/].

* '''<nowiki>modules/video_filter/deinterlace/deinterlace.[ch]</nowiki>'''
** The main files of the filter.
** {{VLCSourceFile|name=modules/video_filter/deinterlace/deinterlace.c}}
** {{VLCSourceFile|name=modules/video_filter/deinterlace/deinterlace.h}}
* '''{{VLCSourceFile|name=modules/video_filter/deinterlace/common.h}}'''
** Common macros (min, max, etc.).
* '''{{VLCSourceFile|name=modules/video_filter/deinterlace/mmx.h}}'''
** Macros for MMX inline assembly support. Safe to include only #ifdef CAN_COMPILE_MMXEXT.
* '''<nowiki>modules/video_filter/deinterlace/helpers.[ch]</nowiki>'''
** General-use helper functions: interlace detection, motion detection, glueing a field pair to make a frame.
** {{VLCSourceFile|name=modules/video_filter/deinterlace/helpers.c}}
** {{VLCSourceFile|name=modules/video_filter/deinterlace/helpers.h}}
* '''<nowiki>modules/video_filter/deinterlace/merge.[ch]</nowiki>'''
** Line-blending functions and (in header) macros for mixing two picture lines into one. (You'll want to use the Merge() and EndMerge() macros.)
** {{VLCSourceFile|name=modules/video_filter/deinterlace/merge.c}}
** {{VLCSourceFile|name=modules/video_filter/deinterlace/merge.h}}
* '''<nowiki>modules/video_filter/deinterlace/algo_basic.[ch]</nowiki>'''
** Basic algorithms: Discard, Bob, Linear, Mean, Blend.
** {{VLCSourceFile|name=modules/video_filter/deinterlace/algo_basic.c}}
** {{VLCSourceFile|name=modules/video_filter/deinterlace/algo_basic.h}}
* '''<nowiki>modules/video_filter/deinterlace/algo_*.[ch]</nowiki>'''
** Advanced algorithms, one algorithm per set of .c and .h.
** If the same algorithm has framerate-doubling and non-doubling versions, it counts here as the same algorithm. For example, both yadif and yadif2x are implemented in algo_yadif.c (in the same render function, even).
** As of <nowiki>[c7d289cad5c9f73d1bfc5b136b503fc3646b6e41]</nowiki>, the following files exist:
** {{VLCSourceFile|name=modules/video_filter/deinterlace/algo_x.c}}
** {{VLCSourceFile|name=modules/video_filter/deinterlace/algo_x.h}}
** {{VLCSourceFile|name=modules/video_filter/deinterlace/algo_yadif.c}}
** {{VLCSourceFile|name=modules/video_filter/deinterlace/algo_yadif.h}}
** {{VLCSourceFile|name=modules/video_filter/deinterlace/algo_phosphor.c}}
** {{VLCSourceFile|name=modules/video_filter/deinterlace/algo_phosphor.h}}
** {{VLCSourceFile|name=modules/video_filter/deinterlace/algo_ivtc.c}}
** {{VLCSourceFile|name=modules/video_filter/deinterlace/algo_ivtc.h}}

The list of files making up the module (for the build system) is in {{VLCSourceFile|name=modules/video_filter/Modules.am}}.

== Understanding the code ==

It is recommended to look at the existing algorithms:

* The simple algorithms in algo_basic.c make good first examples in getting to know the internals of the deinterlacer. But to avoid becoming confused:
** First, make sure to understand the implications of the flags (b_double_rate, b_half_height) for each mode in SetFilterMethod() (deinterlace.c).
** Then, understand the implications of the output chroma choosing logic in GetOutputFormat() (deinterlace.c).
** Note that these flags are nowhere to be seen inside algo_basic.c itself!
** You can also refer to the technical summary table in [[Deinterlacing]]. After reading that, it should all make (more) sense.
* algo_x.c provides an example of an advanced algorithm with lots of private functions.
* algo_yadif.c provides an example of glueing an existing algorithm to VLC. The original file from MPlayer is yadif.h; algo_yadif.* are glue code for VLC.
* ComposeFrame() in helpers.c provides an example of chroma conversions. See algo_phosphor.c for its usage.
* Soft field repeat, which '''must''' be supported by all framerate doublers, is handled differently in different filters. What is a sensible way depends on how the algorithm works. See Bob, Yadif, and Phosphor for very different examples.

Also, generally:

* See the available macros and helper functions in common.h, helpers.h, merge.h, and also {{VLCSourceFile|name=include/vlc_picture.h}}.
* Make sure you understand 4:2:0 and 4:2:2 chroma (explanation and pictures at [http://en.wikipedia.org/wiki/Chroma_subsampling#Sampling_systems_and_ratios Wikipedia]).
** The filter supports I420, J420, YV12, I422 and J422 chroma.
** YV12 is YVU (3 planes), others are YUV (3 planes). Except the plane order, YV12 is identical to I420.
** The Y component of J has full ("digital") scale 0..255, instead of the "analog" 16..240 of I. Usually you don't need to care about this distinction, if you stay within the same chroma, or only convert between 4:2:0 and 4:2:2, preserving I/J.
** Note that if you want to upconvert YV12 into I422, you will need to swap the U and V planes. For an example, see ComposeFrame() in helpers.c.
** See {{VLCSourceFile|name=include/vlc_fourcc.h}}.
** Useful constants: Y_PLANE, U_PLANE, V_PLANE ({{VLCSourceFile|name=include/vlc_picture.h}}). For YV12, U_PLANE actually refers to V, and V_PLANE to U.
* Much of the stuff is thoroughly documented. See the comments (and please keep them up to date when you modify things).

== Adding a new mode ==

So, you have thought up a new algorithm, or maybe you want to add in an existing one from another GPL-compatible project.

Here is a checklist for implementing a new mode:

* Choose a unique internal name (for config system and module internal use), and a user label (for GUI).
** Update the mode lists in {{VLCSourceFile|name=modules/video_filter/deinterlace/deinterlace.h}}
** The internal names must not be localized; use bare strings, e.g. "label"
** The user labels should be localized if they contain English, e.g. N_("My Label")
** In some special cases, user labels might not need localization, e.g. "X"
** If you need unicode characters in the user label, use the \uxxxx feature of C99 (e.g. \u00b2 is the superscript two / math squared symbol, so <nowiki>static const char* const my_label = "X\u00b2";</nowiki> will render as X2 in the GUI).
*** Here unicode = anything that is not 7-bit ASCII
*** This is especially important if you intend it for translation, since gettext does not allow unicode characters in the original C locale strings.
* Write support code in {{VLCSourceFile|name=modules/video_filter/deinterlace/deinterlace.c}}:
** Need configuration options? Search for ''phosphor-chroma'' for an example of making one.
*** If you do add an option, update ppsz_filter_options[]. It is used to validate what options exist.
** Update SetFilterMethod():
*** Setup the flags for your mode. It is up to you to define how you want your algorithm to behave; certain conversions between input and output are allowed. (Keep this in mind when reading existing modes as examples! Refer to this function if necessary.)
**** b_double_rate = framerate doubler/field renderer.
***** If true, it means your mode outputs one frame for each input field.
***** If false, it means your mode outputs one frame for each input frame.
***** These two types behave differently; see e.g. RenderDiscard() (no doubling) and RenderBob() (doubling) for the simplest examples.
***** Framerate doublers '''must''' support soft field repeat (nb_fields == 3). If you're not quite sure how to process the middle field, it's usually safe just to copy the input picture, since a field repeat usually implies no motion during that frame.
***** Soft field repeat is necessary to support some NTSC film sources correctly. These include anything that is soft-telecined, as well as sources where some space has been saved by using field repeat in some frames with no motion. Examples from anime: Sol Bianca (field repeats; this is a 24p/60i hybrid anime), Silent Mobius (field repeats, soft TC), Angel Links (soft TC), Stellvia of the Universe (soft TC).
**** b_half_height = halved vertical resolution in output.
***** If false, the output has as many lines as the input. (This is the usual choice.)
***** If true, the output has only half the number of lines when compared to input. Useful for some special cases, though it's fair to say all useful ones have already been covered in algo_basic.c.
***** Note that this refers to the data resolution; you can still output a picture with doubled (copied) lines even if you use full resolution (RenderBob() does this).
**** b_use_frame_history = use the input frame history buffer.
***** If true, you will have the three (HISTORY_SIZE) latest frames available in the array pp_history[] in filter_sys_t. You can fetch the frames from there; your render function does not need an input picture parameter. This is useful for temporal filtering. See RenderYadif() for an example of using the history buffer.
***** If false, you will need to feed each input picture to your render function directly. This is meant for filters that map the frames in a simple 1 -> 1 (or 1 -> 2) manner, needing no history. See anything in algo_basic.c, or RenderX() (algo_x.c), for an example.
***** It is up to you to decide which frame you want to output at each call; see RenderYadif() (outputs second latest frame) and RenderPhosphor() (outputs latest frame) for examples. See i_frame_offset in filter_sys_t.
** Update GetOutputFormat():
*** Setup which chroma your mode outputs, and under which kind of input.
**** Usually you just want to pass it through. This is the default for 4:2:0 input, but '''do''' add your mode into the first switch statement to support 4:2:2 input correctly.
**** What "correct" means is up to you; see the Phosphor algorithm for an example doing chroma conversions.
*** Half-heighting is already handled by setting the flag in SetFilterMethod(), so there's no need to touch that part.
** Deinterlace() handles some things for you automatically:
*** Allocation of output frames (so you can simply render into them).
*** Setting the correct output timestamps (PTS), also for framerate doublers.
**** You '''can''' use i_frame_offset = CUSTOM_PTS and compute the timestamps yourself, but this is needed only for nontrivial framerate conversions. For an example, see algo_ivtc.c, function IVTCOutputOrDropFrame().
** Update the switch statement in Deinterlace() to handle your new algorithm. See the existing cases for examples.
* Create algo_zzz.c and algo_zzz.h, where zzz is the internal name of your new mode.
** You want to write at least one public function: RenderZzz(), called from Deinterlace(). Most of the parameters of the render functions are self-explanatory, but i_order and i_field may need an explanation.
*** These two parameters are used only for algorithms, which support framerate doubling.
*** i_order is the temporal number, inside the current frame, of the current field being rendered: 0 (first), 1 (second) or 2 (repeated first, used only when nb_fields == 3).
*** i_field indicates which field is being rendered: 0 for top (lines 0, 2, 4, ...), 1 for bottom (lines 1, 3, 5, ...).
**** This needs to match i_order and the b_top_field_first flag ''of the picture you are currently rendering''. (Note that this may not be the latest input picture, if you use the history buffer.)
**** To automatically get it correct, implement your case in Deinterlace() like the existing ones, and use i_frame_offset as instructed in deinterlace.h. Note that if you set i_frame_offset in your render function, it takes effect at the '''next''' input frame (defined as the next time Deinterlace() is called). See the rationale in {{VLCSourceFile|name=modules/video_filter/deinterlace/deinterlace.h}}.
*** If you want to use a framerate doubling algorithm in non-doubling mode, you can pass i_order = 0 and i_field = 0 (like the existing filters, which have both modes).
** If you need to have startup or cleanup code, see the Phosphor and IVTC algorithms for examples.
*** Phosphor configuration options are read in Open().
*** IVTCClearState() is called in Open() and in Flush(). It initializes/resets IVTC state.
*** Note that the filter may be flushed while running under some circumstances. If you allocated something upon startup, don't deallocate from Flush(), but make a separate deallocation function and call that from Close().
*** Note that the filter chain '''never''' changes the input format on the fly; it always Close()s and Open()s again.
** Keep in mind that the deinterlacer is low-level code and needs to be fast.
*** For example, for a 60i stream, a framerate doubler should render one output frame in preferably under 5ms. Note that the field interval in 60i is about 16ms, and some time is needed for decoding, rendering and such. 8ms and above is dangerously slow (will cause skipping).
*** Consider writing the inner processing loops in vectorized inline assembly, such as MMX. This can often gain a factor of 2-8x in speed, depending on how much and what kind of processing your algorithm needs.
* Update {{VLCSourceFile|name=modules/video_filter/Modules.am}}, so that the build system will compile your new files into the module.
See [[Documentation:Hacker's_Guide/How_To_Write_a_Module#Git_and_Repository_integration|new module integration]] for additional help.

Finally, inform the rest of VLC about the existence of your new mode. There are some deinterlacer mode lists and validity checking code that is external to the deinterlacer module itself. Some of these contain the internal name, some the user label. Make sure that these match the ones you put in {{VLCSourceFile|name=modules/video_filter/deinterlace/deinterlace.h}}.

As of this writing, these are located in:

* {{VLCSourceFile|name=src/libvlc-module.c}}
* {{VLCSourceFile|name=src/control/video.c}}
* {{VLCSourceFile|name=src/video_output/interlacing.c}}

Search for the string ''yadif'' to find the relevant parts quickly. The string is unique enough to match even from the whole source tree.

Speaking of which, a pretty good way to keep up to date as to where to find this stuff is to go to the root of the vlc source tree in the terminal, and search for ''yadif'' recursively:

<pre>find . -name "*.[ch]" -exec grep -Hr 'yadif' \{\} \;</pre>

Then just note down anything that matches anywhere else except modules/video_filter/deinterlace/.

{{Documentation}}

YUV

2012-01-27T10:05:26Z

Technologicat: /* J422 */ Let's get the numbers correct, too...

YUV is a class of pixel formats used in Video application and in {{VLC}}.

YUV is also colorspace and the way to represent pixels colors is based on Luminance and Chrominance.

This pages explains the most useful ones in {{VLC}}.

= Planar formats =
== YUV 4:2:0 (I420/J420/YV12)==

=== I420 ===
In I420, you have one luma "luminance" plane '''''Y''''' and 2 chroma planes '''''U''''', '''''V'''''.

In I420, chroma planes (blue and red projections) are subsampled in both the horizontal and vertical dimensions by a factor of 2.

For a 2x2 group of pixels, you have 4 Y samples and 1 U and 1 V sample.

=== J420 ===
J420 is exactly like [[I420]], but with a full range ("digital", 0-255) luma ('''''Y''''') component instead of limited range ("analog", 16-240). The chroma planes are exactly the same as in [[I420]].

=== YV12 ===

YV12 is exactly like [[I420]], but with '''U''' and '''V''' order reversed.

== NV12/NV21 ==

=== NV12 ===
Related to [[I420]], in NV12, you have one luma "luminance" planes '''''Y''''' and one plane with '''''U''''' and '''''V''''' data interleaved. NV12 is considered as ''semiplanar''.

In NV12, chroma planes (blue and red) are subsampled in both the horizontal and vertical dimensions by a factor of 2.

For a 2x2 group of pixels, you have 4 Y samples and 1 U and 1 V sample.

=== NV21 ===

NV21 is like [[NV12]], but with '''U''' and '''V''' order reversed: it starts with '''V'''.

== YUV 4:2:2 (I422/J422) ==
=== I422 ===
In I422, like in [[I420]], you have one luma plane '''''Y''''' and 2 chroma planes '''U''', '''V'''.

However, in I422, chroma planes (blue and red projections) are subsampled just in the horizontal dimension by a factor of 2.
There is the same amount of lines than in the luma.

For a 2x2 group of pixels, you have 4 Y samples and 2 U and 2 V sample.

=== J422 ===
J422 is exactly like [[I422]], but with a full range ("digital", 0-255) luma ('''''Y''''') component instead of limited range ("analog", 16-240). The chroma planes are exactly the same as in [[I422]].

= Packed formats =
In packed formats, you usually have only one plane, with all the luma and chroma data interleaved.

== UYVY ==
Know as ''UYVY'', ''Y422'', ''UYNV''

In UYVY, the chroma samples are sub-sampled by a factor of 2.

In UYVY, the succession for 2 pixels, starts by '''U''' then one luma, then '''V''' again and then another luma.

== YUY2 ==
Know as ''YUY2'', ''YUYV'', ''V422'', ''YUNV''

In YUY2, the chroma samples are sub-sampled by a factor of 2.

In YUY2, the succession for 2 pixels, starts by one luma, then '''U''', then another luma and then '''V''' again.

== YVYU ==
YVYU is just like [[YUY2]], but with '''U''' and '''V''' order reversed.

{{documentation}}

YUV

2012-01-27T10:04:43Z

Technologicat: /* J420 */ Let's get the numbers correct, too...

YUV is a class of pixel formats used in Video application and in {{VLC}}.

YUV is also colorspace and the way to represent pixels colors is based on Luminance and Chrominance.

This pages explains the most useful ones in {{VLC}}.

= Planar formats =
== YUV 4:2:0 (I420/J420/YV12)==

=== I420 ===
In I420, you have one luma "luminance" plane '''''Y''''' and 2 chroma planes '''''U''''', '''''V'''''.

In I420, chroma planes (blue and red projections) are subsampled in both the horizontal and vertical dimensions by a factor of 2.

For a 2x2 group of pixels, you have 4 Y samples and 1 U and 1 V sample.

=== J420 ===
J420 is exactly like [[I420]], but with a full range ("digital", 0-255) luma ('''''Y''''') component instead of limited range ("analog", 16-240). The chroma planes are exactly the same as in [[I420]].

=== YV12 ===

YV12 is exactly like [[I420]], but with '''U''' and '''V''' order reversed.

== NV12/NV21 ==

=== NV12 ===
Related to [[I420]], in NV12, you have one luma "luminance" planes '''''Y''''' and one plane with '''''U''''' and '''''V''''' data interleaved. NV12 is considered as ''semiplanar''.

In NV12, chroma planes (blue and red) are subsampled in both the horizontal and vertical dimensions by a factor of 2.

For a 2x2 group of pixels, you have 4 Y samples and 1 U and 1 V sample.

=== NV21 ===

NV21 is like [[NV12]], but with '''U''' and '''V''' order reversed: it starts with '''V'''.

== YUV 4:2:2 (I422/J422) ==
=== I422 ===
In I422, like in [[I420]], you have one luma plane '''''Y''''' and 2 chroma planes '''U''', '''V'''.

However, in I422, chroma planes (blue and red projections) are subsampled just in the horizontal dimension by a factor of 2.
There is the same amount of lines than in the luma.

For a 2x2 group of pixels, you have 4 Y samples and 2 U and 2 V sample.

=== J422 ===
J422 is exactly like [[I422]], but with a full range ("digital", 0-255) luma ('''''Y''''') component instead of limited range ("analog", 16-235). The chroma planes are exactly the same as in [[I422]].

= Packed formats =
In packed formats, you usually have only one plane, with all the luma and chroma data interleaved.

== UYVY ==
Know as ''UYVY'', ''Y422'', ''UYNV''

In UYVY, the chroma samples are sub-sampled by a factor of 2.

In UYVY, the succession for 2 pixels, starts by '''U''' then one luma, then '''V''' again and then another luma.

== YUY2 ==
Know as ''YUY2'', ''YUYV'', ''V422'', ''YUNV''

In YUY2, the chroma samples are sub-sampled by a factor of 2.

In YUY2, the succession for 2 pixels, starts by one luma, then '''U''', then another luma and then '''V''' again.

== YVYU ==
YVYU is just like [[YUY2]], but with '''U''' and '''V''' order reversed.

{{documentation}}

J420

2012-01-27T09:58:39Z

Technologicat: Set up redirect to YUV/J420

#redirect[[YUV#J420]]

J422

2012-01-27T09:57:51Z

Technologicat: Set up redirect to YUV/J422

#redirect[[YUV#J422]]

YUV

2012-01-27T09:54:22Z

Technologicat: /* YUY2 */ Typo fix: thenU -> then U

YUV is a class of pixel formats used in Video application and in {{VLC}}.

YUV is also colorspace and the way to represent pixels colors is based on Luminance and Chrominance.

This pages explains the most useful ones in {{VLC}}.

= Planar formats =
== YUV 4:2:0 (I420/J420/YV12)==

=== I420 ===
In I420, you have one luma "luminance" plane '''''Y''''' and 2 chroma planes '''''U''''', '''''V'''''.

In I420, chroma planes (blue and red projections) are subsampled in both the horizontal and vertical dimensions by a factor of 2.

For a 2x2 group of pixels, you have 4 Y samples and 1 U and 1 V sample.

=== J420 ===
J420 is exactly like [[I420]], but with a full range ("digital", 0-255) luma ('''''Y''''') component instead of limited range ("analog", 16-235). The chroma planes are exactly the same as in [[I420]].

=== YV12 ===

YV12 is exactly like [[I420]], but with '''U''' and '''V''' order reversed.

== NV12/NV21 ==

=== NV12 ===
Related to [[I420]], in NV12, you have one luma "luminance" planes '''''Y''''' and one plane with '''''U''''' and '''''V''''' data interleaved. NV12 is considered as ''semiplanar''.

In NV12, chroma planes (blue and red) are subsampled in both the horizontal and vertical dimensions by a factor of 2.

For a 2x2 group of pixels, you have 4 Y samples and 1 U and 1 V sample.

=== NV21 ===

NV21 is like [[NV12]], but with '''U''' and '''V''' order reversed: it starts with '''V'''.

== YUV 4:2:2 (I422/J422) ==
=== I422 ===
In I422, like in [[I420]], you have one luma plane '''''Y''''' and 2 chroma planes '''U''', '''V'''.

However, in I422, chroma planes (blue and red projections) are subsampled just in the horizontal dimension by a factor of 2.
There is the same amount of lines than in the luma.

For a 2x2 group of pixels, you have 4 Y samples and 2 U and 2 V sample.

=== J422 ===
J422 is exactly like [[I422]], but with a full range ("digital", 0-255) luma ('''''Y''''') component instead of limited range ("analog", 16-235). The chroma planes are exactly the same as in [[I422]].

= Packed formats =
In packed formats, you usually have only one plane, with all the luma and chroma data interleaved.

== UYVY ==
Know as ''UYVY'', ''Y422'', ''UYNV''

In UYVY, the chroma samples are sub-sampled by a factor of 2.

In UYVY, the succession for 2 pixels, starts by '''U''' then one luma, then '''V''' again and then another luma.

== YUY2 ==
Know as ''YUY2'', ''YUYV'', ''V422'', ''YUNV''

In YUY2, the chroma samples are sub-sampled by a factor of 2.

In YUY2, the succession for 2 pixels, starts by one luma, then '''U''', then another luma and then '''V''' again.

== YVYU ==
YVYU is just like [[YUY2]], but with '''U''' and '''V''' order reversed.

{{documentation}}

YUV

2012-01-27T09:53:05Z

Technologicat: /* YUV 4:2:2 */ Added description of J422

YUV is a class of pixel formats used in Video application and in {{VLC}}.

YUV is also colorspace and the way to represent pixels colors is based on Luminance and Chrominance.

This pages explains the most useful ones in {{VLC}}.

= Planar formats =
== YUV 4:2:0 (I420/J420/YV12)==

=== I420 ===
In I420, you have one luma "luminance" plane '''''Y''''' and 2 chroma planes '''''U''''', '''''V'''''.

In I420, chroma planes (blue and red projections) are subsampled in both the horizontal and vertical dimensions by a factor of 2.

For a 2x2 group of pixels, you have 4 Y samples and 1 U and 1 V sample.

=== J420 ===
J420 is exactly like [[I420]], but with a full range ("digital", 0-255) luma ('''''Y''''') component instead of limited range ("analog", 16-235). The chroma planes are exactly the same as in [[I420]].

=== YV12 ===

YV12 is exactly like [[I420]], but with '''U''' and '''V''' order reversed.

== NV12/NV21 ==

=== NV12 ===
Related to [[I420]], in NV12, you have one luma "luminance" planes '''''Y''''' and one plane with '''''U''''' and '''''V''''' data interleaved. NV12 is considered as ''semiplanar''.

In NV12, chroma planes (blue and red) are subsampled in both the horizontal and vertical dimensions by a factor of 2.

For a 2x2 group of pixels, you have 4 Y samples and 1 U and 1 V sample.

=== NV21 ===

NV21 is like [[NV12]], but with '''U''' and '''V''' order reversed: it starts with '''V'''.

== YUV 4:2:2 (I422/J422) ==
=== I422 ===
In I422, like in [[I420]], you have one luma plane '''''Y''''' and 2 chroma planes '''U''', '''V'''.

However, in I422, chroma planes (blue and red projections) are subsampled just in the horizontal dimension by a factor of 2.
There is the same amount of lines than in the luma.

For a 2x2 group of pixels, you have 4 Y samples and 2 U and 2 V sample.

=== J422 ===
J422 is exactly like [[I422]], but with a full range ("digital", 0-255) luma ('''''Y''''') component instead of limited range ("analog", 16-235). The chroma planes are exactly the same as in [[I422]].

= Packed formats =
In packed formats, you usually have only one plane, with all the luma and chroma data interleaved.

== UYVY ==
Know as ''UYVY'', ''Y422'', ''UYNV''

In UYVY, the chroma samples are sub-sampled by a factor of 2.

In UYVY, the succession for 2 pixels, starts by '''U''' then one luma, then '''V''' again and then another luma.

== YUY2 ==
Know as ''YUY2'', ''YUYV'', ''V422'', ''YUNV''

In YUY2, the chroma samples are sub-sampled by a factor of 2.

In YUY2, the succession for 2 pixels, starts by one luma, then'''U''', then another luma and then '''V''' again.

== YVYU ==
YVYU is just like [[YUY2]], but with '''U''' and '''V''' order reversed.

{{documentation}}

YUV

2012-01-27T09:51:40Z

Technologicat: /* YUV 4:2:0 (I420/YV12) */ Added description of J420

YUV is a class of pixel formats used in Video application and in {{VLC}}.

YUV is also colorspace and the way to represent pixels colors is based on Luminance and Chrominance.

This pages explains the most useful ones in {{VLC}}.

= Planar formats =
== YUV 4:2:0 (I420/J420/YV12)==

=== I420 ===
In I420, you have one luma "luminance" plane '''''Y''''' and 2 chroma planes '''''U''''', '''''V'''''.

In I420, chroma planes (blue and red projections) are subsampled in both the horizontal and vertical dimensions by a factor of 2.

For a 2x2 group of pixels, you have 4 Y samples and 1 U and 1 V sample.

=== J420 ===
J420 is exactly like [[I420]], but with a full range ("digital", 0-255) luma ('''''Y''''') component instead of limited range ("analog", 16-235). The chroma planes are exactly the same as in [[I420]].

=== YV12 ===

YV12 is exactly like [[I420]], but with '''U''' and '''V''' order reversed.

== NV12/NV21 ==

=== NV12 ===
Related to [[I420]], in NV12, you have one luma "luminance" planes '''''Y''''' and one plane with '''''U''''' and '''''V''''' data interleaved. NV12 is considered as ''semiplanar''.

In NV12, chroma planes (blue and red) are subsampled in both the horizontal and vertical dimensions by a factor of 2.

For a 2x2 group of pixels, you have 4 Y samples and 1 U and 1 V sample.

=== NV21 ===

NV21 is like [[NV12]], but with '''U''' and '''V''' order reversed: it starts with '''V'''.

== YUV 4:2:2 ==
=== I422 ===
In I422, like in [[I420]], you have one luma plane '''''Y''''' and 2 chroma planes '''U''', '''V'''.

However, in I422, chroma planes (blue and red projections) are subsampled just in the horizontal dimension by a factor of 2.
There is the same amount of lines than in the luma.

For a 2x2 group of pixels, you have 4 Y samples and 2 U and 2 V sample.

= Packed formats =
In packed formats, you usually have only one plane, with all the luma and chroma data interleaved.

== UYVY ==
Know as ''UYVY'', ''Y422'', ''UYNV''

In UYVY, the chroma samples are sub-sampled by a factor of 2.

In UYVY, the succession for 2 pixels, starts by '''U''' then one luma, then '''V''' again and then another luma.

== YUY2 ==
Know as ''YUY2'', ''YUYV'', ''V422'', ''YUNV''

In YUY2, the chroma samples are sub-sampled by a factor of 2.

In YUY2, the succession for 2 pixels, starts by one luma, then'''U''', then another luma and then '''V''' again.

== YVYU ==
YVYU is just like [[YUY2]], but with '''U''' and '''V''' order reversed.

{{documentation}}

Deinterlacing

2012-01-27T09:40:13Z

Technologicat: /* VLC settings */ Some editing - I think this is what was meant?

[[File:VLC_-_Deinterlace_action_options.png|400px|thumb|right|deinterlace switch menu]]
'''Deinterlacing''' is the process of converting source material that contains alternating half-pictures to a computer screen that displays a full picture at a time.

== Description ==
Traditional TVs do not display one picture-frame at a time. Instead, they display alternately all the odd lines and all the even lines of the picture. That way, they can get a fast display rate of 50 or 60 half-pictures per second, without the bandwidth requirements that full pictures have.

Computers however, do display full pictures, typically at a rate of at least 60 pictures per second. So to get optimum quality when displaying TV-material (such as DVDs) on a computer, the player can convert the alternating half-pictures to full pictures. That is called ''deinterlacing''.

Modern HDTVs operate much like computers in that they display full pictures. Much like computer-based video players, they have deinterlacing built in so that they can display traditional TV material.

A very good description of the problem and the various ways of deinterlacing can be found at [http://www.100fps.com/ 100fps.com].

== VLC settings ==
{{VLC}} has deinterlacing off by default.

You can enable it for the currently playing video by using the main menus, the right-click menus or by using [[hotkey] (by default: '''D'''). It switches deinterlacing on and off (using the mode selected in the Preferences).

To change the settings, go to the Video section in the Simple Preferences. You can choose from On, Automatic or Off. This setting controls whether VLC enables deinterlacing when you open a video. You can also choose the mode (algorithm). See below for available modes and mode recommendations.

The recommended choice is ''Automatic''. VLC will then check the stream flags (technical term for information embedded in the video) and automatically set deinterlacing on or off, depending on if the current video is marked as interlaced. If the setting is ''On'', VLC will apply deinterlacing even if the original is not interlaced (which is a really bad idea). The setting ''Off'', respectively, always keeps deinterlacing off.

'''For advanced users'''

In All Preferences, the settings for deinterlacing can be found in Preferences > Video > Filters > Deinterlace. The available settings are the mode (algorithm), and in v1.2.0+ also some algorithm-specific settings.

Note that if you enable deinterlacing from the menu, the setting will only take effect for the currently playing video. When the video ends, both the on/off/automatic and the mode settings will return to the values set in the preferences. If you want to always enable deinterlacing on every video, type "Deinterlace" in the Video Filter preferences.

The [[Hotkeys|keyboard shortcut]] to switch deinterlacing on and off can be configured in the [[Hotkeys]] section of the Simple Preferences. Its name is ''Cycle deinterlace modes''.

If you use VLC from the command line, you can also set the deinterlace options in your command. To get a list and short description of the available options, run <pre>vlc -p deinterlace --advanced</pre> Note that the deinterlace command-line options are prefixed by "sout" for historical reasons only; they are intended for normal playback use. For more information on using VLC from the command line, see [[VLC_command-line_help|VLC command-line help]].

== Interlaced or telecined? ==
In addition to true interlaced signals, there exists also a process called telecine which produces interlacing. It is a type of artificial interlacing that may be added to sources that were originally progressive. Telecine produces a characteristic look, with a repeating sequence of 3 progressive frames followed by 2 artificially interlaced frames.

Why would anyone want to do that? The answer lies in different framerates, and in transfer of film sources to television and video. Film is commonly shot at 24 pictures per second, while NTSC television and video equipment operates at 60 half-pictures per second. Telecine inserts extra half-pictures so that films can be stored in the NTSC format (for example on DVDs in Japan and USA).

For more on telecine, see for example the following links:
*[http://en.wikipedia.org/wiki/Telecine Telecine (in Wikipedia)]
*[http://arbor.ee.ntu.edu.tw/~jackeikuo/dvd2avi/ivtc/ Inverse Telecine]
*[http://neuron2.net/LVG/telecining1.html NTSC Telecine - Luke's Video Guide]
*[http://users.softlab.ece.ntua.gr/~ttsiod/ivtc.html NTSC, telecine, and how to cope (using telecide and decimate)]

== Which type of interlacing do I have? ==
This depends on the format (PAL, NTSC) and the content of the source. PAL is used in Europe and Australia, NTSC in USA and Japan. This also applies to the DVDs produced in those regions.

Live-action movies in the PAL format typically are not interlaced at all. This is because it is easier to transfer film to PAL than to NTSC.

Home movies shot on a camcorder are typically true interlaced.

Telecine is very common in Asia and especially in Japan. Anything originating in Asia is likely to be telecined. For example, with very few exceptions, all anime in the NTSC format is telecined. This includes the corresponding North American releases.

Source material that was converted from a different format is a special case. Producing a correct transfer for pure video material from NTSC to PAL or vice versa is very difficult due to the differences in framerate and number of lines. Whether a deinterlacer is needed depends on the details of the transfer. In the particular case of anime, a PAL conversion usually spells disaster for deinterlacing; see [http://www.animemusicvideos.org/guides/avtech/videogetb2a.html].

== Algorithm types ==

{{Module|name=deinterlace|type=Video Filter|description=Removes interlacing from display}}

In {{VLC}}, there are several deinterlacing algorithms to choose from. This section presents a general classification. Skip ahead for an individual description of each mode.

Algorithms available from a specific VLC version onward are marked with Name(vx.y.z+).

=== Doublers ===

'''VLC modes''': Bob, Linear, Yadif (2x) (v1.1.0+), Phosphor(v1.2.0+)

These algorithms display the video at the original half-picture rate, which is typically 50 (PAL) or 60 (NTSC) half-pictures per second. This is double the full picture rate, hence the name. This approach to deinterlacing is also known as field rendering.

This group takes into account that the half-pictures of a true interlaced video were intended to be displayed at different times. This can make the motion look very smooth.

Simple doublers (Bob and Linear) display only one half-picture at a time. Nevertheless, the quick alternating display produces a convincing illusion of full vertical resolution while playback is running.

Some more advanced doublers (such as Yadif (2x)) are based on interpolators (see below), and attempt to generate full pictures to display. When interpolators are used in this way, which field is kept, alternates just like in the simple doublers.

The last doubler (Phosphor) does not fit into either of these categories, but attempts to simulate a traditional CRT TV.

All doublers can be used with both true interlaced and telecined video.

=== Interpolators ===

'''VLC modes''': X, Yadif(v1.1.0+), (Discard)

These algorithms analyze the picture, detecting progressive and interlaced parts. Typically the progressive parts of the picture are passed through unchanged, although in some algorithms various kinds of filtering may be used.

For the interlaced parts, one half-picture is kept, while the other is generated out of thin air with various mathematical methods, based on the information in one or both of the original half-pictures.

The Discard algorithm is a degenerate case of this type, which does not analyze the picture. It simply keeps one half-picture and discards the other one.

The output runs at the full-picture rate, which is typically 25 (PAL) or 30 (NTSC) pictures per second. The different intended display times for the half-pictures in a true interlaced video are ignored.

Interpolators are designed for use with true interlaced video only. They will stutter if applied to telecined video.

Some doublers exist, such as Yadif (2x), which are based on interpolators. These hybrid algorithms behave like doublers (see above). They may be able to improve the picture quality compared to simple doublers, at the cost of more CPU cycles (requiring a faster processor).

=== Blenders ===

'''VLC modes''': Mean, Blend

These algorithms mix information from both half-pictures to produce a blended (mixed) full picture. This is simple and removes interlacing, but causes ghostlike trails for fast motion.

The output runs at the full-picture rate, which is typically 25 (PAL) or 30 (NTSC) pictures per second. The different intended display times for the half-pictures in a true interlaced video are ignored.

Blenders are designed for use with true interlaced video only, and will stutter with telecined video.

=== Inverse telecine ===

'''VLC modes''': IVTC(v1.2.0+)

This group of algorithms is specifically designed for removing telecine from NTSC telecined video. Inverse telecine is also known as ''IVTC'', ''film mode'' and ''3:2 reverse pulldown''.

Inverse telecine algorithms try to extract the original progressive film frames and to display them at the original framerate (24 pictures per second). In the ideal case, this perfectly restores the progressive signal. There is no loss of information and no need to generate anything out of thin air.

Due to practical reasons, these algorithms are always based on analyzing the picture. They are not and cannot be perfectly accurate, but in practice they work well for most telecined sources.

== VLC deinterlace modes ==
[[File:VLC_-_Deinterlace_Mode.png|250px|thumb|right|The deinterlace options selection]]
VLC has the following deinterlace modes. Refer to 100fps.com for illustrations. VLC does not have anything like what 100fps.com calls Motion blur, Hybrid, and Motion compensation.

=== Disabled ===
This is what 100fps.com calls "do nothing". Other names: "weave" or "no deinterlacing". Should be used for PsF ([http://en.wikipedia.org/wiki/Progressive_segmented_Frame]) content.

=== Blend ===
Blender (full resolution). Each line of the picture is created as the average of a line from the odd and a line from the even half-pictures. This ignores the fact that they are supposed to be displayed at different times.

=== Bob ===
Doubler. Display each half-picture like a full picture, by simply displaying each line twice. Preserves temporal resolution of interlaced video.

=== Discard ===
Only display one of the half-pictures, discard the other. Other name: "single field". Both temporal and vertical spatial resolutions are halved. Can be used for slower computers or to give interlaced video movie-like look with characteristic judder.

=== Linear ===
Doubler. Bob with linear interpolation: instead of displaying each line twice, line 2 is created as the average of line 1 and 3, etc.

=== Mean ===
Blender (half resolution). Display a half-picture that is created as the average of the two original half-pictures.

=== X ===
Interpolator. Generates a full picture taking the odd lines from the odd half-picture, and creating the even lines through a complicated algorithm (involving ME, MC, edge-oriented interpolation) that uses information from both half-pictures.

This is similar to what 100fps.com calls Area based.

=== Yadif (v1.1.0+)===
Interpolator. The ''Yet Another DeInterlacing Filter'' from the MPlayer project. Generates a full picture taking the odd lines from the odd half-picture, and creating the even lines through a complicated algorithm that includes both temporal and spatial interpolation.

=== Yadif (2x) (v1.1.0+)===
Doubler. Bob with Yadif interpolation. '''Caution''': Very heavy on the CPU.

=== Phosphor (v1.2.0+) ===
Doubler. This filter attempts to simulate the rendering mechanism of a traditional CRT TV. The latest two half-pictures are displayed, the old one fading out. The strength of the fade effect can be configured in All Preferences, see Video > Filters > Deinterlace.

=== IVTC (v1.2.0+) ===
Inverse telecine. Removes telecine from NTSC telecined video in realtime, losslessly recovering the progressive signal. '''Note''': Only applicable to telecined sources. Particularly useful for NTSC anime DVDs.

== Examples ==

<gallery>
File:Example- Disabled Deinterlace.png|Interlaced
File:Example- Discard interpolate.png|Discard
File:Example- Blend Deinterlace.png|Blend
File:Example- Mean Blender.png|Mean
File:Example- Bob Doubler.png|Bob
File:Example- Linear Doubler.png|Linear
File:Example-X interpolate.png|X
File:Example-Yadif 1.png|Yadif
File:Example-Yadif 2.png|Yadif (2x)
File:Example-Phospor Doubler.png|Phosphor
File:Example-Invert Telecine.png|IVTC
</gallery>

== Recommendation ==
For telecined sources, use '''IVTC'''.

For true interlaced sources, use a doubler. Which one is a matter of taste. Try '''Linear''' first; it is pretty good while simple and light on the CPU.

The classic '''Bob''' gives the lowest CPU load and is very simple.

You can use '''Yadif (2x)''' if you have a very fast CPU or fastest graphics cards such as NVIDIA Geforce, AMD Radeon or other best graphics cards.

The '''Phosphor''' (v1.2.0+) mode is different, and may be worth a try if you want a "TV look".

== Disclaimer ==
Please update this page if it contains any errors, is incomplete, or when it goes out of date.

== Appendix: Technical summary ==

This table summarizes various technical information on the algorithms. Don't be afraid to dive in - it's intended for users.

If you are a developer looking for more detailed technical understanding, this module also has a section in the [[Documentation:Hacker's Guide/Video Filters/Deinterlace|Hacker's Guide]].

'''Terminology'''

* "field" = half-picture
* "top field" = "odd lines" in other sections above (note that the numbering started from 1)
* "bottom field" = "even lines"

'''Legend'''

* Column 4:2:0 in the table = "if input has 4:2:0 chroma, then..." See [http://en.wikipedia.org/wiki/Chroma_subsampling#Sampling_systems_and_ratios] for an explanation and pictures. This format is common on DVDs.
* Column 4:2:2 in the table = "if input has 4:2:2 chroma, then..."
* C, H, FR = output Chroma, Height, Framerate
* 0 = output has 4:2:0 chroma
* 2 = output has 4:2:2 chroma
* h = half height; output has as many lines as an input half-picture
* f = full height; output has as many lines as an input full picture
* 1x = output has original framerate (original full-picture rate; typically 25 (PAL) or 30 (NTSC) times per second)
* 2x = output has double framerate (original half-picture rate; typically 50 (PAL) or 60 (NTSC) times per second)
* '''n)''' = numbered note, read below for details

Note that the picture will be scaled before being displayed, so the number of lines just tells us how detailed the picture could potentially be.

{| align="left "valign="center" border="1"
! Algo
! 4:2:0
! 4:2:2
! Algo type
! Interpolation (if applic.)
! Notes
|-
|
| C, H, FR
| C, H, FR
|
|
|
|-
| Discard
| 0, h, 1x
| 0, f, 1x
| interpolator
| none
| keeps only top field; each line is repeated
|-
| Mean
| 0, h, 1x
| 2, h, 1x
| blender
|
| half-resolution blender; '''1)'''
|-
| Blend
| 0, f, 1x
| 0, f, 1x
| blender
|
| full-resolution blender; '''2)'''
|-
| Bob
| 0, f, 2x
| 0, f, 2x
| doubler
| none
| each line is repeated; '''3)'''
|-
| Linear
| 0, f, 2x
| 2, f, 2x
| doubler
| simple linear
| first/last line copied; others interpolated; '''3)'''
|-
| X
| 0, f, 1x
| 2, f, 1x
| interpolator
| MC + edge-oriented
| keeps only top field in interlaced parts; '''4)'''
|-
| Yadif
| 0, f, 1x
| 2, f, 1x
| interpolator
| spatial/temporal
| keeps only top field in interlaced parts
|-
| Yadif (2x)
| 0, f, 2x
| 2, f, 2x
| doubler
| spatial/temporal
| Yadif and Yadif (2x) come from MPlayer
|-
| Phosphor
| '''5)''', f, 2x
| 2, f, 2x
| doubler
|
| CRT TV simulator; '''6)'''
|-
| IVTC
| 0, f, '''7)'''
| 2, f, '''7)'''
| inverse telecine
|
|
|}

The luma (Y) component scale I ("analog", 16..240) vs. J (full scale, "digital", 0..255) is preserved in all format conversions.

'''Notes'''

'''1)''' The Mean algorithm simply pairs the original lines, and averages each pair into one output line. Line 1 of output is the mean of lines 1 and 2 in input, line 2 of output is the mean of lines 3 and 4 in input, and so on.

'''2)''' Blend is slightly more sophisticated than Mean. The first line of output is copied from the first line of input. For any other output line N, the line is the mean of input lines N and N-1. That is, the second line of output is the mean of lines 1 and 2 in input, the third line is the mean of lines 2 and 3, ... and finally, the last line of output is the mean of the last two lines in input. The sliding averaging procedure preserves the original vertical resolution.

'''3)''' Bob converts the half-pictures to full-pictures by simply showing each line twice; so while the full number of lines are displayed, the true resolution is only half. The same goes for Linear, though it's a bit smarter about the process.

Still, even the basic Bob is slightly more clever than just a 2x version of Discard. In the output, the bottom field is offset by one line with respect to the top field. This is the technical reason behind the perceived full resolution magic. (As a side effect, it also causes perfectly horizontal lines to flicker in a way characteristic to the Bob algorithm. The other doublers try to overcome this.)

'''4)''' The X algorithm divides the video into blocks of 8x8 pixels, and analyzes them. It blurs the progressive blocks lightly. In the interlaced blocks, it keeps only the top field, and uses MC and edge-oriented interpolation to create the bottom field.

'''5)''' Output from Phosphor is 4:2:2 if the Upconvert mode is chosen in the settings, otherwise 4:2:0 is used.

'''6)''' Phosphor displays the latest two fields, regardless of temporal frame boundaries. To simulate phosphor light output decay, the old field is darkened by an amount that can be configured in All settings > Video > Filters > Deinterlace.

'''7)''' IVTC outputs at (4/5)x of original framerate, when inverting telecine on a locked-on cadence, and at 1x while trying to acquire lock-on.

[[Category:Video Filters]]
{{Documentation}}

Hacker Guide/How To Write a Module

2011-05-16T13:31:25Z

Technologicat: /* Troubleshooting */

== Introduction to modules writer guide ==

VLC is based on many independent modules, like most multimedia frameworks. Each module introduces a new functionnality.

You '''MUST''' '''read''' [[Documentation:Hacker%27s_Guide/Core#VLC_Pipeline_and_Modularity|VLC Core and Modules ]] and [[Documentation:VLC_Modules_Loading|How VLC loads modules]] '''before''' reading this howto.

This is a description about how you can write a new module for VLC.

== Git and Repository integration ==

=== Git ===
If you plan to submit your work to VLC upstream, be sure to look at [[Git |the git introduction]] and check the [[Git#Submitting_patches_to_the_vlc-devel_or_x264-devel|send patches part]].

=== Compiling your module ===

==== Modules.am ====
First, find the right place under [[Modules_source_tree| modules/]] to add your new code.

# If you are adding a one-file module to the repository, just add its sources to the Modules.am in the same folder.
# If you are adding a bigger modules to the repository:
#* create a new directory
#* create a new Modules.am, inside that directory.
#* Add a corresponding Makefile line at the end of configure.ac for this new Modules.am.

For example, the file {{VLCSourceFile|name=modules/video_filter/Modules.am}} tells the build system which source files go into which video filter module.

Note that the indentation in Modules.am uses a tab (ASCII 0x09), not spaces.

==== configure.ac ====

To have the module built, you need to use ''VLC_ADD_PLUGIN'' or ''PKG_ENABLE_MODULES_VLC'' inside the {{VLCSourceFile|name=configure.ac}} with your new module name as argument. Look at other modules for guidelines on how to add build and linkage options.

After changing {{VLCSourceFile|name=configure.ac}}, you will always need to rerun
./bootstrap && ./configure

=== Troubleshooting ===

Sometimes, it doesn't work, because of some libtool bug.

You probably need to go to the root of your VLC source tree, and do something akin to the following. The examples here assume the ''bash'' shell.

'''Medium version''' (try this first)

<pre>find . -name .deps -exec rm -rf \{\} \;</pre>
<pre>./bootstrap</pre>
<pre>./configure</pre>

The dangerous-looking recursive ''rm -rf'' deletes the dependency caches in the source tree.

This ensures that the build system finds your new files. Then you can go on as usual:
<pre>./compile</pre>

And finally:
<pre>./vlc --reset-plugins-cache</pre>

'''Mild version'''

Sometimes ''config.status'' after resetting the .deps cache is enough:

<pre>find . -name .deps -exec rm -rf \{\} \;</pre>
<pre>./config.status</pre>
<pre>./compile</pre>
<pre>./vlc --reset-plugins-cache</pre>

...but not always, so it may save some headaches to always use the "medium version" above.

'''Extreme version'''

If the above didn't help, you can use this more extreme version:

<pre>find . -name .deps -exec rm -rf \{\} \;</pre>
<pre>find . -name .libs -exec rm -rf \{\} \;</pre>
<pre>./bootstrap</pre>
<pre>./configure</pre>
<pre>./compile</pre>
<pre>./vlc --reset-plugins-cache</pre>

...which also deletes library caches. Note that when you do this, you '''must''' run ''configure'' and not just ''config.status'', because ''config.status'' runs configure in ''--no-create'' mode, which will not re-create the .libs caches.

==== Out of Tree Compile ====

If you need to compile just a module, please read [[OutOfTreeCompile|out of tree compilation]].

=== Loading your module ===

VLC keeps the cache of its modules (in ~/.cache/vlc/ on Linux), you'll have to delete it (or use vlc --reset-plugins-cache) so that your module gets loaded.

Then use
vlc -vvv --color --list
to check that your plugin is seen by {{VLC}}.

You should also see it in the plugins dialog of the [[Qt interface]] (Linux and Windows).

== Example of an empty module ==

We will start with a small example, so you can understand it better.
<pre>/*****************************************************************************
* Preamble
*****************************************************************************/

#ifdef HAVE_CONFIG_H
# include "config.h"
#endif

#include <vlc_common.h>
#include <vlc_plugin.h>

/*****************************************************************************
* Local prototypes.
*****************************************************************************/

static int Open ( vlc_object_t * );
static void Close ( vlc_object_t * );

/*****************************************************************************
* Module descriptor
*****************************************************************************/

vlc_module_begin()
set_shortname( _("testmodule") )
set_description( _("testmodle plug-in") )
set_capability( "testing", 0 )
set_callbacks( Open, Close )
set_category( CAT_INTERFACE ) /* choose relevant category, default categories in vlc/include/vlc_configuration.h */

add_integer( "test-angle", 1, "DVD Angle", "Default DVD Angle", false)
vlc_module_end ()

/*****************************************************************************
* Open: initialize interface
*****************************************************************************/
static int Open( vlc_object_t *p_this )
{
}

/*****************************************************************************
* Close: destroy interface
*****************************************************************************/
static void Close( vlc_object_t *p_this )
{
}</pre>

== How to write the module, aka Explanations about the empty module ==

=== Module Descriptor ===

A {{VLC}} module '''must''' include a ''description'' of itself, and the ''parameters'' it accepts.

The module descriptor begins with:
<pre>vlc_module_begin();
</pre>
You should set some basic information about your module:
<pre> set_shortname( _("DVD without menus") );
set_description( _("DVDRead Input") );
set_category( CAT_INPUT );
set_subcategory( SUBCAT_INPUT_ACCESS );
</pre>
''Note'' the use of _("") to create a string that needs to be translated.

==== Capability and score ====

'''The capability is key to the module loading'''.

It defines the type of module we are dealing with, whether it is an access, a demux, a decoder or else...

Now is the time to '''re-read [[Documentation:VLC Modules Loading#How_does_the_loading_of_modules_happens|how VLC loads modules]]'''.

 

*If VLC needs to load a specific name, it will load it by its name and VLC directly opens this module
*If VLC needs a type of modules ("I need a decoder"), VLC will load all modules matching this capability by decreasing score order until one modules's Open() function (see later) returns VLC_SUCCESS.

See the major types of [[Documentation:VLC Modules Loading#Major_Capabilities_of_Modules|capabilities of VLC]].

'''Score''' should be an integer, and related to other scores in the same category. Score 0 is a [[Documentation:VLC_Modules_Loading#Score_of_0|special case]].

Definition '''Example''':
<pre> set_capability( "testing", 10 )
</pre>
This defines a module of "testing" capability and a score of 10.

==== Configuration Categories and SubCategories ====

You should use '''one''' of the '''predefined categories''' for configuration.

The Configuration Categories and Subcategories define '''where''' the module will appear in the '''preferences'''.

The configuration categories include:

*CAT_INTERFACE
*CAT_AUDIO
*CAT_VIDEO
*CAT_INPUT
*CAT_SOUT
*CAT_ADVANCED
*CAT_PLAYLIST

You should use one of '''predefined sub-categories''' too.

See {{VLCSourceFile|name=include/vlc_configuration.h}} for definition of all configuration categories and sub-categories.

==== Configuration and Parameters Options ====

You will need options for your module. Defining new options is easy.

All options definitions take the following argument list:
<pre> add_integer(name, value, text, longtext, advanced) </pre>
*'''name''' is the string that identifies this parameter in the configuration. This name is used at the command prompt to set the configuration value.
*'''value''' is the default value for this parameter,
*'''text''' A short description of the parameter, use _("") to create a string that needs to be translated,
*'''longtext''' A complete description of the parameter, use _("") to create a string that needs to be translated,
*'''advanced''' Boolean, ADVanced Configuration. If TRUE, this parameter will only be displayed when using the --advanced flag. e.g.

 You may add the following options/parameter types to your module:

*add_integer,
*add_string,
*add_float,
*add_bool,
*add_key,
*add_file,
*add_directory,

For complete definitions, see {{VLCSourceFile|name=include/vlc_plugin.h}}

==== Callback ====

The opening and closing static functions, detailed afterwards, need to be defined in the descriptor, so VLC can discuss with the module.

=== Open(vlc_object_t *) ===

The most important function of a module, is the opening: the usually-named Open() function.
<pre>static int Open ( vlc_object_t * );</pre>
The Open() function is called when the VLC core tries to open the module, and wants to load it.

During Open(), setup of structures, devices or I/O, checks should be done. A working open should return VLC_SUCCESS. In all other cases, it will not be considered as loaded.

The '''Open()''' function is expected to allocate private data (if any), and set up the private structure.

If the Opening fails, you need to clean-up your own structures.

=== Close(vlc_object_t *) ===
The second most important function of a module, is the closing: the usually-named Close() function.
<pre>static int Close ( vlc_object_t * );</pre>

The Close() function is called when the VLC core tries to close or unload an '''already-loaded''' module.

'''NB:''' If the Open() function failed, Close() will not get called.

During Close(), close devices or I/O and cleaning of structures should be done. Do not leak memory here!

The '''Close()''' function should deallocate private data.

== SubModules ==

Submodules, created in some modules with

add_submodule()

work exactly the same way as modules. They are useful to parent modules, and to share code between modules.

The submodules will be separated in the cache, and will be loaded by VLC in the exact same way as a module.

== Modules types ==

Depending on the module capability, you will need more information, about the necessary functions to implement.

We will detail those here:
* [[Documentation:Hacker's_Guide/Access|Access]]
* [[Documentation:Hacker's_Guide/Demux|Demux]]
* [[Documentation:Hacker's_Guide/Access_Demux|Access_Demux]]
* Decoder
* Interface
* [[Documentation:Hacker's_Guide/Video_Filters|Video Filter]]
* [[Documentation:Hacker's_Guide/Audio_Filters|Audio Filter]]

{{Documentation}}

[[Category:Coding]]

Hacker Guide/How To Write a Module

2011-05-16T13:30:05Z

Technologicat: /* Troubleshooting */

== Introduction to modules writer guide ==

VLC is based on many independent modules, like most multimedia frameworks. Each module introduces a new functionnality.

You '''MUST''' '''read''' [[Documentation:Hacker%27s_Guide/Core#VLC_Pipeline_and_Modularity|VLC Core and Modules ]] and [[Documentation:VLC_Modules_Loading|How VLC loads modules]] '''before''' reading this howto.

This is a description about how you can write a new module for VLC.

== Git and Repository integration ==

=== Git ===
If you plan to submit your work to VLC upstream, be sure to look at [[Git |the git introduction]] and check the [[Git#Submitting_patches_to_the_vlc-devel_or_x264-devel|send patches part]].

=== Compiling your module ===

==== Modules.am ====
First, find the right place under [[Modules_source_tree| modules/]] to add your new code.

# If you are adding a one-file module to the repository, just add its sources to the Modules.am in the same folder.
# If you are adding a bigger modules to the repository:
#* create a new directory
#* create a new Modules.am, inside that directory.
#* Add a corresponding Makefile line at the end of configure.ac for this new Modules.am.

For example, the file {{VLCSourceFile|name=modules/video_filter/Modules.am}} tells the build system which source files go into which video filter module.

Note that the indentation in Modules.am uses a tab (ASCII 0x09), not spaces.

==== configure.ac ====

To have the module built, you need to use ''VLC_ADD_PLUGIN'' or ''PKG_ENABLE_MODULES_VLC'' inside the {{VLCSourceFile|name=configure.ac}} with your new module name as argument. Look at other modules for guidelines on how to add build and linkage options.

After changing {{VLCSourceFile|name=configure.ac}}, you will always need to rerun
./bootstrap && ./configure

=== Troubleshooting ===

Sometimes, it doesn't work, because of some libtool bug.

You probably need to go to the root of your VLC source tree, and do something akin to the following. The examples here assume the ''bash'' shell.

'''Medium version''' (try this first)

<pre>find . -name .deps -exec rm -rf \{\} \;</pre>
<pre>./bootstrap</pre>
<pre>./configure</pre>

The dangerous-looking recursive ''rm -rf'' deletes the dependency caches in the source tree.

This ensures that the build system finds your new files. Then you can go on as usual:
<pre>./compile</pre>

And finally:
<pre>./vlc --reset-plugins-cache</pre>

'''Mild version'''

Sometimes ''config.status'' is enough:

<pre>find . -name .deps -exec rm -rf \{\} \;</pre>
<pre>./config.status</pre>
<pre>./compile</pre>
<pre>./vlc --reset-plugins-cache</pre>

...but not always, so it may save some headaches to always use the "medium version" above.

'''Extreme version'''

If the above didn't help, you can use this more extreme version:

<pre>find . -name .deps -exec rm -rf \{\} \;</pre>
<pre>find . -name .libs -exec rm -rf \{\} \;</pre>
<pre>./bootstrap</pre>
<pre>./configure</pre>
<pre>./compile</pre>
<pre>./vlc --reset-plugins-cache</pre>

...which also deletes library caches. Note that when you do this, you '''must''' run ''configure'' and not just ''config.status'', because ''config.status'' runs configure in ''--no-create'' mode, which will not re-create the .libs caches.

==== Out of Tree Compile ====

If you need to compile just a module, please read [[OutOfTreeCompile|out of tree compilation]].

=== Loading your module ===

VLC keeps the cache of its modules (in ~/.cache/vlc/ on Linux), you'll have to delete it (or use vlc --reset-plugins-cache) so that your module gets loaded.

Then use
vlc -vvv --color --list
to check that your plugin is seen by {{VLC}}.

You should also see it in the plugins dialog of the [[Qt interface]] (Linux and Windows).

== Example of an empty module ==

We will start with a small example, so you can understand it better.
<pre>/*****************************************************************************
* Preamble
*****************************************************************************/

#ifdef HAVE_CONFIG_H
# include "config.h"
#endif

#include <vlc_common.h>
#include <vlc_plugin.h>

/*****************************************************************************
* Local prototypes.
*****************************************************************************/

static int Open ( vlc_object_t * );
static void Close ( vlc_object_t * );

/*****************************************************************************
* Module descriptor
*****************************************************************************/

vlc_module_begin()
set_shortname( _("testmodule") )
set_description( _("testmodle plug-in") )
set_capability( "testing", 0 )
set_callbacks( Open, Close )
set_category( CAT_INTERFACE ) /* choose relevant category, default categories in vlc/include/vlc_configuration.h */

add_integer( "test-angle", 1, "DVD Angle", "Default DVD Angle", false)
vlc_module_end ()

/*****************************************************************************
* Open: initialize interface
*****************************************************************************/
static int Open( vlc_object_t *p_this )
{
}

/*****************************************************************************
* Close: destroy interface
*****************************************************************************/
static void Close( vlc_object_t *p_this )
{
}</pre>

== How to write the module, aka Explanations about the empty module ==

=== Module Descriptor ===

A {{VLC}} module '''must''' include a ''description'' of itself, and the ''parameters'' it accepts.

The module descriptor begins with:
<pre>vlc_module_begin();
</pre>
You should set some basic information about your module:
<pre> set_shortname( _("DVD without menus") );
set_description( _("DVDRead Input") );
set_category( CAT_INPUT );
set_subcategory( SUBCAT_INPUT_ACCESS );
</pre>
''Note'' the use of _("") to create a string that needs to be translated.

==== Capability and score ====

'''The capability is key to the module loading'''.

It defines the type of module we are dealing with, whether it is an access, a demux, a decoder or else...

Now is the time to '''re-read [[Documentation:VLC Modules Loading#How_does_the_loading_of_modules_happens|how VLC loads modules]]'''.

 

*If VLC needs to load a specific name, it will load it by its name and VLC directly opens this module
*If VLC needs a type of modules ("I need a decoder"), VLC will load all modules matching this capability by decreasing score order until one modules's Open() function (see later) returns VLC_SUCCESS.

See the major types of [[Documentation:VLC Modules Loading#Major_Capabilities_of_Modules|capabilities of VLC]].

'''Score''' should be an integer, and related to other scores in the same category. Score 0 is a [[Documentation:VLC_Modules_Loading#Score_of_0|special case]].

Definition '''Example''':
<pre> set_capability( "testing", 10 )
</pre>
This defines a module of "testing" capability and a score of 10.

==== Configuration Categories and SubCategories ====

You should use '''one''' of the '''predefined categories''' for configuration.

The Configuration Categories and Subcategories define '''where''' the module will appear in the '''preferences'''.

The configuration categories include:

*CAT_INTERFACE
*CAT_AUDIO
*CAT_VIDEO
*CAT_INPUT
*CAT_SOUT
*CAT_ADVANCED
*CAT_PLAYLIST

You should use one of '''predefined sub-categories''' too.

See {{VLCSourceFile|name=include/vlc_configuration.h}} for definition of all configuration categories and sub-categories.

==== Configuration and Parameters Options ====

You will need options for your module. Defining new options is easy.

All options definitions take the following argument list:
<pre> add_integer(name, value, text, longtext, advanced) </pre>
*'''name''' is the string that identifies this parameter in the configuration. This name is used at the command prompt to set the configuration value.
*'''value''' is the default value for this parameter,
*'''text''' A short description of the parameter, use _("") to create a string that needs to be translated,
*'''longtext''' A complete description of the parameter, use _("") to create a string that needs to be translated,
*'''advanced''' Boolean, ADVanced Configuration. If TRUE, this parameter will only be displayed when using the --advanced flag. e.g.

 You may add the following options/parameter types to your module:

*add_integer,
*add_string,
*add_float,
*add_bool,
*add_key,
*add_file,
*add_directory,

For complete definitions, see {{VLCSourceFile|name=include/vlc_plugin.h}}

==== Callback ====

The opening and closing static functions, detailed afterwards, need to be defined in the descriptor, so VLC can discuss with the module.

=== Open(vlc_object_t *) ===

The most important function of a module, is the opening: the usually-named Open() function.
<pre>static int Open ( vlc_object_t * );</pre>
The Open() function is called when the VLC core tries to open the module, and wants to load it.

During Open(), setup of structures, devices or I/O, checks should be done. A working open should return VLC_SUCCESS. In all other cases, it will not be considered as loaded.

The '''Open()''' function is expected to allocate private data (if any), and set up the private structure.

If the Opening fails, you need to clean-up your own structures.

=== Close(vlc_object_t *) ===
The second most important function of a module, is the closing: the usually-named Close() function.
<pre>static int Close ( vlc_object_t * );</pre>

The Close() function is called when the VLC core tries to close or unload an '''already-loaded''' module.

'''NB:''' If the Open() function failed, Close() will not get called.

During Close(), close devices or I/O and cleaning of structures should be done. Do not leak memory here!

The '''Close()''' function should deallocate private data.

== SubModules ==

Submodules, created in some modules with

add_submodule()

work exactly the same way as modules. They are useful to parent modules, and to share code between modules.

The submodules will be separated in the cache, and will be loaded by VLC in the exact same way as a module.

== Modules types ==

Depending on the module capability, you will need more information, about the necessary functions to implement.

We will detail those here:
* [[Documentation:Hacker's_Guide/Access|Access]]
* [[Documentation:Hacker's_Guide/Demux|Demux]]
* [[Documentation:Hacker's_Guide/Access_Demux|Access_Demux]]
* Decoder
* Interface
* [[Documentation:Hacker's_Guide/Video_Filters|Video Filter]]
* [[Documentation:Hacker's_Guide/Audio_Filters|Audio Filter]]

{{Documentation}}

[[Category:Coding]]

Hacker Guide/How To Write a Module

2011-05-16T13:29:26Z

Technologicat: /* Troubleshooting */

== Introduction to modules writer guide ==

VLC is based on many independent modules, like most multimedia frameworks. Each module introduces a new functionnality.

You '''MUST''' '''read''' [[Documentation:Hacker%27s_Guide/Core#VLC_Pipeline_and_Modularity|VLC Core and Modules ]] and [[Documentation:VLC_Modules_Loading|How VLC loads modules]] '''before''' reading this howto.

This is a description about how you can write a new module for VLC.

== Git and Repository integration ==

=== Git ===
If you plan to submit your work to VLC upstream, be sure to look at [[Git |the git introduction]] and check the [[Git#Submitting_patches_to_the_vlc-devel_or_x264-devel|send patches part]].

=== Compiling your module ===

==== Modules.am ====
First, find the right place under [[Modules_source_tree| modules/]] to add your new code.

# If you are adding a one-file module to the repository, just add its sources to the Modules.am in the same folder.
# If you are adding a bigger modules to the repository:
#* create a new directory
#* create a new Modules.am, inside that directory.
#* Add a corresponding Makefile line at the end of configure.ac for this new Modules.am.

For example, the file {{VLCSourceFile|name=modules/video_filter/Modules.am}} tells the build system which source files go into which video filter module.

Note that the indentation in Modules.am uses a tab (ASCII 0x09), not spaces.

==== configure.ac ====

To have the module built, you need to use ''VLC_ADD_PLUGIN'' or ''PKG_ENABLE_MODULES_VLC'' inside the {{VLCSourceFile|name=configure.ac}} with your new module name as argument. Look at other modules for guidelines on how to add build and linkage options.

After changing {{VLCSourceFile|name=configure.ac}}, you will always need to rerun
./bootstrap && ./configure

=== Troubleshooting ===

Sometimes, it doesn't work, because of some libtool bug.

You probably need to go to the root of your VLC source tree, and do something akin to the following. The examples here assume the ''bash'' shell.

'''Medium version''' (try this first)

<pre>find . -name .deps -exec rm -rf \{\} \;</pre>
<pre>./bootstrap</pre>
<pre>./configure</pre>

The dangerous-looking recursive ''rm -rf'' deletes the dependency caches in the source tree.

This ensures that the build system finds your new files. Then you can go on as usual:
<pre>./compile</pre>

And finally:
<pre>./vlc --reset-plugins-cache</pre>

'''Mild version'''

Sometimes ''config.status'' is enough:

<pre>find . -name .deps -exec rm -rf \{\} \;</pre>
<pre>./config.status<pre>
<pre>./compile</pre>
<pre>./vlc --reset-plugins-cache</pre>

...but not always, so it may save some headaches to always use the "medium version" above.

'''Extreme version'''

If the above didn't help, you can use this more extreme version:

<pre>find . -name .deps -exec rm -rf \{\} \;</pre>
<pre>find . -name .libs -exec rm -rf \{\} \;</pre>
<pre>./bootstrap</pre>
<pre>./configure</pre>
<pre>./compile</pre>
<pre>./vlc --reset-plugins-cache</pre>

...which also deletes library caches. Note that when you do this, you '''must''' run ''configure'' and not just ''config.status'', because ''config.status'' runs configure in ''--no-create'' mode, which will not re-create the .libs caches.

==== Out of Tree Compile ====

If you need to compile just a module, please read [[OutOfTreeCompile|out of tree compilation]].

=== Loading your module ===

VLC keeps the cache of its modules (in ~/.cache/vlc/ on Linux), you'll have to delete it (or use vlc --reset-plugins-cache) so that your module gets loaded.

Then use
vlc -vvv --color --list
to check that your plugin is seen by {{VLC}}.

You should also see it in the plugins dialog of the [[Qt interface]] (Linux and Windows).

== Example of an empty module ==

We will start with a small example, so you can understand it better.
<pre>/*****************************************************************************
* Preamble
*****************************************************************************/

#ifdef HAVE_CONFIG_H
# include "config.h"
#endif

#include <vlc_common.h>
#include <vlc_plugin.h>

/*****************************************************************************
* Local prototypes.
*****************************************************************************/

static int Open ( vlc_object_t * );
static void Close ( vlc_object_t * );

/*****************************************************************************
* Module descriptor
*****************************************************************************/

vlc_module_begin()
set_shortname( _("testmodule") )
set_description( _("testmodle plug-in") )
set_capability( "testing", 0 )
set_callbacks( Open, Close )
set_category( CAT_INTERFACE ) /* choose relevant category, default categories in vlc/include/vlc_configuration.h */

add_integer( "test-angle", 1, "DVD Angle", "Default DVD Angle", false)
vlc_module_end ()

/*****************************************************************************
* Open: initialize interface
*****************************************************************************/
static int Open( vlc_object_t *p_this )
{
}

/*****************************************************************************
* Close: destroy interface
*****************************************************************************/
static void Close( vlc_object_t *p_this )
{
}</pre>

== How to write the module, aka Explanations about the empty module ==

=== Module Descriptor ===

A {{VLC}} module '''must''' include a ''description'' of itself, and the ''parameters'' it accepts.

The module descriptor begins with:
<pre>vlc_module_begin();
</pre>
You should set some basic information about your module:
<pre> set_shortname( _("DVD without menus") );
set_description( _("DVDRead Input") );
set_category( CAT_INPUT );
set_subcategory( SUBCAT_INPUT_ACCESS );
</pre>
''Note'' the use of _("") to create a string that needs to be translated.

==== Capability and score ====

'''The capability is key to the module loading'''.

It defines the type of module we are dealing with, whether it is an access, a demux, a decoder or else...

Now is the time to '''re-read [[Documentation:VLC Modules Loading#How_does_the_loading_of_modules_happens|how VLC loads modules]]'''.

 

*If VLC needs to load a specific name, it will load it by its name and VLC directly opens this module
*If VLC needs a type of modules ("I need a decoder"), VLC will load all modules matching this capability by decreasing score order until one modules's Open() function (see later) returns VLC_SUCCESS.

See the major types of [[Documentation:VLC Modules Loading#Major_Capabilities_of_Modules|capabilities of VLC]].

'''Score''' should be an integer, and related to other scores in the same category. Score 0 is a [[Documentation:VLC_Modules_Loading#Score_of_0|special case]].

Definition '''Example''':
<pre> set_capability( "testing", 10 )
</pre>
This defines a module of "testing" capability and a score of 10.

==== Configuration Categories and SubCategories ====

You should use '''one''' of the '''predefined categories''' for configuration.

The Configuration Categories and Subcategories define '''where''' the module will appear in the '''preferences'''.

The configuration categories include:

*CAT_INTERFACE
*CAT_AUDIO
*CAT_VIDEO
*CAT_INPUT
*CAT_SOUT
*CAT_ADVANCED
*CAT_PLAYLIST

You should use one of '''predefined sub-categories''' too.

See {{VLCSourceFile|name=include/vlc_configuration.h}} for definition of all configuration categories and sub-categories.

==== Configuration and Parameters Options ====

You will need options for your module. Defining new options is easy.

All options definitions take the following argument list:
<pre> add_integer(name, value, text, longtext, advanced) </pre>
*'''name''' is the string that identifies this parameter in the configuration. This name is used at the command prompt to set the configuration value.
*'''value''' is the default value for this parameter,
*'''text''' A short description of the parameter, use _("") to create a string that needs to be translated,
*'''longtext''' A complete description of the parameter, use _("") to create a string that needs to be translated,
*'''advanced''' Boolean, ADVanced Configuration. If TRUE, this parameter will only be displayed when using the --advanced flag. e.g.

 You may add the following options/parameter types to your module:

*add_integer,
*add_string,
*add_float,
*add_bool,
*add_key,
*add_file,
*add_directory,

For complete definitions, see {{VLCSourceFile|name=include/vlc_plugin.h}}

==== Callback ====

The opening and closing static functions, detailed afterwards, need to be defined in the descriptor, so VLC can discuss with the module.

=== Open(vlc_object_t *) ===

The most important function of a module, is the opening: the usually-named Open() function.
<pre>static int Open ( vlc_object_t * );</pre>
The Open() function is called when the VLC core tries to open the module, and wants to load it.

During Open(), setup of structures, devices or I/O, checks should be done. A working open should return VLC_SUCCESS. In all other cases, it will not be considered as loaded.

The '''Open()''' function is expected to allocate private data (if any), and set up the private structure.

If the Opening fails, you need to clean-up your own structures.

=== Close(vlc_object_t *) ===
The second most important function of a module, is the closing: the usually-named Close() function.
<pre>static int Close ( vlc_object_t * );</pre>

The Close() function is called when the VLC core tries to close or unload an '''already-loaded''' module.

'''NB:''' If the Open() function failed, Close() will not get called.

During Close(), close devices or I/O and cleaning of structures should be done. Do not leak memory here!

The '''Close()''' function should deallocate private data.

== SubModules ==

Submodules, created in some modules with

add_submodule()

work exactly the same way as modules. They are useful to parent modules, and to share code between modules.

The submodules will be separated in the cache, and will be loaded by VLC in the exact same way as a module.

== Modules types ==

Depending on the module capability, you will need more information, about the necessary functions to implement.

We will detail those here:
* [[Documentation:Hacker's_Guide/Access|Access]]
* [[Documentation:Hacker's_Guide/Demux|Demux]]
* [[Documentation:Hacker's_Guide/Access_Demux|Access_Demux]]
* Decoder
* Interface
* [[Documentation:Hacker's_Guide/Video_Filters|Video Filter]]
* [[Documentation:Hacker's_Guide/Audio_Filters|Audio Filter]]

{{Documentation}}

[[Category:Coding]]

Hacker Guide/How To Write a Module

2011-05-16T13:26:27Z

Technologicat: /* Troubleshooting */ note: bash

== Introduction to modules writer guide ==

VLC is based on many independent modules, like most multimedia frameworks. Each module introduces a new functionnality.

You '''MUST''' '''read''' [[Documentation:Hacker%27s_Guide/Core#VLC_Pipeline_and_Modularity|VLC Core and Modules ]] and [[Documentation:VLC_Modules_Loading|How VLC loads modules]] '''before''' reading this howto.

This is a description about how you can write a new module for VLC.

== Git and Repository integration ==

=== Git ===
If you plan to submit your work to VLC upstream, be sure to look at [[Git |the git introduction]] and check the [[Git#Submitting_patches_to_the_vlc-devel_or_x264-devel|send patches part]].

=== Compiling your module ===

==== Modules.am ====
First, find the right place under [[Modules_source_tree| modules/]] to add your new code.

# If you are adding a one-file module to the repository, just add its sources to the Modules.am in the same folder.
# If you are adding a bigger modules to the repository:
#* create a new directory
#* create a new Modules.am, inside that directory.
#* Add a corresponding Makefile line at the end of configure.ac for this new Modules.am.

For example, the file {{VLCSourceFile|name=modules/video_filter/Modules.am}} tells the build system which source files go into which video filter module.

Note that the indentation in Modules.am uses a tab (ASCII 0x09), not spaces.

==== configure.ac ====

To have the module built, you need to use ''VLC_ADD_PLUGIN'' or ''PKG_ENABLE_MODULES_VLC'' inside the {{VLCSourceFile|name=configure.ac}} with your new module name as argument. Look at other modules for guidelines on how to add build and linkage options.

After changing {{VLCSourceFile|name=configure.ac}}, you will always need to rerun
./bootstrap && ./configure

=== Troubleshooting ===

Sometimes, it doesn't work, because of some libtool bug.

You probably need to go to the root of your VLC source tree, and do something akin to the following. The examples here assume the ''bash'' shell.

'''Version 1''' (try this first)

<pre>find . -name .deps -exec rm -rf \{\} \;</pre>
<pre>./bootstrap</pre>
<pre>./configure</pre>

The dangerous-looking recursive ''rm -rf'' deletes the dependency caches in the source tree.

This ensures that the build system finds your new files. Then you can go on as usual:
<pre>./compile</pre>

And finally:
<pre>./vlc --reset-plugins-cache</pre>

'''Version 2''' (more extreme)

If the above didn't help, you can use this more extreme version:

<pre>find . -name .deps -exec rm -rf \{\} \;</pre>
<pre>find . -name .libs -exec rm -rf \{\} \;</pre>
<pre>./bootstrap</pre>
<pre>./configure</pre>
<pre>./compile</pre>
<pre>./vlc --reset-plugins-cache</pre>

...which also deletes library caches. Note that when you do this, you '''must''' run ''configure'' and not just ''config.status'', because ''config.status'' runs configure in ''--no-create'' mode, which will not re-create the .libs caches.

==== Out of Tree Compile ====

If you need to compile just a module, please read [[OutOfTreeCompile|out of tree compilation]].

=== Loading your module ===

VLC keeps the cache of its modules (in ~/.cache/vlc/ on Linux), you'll have to delete it (or use vlc --reset-plugins-cache) so that your module gets loaded.

Then use
vlc -vvv --color --list
to check that your plugin is seen by {{VLC}}.

You should also see it in the plugins dialog of the [[Qt interface]] (Linux and Windows).

== Example of an empty module ==

We will start with a small example, so you can understand it better.
<pre>/*****************************************************************************
* Preamble
*****************************************************************************/

#ifdef HAVE_CONFIG_H
# include "config.h"
#endif

#include <vlc_common.h>
#include <vlc_plugin.h>

/*****************************************************************************
* Local prototypes.
*****************************************************************************/

static int Open ( vlc_object_t * );
static void Close ( vlc_object_t * );

/*****************************************************************************
* Module descriptor
*****************************************************************************/

vlc_module_begin()
set_shortname( _("testmodule") )
set_description( _("testmodle plug-in") )
set_capability( "testing", 0 )
set_callbacks( Open, Close )
set_category( CAT_INTERFACE ) /* choose relevant category, default categories in vlc/include/vlc_configuration.h */

add_integer( "test-angle", 1, "DVD Angle", "Default DVD Angle", false)
vlc_module_end ()

/*****************************************************************************
* Open: initialize interface
*****************************************************************************/
static int Open( vlc_object_t *p_this )
{
}

/*****************************************************************************
* Close: destroy interface
*****************************************************************************/
static void Close( vlc_object_t *p_this )
{
}</pre>

== How to write the module, aka Explanations about the empty module ==

=== Module Descriptor ===

A {{VLC}} module '''must''' include a ''description'' of itself, and the ''parameters'' it accepts.

The module descriptor begins with:
<pre>vlc_module_begin();
</pre>
You should set some basic information about your module:
<pre> set_shortname( _("DVD without menus") );
set_description( _("DVDRead Input") );
set_category( CAT_INPUT );
set_subcategory( SUBCAT_INPUT_ACCESS );
</pre>
''Note'' the use of _("") to create a string that needs to be translated.

==== Capability and score ====

'''The capability is key to the module loading'''.

It defines the type of module we are dealing with, whether it is an access, a demux, a decoder or else...

Now is the time to '''re-read [[Documentation:VLC Modules Loading#How_does_the_loading_of_modules_happens|how VLC loads modules]]'''.

 

*If VLC needs to load a specific name, it will load it by its name and VLC directly opens this module
*If VLC needs a type of modules ("I need a decoder"), VLC will load all modules matching this capability by decreasing score order until one modules's Open() function (see later) returns VLC_SUCCESS.

See the major types of [[Documentation:VLC Modules Loading#Major_Capabilities_of_Modules|capabilities of VLC]].

'''Score''' should be an integer, and related to other scores in the same category. Score 0 is a [[Documentation:VLC_Modules_Loading#Score_of_0|special case]].

Definition '''Example''':
<pre> set_capability( "testing", 10 )
</pre>
This defines a module of "testing" capability and a score of 10.

==== Configuration Categories and SubCategories ====

You should use '''one''' of the '''predefined categories''' for configuration.

The Configuration Categories and Subcategories define '''where''' the module will appear in the '''preferences'''.

The configuration categories include:

*CAT_INTERFACE
*CAT_AUDIO
*CAT_VIDEO
*CAT_INPUT
*CAT_SOUT
*CAT_ADVANCED
*CAT_PLAYLIST

You should use one of '''predefined sub-categories''' too.

See {{VLCSourceFile|name=include/vlc_configuration.h}} for definition of all configuration categories and sub-categories.

==== Configuration and Parameters Options ====

You will need options for your module. Defining new options is easy.

All options definitions take the following argument list:
<pre> add_integer(name, value, text, longtext, advanced) </pre>
*'''name''' is the string that identifies this parameter in the configuration. This name is used at the command prompt to set the configuration value.
*'''value''' is the default value for this parameter,
*'''text''' A short description of the parameter, use _("") to create a string that needs to be translated,
*'''longtext''' A complete description of the parameter, use _("") to create a string that needs to be translated,
*'''advanced''' Boolean, ADVanced Configuration. If TRUE, this parameter will only be displayed when using the --advanced flag. e.g.

 You may add the following options/parameter types to your module:

*add_integer,
*add_string,
*add_float,
*add_bool,
*add_key,
*add_file,
*add_directory,

For complete definitions, see {{VLCSourceFile|name=include/vlc_plugin.h}}

==== Callback ====

The opening and closing static functions, detailed afterwards, need to be defined in the descriptor, so VLC can discuss with the module.

=== Open(vlc_object_t *) ===

The most important function of a module, is the opening: the usually-named Open() function.
<pre>static int Open ( vlc_object_t * );</pre>
The Open() function is called when the VLC core tries to open the module, and wants to load it.

During Open(), setup of structures, devices or I/O, checks should be done. A working open should return VLC_SUCCESS. In all other cases, it will not be considered as loaded.

The '''Open()''' function is expected to allocate private data (if any), and set up the private structure.

If the Opening fails, you need to clean-up your own structures.

=== Close(vlc_object_t *) ===
The second most important function of a module, is the closing: the usually-named Close() function.
<pre>static int Close ( vlc_object_t * );</pre>

The Close() function is called when the VLC core tries to close or unload an '''already-loaded''' module.

'''NB:''' If the Open() function failed, Close() will not get called.

During Close(), close devices or I/O and cleaning of structures should be done. Do not leak memory here!

The '''Close()''' function should deallocate private data.

== SubModules ==

Submodules, created in some modules with

add_submodule()

work exactly the same way as modules. They are useful to parent modules, and to share code between modules.

The submodules will be separated in the cache, and will be loaded by VLC in the exact same way as a module.

== Modules types ==

Depending on the module capability, you will need more information, about the necessary functions to implement.

We will detail those here:
* [[Documentation:Hacker's_Guide/Access|Access]]
* [[Documentation:Hacker's_Guide/Demux|Demux]]
* [[Documentation:Hacker's_Guide/Access_Demux|Access_Demux]]
* Decoder
* Interface
* [[Documentation:Hacker's_Guide/Video_Filters|Video Filter]]
* [[Documentation:Hacker's_Guide/Audio_Filters|Audio Filter]]

{{Documentation}}

[[Category:Coding]]

Hacker Guide/How To Write a Module

2011-05-16T13:25:48Z

Technologicat: /* Troubleshooting */ Added moar extreme version as alternative (delete .libs caches too; I needed this once)

== Introduction to modules writer guide ==

VLC is based on many independent modules, like most multimedia frameworks. Each module introduces a new functionnality.

You '''MUST''' '''read''' [[Documentation:Hacker%27s_Guide/Core#VLC_Pipeline_and_Modularity|VLC Core and Modules ]] and [[Documentation:VLC_Modules_Loading|How VLC loads modules]] '''before''' reading this howto.

This is a description about how you can write a new module for VLC.

== Git and Repository integration ==

=== Git ===
If you plan to submit your work to VLC upstream, be sure to look at [[Git |the git introduction]] and check the [[Git#Submitting_patches_to_the_vlc-devel_or_x264-devel|send patches part]].

=== Compiling your module ===

==== Modules.am ====
First, find the right place under [[Modules_source_tree| modules/]] to add your new code.

# If you are adding a one-file module to the repository, just add its sources to the Modules.am in the same folder.
# If you are adding a bigger modules to the repository:
#* create a new directory
#* create a new Modules.am, inside that directory.
#* Add a corresponding Makefile line at the end of configure.ac for this new Modules.am.

For example, the file {{VLCSourceFile|name=modules/video_filter/Modules.am}} tells the build system which source files go into which video filter module.

Note that the indentation in Modules.am uses a tab (ASCII 0x09), not spaces.

==== configure.ac ====

To have the module built, you need to use ''VLC_ADD_PLUGIN'' or ''PKG_ENABLE_MODULES_VLC'' inside the {{VLCSourceFile|name=configure.ac}} with your new module name as argument. Look at other modules for guidelines on how to add build and linkage options.

After changing {{VLCSourceFile|name=configure.ac}}, you will always need to rerun
./bootstrap && ./configure

=== Troubleshooting ===

Sometimes, it doesn't work, because of some libtool bug.

You probably need to go to the root of your VLC source tree, and do something akin to the following.

'''Version 1''' (try this first)

<pre>find . -name .deps -exec rm -rf \{\} \;</pre>
<pre>./bootstrap</pre>
<pre>./configure</pre>

The dangerous-looking recursive ''rm -rf'' deletes the dependency caches in the source tree.

This ensures that the build system finds your new files. Then you can go on as usual:
<pre>./compile</pre>

And finally:
<pre>./vlc --reset-plugins-cache</pre>

'''Version 2''' (more extreme)

If the above didn't help, you can use this more extreme version:

<pre>find . -name .deps -exec rm -rf \{\} \;</pre>
<pre>find . -name .libs -exec rm -rf \{\} \;</pre>
<pre>./bootstrap</pre>
<pre>./configure</pre>
<pre>./compile</pre>
<pre>./vlc --reset-plugins-cache</pre>

...which also deletes library caches. Note that when you do this, you '''must''' run ''configure'' and not just ''config.status'', because ''config.status'' runs configure in ''--no-create'' mode, which will not re-create the .libs caches.

==== Out of Tree Compile ====

If you need to compile just a module, please read [[OutOfTreeCompile|out of tree compilation]].

=== Loading your module ===

VLC keeps the cache of its modules (in ~/.cache/vlc/ on Linux), you'll have to delete it (or use vlc --reset-plugins-cache) so that your module gets loaded.

Then use
vlc -vvv --color --list
to check that your plugin is seen by {{VLC}}.

You should also see it in the plugins dialog of the [[Qt interface]] (Linux and Windows).

== Example of an empty module ==

We will start with a small example, so you can understand it better.
<pre>/*****************************************************************************
* Preamble
*****************************************************************************/

#ifdef HAVE_CONFIG_H
# include "config.h"
#endif

#include <vlc_common.h>
#include <vlc_plugin.h>

/*****************************************************************************
* Local prototypes.
*****************************************************************************/

static int Open ( vlc_object_t * );
static void Close ( vlc_object_t * );

/*****************************************************************************
* Module descriptor
*****************************************************************************/

vlc_module_begin()
set_shortname( _("testmodule") )
set_description( _("testmodle plug-in") )
set_capability( "testing", 0 )
set_callbacks( Open, Close )
set_category( CAT_INTERFACE ) /* choose relevant category, default categories in vlc/include/vlc_configuration.h */

add_integer( "test-angle", 1, "DVD Angle", "Default DVD Angle", false)
vlc_module_end ()

/*****************************************************************************
* Open: initialize interface
*****************************************************************************/
static int Open( vlc_object_t *p_this )
{
}

/*****************************************************************************
* Close: destroy interface
*****************************************************************************/
static void Close( vlc_object_t *p_this )
{
}</pre>

== How to write the module, aka Explanations about the empty module ==

=== Module Descriptor ===

A {{VLC}} module '''must''' include a ''description'' of itself, and the ''parameters'' it accepts.

The module descriptor begins with:
<pre>vlc_module_begin();
</pre>
You should set some basic information about your module:
<pre> set_shortname( _("DVD without menus") );
set_description( _("DVDRead Input") );
set_category( CAT_INPUT );
set_subcategory( SUBCAT_INPUT_ACCESS );
</pre>
''Note'' the use of _("") to create a string that needs to be translated.

==== Capability and score ====

'''The capability is key to the module loading'''.

It defines the type of module we are dealing with, whether it is an access, a demux, a decoder or else...

Now is the time to '''re-read [[Documentation:VLC Modules Loading#How_does_the_loading_of_modules_happens|how VLC loads modules]]'''.

 

*If VLC needs to load a specific name, it will load it by its name and VLC directly opens this module
*If VLC needs a type of modules ("I need a decoder"), VLC will load all modules matching this capability by decreasing score order until one modules's Open() function (see later) returns VLC_SUCCESS.

See the major types of [[Documentation:VLC Modules Loading#Major_Capabilities_of_Modules|capabilities of VLC]].

'''Score''' should be an integer, and related to other scores in the same category. Score 0 is a [[Documentation:VLC_Modules_Loading#Score_of_0|special case]].

Definition '''Example''':
<pre> set_capability( "testing", 10 )
</pre>
This defines a module of "testing" capability and a score of 10.

==== Configuration Categories and SubCategories ====

You should use '''one''' of the '''predefined categories''' for configuration.

The Configuration Categories and Subcategories define '''where''' the module will appear in the '''preferences'''.

The configuration categories include:

*CAT_INTERFACE
*CAT_AUDIO
*CAT_VIDEO
*CAT_INPUT
*CAT_SOUT
*CAT_ADVANCED
*CAT_PLAYLIST

You should use one of '''predefined sub-categories''' too.

See {{VLCSourceFile|name=include/vlc_configuration.h}} for definition of all configuration categories and sub-categories.

==== Configuration and Parameters Options ====

You will need options for your module. Defining new options is easy.

All options definitions take the following argument list:
<pre> add_integer(name, value, text, longtext, advanced) </pre>
*'''name''' is the string that identifies this parameter in the configuration. This name is used at the command prompt to set the configuration value.
*'''value''' is the default value for this parameter,
*'''text''' A short description of the parameter, use _("") to create a string that needs to be translated,
*'''longtext''' A complete description of the parameter, use _("") to create a string that needs to be translated,
*'''advanced''' Boolean, ADVanced Configuration. If TRUE, this parameter will only be displayed when using the --advanced flag. e.g.

 You may add the following options/parameter types to your module:

*add_integer,
*add_string,
*add_float,
*add_bool,
*add_key,
*add_file,
*add_directory,

For complete definitions, see {{VLCSourceFile|name=include/vlc_plugin.h}}

==== Callback ====

The opening and closing static functions, detailed afterwards, need to be defined in the descriptor, so VLC can discuss with the module.

=== Open(vlc_object_t *) ===

The most important function of a module, is the opening: the usually-named Open() function.
<pre>static int Open ( vlc_object_t * );</pre>
The Open() function is called when the VLC core tries to open the module, and wants to load it.

During Open(), setup of structures, devices or I/O, checks should be done. A working open should return VLC_SUCCESS. In all other cases, it will not be considered as loaded.

The '''Open()''' function is expected to allocate private data (if any), and set up the private structure.

If the Opening fails, you need to clean-up your own structures.

=== Close(vlc_object_t *) ===
The second most important function of a module, is the closing: the usually-named Close() function.
<pre>static int Close ( vlc_object_t * );</pre>

The Close() function is called when the VLC core tries to close or unload an '''already-loaded''' module.

'''NB:''' If the Open() function failed, Close() will not get called.

During Close(), close devices or I/O and cleaning of structures should be done. Do not leak memory here!

The '''Close()''' function should deallocate private data.

== SubModules ==

Submodules, created in some modules with

add_submodule()

work exactly the same way as modules. They are useful to parent modules, and to share code between modules.

The submodules will be separated in the cache, and will be loaded by VLC in the exact same way as a module.

== Modules types ==

Depending on the module capability, you will need more information, about the necessary functions to implement.

We will detail those here:
* [[Documentation:Hacker's_Guide/Access|Access]]
* [[Documentation:Hacker's_Guide/Demux|Demux]]
* [[Documentation:Hacker's_Guide/Access_Demux|Access_Demux]]
* Decoder
* Interface
* [[Documentation:Hacker's_Guide/Video_Filters|Video Filter]]
* [[Documentation:Hacker's_Guide/Audio_Filters|Audio Filter]]

{{Documentation}}

[[Category:Coding]]

Hacker Guide/How To Write a Module

2011-05-16T13:18:28Z

Technologicat: /* Troubleshooting */ removed bullet points from pre items to be consistent with the rest of the page

== Introduction to modules writer guide ==

VLC is based on many independent modules, like most multimedia frameworks. Each module introduces a new functionnality.

You '''MUST''' '''read''' [[Documentation:Hacker%27s_Guide/Core#VLC_Pipeline_and_Modularity|VLC Core and Modules ]] and [[Documentation:VLC_Modules_Loading|How VLC loads modules]] '''before''' reading this howto.

This is a description about how you can write a new module for VLC.

== Git and Repository integration ==

=== Git ===
If you plan to submit your work to VLC upstream, be sure to look at [[Git |the git introduction]] and check the [[Git#Submitting_patches_to_the_vlc-devel_or_x264-devel|send patches part]].

=== Compiling your module ===

==== Modules.am ====
First, find the right place under [[Modules_source_tree| modules/]] to add your new code.

# If you are adding a one-file module to the repository, just add its sources to the Modules.am in the same folder.
# If you are adding a bigger modules to the repository:
#* create a new directory
#* create a new Modules.am, inside that directory.
#* Add a corresponding Makefile line at the end of configure.ac for this new Modules.am.

For example, the file {{VLCSourceFile|name=modules/video_filter/Modules.am}} tells the build system which source files go into which video filter module.

Note that the indentation in Modules.am uses a tab (ASCII 0x09), not spaces.

==== configure.ac ====

To have the module built, you need to use ''VLC_ADD_PLUGIN'' or ''PKG_ENABLE_MODULES_VLC'' inside the {{VLCSourceFile|name=configure.ac}} with your new module name as argument. Look at other modules for guidelines on how to add build and linkage options.

After changing {{VLCSourceFile|name=configure.ac}}, you will always need to rerun
./bootstrap && ./configure

=== Troubleshooting ===

Sometimes, it doesn't work, because of some libtool bug.

You probably need to go to the root of your VLC source tree, and do something akin to the following:

<pre>find . -name .deps -exec rm -rf \{\} \;</pre>
<pre>./bootstrap</pre>
<pre>./configure</pre>

This ensures that the build system finds your new files. Then you can go on as usual:
<pre>./compile</pre>

==== Out of Tree Compile ====

If you need to compile just a module, please read [[OutOfTreeCompile|out of tree compilation]].

=== Loading your module ===

VLC keeps the cache of its modules (in ~/.cache/vlc/ on Linux), you'll have to delete it (or use vlc --reset-plugins-cache) so that your module gets loaded.

Then use
vlc -vvv --color --list
to check that your plugin is seen by {{VLC}}.

You should also see it in the plugins dialog of the [[Qt interface]] (Linux and Windows).

== Example of an empty module ==

We will start with a small example, so you can understand it better.
<pre>/*****************************************************************************
* Preamble
*****************************************************************************/

#ifdef HAVE_CONFIG_H
# include "config.h"
#endif

#include <vlc_common.h>
#include <vlc_plugin.h>

/*****************************************************************************
* Local prototypes.
*****************************************************************************/

static int Open ( vlc_object_t * );
static void Close ( vlc_object_t * );

/*****************************************************************************
* Module descriptor
*****************************************************************************/

vlc_module_begin()
set_shortname( _("testmodule") )
set_description( _("testmodle plug-in") )
set_capability( "testing", 0 )
set_callbacks( Open, Close )
set_category( CAT_INTERFACE ) /* choose relevant category, default categories in vlc/include/vlc_configuration.h */

add_integer( "test-angle", 1, "DVD Angle", "Default DVD Angle", false)
vlc_module_end ()

/*****************************************************************************
* Open: initialize interface
*****************************************************************************/
static int Open( vlc_object_t *p_this )
{
}

/*****************************************************************************
* Close: destroy interface
*****************************************************************************/
static void Close( vlc_object_t *p_this )
{
}</pre>

== How to write the module, aka Explanations about the empty module ==

=== Module Descriptor ===

A {{VLC}} module '''must''' include a ''description'' of itself, and the ''parameters'' it accepts.

The module descriptor begins with:
<pre>vlc_module_begin();
</pre>
You should set some basic information about your module:
<pre> set_shortname( _("DVD without menus") );
set_description( _("DVDRead Input") );
set_category( CAT_INPUT );
set_subcategory( SUBCAT_INPUT_ACCESS );
</pre>
''Note'' the use of _("") to create a string that needs to be translated.

==== Capability and score ====

'''The capability is key to the module loading'''.

It defines the type of module we are dealing with, whether it is an access, a demux, a decoder or else...

Now is the time to '''re-read [[Documentation:VLC Modules Loading#How_does_the_loading_of_modules_happens|how VLC loads modules]]'''.

 

*If VLC needs to load a specific name, it will load it by its name and VLC directly opens this module
*If VLC needs a type of modules ("I need a decoder"), VLC will load all modules matching this capability by decreasing score order until one modules's Open() function (see later) returns VLC_SUCCESS.

See the major types of [[Documentation:VLC Modules Loading#Major_Capabilities_of_Modules|capabilities of VLC]].

'''Score''' should be an integer, and related to other scores in the same category. Score 0 is a [[Documentation:VLC_Modules_Loading#Score_of_0|special case]].

Definition '''Example''':
<pre> set_capability( "testing", 10 )
</pre>
This defines a module of "testing" capability and a score of 10.

==== Configuration Categories and SubCategories ====

You should use '''one''' of the '''predefined categories''' for configuration.

The Configuration Categories and Subcategories define '''where''' the module will appear in the '''preferences'''.

The configuration categories include:

*CAT_INTERFACE
*CAT_AUDIO
*CAT_VIDEO
*CAT_INPUT
*CAT_SOUT
*CAT_ADVANCED
*CAT_PLAYLIST

You should use one of '''predefined sub-categories''' too.

See {{VLCSourceFile|name=include/vlc_configuration.h}} for definition of all configuration categories and sub-categories.

==== Configuration and Parameters Options ====

You will need options for your module. Defining new options is easy.

All options definitions take the following argument list:
<pre> add_integer(name, value, text, longtext, advanced) </pre>
*'''name''' is the string that identifies this parameter in the configuration. This name is used at the command prompt to set the configuration value.
*'''value''' is the default value for this parameter,
*'''text''' A short description of the parameter, use _("") to create a string that needs to be translated,
*'''longtext''' A complete description of the parameter, use _("") to create a string that needs to be translated,
*'''advanced''' Boolean, ADVanced Configuration. If TRUE, this parameter will only be displayed when using the --advanced flag. e.g.

 You may add the following options/parameter types to your module:

*add_integer,
*add_string,
*add_float,
*add_bool,
*add_key,
*add_file,
*add_directory,

For complete definitions, see {{VLCSourceFile|name=include/vlc_plugin.h}}

==== Callback ====

The opening and closing static functions, detailed afterwards, need to be defined in the descriptor, so VLC can discuss with the module.

=== Open(vlc_object_t *) ===

The most important function of a module, is the opening: the usually-named Open() function.
<pre>static int Open ( vlc_object_t * );</pre>
The Open() function is called when the VLC core tries to open the module, and wants to load it.

During Open(), setup of structures, devices or I/O, checks should be done. A working open should return VLC_SUCCESS. In all other cases, it will not be considered as loaded.

The '''Open()''' function is expected to allocate private data (if any), and set up the private structure.

If the Opening fails, you need to clean-up your own structures.

=== Close(vlc_object_t *) ===
The second most important function of a module, is the closing: the usually-named Close() function.
<pre>static int Close ( vlc_object_t * );</pre>

The Close() function is called when the VLC core tries to close or unload an '''already-loaded''' module.

'''NB:''' If the Open() function failed, Close() will not get called.

During Close(), close devices or I/O and cleaning of structures should be done. Do not leak memory here!

The '''Close()''' function should deallocate private data.

== SubModules ==

Submodules, created in some modules with

add_submodule()

work exactly the same way as modules. They are useful to parent modules, and to share code between modules.

The submodules will be separated in the cache, and will be loaded by VLC in the exact same way as a module.

== Modules types ==

Depending on the module capability, you will need more information, about the necessary functions to implement.

We will detail those here:
* [[Documentation:Hacker's_Guide/Access|Access]]
* [[Documentation:Hacker's_Guide/Demux|Demux]]
* [[Documentation:Hacker's_Guide/Access_Demux|Access_Demux]]
* Decoder
* Interface
* [[Documentation:Hacker's_Guide/Video_Filters|Video Filter]]
* [[Documentation:Hacker's_Guide/Audio_Filters|Audio Filter]]

{{Documentation}}

[[Category:Coding]]

Hacker Guide/Video Filters

2011-05-09T14:41:05Z

Technologicat: /* Some notes on video filter API */ mouse handler function notes v2

This is the [[Documentation:Hacker's_Guide|Hacker's Guide]] for video filters ('''video filter2''', VLC 1.2).

== Location in source tree, compilation ==

The right place for video filter is in the {{VLCSourceFile|name=modules/video_filter/}} folder.

See {{VLCSourceFile|name=modules/video_filter/Modules.am}} for a list of video filters currently in VLC, and which source files belong to which.

See [[Documentation:Hacker's_Guide/How_To_Write_a_Module#Git_and_Repository_integration|new module integration]] for additional help on how to integrate a new module in the build-system.

== Additional documentation ==

Reading [[Documentation:Hacker's_Guide/How_To_Write_a_Module|How to write a module]] before is very advised.

Some ''video filters'' may have their own [[Documentation:Hacker's Guide|Hacker's Guide]] sections.

Currently, the '''deinterlace''' filter has both a [[Documentation:Hacker's Guide/Video Filters/Deinterlace|Hacker's Guide section]] and [[Deinterlace|user documentation]] with some useful additional technical information.

== Programming conventions and tips ==

{{VLC}} [[video filters]] are usually written in C99, like the rest of VLC. Some use C++, like AtmoLight plugin.

The usual stuff holds here too:
* Use '''assert()''' for checking conditions that, in presence of no bugs, always hold.
** ''Don't'' use '''assert()''' for checking stuff that may fail at runtime. Handle errors gracefully.
* Keep your code readable. Make private functions when it simplifies things.
* Prefer 80 characters per line at maximum, if possible.

Particularly for video filters: keep in mind that this is low-level stuff and needs to be fast.
* How fast? Note that the source may be interlaced, and someone may have thrown a framerate doubler into the filter chain... this means that for a 60i signal, the entire processing chain from decode to render has about 16ms per output frame.
* Keep the required memory bandwidth down.
** Don't copy whole pictures unless you absolutely must. Moving pixels costs time.
** Allocate an output picture and render directly into it.
** If you need more than one pass, do in-place processing in the subsequent ones if possible.
** If your processing cannot avoid multiple passes, consider doing all passes for a picture line (or a few) at a time, instead of iterating over the whole picture separately for each pass. This will help to keep down the pressure on the CPU cache.
* Consider writing the inner processing loops in vectorized inline assembly, such as MMX. This can often gain a factor of 2-8x in speed, depending on how much and what kind of processing your algorithm needs.

== Some notes on video filter API ==

'''Note''': See {{VLCSourceFile|name=modules/video_filter/deinterlace/deinterlace.c}} and {{VLCSourceFile|name=modules/video_filter/deinterlace/deinterlace.h}} for an example of everything discussed in this section. See also the [[Documentation:Hacker's Guide/Video Filters/Deinterlace|Deinterlace Hacker's Guide]] for some further details.

The video filters use an object-based model. Calls to the filter will come with an instance pointer, so each filter instance can keep local data.

Usually, each filter defines its own internal data structure called '''filter_sys_t'''. It is completely private to the filter.

The API for video filters in VLC 1.2 is called '''video filter2'''. This requires the functions '''Open()''' and '''Close()'''.

The '''Open()''' function is expected to allocate private data (if any), and set up the filter structure with pointers to a frame processing function ('''p_filter->pf_video_filter''', which is the actual filter), a flush function ('''p_filter->pf_video_flush'''), and a mouse event function ('''p_filter->pf_video_mouse'''). The relevant types are defined in {{VLCSourceFile|name=include/vlc_filter.h}}.

The '''Close()''' function should deallocate private data.

The mouse function should convert mouse events, from filter output format to filter input format (note the direction!). This is needed when doing scaling or geometrical distortions, so that the mouse position will map correctly. See {{VLCSourceFile|name=include/vlc_filter.h}} for the definition and '''Mouse()''' in {{VLCSourceFile|name=modules/video_filter/deinterlace/deinterlace.c}} for an example. If you do not need to remap mouse events, you can leave '''p_filter->pf_video_mouse''' as '''NULL'''.

=== Filter work (frame processing function) ===
The frame processing function takes a frame as input ('''picture_t*''', {{VLCSourceFile|name=include/vlc_picture.h}}), and optionally outputs one or more frames ('''picture_t*''') as a linked list. The usual case is one frame in, one frame out, but this is not a requirement. (See framerate doublers and IVTC in the [[deinterlace]] module for counterexamples.)

If you do not wish to output a frame at a particular call to your processing function, you can return '''NULL'''. If you wish to output several, make a linked list by using the '''p_next''' member of '''picture_t'''.

You don't have to necessarily output the ''same'' frame (after processing) that just came in. You can keep a private input frame history in your '''filter_sys_t''', pick the base frame for your output from that, and adjust the PTSs (presentation timestamps, member '''date''' in '''picture_t''') of your output as you deem necessary. The deinterlacer provides an example of how to do this.

Note however that there may be other filters in the chain that already set up an offset; this leaves less time (until the designated PTS) for the rest of the filters to do their processing.

The input picture coming in to the processing function is considered as owned by the processing function, and should be released before returning. To do this, use
<pre>picture_Release( p_inpic );</pre>

=== Frame allocation ===
Be sure to allocate your output pictures using <pre>picture_t *p_outpic = filter_NewPicture( p_filter );</pre>
This function requests a new picture from the private pool (see {{VLCSourceFile|name=src/video_output/vout_wrapper.c}}). This is very important! Be aware that pictures created like
<pre>picture_t *p_temp = picture_NewFromFormat( &p_input_pic->format );</pre>
lack a shared memory context (in Linux; or the equivalent in other OS), and thus cannot be passed to the video output logic. If your filter attempts to do so, VLC will crash.

If you need temporary pictures, you can allocate those using '''picture_NewFromFormat()''', but remember to ''always create output pictures using '''filter_NewPicture()'''''. Here an ''output picture'' is defined as a picture which goes out from the filter to the caller.

There is a limit to the number of simultaneous picture slots available in the private pool. Currently (<nowiki>[71992c5fbc75ee2c94cd2a3ab1aaca70bfc688a9]</nowiki>) this is 3 pictures for the filter layer. See constant '''private_picture''' in {{VLCSourceFile|name=src/video_output/vout_wrapper.c}}. If you try to allocate more output pictures than this (during a single processing call), the allocation will fail.

=== Notes ===
* In VLC 1.2, the video format (resolution, chroma) never changes on the fly. The whole filter chain is '''Close()'''d and then '''Open()'''ed again if the format changes.

* Note that it ''is'' allowed to flush the filter without closing it. This actually happens under some circumstances, so ''don't deallocate stuff in your flush function''. Rather, allocate dynamic resources in '''Open()''', and deallocate them in '''Close()''' (or in private functions called from those).

* Supported input and output formats are left up to each filter. The usual thing is to support YUV formats, e.g. I420, J420, YV12, I422 and J422. See {{VLCSourceFile|name=include/vlc_fourcc.h}}. The most important ones to support are I420 and I422.

== Pitch, visible pitch, planes et al. ==

=== Pitch and visible pitch ===
Important to know:

* i_pitch = number of (macro)pixels on one line
* i_visible_pitch = number of (macro)pixels on one line, adjusted for memory alignment constraints etc. (see below)

Note that the pitches may be different for each plane, and indeed in YUV formats the luma ('''Y_PLANE''') and chroma ('''U_PLANE''', '''V_PLANE''') have different pitches due to chroma subsampling. (In 4:2:0 formats, the number of lines differs, too.)

Pitches also tend to slightly differ depending on how the picture was allocated, even if the visible size is the same. The input picture to the filter may have one pitch, temporary pictures ('''picture_NewFromFormat()''') another, and output pictures ('''filter_NewPicture()''') yet another.

''Be sure to always use the correct pitch when you handle the pixels of a picture.'' That is, always use the '''i_pitch''' member of the actual '''plane_t''' you are working on.

If you absolutely need matching pitches (e.g. if you are glueing in processing code from another GPL-compatible project which assumes this and you don't want to rewrite it...), consider making temporary copies with '''picture_NewFromFormat()'''. See {{VLCSourceFile|name=modules/video_filter/deinterlace/deinterlace.c}} (the history mechanism <nowiki>pp_history[]</nowiki>) and {{VLCSourceFile|name=modules/video_filter/deinterlace/algo_yadif.c}} (its usage) for an example.

If you are processing pixels from one picture to another, the safe thing to do is to take the smallest '''i_visible_pitch''', and loop from x = 0 until the visible pitch has been reached, but use the individual '''i_pitch'''es for computing the pixel locations. See {{VLCSourceFile|name=modules/video_filter/deinterlace/helpers.c}} for examples.

=== Video_format_t vs. Plane_t ===

In '''video_format_t''', the '''i_visible_width''' and '''i_visible_height''' members go with '''i_x_offset''' and '''i_y_offset''', but they are not related to '''plane_t::i_visible_*'''.

The '''plane_t::i_visible_*''' fields are there to know which part of the memory planes can at most be displayed. Usually these are different from the whole surface due to memory alignment constraints (like mod 16 for SSE2 or to be aligned on a macroblock). Those values are decided when the picture is allocated and they usually don't change for a given picture pointer (but they can change, like in direct3d buffers).

The '''video_format_t:i_visible_width/height''' and '''i_x/y_offset''' are more a 'hint' regarding what will be actually displayed. A filter should update '''filter_t::fmt_out''' if needed to ensure that the allocated pictures get the right value. A filter should, however, not limit its processing to this area, because VLC supports dynamic cropping.

Thus, the correct thing to do in a filter is to process the whole picture area, or at least the first i_visible_* lines and pixels ''according to the '''plane_t'''s''. Filters do not need to care about margin positioning or that there even exists such a thing as margins.

== Disclaimer ==

The information is mostly based on a few months' hacking on the deinterlacer.

This page is still a stub. Please add new relevant stuff.

Also, the stuff that already exists could use better organization and formatting.

{{Documentation}}

Hacker Guide/Video Filters

2011-05-09T14:23:40Z

Technologicat: /* Some notes on video filter API */ mouse handler function notes

This is the [[Documentation:Hacker's_Guide|Hacker's Guide]] for video filters ('''video filter2''', VLC 1.2).

== Location in source tree, compilation ==

The right place for video filter is in the {{VLCSourceFile|name=modules/video_filter/}} folder.

See {{VLCSourceFile|name=modules/video_filter/Modules.am}} for a list of video filters currently in VLC, and which source files belong to which.

See [[Documentation:Hacker's_Guide/How_To_Write_a_Module#Git_and_Repository_integration|new module integration]] for additional help on how to integrate a new module in the build-system.

== Additional documentation ==

Reading [[Documentation:Hacker's_Guide/How_To_Write_a_Module|How to write a module]] before is very advised.

Some ''video filters'' may have their own [[Documentation:Hacker's Guide|Hacker's Guide]] sections.

Currently, the '''deinterlace''' filter has both a [[Documentation:Hacker's Guide/Video Filters/Deinterlace|Hacker's Guide section]] and [[Deinterlace|user documentation]] with some useful additional technical information.

== Programming conventions and tips ==

{{VLC}} [[video filters]] are usually written in C99, like the rest of VLC. Some use C++, like AtmoLight plugin.

The usual stuff holds here too:
* Use '''assert()''' for checking conditions that, in presence of no bugs, always hold.
** ''Don't'' use '''assert()''' for checking stuff that may fail at runtime. Handle errors gracefully.
* Keep your code readable. Make private functions when it simplifies things.
* Prefer 80 characters per line at maximum, if possible.

Particularly for video filters: keep in mind that this is low-level stuff and needs to be fast.
* How fast? Note that the source may be interlaced, and someone may have thrown a framerate doubler into the filter chain... this means that for a 60i signal, the entire processing chain from decode to render has about 16ms per output frame.
* Keep the required memory bandwidth down.
** Don't copy whole pictures unless you absolutely must. Moving pixels costs time.
** Allocate an output picture and render directly into it.
** If you need more than one pass, do in-place processing in the subsequent ones if possible.
** If your processing cannot avoid multiple passes, consider doing all passes for a picture line (or a few) at a time, instead of iterating over the whole picture separately for each pass. This will help to keep down the pressure on the CPU cache.
* Consider writing the inner processing loops in vectorized inline assembly, such as MMX. This can often gain a factor of 2-8x in speed, depending on how much and what kind of processing your algorithm needs.

== Some notes on video filter API ==

'''Note''': See {{VLCSourceFile|name=modules/video_filter/deinterlace/deinterlace.c}} and {{VLCSourceFile|name=modules/video_filter/deinterlace/deinterlace.h}} for an example of everything discussed in this section. See also the [[Documentation:Hacker's Guide/Video Filters/Deinterlace|Deinterlace Hacker's Guide]] for some further details.

The video filters use an object-based model. Calls to the filter will come with an instance pointer, so each filter instance can keep local data.

Usually, each filter defines its own internal data structure called '''filter_sys_t'''. It is completely private to the filter.

The API for video filters in VLC 1.2 is called '''video filter2'''. This requires the functions '''Open()''' and '''Close()'''.

The '''Open()''' function is expected to allocate private data (if any), and set up the filter structure with pointers to a frame processing function ('''p_filter->pf_video_filter''', which is the actual filter), a flush function ('''p_filter->pf_video_flush'''), and a mouse event function ('''p_filter->pf_video_mouse'''). The relevant types are defined in {{VLCSourceFile|name=include/vlc_filter.h}}.

The '''Close()''' function should deallocate private data.

The mouse function should convert mouse events, from filter output format to filter input format. This is needed when doing scaling or geometrical distortions, so that the mouse position will map correctly. See {{VLCSourceFile|name=include/vlc_filter.h}} for the definition and '''Mouse()''' in {{VLCSourceFile|name=modules/video_filter/deinterlace/deinterlace.c}} for an example.

=== Filter work (frame processing function) ===
The frame processing function takes a frame as input ('''picture_t*''', {{VLCSourceFile|name=include/vlc_picture.h}}), and optionally outputs one or more frames ('''picture_t*''') as a linked list. The usual case is one frame in, one frame out, but this is not a requirement. (See framerate doublers and IVTC in the [[deinterlace]] module for counterexamples.)

If you do not wish to output a frame at a particular call to your processing function, you can return '''NULL'''. If you wish to output several, make a linked list by using the '''p_next''' member of '''picture_t'''.

You don't have to necessarily output the ''same'' frame (after processing) that just came in. You can keep a private input frame history in your '''filter_sys_t''', pick the base frame for your output from that, and adjust the PTSs (presentation timestamps, member '''date''' in '''picture_t''') of your output as you deem necessary. The deinterlacer provides an example of how to do this.

Note however that there may be other filters in the chain that already set up an offset; this leaves less time (until the designated PTS) for the rest of the filters to do their processing.

The input picture coming in to the processing function is considered as owned by the processing function, and should be released before returning. To do this, use
<pre>picture_Release( p_inpic );</pre>

=== Frame allocation ===
Be sure to allocate your output pictures using <pre>picture_t *p_outpic = filter_NewPicture( p_filter );</pre>
This function requests a new picture from the private pool (see {{VLCSourceFile|name=src/video_output/vout_wrapper.c}}). This is very important! Be aware that pictures created like
<pre>picture_t *p_temp = picture_NewFromFormat( &p_input_pic->format );</pre>
lack a shared memory context (in Linux; or the equivalent in other OS), and thus cannot be passed to the video output logic. If your filter attempts to do so, VLC will crash.

If you need temporary pictures, you can allocate those using '''picture_NewFromFormat()''', but remember to ''always create output pictures using '''filter_NewPicture()'''''. Here an ''output picture'' is defined as a picture which goes out from the filter to the caller.

There is a limit to the number of simultaneous picture slots available in the private pool. Currently (<nowiki>[71992c5fbc75ee2c94cd2a3ab1aaca70bfc688a9]</nowiki>) this is 3 pictures for the filter layer. See constant '''private_picture''' in {{VLCSourceFile|name=src/video_output/vout_wrapper.c}}. If you try to allocate more output pictures than this (during a single processing call), the allocation will fail.

=== Notes ===
* In VLC 1.2, the video format (resolution, chroma) never changes on the fly. The whole filter chain is '''Close()'''d and then '''Open()'''ed again if the format changes.

* Note that it ''is'' allowed to flush the filter without closing it. This actually happens under some circumstances, so ''don't deallocate stuff in your flush function''. Rather, allocate dynamic resources in '''Open()''', and deallocate them in '''Close()''' (or in private functions called from those).

* Supported input and output formats are left up to each filter. The usual thing is to support YUV formats, e.g. I420, J420, YV12, I422 and J422. See {{VLCSourceFile|name=include/vlc_fourcc.h}}. The most important ones to support are I420 and I422.

== Pitch, visible pitch, planes et al. ==

=== Pitch and visible pitch ===
Important to know:

* i_pitch = number of (macro)pixels on one line
* i_visible_pitch = number of (macro)pixels on one line, adjusted for memory alignment constraints etc. (see below)

Note that the pitches may be different for each plane, and indeed in YUV formats the luma ('''Y_PLANE''') and chroma ('''U_PLANE''', '''V_PLANE''') have different pitches due to chroma subsampling. (In 4:2:0 formats, the number of lines differs, too.)

Pitches also tend to slightly differ depending on how the picture was allocated, even if the visible size is the same. The input picture to the filter may have one pitch, temporary pictures ('''picture_NewFromFormat()''') another, and output pictures ('''filter_NewPicture()''') yet another.

''Be sure to always use the correct pitch when you handle the pixels of a picture.'' That is, always use the '''i_pitch''' member of the actual '''plane_t''' you are working on.

If you absolutely need matching pitches (e.g. if you are glueing in processing code from another GPL-compatible project which assumes this and you don't want to rewrite it...), consider making temporary copies with '''picture_NewFromFormat()'''. See {{VLCSourceFile|name=modules/video_filter/deinterlace/deinterlace.c}} (the history mechanism <nowiki>pp_history[]</nowiki>) and {{VLCSourceFile|name=modules/video_filter/deinterlace/algo_yadif.c}} (its usage) for an example.

If you are processing pixels from one picture to another, the safe thing to do is to take the smallest '''i_visible_pitch''', and loop from x = 0 until the visible pitch has been reached, but use the individual '''i_pitch'''es for computing the pixel locations. See {{VLCSourceFile|name=modules/video_filter/deinterlace/helpers.c}} for examples.

=== Video_format_t vs. Plane_t ===

In '''video_format_t''', the '''i_visible_width''' and '''i_visible_height''' members go with '''i_x_offset''' and '''i_y_offset''', but they are not related to '''plane_t::i_visible_*'''.

The '''plane_t::i_visible_*''' fields are there to know which part of the memory planes can at most be displayed. Usually these are different from the whole surface due to memory alignment constraints (like mod 16 for SSE2 or to be aligned on a macroblock). Those values are decided when the picture is allocated and they usually don't change for a given picture pointer (but they can change, like in direct3d buffers).

The '''video_format_t:i_visible_width/height''' and '''i_x/y_offset''' are more a 'hint' regarding what will be actually displayed. A filter should update '''filter_t::fmt_out''' if needed to ensure that the allocated pictures get the right value. A filter should, however, not limit its processing to this area, because VLC supports dynamic cropping.

Thus, the correct thing to do in a filter is to process the whole picture area, or at least the first i_visible_* lines and pixels ''according to the '''plane_t'''s''. Filters do not need to care about margin positioning or that there even exists such a thing as margins.

== Disclaimer ==

The information is mostly based on a few months' hacking on the deinterlacer.

This page is still a stub. Please add new relevant stuff.

Also, the stuff that already exists could use better organization and formatting.

{{Documentation}}

Hacker Guide/Video Filters

2011-05-09T14:17:46Z

Technologicat: /* Filter work (frame processing function) */ input picture ownership

This is the [[Documentation:Hacker's_Guide|Hacker's Guide]] for video filters ('''video filter2''', VLC 1.2).

== Location in source tree, compilation ==

The right place for video filter is in the {{VLCSourceFile|name=modules/video_filter/}} folder.

See {{VLCSourceFile|name=modules/video_filter/Modules.am}} for a list of video filters currently in VLC, and which source files belong to which.

See [[Documentation:Hacker's_Guide/How_To_Write_a_Module#Git_and_Repository_integration|new module integration]] for additional help on how to integrate a new module in the build-system.

== Additional documentation ==

Reading [[Documentation:Hacker's_Guide/How_To_Write_a_Module|How to write a module]] before is very advised.

Some ''video filters'' may have their own [[Documentation:Hacker's Guide|Hacker's Guide]] sections.

Currently, the '''deinterlace''' filter has both a [[Documentation:Hacker's Guide/Video Filters/Deinterlace|Hacker's Guide section]] and [[Deinterlace|user documentation]] with some useful additional technical information.

== Programming conventions and tips ==

{{VLC}} [[video filters]] are usually written in C99, like the rest of VLC. Some use C++, like AtmoLight plugin.

The usual stuff holds here too:
* Use '''assert()''' for checking conditions that, in presence of no bugs, always hold.
** ''Don't'' use '''assert()''' for checking stuff that may fail at runtime. Handle errors gracefully.
* Keep your code readable. Make private functions when it simplifies things.
* Prefer 80 characters per line at maximum, if possible.

Particularly for video filters: keep in mind that this is low-level stuff and needs to be fast.
* How fast? Note that the source may be interlaced, and someone may have thrown a framerate doubler into the filter chain... this means that for a 60i signal, the entire processing chain from decode to render has about 16ms per output frame.
* Keep the required memory bandwidth down.
** Don't copy whole pictures unless you absolutely must. Moving pixels costs time.
** Allocate an output picture and render directly into it.
** If you need more than one pass, do in-place processing in the subsequent ones if possible.
** If your processing cannot avoid multiple passes, consider doing all passes for a picture line (or a few) at a time, instead of iterating over the whole picture separately for each pass. This will help to keep down the pressure on the CPU cache.
* Consider writing the inner processing loops in vectorized inline assembly, such as MMX. This can often gain a factor of 2-8x in speed, depending on how much and what kind of processing your algorithm needs.

== Some notes on video filter API ==

'''Note''': See {{VLCSourceFile|name=modules/video_filter/deinterlace/deinterlace.c}} and {{VLCSourceFile|name=modules/video_filter/deinterlace/deinterlace.h}} for an example of everything discussed in this section. See also the [[Documentation:Hacker's Guide/Video Filters/Deinterlace|Deinterlace Hacker's Guide]] for some further details.

The video filters use an object-based model. Calls to the filter will come with an instance pointer, so each filter instance can keep local data.

Usually, each filter defines its own internal data structure called '''filter_sys_t'''. It is completely private to the filter.

The API for video filters in VLC 1.2 is called '''video filter2'''. This requires the functions '''Open()''' and '''Close()'''. The '''Open()''' function is expected to allocate private data (if any), and set up the filter structure with pointers to a frame processing function ('''p_filter->pf_video_filter''', which is the actual filter), a flush function ('''p_filter->pf_video_flush'''), and a mouse event function ('''p_filter->pf_video_mouse'''). The '''Close()''' function should deallocate private data.

=== Filter work (frame processing function) ===
The frame processing function takes a frame as input ('''picture_t*''', {{VLCSourceFile|name=include/vlc_picture.h}}), and optionally outputs one or more frames ('''picture_t*''') as a linked list. The usual case is one frame in, one frame out, but this is not a requirement. (See framerate doublers and IVTC in the [[deinterlace]] module for counterexamples.)

If you do not wish to output a frame at a particular call to your processing function, you can return '''NULL'''. If you wish to output several, make a linked list by using the '''p_next''' member of '''picture_t'''.

You don't have to necessarily output the ''same'' frame (after processing) that just came in. You can keep a private input frame history in your '''filter_sys_t''', pick the base frame for your output from that, and adjust the PTSs (presentation timestamps, member '''date''' in '''picture_t''') of your output as you deem necessary. The deinterlacer provides an example of how to do this.

Note however that there may be other filters in the chain that already set up an offset; this leaves less time (until the designated PTS) for the rest of the filters to do their processing.

The input picture coming in to the processing function is considered as owned by the processing function, and should be released before returning. To do this, use
<pre>picture_Release( p_inpic );</pre>

=== Frame allocation ===
Be sure to allocate your output pictures using <pre>picture_t *p_outpic = filter_NewPicture( p_filter );</pre>
This function requests a new picture from the private pool (see {{VLCSourceFile|name=src/video_output/vout_wrapper.c}}). This is very important! Be aware that pictures created like
<pre>picture_t *p_temp = picture_NewFromFormat( &p_input_pic->format );</pre>
lack a shared memory context (in Linux; or the equivalent in other OS), and thus cannot be passed to the video output logic. If your filter attempts to do so, VLC will crash.

If you need temporary pictures, you can allocate those using '''picture_NewFromFormat()''', but remember to ''always create output pictures using '''filter_NewPicture()'''''. Here an ''output picture'' is defined as a picture which goes out from the filter to the caller.

There is a limit to the number of simultaneous picture slots available in the private pool. Currently (<nowiki>[71992c5fbc75ee2c94cd2a3ab1aaca70bfc688a9]</nowiki>) this is 3 pictures for the filter layer. See constant '''private_picture''' in {{VLCSourceFile|name=src/video_output/vout_wrapper.c}}. If you try to allocate more output pictures than this (during a single processing call), the allocation will fail.

=== Notes ===
* In VLC 1.2, the video format (resolution, chroma) never changes on the fly. The whole filter chain is '''Close()'''d and then '''Open()'''ed again if the format changes.

* Note that it ''is'' allowed to flush the filter without closing it. This actually happens under some circumstances, so ''don't deallocate stuff in your flush function''. Rather, allocate dynamic resources in '''Open()''', and deallocate them in '''Close()''' (or in private functions called from those).

* Supported input and output formats are left up to each filter. The usual thing is to support YUV formats, e.g. I420, J420, YV12, I422 and J422. See {{VLCSourceFile|name=include/vlc_fourcc.h}}. The most important ones to support are I420 and I422.

== Pitch, visible pitch, planes et al. ==

=== Pitch and visible pitch ===
Important to know:

* i_pitch = number of (macro)pixels on one line
* i_visible_pitch = number of (macro)pixels on one line, adjusted for memory alignment constraints etc. (see below)

Note that the pitches may be different for each plane, and indeed in YUV formats the luma ('''Y_PLANE''') and chroma ('''U_PLANE''', '''V_PLANE''') have different pitches due to chroma subsampling. (In 4:2:0 formats, the number of lines differs, too.)

Pitches also tend to slightly differ depending on how the picture was allocated, even if the visible size is the same. The input picture to the filter may have one pitch, temporary pictures ('''picture_NewFromFormat()''') another, and output pictures ('''filter_NewPicture()''') yet another.

''Be sure to always use the correct pitch when you handle the pixels of a picture.'' That is, always use the '''i_pitch''' member of the actual '''plane_t''' you are working on.

If you absolutely need matching pitches (e.g. if you are glueing in processing code from another GPL-compatible project which assumes this and you don't want to rewrite it...), consider making temporary copies with '''picture_NewFromFormat()'''. See {{VLCSourceFile|name=modules/video_filter/deinterlace/deinterlace.c}} (the history mechanism <nowiki>pp_history[]</nowiki>) and {{VLCSourceFile|name=modules/video_filter/deinterlace/algo_yadif.c}} (its usage) for an example.

If you are processing pixels from one picture to another, the safe thing to do is to take the smallest '''i_visible_pitch''', and loop from x = 0 until the visible pitch has been reached, but use the individual '''i_pitch'''es for computing the pixel locations. See {{VLCSourceFile|name=modules/video_filter/deinterlace/helpers.c}} for examples.

=== Video_format_t vs. Plane_t ===

In '''video_format_t''', the '''i_visible_width''' and '''i_visible_height''' members go with '''i_x_offset''' and '''i_y_offset''', but they are not related to '''plane_t::i_visible_*'''.

The '''plane_t::i_visible_*''' fields are there to know which part of the memory planes can at most be displayed. Usually these are different from the whole surface due to memory alignment constraints (like mod 16 for SSE2 or to be aligned on a macroblock). Those values are decided when the picture is allocated and they usually don't change for a given picture pointer (but they can change, like in direct3d buffers).

The '''video_format_t:i_visible_width/height''' and '''i_x/y_offset''' are more a 'hint' regarding what will be actually displayed. A filter should update '''filter_t::fmt_out''' if needed to ensure that the allocated pictures get the right value. A filter should, however, not limit its processing to this area, because VLC supports dynamic cropping.

Thus, the correct thing to do in a filter is to process the whole picture area, or at least the first i_visible_* lines and pixels ''according to the '''plane_t'''s''. Filters do not need to care about margin positioning or that there even exists such a thing as margins.

== Disclaimer ==

The information is mostly based on a few months' hacking on the deinterlacer.

This page is still a stub. Please add new relevant stuff.

Also, the stuff that already exists could use better organization and formatting.

{{Documentation}}

Hacker Guide/Video Filters

2011-05-09T14:16:08Z

Technologicat: /* Filter work (frame processing function) */ correct member name (p_next)

This is the [[Documentation:Hacker's_Guide|Hacker's Guide]] for video filters ('''video filter2''', VLC 1.2).

== Location in source tree, compilation ==

The right place for video filter is in the {{VLCSourceFile|name=modules/video_filter/}} folder.

See {{VLCSourceFile|name=modules/video_filter/Modules.am}} for a list of video filters currently in VLC, and which source files belong to which.

See [[Documentation:Hacker's_Guide/How_To_Write_a_Module#Git_and_Repository_integration|new module integration]] for additional help on how to integrate a new module in the build-system.

== Additional documentation ==

Reading [[Documentation:Hacker's_Guide/How_To_Write_a_Module|How to write a module]] before is very advised.

Some ''video filters'' may have their own [[Documentation:Hacker's Guide|Hacker's Guide]] sections.

Currently, the '''deinterlace''' filter has both a [[Documentation:Hacker's Guide/Video Filters/Deinterlace|Hacker's Guide section]] and [[Deinterlace|user documentation]] with some useful additional technical information.

== Programming conventions and tips ==

{{VLC}} [[video filters]] are usually written in C99, like the rest of VLC. Some use C++, like AtmoLight plugin.

The usual stuff holds here too:
* Use '''assert()''' for checking conditions that, in presence of no bugs, always hold.
** ''Don't'' use '''assert()''' for checking stuff that may fail at runtime. Handle errors gracefully.
* Keep your code readable. Make private functions when it simplifies things.
* Prefer 80 characters per line at maximum, if possible.

Particularly for video filters: keep in mind that this is low-level stuff and needs to be fast.
* How fast? Note that the source may be interlaced, and someone may have thrown a framerate doubler into the filter chain... this means that for a 60i signal, the entire processing chain from decode to render has about 16ms per output frame.
* Keep the required memory bandwidth down.
** Don't copy whole pictures unless you absolutely must. Moving pixels costs time.
** Allocate an output picture and render directly into it.
** If you need more than one pass, do in-place processing in the subsequent ones if possible.
** If your processing cannot avoid multiple passes, consider doing all passes for a picture line (or a few) at a time, instead of iterating over the whole picture separately for each pass. This will help to keep down the pressure on the CPU cache.
* Consider writing the inner processing loops in vectorized inline assembly, such as MMX. This can often gain a factor of 2-8x in speed, depending on how much and what kind of processing your algorithm needs.

== Some notes on video filter API ==

'''Note''': See {{VLCSourceFile|name=modules/video_filter/deinterlace/deinterlace.c}} and {{VLCSourceFile|name=modules/video_filter/deinterlace/deinterlace.h}} for an example of everything discussed in this section. See also the [[Documentation:Hacker's Guide/Video Filters/Deinterlace|Deinterlace Hacker's Guide]] for some further details.

The video filters use an object-based model. Calls to the filter will come with an instance pointer, so each filter instance can keep local data.

Usually, each filter defines its own internal data structure called '''filter_sys_t'''. It is completely private to the filter.

The API for video filters in VLC 1.2 is called '''video filter2'''. This requires the functions '''Open()''' and '''Close()'''. The '''Open()''' function is expected to allocate private data (if any), and set up the filter structure with pointers to a frame processing function ('''p_filter->pf_video_filter''', which is the actual filter), a flush function ('''p_filter->pf_video_flush'''), and a mouse event function ('''p_filter->pf_video_mouse'''). The '''Close()''' function should deallocate private data.

=== Filter work (frame processing function) ===
The frame processing function takes a frame as input ('''picture_t*''', {{VLCSourceFile|name=include/vlc_picture.h}}), and optionally outputs one or more frames ('''picture_t*''') as a linked list. The usual case is one frame in, one frame out, but this is not a requirement. (See framerate doublers and IVTC in the [[deinterlace]] module for counterexamples.)

If you do not wish to output a frame at a particular call to your processing function, you can return '''NULL'''. If you wish to output several, make a linked list by using the '''p_next''' member of '''picture_t'''.

You don't have to necessarily output the ''same'' frame (after processing) that just came in. You can keep a private input frame history in your '''filter_sys_t''', pick the base frame for your output from that, and adjust the PTSs (presentation timestamps, member '''date''' in '''picture_t''') of your output as you deem necessary. The deinterlacer provides an example of how to do this.

Note however that there may be other filters in the chain that already set up an offset; this leaves less time (until the designated PTS) for the rest of the filters to do their processing.

=== Frame allocation ===
Be sure to allocate your output pictures using <pre>picture_t *p_outpic = filter_NewPicture( p_filter );</pre>
This function requests a new picture from the private pool (see {{VLCSourceFile|name=src/video_output/vout_wrapper.c}}). This is very important! Be aware that pictures created like
<pre>picture_t *p_temp = picture_NewFromFormat( &p_input_pic->format );</pre>
lack a shared memory context (in Linux; or the equivalent in other OS), and thus cannot be passed to the video output logic. If your filter attempts to do so, VLC will crash.

If you need temporary pictures, you can allocate those using '''picture_NewFromFormat()''', but remember to ''always create output pictures using '''filter_NewPicture()'''''. Here an ''output picture'' is defined as a picture which goes out from the filter to the caller.

There is a limit to the number of simultaneous picture slots available in the private pool. Currently (<nowiki>[71992c5fbc75ee2c94cd2a3ab1aaca70bfc688a9]</nowiki>) this is 3 pictures for the filter layer. See constant '''private_picture''' in {{VLCSourceFile|name=src/video_output/vout_wrapper.c}}. If you try to allocate more output pictures than this (during a single processing call), the allocation will fail.

=== Notes ===
* In VLC 1.2, the video format (resolution, chroma) never changes on the fly. The whole filter chain is '''Close()'''d and then '''Open()'''ed again if the format changes.

* Note that it ''is'' allowed to flush the filter without closing it. This actually happens under some circumstances, so ''don't deallocate stuff in your flush function''. Rather, allocate dynamic resources in '''Open()''', and deallocate them in '''Close()''' (or in private functions called from those).

* Supported input and output formats are left up to each filter. The usual thing is to support YUV formats, e.g. I420, J420, YV12, I422 and J422. See {{VLCSourceFile|name=include/vlc_fourcc.h}}. The most important ones to support are I420 and I422.

== Pitch, visible pitch, planes et al. ==

=== Pitch and visible pitch ===
Important to know:

* i_pitch = number of (macro)pixels on one line
* i_visible_pitch = number of (macro)pixels on one line, adjusted for memory alignment constraints etc. (see below)

Note that the pitches may be different for each plane, and indeed in YUV formats the luma ('''Y_PLANE''') and chroma ('''U_PLANE''', '''V_PLANE''') have different pitches due to chroma subsampling. (In 4:2:0 formats, the number of lines differs, too.)

Pitches also tend to slightly differ depending on how the picture was allocated, even if the visible size is the same. The input picture to the filter may have one pitch, temporary pictures ('''picture_NewFromFormat()''') another, and output pictures ('''filter_NewPicture()''') yet another.

''Be sure to always use the correct pitch when you handle the pixels of a picture.'' That is, always use the '''i_pitch''' member of the actual '''plane_t''' you are working on.

If you absolutely need matching pitches (e.g. if you are glueing in processing code from another GPL-compatible project which assumes this and you don't want to rewrite it...), consider making temporary copies with '''picture_NewFromFormat()'''. See {{VLCSourceFile|name=modules/video_filter/deinterlace/deinterlace.c}} (the history mechanism <nowiki>pp_history[]</nowiki>) and {{VLCSourceFile|name=modules/video_filter/deinterlace/algo_yadif.c}} (its usage) for an example.

If you are processing pixels from one picture to another, the safe thing to do is to take the smallest '''i_visible_pitch''', and loop from x = 0 until the visible pitch has been reached, but use the individual '''i_pitch'''es for computing the pixel locations. See {{VLCSourceFile|name=modules/video_filter/deinterlace/helpers.c}} for examples.

=== Video_format_t vs. Plane_t ===

In '''video_format_t''', the '''i_visible_width''' and '''i_visible_height''' members go with '''i_x_offset''' and '''i_y_offset''', but they are not related to '''plane_t::i_visible_*'''.

The '''plane_t::i_visible_*''' fields are there to know which part of the memory planes can at most be displayed. Usually these are different from the whole surface due to memory alignment constraints (like mod 16 for SSE2 or to be aligned on a macroblock). Those values are decided when the picture is allocated and they usually don't change for a given picture pointer (but they can change, like in direct3d buffers).

The '''video_format_t:i_visible_width/height''' and '''i_x/y_offset''' are more a 'hint' regarding what will be actually displayed. A filter should update '''filter_t::fmt_out''' if needed to ensure that the allocated pictures get the right value. A filter should, however, not limit its processing to this area, because VLC supports dynamic cropping.

Thus, the correct thing to do in a filter is to process the whole picture area, or at least the first i_visible_* lines and pixels ''according to the '''plane_t'''s''. Filters do not need to care about margin positioning or that there even exists such a thing as margins.

== Disclaimer ==

The information is mostly based on a few months' hacking on the deinterlacer.

This page is still a stub. Please add new relevant stuff.

Also, the stuff that already exists could use better organization and formatting.

{{Documentation}}

Hacker Guide/How To Write a Module

2011-05-09T14:13:37Z

Technologicat: /* Modules.am */ video filter example fixed

== Introduction to modules writer guide ==

VLC is based on many independent modules, like most multimedia frameworks. Each module introduces a new functionnality.

You '''MUST''' '''read''' [[Documentation:Hacker%27s_Guide/Core#VLC_Pipeline_and_Modularity|VLC Core and Modules ]] and [[Documentation:VLC_Modules_Loading|How VLC loads modules]] '''before''' reading this howto.

This is a description about how you can write a new module for VLC.

== Git and Repository integration ==

=== Git ===
If you plan to submit your work to VLC upstream, be sure to look at [[Git |the git introduction]] and check the [[Git#Submitting_patches_to_the_vlc-devel_or_x264-devel|send patches part]].

=== Compiling your module ===

==== Modules.am ====
First, find the right place under [[Modules_source_tree| modules/]] to add your new code.

# If you are adding a one-file module to the repository, just add its sources to the Modules.am in the same folder.
# If you are adding a bigger modules to the repository:
#* create a new directory
#* create a new Modules.am, inside that directory.
#* Add a corresponding Makefile line at the end of configure.ac for this new Modules.am.

For example, the file {{VLCSourceFile|name=modules/video_filter/Modules.am}} tells the build system which source files go into which video filter module.

Note that the indentation in Modules.am uses a tab (ASCII 0x09), not spaces.

==== configure.ac ====

To have the module built, you need to use ''VLC_ADD_PLUGIN'' or ''PKG_ENABLE_MODULES_VLC'' inside the {{VLCSourceFile|name=configure.ac}} with your new module name as argument. Look at other modules for guidelines on how to add build and linkage options.

After changing {{VLCSourceFile|name=configure.ac}}, you will always need to rerun
./bootstrap && ./configure

=== Troubleshooting ===

Sometimes, it doesn't work, because of some libtool bug.

You probably need to go to the root of your VLC source tree, and do something akin to the following:

* <pre>find . -name .deps -exec rm -rf \{\} \;</pre>
* <pre>./bootstrap</pre>
* <pre>./configure</pre>

This ensures that the build system finds your new files. Then you can go on as usual:
* <pre>./compile</pre>

==== Out of Tree Compile ====

If you need to compile just a module, please read [[OutOfTreeCompile|out of tree compilation]].

=== Loading your module ===

VLC keeps the cache of its modules (in ~/.cache/vlc/ on Linux), you'll have to delete it (or use vlc --reset-plugins-cache) so that your module gets loaded.

Then use
vlc -vvv --color --list
to check that your plugin is seen by {{VLC}}.

You should also see it in the plugins dialog of the [[Qt interface]] (Linux and Windows).

== Example of an empty module ==

We will start with a small example, so you can understand it better.
<pre>/*****************************************************************************
* Preamble
*****************************************************************************/

#ifdef HAVE_CONFIG_H
# include "config.h"
#endif

#include <vlc_common.h>
#include <vlc_plugin.h>

/*****************************************************************************
* Local prototypes.
*****************************************************************************/

static int Open ( vlc_object_t * );
static void Close ( vlc_object_t * );

/*****************************************************************************
* Module descriptor
*****************************************************************************/

vlc_module_begin()
set_shortname( _("testmodule") )
set_description( _("testmodle plug-in") )
set_capability( "testing", 0 )
set_callbacks( Open, Close )
set_category( CAT_INTERFACE ) /* choose relevant category, default categories in vlc/include/vlc_configuration.h */

add_integer( "test-angle", 1, "DVD Angle", "Default DVD Angle", false)
vlc_module_end ()

/*****************************************************************************
* Open: initialize interface
*****************************************************************************/
static int Open( vlc_object_t *p_this )
{
}

/*****************************************************************************
* Close: destroy interface
*****************************************************************************/
static void Close( vlc_object_t *p_this )
{
}</pre>

== How to write the module, aka Explanations about the empty module ==

=== Module Descriptor ===

A {{VLC}} module '''must''' include a ''description'' of itself, and the ''parameters'' it accepts.

The module descriptor begins with:
<pre>vlc_module_begin();
</pre>
You should set some basic information about your module:
<pre> set_shortname( _("DVD without menus") );
set_description( _("DVDRead Input") );
set_category( CAT_INPUT );
set_subcategory( SUBCAT_INPUT_ACCESS );
</pre>
''Note'' the use of _("") to create a string that needs to be translated.

==== Capability and score ====

'''The capability is key to the module loading'''.

It defines the type of module we are dealing with, whether it is an access, a demux, a decoder or else...

Now is the time to '''re-read [[Documentation:VLC Modules Loading#How_does_the_loading_of_modules_happens|how VLC loads modules]]'''.

 

*If VLC needs to load a specific name, it will load it by its name and VLC directly opens this module
*If VLC needs a type of modules ("I need a decoder"), VLC will load all modules matching this capability by decreasing score order until one modules's Open() function (see later) returns VLC_SUCCESS.

See the major types of [[Documentation:VLC Modules Loading#Major_Capabilities_of_Modules|capabilities of VLC]].

'''Score''' should be an integer, and related to other scores in the same category. Score 0 is a [[Documentation:VLC_Modules_Loading#Score_of_0|special case]].

Definition '''Example''':
<pre> set_capability( "testing", 10 )
</pre>
This defines a module of "testing" capability and a score of 10.

==== Configuration Categories and SubCategories ====

You should use '''one''' of the '''predefined categories''' for configuration.

The Configuration Categories and Subcategories define '''where''' the module will appear in the '''preferences'''.

The configuration categories include:

*CAT_INTERFACE
*CAT_AUDIO
*CAT_VIDEO
*CAT_INPUT
*CAT_SOUT
*CAT_ADVANCED
*CAT_PLAYLIST

You should use one of '''predefined sub-categories''' too.

See {{VLCSourceFile|name=include/vlc_configuration.h}} for definition of all configuration categories and sub-categories.

==== Configuration and Parameters Options ====

You will need options for your module. Defining new options is easy.

All options definitions take the following argument list:
<pre> add_integer(name, value, text, longtext, advanced) </pre>
*'''name''' is the string that identifies this parameter in the configuration. This name is used at the command prompt to set the configuration value.
*'''value''' is the default value for this parameter,
*'''text''' A short description of the parameter, use _("") to create a string that needs to be translated,
*'''longtext''' A complete description of the parameter, use _("") to create a string that needs to be translated,
*'''advanced''' Boolean, ADVanced Configuration. If TRUE, this parameter will only be displayed when using the --advanced flag. e.g.

 You may add the following options/parameter types to your module:

*add_integer,
*add_string,
*add_float,
*add_bool,
*add_key,
*add_file,
*add_directory,

For complete definitions, see {{VLCSourceFile|name=include/vlc_plugin.h}}

==== Callback ====

The opening and closing static functions, detailed afterwards, need to be defined in the descriptor, so VLC can discuss with the module.

=== Open(vlc_object_t *) ===

The most important function of a module, is the opening: the usually-named Open() function.
<pre>static int Open ( vlc_object_t * );</pre>
The Open() function is called when the VLC core tries to open the module, and wants to load it.

During Open(), setup of structures, devices or I/O, checks should be done. A working open should return VLC_SUCCESS. In all other cases, it will not be considered as loaded.

The '''Open()''' function is expected to allocate private data (if any), and set up the private structure.

If the Opening fails, you need to clean-up your own structures.

=== Close(vlc_object_t *) ===
The second most important function of a module, is the closing: the usually-named Close() function.
<pre>static int Close ( vlc_object_t * );</pre>

The Close() function is called when the VLC core tries to close or unload an '''already-loaded''' module.

'''NB:''' If the Open() function failed, Close() will not get called.

During Close(), close devices or I/O and cleaning of structures should be done. Do not leak memory here!

The '''Close()''' function should deallocate private data.

== SubModules ==

Submodules, created in some modules with

add_submodule()

work exactly the same way as modules. They are useful to parent modules, and to share code between modules.

The submodules will be separated in the cache, and will be loaded by VLC in the exact same way as a module.

== Modules types ==

Depending on the module capability, you will need more information, about the necessary functions to implement.

We will detail those here:
* [[Documentation:Hacker's_Guide/Access|Access]]
* [[Documentation:Hacker's_Guide/Demux|Demux]]
* [[Documentation:Hacker's_Guide/Access_Demux|Access_Demux]]
* Decoder
* Interface
* [[Documentation:Hacker's_Guide/Video_Filters|Video Filter]]
* [[Documentation:Hacker's_Guide/Audio_Filters|Audio Filter]]

{{Documentation}}

[[Category:Coding]]

Hacker Guide/Video Filters

2011-05-09T02:13:15Z

Technologicat: /* Individual filter docs */ highlight

This is the [[Documentation:Hacker's_Guide|Hacker's Guide]] for video filters ('''video filter2''', VLC 1.2).

== Location in source tree ==

Look in {{VLCSourceFile|name=modules/video_filter/}}.

See {{VLCSourceFile|name=modules/video_filter/Modules.am}} for a list of video filters currently in VLC, and which source files belong to which.

== Adding new source files ==

Want to develop a new filter or add new source files to an existing one?

The file {{VLCSourceFile|name=modules/video_filter/Modules.am}} tells the build system which source files go into which filter, so go ahead and update it. Note that the indentation in the file uses a tab (ASCII 0x09), not spaces.

When done, you'll probably need to go to the root of your VLC source tree, and do something akin to the following:
* <pre>find . -name .deps -exec rm -rf \{\} \;</pre>
* <pre>./bootstrap</pre>
* <pre>./configure</pre>

This ensures that the build system finds your new files. Then you can go on as usual:
* <pre>./compile</pre>

== Individual filter docs ==

Some filters may have their own [[Documentation:Hacker's Guide|Hacker's Guide]] sections.

Currently, the '''deinterlace''' filter has both a [[Documentation:Hacker's Guide/Video Filters/Deinterlace|Hacker's Guide section]] and [[Deinterlace|user documentation]] with some useful technical information.

== Programming conventions and tips ==

VLC video filters are usually written in C99, like the rest of VLC. The AtmoLight plugin uses C++, though.

The usual stuff holds here too:
* Use '''assert()''' for checking conditions that, in presence of no bugs, always hold.
** ''Don't'' use '''assert()''' for checking stuff that may fail at runtime. Handle errors gracefully.
* Keep your code readable. Make private functions when it simplifies things.
* Prefer 80 characters per line at maximum, if possible.

Particularly for video filters: keep in mind that this is low-level stuff and needs to be fast.
* How fast? Note that the source may be interlaced, and someone may have thrown a framerate doubler into the filter chain... this means that for a 60i signal, the entire processing chain from decode to render has about 16ms per output frame.
* Keep the required memory bandwidth down.
** Don't copy whole pictures unless you absolutely must. Moving pixels costs time.
** Allocate an output picture and render directly into it.
** If you need more than one pass, do in-place processing in the subsequent ones if possible.
** If your processing cannot avoid multiple passes, consider doing all passes for a picture line (or a few) at a time, instead of iterating over the whole picture separately for each pass. This will help to keep down the pressure on the CPU cache.
* Consider writing the inner processing loops in vectorized inline assembly, such as MMX. This can often gain a factor of 2-8x in speed, depending on how much and what kind of processing your algorithm needs.

== Some notes on the API ==

'''Note''': See {{VLCSourceFile|name=modules/video_filter/deinterlace/deinterlace.c}} and {{VLCSourceFile|name=modules/video_filter/deinterlace/deinterlace.h}} for an example of everything discussed in this section. See also the [[Documentation:Hacker's Guide/Video Filters/Deinterlace|Deinterlace Hacker's Guide]] for some further details.

The video filters use an object-based model. Calls to the filter will come with an instance pointer, so each filter instance can keep local data.

Usually, each filter defines its own internal data structure called '''filter_sys_t'''. It is completely private to the filter.

The API for video filters in VLC 1.2 is called '''video filter2'''. This requires the functions '''Open()''' and '''Close()'''. The '''Open()''' function is expected to allocate private data (if any), and set up the filter structure with pointers to a frame processing function ('''p_filter->pf_video_filter''', which is the actual filter), a flush function ('''p_filter->pf_video_flush'''), and a mouse event function ('''p_filter->pf_video_mouse'''). The '''Close()''' function should deallocate private data.

The frame processing function takes a frame as input ('''picture_t*''', {{VLCSourceFile|name=include/vlc_picture.h}}), and optionally outputs one or more frames ('''picture_t*''') as a linked list. The usual case is one frame in, one frame out, but this is not a requirement. (See framerate doublers and IVTC in the deinterlace module for counterexamples.)

If you do not wish to output a frame at a particular call to your processing function, you can return '''NULL'''. If you wish to output several, make a linked list by using the '''next''' member of '''picture_t'''.

You don't have to necessarily output the ''same'' frame (after processing) that just came in. You can keep a private input frame history in your '''filter_sys_t''', pick the base frame for your output from that, and adjust the PTSs (presentation timestamps, member '''date''' in '''picture_t''') of your output as you deem necessary. The deinterlacer provides an example of how to do this.

Note however that there may be other filters in the chain that already set up an offset; this leaves less time (until the designated PTS) for the rest of the filters to do their processing.

Be sure to allocate your output pictures using <pre>picture_t *p_outpic = filter_NewPicture( p_filter );</pre>
This function requests a new picture from the private pool (see {{VLCSourceFile|name=src/video_output/vout_wrapper.c}}). This is very important! Be aware that pictures created like
<pre>picture_t *p_temp = picture_NewFromFormat( &p_input_pic->format );</pre>
lack a shared memory context (in Linux; or the equivalent in other OS), and thus cannot be passed to the video output logic. If your filter attempts to do so, VLC will crash.

If you need temporary pictures, you can allocate those using '''picture_NewFromFormat()''', but remember to ''always create output pictures using '''filter_NewPicture()'''''. Here an ''output picture'' is defined as a picture which goes out from the filter to the caller.

There is a limit to the number of simultaneous picture slots available in the private pool. Currently (<nowiki>[71992c5fbc75ee2c94cd2a3ab1aaca70bfc688a9]</nowiki>) this is 3 pictures for the filter layer. See constant '''private_picture''' in {{VLCSourceFile|name=src/video_output/vout_wrapper.c}}. If you try to allocate more output pictures than this (during a single processing call), the allocation will fail.

In VLC 1.2, the video format (resolution, chroma) never changes on the fly. The whole filter chain is '''Close()'''d and then '''Open()'''ed again if the format changes.

Note that it ''is'' allowed to flush the filter without closing it. This actually happens under some circumstances, so ''don't deallocate stuff in your flush function''. Rather, allocate dynamic resources in '''Open()''', and deallocate them in '''Close()''' (or in private functions called from those).

Supported input and output formats are left up to each filter. The usual thing is to support YUV formats, e.g. I420, J420, YV12, I422 and J422. See {{VLCSourceFile|name=include/vlc_fourcc.h}}.

== Pitch, visible pitch, et al. ==

Important to know:

* i_pitch = number of (macro)pixels on one line
* i_visible_pitch = number of (macro)pixels on one line, adjusted for memory alignment constraints etc. (see below)

Note that the pitches may be different for each plane, and indeed in YUV formats the luma ('''Y_PLANE''') and chroma ('''U_PLANE''', '''V_PLANE''') have different pitches due to chroma subsampling. (In 4:2:0 formats, the number of lines differs, too.)

Pitches also tend to slightly differ depending on how the picture was allocated, even if the visible size is the same. The input picture to the filter may have one pitch, temporary pictures ('''picture_NewFromFormat()''') another, and output pictures ('''filter_NewPicture()''') yet another.

''Be sure to always use the correct pitch when you handle the pixels of a picture.'' That is, always use the '''i_pitch''' member of the actual '''plane_t''' you are working on.

If you absolutely need matching pitches (e.g. if you are glueing in processing code from another GPL-compatible project which assumes this and you don't want to rewrite it...), consider making temporary copies with '''picture_NewFromFormat()'''. See {{VLCSourceFile|name=modules/video_filter/deinterlace/deinterlace.c}} (the history mechanism <nowiki>pp_history[]</nowiki>) and {{VLCSourceFile|name=modules/video_filter/deinterlace/algo_yadif.c}} (its usage) for an example.

If you are processing pixels from one picture to another, the safe thing to do is to take the smallest '''i_visible_pitch''', and loop from x = 0 until the visible pitch has been reached, but use the individual '''i_pitch'''es for computing the pixel locations. See {{VLCSourceFile|name=modules/video_filter/deinterlace/helpers.c}} for examples.

=== video_format_t vs. plane_t ===

In '''video_format_t''', the '''i_visible_width''' and '''i_visible_height''' members go with '''i_x_offset''' and '''i_y_offset''', but they are not related to '''plane_t::i_visible_*'''.

The '''plane_t::i_visible_*''' fields are there to know which part of the memory planes can at most be displayed. Usually these are different from the whole surface due to memory alignment constraints (like mod 16 for SSE2 or to be aligned on a macroblock). Those values are decided when the picture is allocated and they usually don't change for a given picture pointer (but they can change, like in direct3d buffers).

The '''video_format_t:i_visible_width/height''' and '''i_x/y_offset''' are more a 'hint' regarding what will be actually displayed. A filter should update '''filter_t::fmt_out''' if needed to ensure that the allocated pictures get the right value. A filter should, however, not limit its processing to this area, because VLC supports dynamic cropping.

Thus, the correct thing to do in a filter is to process the whole picture area, or at least the first i_visible_* lines and pixels ''according to the '''plane_t'''s''. Filters do not need to care about margin positioning or that there even exists such a thing as margins.

== Disclaimer ==

The information is mostly based on a few months' hacking on the deinterlacer.

This page is still a stub. Please add new relevant stuff.

Also, the stuff that already exists could use better organization and formatting.

{{Documentation}}

Hacker Guide/Video Filters

2011-05-09T02:12:12Z

Technologicat: /* Adding new source files */

This is the [[Documentation:Hacker's_Guide|Hacker's Guide]] for video filters ('''video filter2''', VLC 1.2).

== Location in source tree ==

Look in {{VLCSourceFile|name=modules/video_filter/}}.

See {{VLCSourceFile|name=modules/video_filter/Modules.am}} for a list of video filters currently in VLC, and which source files belong to which.

== Adding new source files ==

Want to develop a new filter or add new source files to an existing one?

The file {{VLCSourceFile|name=modules/video_filter/Modules.am}} tells the build system which source files go into which filter, so go ahead and update it. Note that the indentation in the file uses a tab (ASCII 0x09), not spaces.

When done, you'll probably need to go to the root of your VLC source tree, and do something akin to the following:
* <pre>find . -name .deps -exec rm -rf \{\} \;</pre>
* <pre>./bootstrap</pre>
* <pre>./configure</pre>

This ensures that the build system finds your new files. Then you can go on as usual:
* <pre>./compile</pre>

== Individual filter docs ==

Some filters may have their own [[Documentation:Hacker's Guide|Hacker's Guide]] sections.

Currently, the deinterlace filter has both a [[Documentation:Hacker's Guide/Video Filters/Deinterlace|Hacker's Guide section]] and [[Deinterlace|user documentation]] with some useful technical information.

== Programming conventions and tips ==

VLC video filters are usually written in C99, like the rest of VLC. The AtmoLight plugin uses C++, though.

The usual stuff holds here too:
* Use '''assert()''' for checking conditions that, in presence of no bugs, always hold.
** ''Don't'' use '''assert()''' for checking stuff that may fail at runtime. Handle errors gracefully.
* Keep your code readable. Make private functions when it simplifies things.
* Prefer 80 characters per line at maximum, if possible.

Particularly for video filters: keep in mind that this is low-level stuff and needs to be fast.
* How fast? Note that the source may be interlaced, and someone may have thrown a framerate doubler into the filter chain... this means that for a 60i signal, the entire processing chain from decode to render has about 16ms per output frame.
* Keep the required memory bandwidth down.
** Don't copy whole pictures unless you absolutely must. Moving pixels costs time.
** Allocate an output picture and render directly into it.
** If you need more than one pass, do in-place processing in the subsequent ones if possible.
** If your processing cannot avoid multiple passes, consider doing all passes for a picture line (or a few) at a time, instead of iterating over the whole picture separately for each pass. This will help to keep down the pressure on the CPU cache.
* Consider writing the inner processing loops in vectorized inline assembly, such as MMX. This can often gain a factor of 2-8x in speed, depending on how much and what kind of processing your algorithm needs.

== Some notes on the API ==

'''Note''': See {{VLCSourceFile|name=modules/video_filter/deinterlace/deinterlace.c}} and {{VLCSourceFile|name=modules/video_filter/deinterlace/deinterlace.h}} for an example of everything discussed in this section. See also the [[Documentation:Hacker's Guide/Video Filters/Deinterlace|Deinterlace Hacker's Guide]] for some further details.

The video filters use an object-based model. Calls to the filter will come with an instance pointer, so each filter instance can keep local data.

Usually, each filter defines its own internal data structure called '''filter_sys_t'''. It is completely private to the filter.

The API for video filters in VLC 1.2 is called '''video filter2'''. This requires the functions '''Open()''' and '''Close()'''. The '''Open()''' function is expected to allocate private data (if any), and set up the filter structure with pointers to a frame processing function ('''p_filter->pf_video_filter''', which is the actual filter), a flush function ('''p_filter->pf_video_flush'''), and a mouse event function ('''p_filter->pf_video_mouse'''). The '''Close()''' function should deallocate private data.

The frame processing function takes a frame as input ('''picture_t*''', {{VLCSourceFile|name=include/vlc_picture.h}}), and optionally outputs one or more frames ('''picture_t*''') as a linked list. The usual case is one frame in, one frame out, but this is not a requirement. (See framerate doublers and IVTC in the deinterlace module for counterexamples.)

If you do not wish to output a frame at a particular call to your processing function, you can return '''NULL'''. If you wish to output several, make a linked list by using the '''next''' member of '''picture_t'''.

You don't have to necessarily output the ''same'' frame (after processing) that just came in. You can keep a private input frame history in your '''filter_sys_t''', pick the base frame for your output from that, and adjust the PTSs (presentation timestamps, member '''date''' in '''picture_t''') of your output as you deem necessary. The deinterlacer provides an example of how to do this.

Note however that there may be other filters in the chain that already set up an offset; this leaves less time (until the designated PTS) for the rest of the filters to do their processing.

Be sure to allocate your output pictures using <pre>picture_t *p_outpic = filter_NewPicture( p_filter );</pre>
This function requests a new picture from the private pool (see {{VLCSourceFile|name=src/video_output/vout_wrapper.c}}). This is very important! Be aware that pictures created like
<pre>picture_t *p_temp = picture_NewFromFormat( &p_input_pic->format );</pre>
lack a shared memory context (in Linux; or the equivalent in other OS), and thus cannot be passed to the video output logic. If your filter attempts to do so, VLC will crash.

If you need temporary pictures, you can allocate those using '''picture_NewFromFormat()''', but remember to ''always create output pictures using '''filter_NewPicture()'''''. Here an ''output picture'' is defined as a picture which goes out from the filter to the caller.

There is a limit to the number of simultaneous picture slots available in the private pool. Currently (<nowiki>[71992c5fbc75ee2c94cd2a3ab1aaca70bfc688a9]</nowiki>) this is 3 pictures for the filter layer. See constant '''private_picture''' in {{VLCSourceFile|name=src/video_output/vout_wrapper.c}}. If you try to allocate more output pictures than this (during a single processing call), the allocation will fail.

In VLC 1.2, the video format (resolution, chroma) never changes on the fly. The whole filter chain is '''Close()'''d and then '''Open()'''ed again if the format changes.

Note that it ''is'' allowed to flush the filter without closing it. This actually happens under some circumstances, so ''don't deallocate stuff in your flush function''. Rather, allocate dynamic resources in '''Open()''', and deallocate them in '''Close()''' (or in private functions called from those).

Supported input and output formats are left up to each filter. The usual thing is to support YUV formats, e.g. I420, J420, YV12, I422 and J422. See {{VLCSourceFile|name=include/vlc_fourcc.h}}.

== Pitch, visible pitch, et al. ==

Important to know:

* i_pitch = number of (macro)pixels on one line
* i_visible_pitch = number of (macro)pixels on one line, adjusted for memory alignment constraints etc. (see below)

Note that the pitches may be different for each plane, and indeed in YUV formats the luma ('''Y_PLANE''') and chroma ('''U_PLANE''', '''V_PLANE''') have different pitches due to chroma subsampling. (In 4:2:0 formats, the number of lines differs, too.)

Pitches also tend to slightly differ depending on how the picture was allocated, even if the visible size is the same. The input picture to the filter may have one pitch, temporary pictures ('''picture_NewFromFormat()''') another, and output pictures ('''filter_NewPicture()''') yet another.

''Be sure to always use the correct pitch when you handle the pixels of a picture.'' That is, always use the '''i_pitch''' member of the actual '''plane_t''' you are working on.

If you absolutely need matching pitches (e.g. if you are glueing in processing code from another GPL-compatible project which assumes this and you don't want to rewrite it...), consider making temporary copies with '''picture_NewFromFormat()'''. See {{VLCSourceFile|name=modules/video_filter/deinterlace/deinterlace.c}} (the history mechanism <nowiki>pp_history[]</nowiki>) and {{VLCSourceFile|name=modules/video_filter/deinterlace/algo_yadif.c}} (its usage) for an example.

If you are processing pixels from one picture to another, the safe thing to do is to take the smallest '''i_visible_pitch''', and loop from x = 0 until the visible pitch has been reached, but use the individual '''i_pitch'''es for computing the pixel locations. See {{VLCSourceFile|name=modules/video_filter/deinterlace/helpers.c}} for examples.

=== video_format_t vs. plane_t ===

In '''video_format_t''', the '''i_visible_width''' and '''i_visible_height''' members go with '''i_x_offset''' and '''i_y_offset''', but they are not related to '''plane_t::i_visible_*'''.

The '''plane_t::i_visible_*''' fields are there to know which part of the memory planes can at most be displayed. Usually these are different from the whole surface due to memory alignment constraints (like mod 16 for SSE2 or to be aligned on a macroblock). Those values are decided when the picture is allocated and they usually don't change for a given picture pointer (but they can change, like in direct3d buffers).

The '''video_format_t:i_visible_width/height''' and '''i_x/y_offset''' are more a 'hint' regarding what will be actually displayed. A filter should update '''filter_t::fmt_out''' if needed to ensure that the allocated pictures get the right value. A filter should, however, not limit its processing to this area, because VLC supports dynamic cropping.

Thus, the correct thing to do in a filter is to process the whole picture area, or at least the first i_visible_* lines and pixels ''according to the '''plane_t'''s''. Filters do not need to care about margin positioning or that there even exists such a thing as margins.

== Disclaimer ==

The information is mostly based on a few months' hacking on the deinterlacer.

This page is still a stub. Please add new relevant stuff.

Also, the stuff that already exists could use better organization and formatting.

{{Documentation}}

Hacker Guide/Video Filters

2011-05-09T02:10:33Z

Technologicat: Added a lead-in sentence at the beginning

This is the [[Documentation:Hacker's_Guide|Hacker's Guide]] for video filters ('''video filter2''', VLC 1.2).

== Location in source tree ==

Look in {{VLCSourceFile|name=modules/video_filter/}}.

See {{VLCSourceFile|name=modules/video_filter/Modules.am}} for a list of video filters currently in VLC, and which source files belong to which.

== Adding new source files ==

Want to develop a new filter or add new source files to an existing one?

The file {{VLCSourceFile|name=modules/video_filter/Modules.am}} tells the build system which source files go into which filter, so go ahead and update it.

When done, you'll probably need to go to the root of your VLC source tree, and do something akin to the following:
* <pre>find . -name .deps -exec rm -rf \{\} \;</pre>
* <pre>./bootstrap</pre>
* <pre>./configure</pre>

This ensures that the build system finds your new files. Then you can go on as usual:
* <pre>./compile</pre>

== Individual filter docs ==

Some filters may have their own [[Documentation:Hacker's Guide|Hacker's Guide]] sections.

Currently, the deinterlace filter has both a [[Documentation:Hacker's Guide/Video Filters/Deinterlace|Hacker's Guide section]] and [[Deinterlace|user documentation]] with some useful technical information.

== Programming conventions and tips ==

VLC video filters are usually written in C99, like the rest of VLC. The AtmoLight plugin uses C++, though.

The usual stuff holds here too:
* Use '''assert()''' for checking conditions that, in presence of no bugs, always hold.
** ''Don't'' use '''assert()''' for checking stuff that may fail at runtime. Handle errors gracefully.
* Keep your code readable. Make private functions when it simplifies things.
* Prefer 80 characters per line at maximum, if possible.

Particularly for video filters: keep in mind that this is low-level stuff and needs to be fast.
* How fast? Note that the source may be interlaced, and someone may have thrown a framerate doubler into the filter chain... this means that for a 60i signal, the entire processing chain from decode to render has about 16ms per output frame.
* Keep the required memory bandwidth down.
** Don't copy whole pictures unless you absolutely must. Moving pixels costs time.
** Allocate an output picture and render directly into it.
** If you need more than one pass, do in-place processing in the subsequent ones if possible.
** If your processing cannot avoid multiple passes, consider doing all passes for a picture line (or a few) at a time, instead of iterating over the whole picture separately for each pass. This will help to keep down the pressure on the CPU cache.
* Consider writing the inner processing loops in vectorized inline assembly, such as MMX. This can often gain a factor of 2-8x in speed, depending on how much and what kind of processing your algorithm needs.

== Some notes on the API ==

'''Note''': See {{VLCSourceFile|name=modules/video_filter/deinterlace/deinterlace.c}} and {{VLCSourceFile|name=modules/video_filter/deinterlace/deinterlace.h}} for an example of everything discussed in this section. See also the [[Documentation:Hacker's Guide/Video Filters/Deinterlace|Deinterlace Hacker's Guide]] for some further details.

The video filters use an object-based model. Calls to the filter will come with an instance pointer, so each filter instance can keep local data.

Usually, each filter defines its own internal data structure called '''filter_sys_t'''. It is completely private to the filter.

The API for video filters in VLC 1.2 is called '''video filter2'''. This requires the functions '''Open()''' and '''Close()'''. The '''Open()''' function is expected to allocate private data (if any), and set up the filter structure with pointers to a frame processing function ('''p_filter->pf_video_filter''', which is the actual filter), a flush function ('''p_filter->pf_video_flush'''), and a mouse event function ('''p_filter->pf_video_mouse'''). The '''Close()''' function should deallocate private data.

The frame processing function takes a frame as input ('''picture_t*''', {{VLCSourceFile|name=include/vlc_picture.h}}), and optionally outputs one or more frames ('''picture_t*''') as a linked list. The usual case is one frame in, one frame out, but this is not a requirement. (See framerate doublers and IVTC in the deinterlace module for counterexamples.)

If you do not wish to output a frame at a particular call to your processing function, you can return '''NULL'''. If you wish to output several, make a linked list by using the '''next''' member of '''picture_t'''.

You don't have to necessarily output the ''same'' frame (after processing) that just came in. You can keep a private input frame history in your '''filter_sys_t''', pick the base frame for your output from that, and adjust the PTSs (presentation timestamps, member '''date''' in '''picture_t''') of your output as you deem necessary. The deinterlacer provides an example of how to do this.

Note however that there may be other filters in the chain that already set up an offset; this leaves less time (until the designated PTS) for the rest of the filters to do their processing.

Be sure to allocate your output pictures using <pre>picture_t *p_outpic = filter_NewPicture( p_filter );</pre>
This function requests a new picture from the private pool (see {{VLCSourceFile|name=src/video_output/vout_wrapper.c}}). This is very important! Be aware that pictures created like
<pre>picture_t *p_temp = picture_NewFromFormat( &p_input_pic->format );</pre>
lack a shared memory context (in Linux; or the equivalent in other OS), and thus cannot be passed to the video output logic. If your filter attempts to do so, VLC will crash.

If you need temporary pictures, you can allocate those using '''picture_NewFromFormat()''', but remember to ''always create output pictures using '''filter_NewPicture()'''''. Here an ''output picture'' is defined as a picture which goes out from the filter to the caller.

There is a limit to the number of simultaneous picture slots available in the private pool. Currently (<nowiki>[71992c5fbc75ee2c94cd2a3ab1aaca70bfc688a9]</nowiki>) this is 3 pictures for the filter layer. See constant '''private_picture''' in {{VLCSourceFile|name=src/video_output/vout_wrapper.c}}. If you try to allocate more output pictures than this (during a single processing call), the allocation will fail.

In VLC 1.2, the video format (resolution, chroma) never changes on the fly. The whole filter chain is '''Close()'''d and then '''Open()'''ed again if the format changes.

Note that it ''is'' allowed to flush the filter without closing it. This actually happens under some circumstances, so ''don't deallocate stuff in your flush function''. Rather, allocate dynamic resources in '''Open()''', and deallocate them in '''Close()''' (or in private functions called from those).

Supported input and output formats are left up to each filter. The usual thing is to support YUV formats, e.g. I420, J420, YV12, I422 and J422. See {{VLCSourceFile|name=include/vlc_fourcc.h}}.

== Pitch, visible pitch, et al. ==

Important to know:

* i_pitch = number of (macro)pixels on one line
* i_visible_pitch = number of (macro)pixels on one line, adjusted for memory alignment constraints etc. (see below)

Note that the pitches may be different for each plane, and indeed in YUV formats the luma ('''Y_PLANE''') and chroma ('''U_PLANE''', '''V_PLANE''') have different pitches due to chroma subsampling. (In 4:2:0 formats, the number of lines differs, too.)

Pitches also tend to slightly differ depending on how the picture was allocated, even if the visible size is the same. The input picture to the filter may have one pitch, temporary pictures ('''picture_NewFromFormat()''') another, and output pictures ('''filter_NewPicture()''') yet another.

''Be sure to always use the correct pitch when you handle the pixels of a picture.'' That is, always use the '''i_pitch''' member of the actual '''plane_t''' you are working on.

If you absolutely need matching pitches (e.g. if you are glueing in processing code from another GPL-compatible project which assumes this and you don't want to rewrite it...), consider making temporary copies with '''picture_NewFromFormat()'''. See {{VLCSourceFile|name=modules/video_filter/deinterlace/deinterlace.c}} (the history mechanism <nowiki>pp_history[]</nowiki>) and {{VLCSourceFile|name=modules/video_filter/deinterlace/algo_yadif.c}} (its usage) for an example.

If you are processing pixels from one picture to another, the safe thing to do is to take the smallest '''i_visible_pitch''', and loop from x = 0 until the visible pitch has been reached, but use the individual '''i_pitch'''es for computing the pixel locations. See {{VLCSourceFile|name=modules/video_filter/deinterlace/helpers.c}} for examples.

=== video_format_t vs. plane_t ===

In '''video_format_t''', the '''i_visible_width''' and '''i_visible_height''' members go with '''i_x_offset''' and '''i_y_offset''', but they are not related to '''plane_t::i_visible_*'''.

The '''plane_t::i_visible_*''' fields are there to know which part of the memory planes can at most be displayed. Usually these are different from the whole surface due to memory alignment constraints (like mod 16 for SSE2 or to be aligned on a macroblock). Those values are decided when the picture is allocated and they usually don't change for a given picture pointer (but they can change, like in direct3d buffers).

The '''video_format_t:i_visible_width/height''' and '''i_x/y_offset''' are more a 'hint' regarding what will be actually displayed. A filter should update '''filter_t::fmt_out''' if needed to ensure that the allocated pictures get the right value. A filter should, however, not limit its processing to this area, because VLC supports dynamic cropping.

Thus, the correct thing to do in a filter is to process the whole picture area, or at least the first i_visible_* lines and pixels ''according to the '''plane_t'''s''. Filters do not need to care about margin positioning or that there even exists such a thing as margins.

== Disclaimer ==

The information is mostly based on a few months' hacking on the deinterlacer.

This page is still a stub. Please add new relevant stuff.

Also, the stuff that already exists could use better organization and formatting.

{{Documentation}}

Hacker Guide/Video Filters

2011-05-09T01:51:12Z

Technologicat: /* Pitch, visible pitch, et al. */

== Location in source tree ==

Look in {{VLCSourceFile|name=modules/video_filter/}}.

See {{VLCSourceFile|name=modules/video_filter/Modules.am}} for a list of video filters currently in VLC, and which source files belong to which.

== Adding new source files ==

Want to develop a new filter or add new source files to an existing one?

The file {{VLCSourceFile|name=modules/video_filter/Modules.am}} tells the build system which source files go into which filter, so go ahead and update it.

When done, you'll probably need to go to the root of your VLC source tree, and do something akin to the following:
* <pre>find . -name .deps -exec rm -rf \{\} \;</pre>
* <pre>./bootstrap</pre>
* <pre>./configure</pre>

This ensures that the build system finds your new files. Then you can go on as usual:
* <pre>./compile</pre>

== Individual filter docs ==

Some filters may have their own [[Documentation:Hacker's Guide|Hacker's Guide]] sections.

Currently, the deinterlace filter has both a [[Documentation:Hacker's Guide/Video Filters/Deinterlace|Hacker's Guide section]] and [[Deinterlace|user documentation]] with some useful technical information.

== Programming conventions and tips ==

VLC video filters are usually written in C99, like the rest of VLC. The AtmoLight plugin uses C++, though.

The usual stuff holds here too:
* Use '''assert()''' for checking conditions that, in presence of no bugs, always hold.
** ''Don't'' use '''assert()''' for checking stuff that may fail at runtime. Handle errors gracefully.
* Keep your code readable. Make private functions when it simplifies things.
* Prefer 80 characters per line at maximum, if possible.

Particularly for video filters: keep in mind that this is low-level stuff and needs to be fast.
* How fast? Note that the source may be interlaced, and someone may have thrown a framerate doubler into the filter chain... this means that for a 60i signal, the entire processing chain from decode to render has about 16ms per output frame.
* Keep the required memory bandwidth down.
** Don't copy whole pictures unless you absolutely must. Moving pixels costs time.
** Allocate an output picture and render directly into it.
** If you need more than one pass, do in-place processing in the subsequent ones if possible.
** If your processing cannot avoid multiple passes, consider doing all passes for a picture line (or a few) at a time, instead of iterating over the whole picture separately for each pass. This will help to keep down the pressure on the CPU cache.
* Consider writing the inner processing loops in vectorized inline assembly, such as MMX. This can often gain a factor of 2-8x in speed, depending on how much and what kind of processing your algorithm needs.

== Some notes on the API ==

'''Note''': See {{VLCSourceFile|name=modules/video_filter/deinterlace/deinterlace.c}} and {{VLCSourceFile|name=modules/video_filter/deinterlace/deinterlace.h}} for an example of everything discussed in this section. See also the [[Documentation:Hacker's Guide/Video Filters/Deinterlace|Deinterlace Hacker's Guide]] for some further details.

The video filters use an object-based model. Calls to the filter will come with an instance pointer, so each filter instance can keep local data.

Usually, each filter defines its own internal data structure called '''filter_sys_t'''. It is completely private to the filter.

The API for video filters in VLC 1.2 is called '''video filter2'''. This requires the functions '''Open()''' and '''Close()'''. The '''Open()''' function is expected to allocate private data (if any), and set up the filter structure with pointers to a frame processing function ('''p_filter->pf_video_filter''', which is the actual filter), a flush function ('''p_filter->pf_video_flush'''), and a mouse event function ('''p_filter->pf_video_mouse'''). The '''Close()''' function should deallocate private data.

The frame processing function takes a frame as input ('''picture_t*''', {{VLCSourceFile|name=include/vlc_picture.h}}), and optionally outputs one or more frames ('''picture_t*''') as a linked list. The usual case is one frame in, one frame out, but this is not a requirement. (See framerate doublers and IVTC in the deinterlace module for counterexamples.)

If you do not wish to output a frame at a particular call to your processing function, you can return '''NULL'''. If you wish to output several, make a linked list by using the '''next''' member of '''picture_t'''.

You don't have to necessarily output the ''same'' frame (after processing) that just came in. You can keep a private input frame history in your '''filter_sys_t''', pick the base frame for your output from that, and adjust the PTSs (presentation timestamps, member '''date''' in '''picture_t''') of your output as you deem necessary. The deinterlacer provides an example of how to do this.

Note however that there may be other filters in the chain that already set up an offset; this leaves less time (until the designated PTS) for the rest of the filters to do their processing.

Be sure to allocate your output pictures using <pre>picture_t *p_outpic = filter_NewPicture( p_filter );</pre>
This function requests a new picture from the private pool (see {{VLCSourceFile|name=src/video_output/vout_wrapper.c}}). This is very important! Be aware that pictures created like
<pre>picture_t *p_temp = picture_NewFromFormat( &p_input_pic->format );</pre>
lack a shared memory context (in Linux; or the equivalent in other OS), and thus cannot be passed to the video output logic. If your filter attempts to do so, VLC will crash.

If you need temporary pictures, you can allocate those using '''picture_NewFromFormat()''', but remember to ''always create output pictures using '''filter_NewPicture()'''''. Here an ''output picture'' is defined as a picture which goes out from the filter to the caller.

There is a limit to the number of simultaneous picture slots available in the private pool. Currently (<nowiki>[71992c5fbc75ee2c94cd2a3ab1aaca70bfc688a9]</nowiki>) this is 3 pictures for the filter layer. See constant '''private_picture''' in {{VLCSourceFile|name=src/video_output/vout_wrapper.c}}. If you try to allocate more output pictures than this (during a single processing call), the allocation will fail.

In VLC 1.2, the video format (resolution, chroma) never changes on the fly. The whole filter chain is '''Close()'''d and then '''Open()'''ed again if the format changes.

Note that it ''is'' allowed to flush the filter without closing it. This actually happens under some circumstances, so ''don't deallocate stuff in your flush function''. Rather, allocate dynamic resources in '''Open()''', and deallocate them in '''Close()''' (or in private functions called from those).

Supported input and output formats are left up to each filter. The usual thing is to support YUV formats, e.g. I420, J420, YV12, I422 and J422. See {{VLCSourceFile|name=include/vlc_fourcc.h}}.

== Pitch, visible pitch, et al. ==

Important to know:

* i_pitch = number of (macro)pixels on one line
* i_visible_pitch = number of (macro)pixels on one line, adjusted for memory alignment constraints etc. (see below)

Note that the pitches may be different for each plane, and indeed in YUV formats the luma ('''Y_PLANE''') and chroma ('''U_PLANE''', '''V_PLANE''') have different pitches due to chroma subsampling. (In 4:2:0 formats, the number of lines differs, too.)

Pitches also tend to slightly differ depending on how the picture was allocated, even if the visible size is the same. The input picture to the filter may have one pitch, temporary pictures ('''picture_NewFromFormat()''') another, and output pictures ('''filter_NewPicture()''') yet another.

''Be sure to always use the correct pitch when you handle the pixels of a picture.'' That is, always use the '''i_pitch''' member of the actual '''plane_t''' you are working on.

If you absolutely need matching pitches (e.g. if you are glueing in processing code from another GPL-compatible project which assumes this and you don't want to rewrite it...), consider making temporary copies with '''picture_NewFromFormat()'''. See {{VLCSourceFile|name=modules/video_filter/deinterlace/deinterlace.c}} (the history mechanism <nowiki>pp_history[]</nowiki>) and {{VLCSourceFile|name=modules/video_filter/deinterlace/algo_yadif.c}} (its usage) for an example.

If you are processing pixels from one picture to another, the safe thing to do is to take the smallest '''i_visible_pitch''', and loop from x = 0 until the visible pitch has been reached, but use the individual '''i_pitch'''es for computing the pixel locations. See {{VLCSourceFile|name=modules/video_filter/deinterlace/helpers.c}} for examples.

=== video_format_t vs. plane_t ===

In '''video_format_t''', the '''i_visible_width''' and '''i_visible_height''' members go with '''i_x_offset''' and '''i_y_offset''', but they are not related to '''plane_t::i_visible_*'''.

The '''plane_t::i_visible_*''' fields are there to know which part of the memory planes can at most be displayed. Usually these are different from the whole surface due to memory alignment constraints (like mod 16 for SSE2 or to be aligned on a macroblock). Those values are decided when the picture is allocated and they usually don't change for a given picture pointer (but they can change, like in direct3d buffers).

The '''video_format_t:i_visible_width/height''' and '''i_x/y_offset''' are more a 'hint' regarding what will be actually displayed. A filter should update '''filter_t::fmt_out''' if needed to ensure that the allocated pictures get the right value. A filter should, however, not limit its processing to this area, because VLC supports dynamic cropping.

Thus, the correct thing to do in a filter is to process the whole picture area, or at least the first i_visible_* lines and pixels ''according to the '''plane_t'''s''. Filters do not need to care about margin positioning or that there even exists such a thing as margins.

== Disclaimer ==

The information is mostly based on a few months' hacking on the deinterlacer.

This page is still a stub. Please add new relevant stuff.

Also, the stuff that already exists could use better organization and formatting.

{{Documentation}}