Hacker Guide/Decoder - Revision history

DoesItReallyMatter: /* Packet changes and alignment issues */ Add Wikipedia link inline: PTS

2019-05-31T09:09:13Z

Packet changes and alignment issues: Add Wikipedia link inline: PTS

← Older revision		Revision as of 09:09, 31 May 2019
Line 60:		Line 60:
	For instance <code>GetBits()</code> will call <code>UnalignedGetBits()</code> from {{VLCSourceFile\|src/input/input_ext-dec.c}}. Basically it will read byte after byte until the stream gets realigned. <code>UnalignedShowBits()</code> is a bit more complicated and may require a temporary packet (<code>p_bit_stream->showbits_data</code>).		For instance <code>GetBits()</code> will call <code>UnalignedGetBits()</code> from {{VLCSourceFile\|src/input/input_ext-dec.c}}. Basically it will read byte after byte until the stream gets realigned. <code>UnalignedShowBits()</code> is a bit more complicated and may require a temporary packet (<code>p_bit_stream->showbits_data</code>).

	To use the bit stream, you have to call <code>p_decoder_config->pf_init_bit_stream( bit_stream_t * p_bit_stream, decoder_fifo_t * p_fifo )</code> to set up all variables. You will probably need to regularly fetch specific information from the packet, for instance the PTS. If <code>p_bit_stream->pf_bit_stream_callback</code> is not <code>NULL</code>, it will be called on a packet change. See {{VLCSourceFile\|src/video_parser/video_parser.c}} for an example. The second argument indicates whether it is just a new <var>data_packet_t</var> or also a new <var>pes_packet_t</var>. You can store your own structure in <code>p_bit_stream->p_callback_arg</code>.		To use the bit stream, you have to call <code>p_decoder_config->pf_init_bit_stream( bit_stream_t * p_bit_stream, decoder_fifo_t * p_fifo )</code> to set up all variables. You will probably need to regularly fetch specific information from the packet, for instance the [[wikipedia:Presentation timestamp\|PTS]]. If <code>p_bit_stream->pf_bit_stream_callback</code> is not <code>NULL</code>, it will be called on a packet change. See {{VLCSourceFile\|src/video_parser/video_parser.c}} for an example. The second argument indicates whether it is just a new <var>data_packet_t</var> or also a new <var>pes_packet_t</var>. You can store your own structure in <code>p_bit_stream->p_callback_arg</code>.

	====Warning====		====Warning====

DoesItReallyMatter: Format

2019-05-30T22:09:20Z

Format

Show changes

DoesItReallyMatter: +{{Back to|Hacker Guide}}

2019-04-17T06:37:22Z

+{{Back to|Hacker Guide}}

← Older revision		Revision as of 06:37, 17 April 2019
Line 1:		Line 1:
			{{Back to\|Hacker Guide}}
	=How to write a decoder=		=How to write a decoder=

DoesItReallyMatter: Correct spelling/grammar, add redlink to packetize

2019-02-15T06:43:20Z

Correct spelling/grammar, add redlink to packetize

← Older revision		Revision as of 06:43, 15 February 2019
Line 13:		Line 13:
	set_capability( "decoder", 100 );		set_capability( "decoder", 100 );

	which means its score is 100 when we request a decoder. The same system is used in each part of VLC to chose which module will be used. Then, it verifies if the decoder needs ~~packetized~~ data (in that case, it finds and loads a packetizer module). In the end, it starts the decoder thread by a call to:		which means its score is 100 when we request a decoder. The same system is used in each part of VLC to chose which module will be used. Then, it verifies if the decoder needs [[packetize]]d data (in that case, it finds and loads a packetizer module). In the end, it starts the decoder thread by a call to:
	vlc_thread_create( p_dec, "decoder", DecoderThread, i_priority, false )		vlc_thread_create( p_dec, "decoder", DecoderThread, i_priority, false )

Line 51:		Line 51:
	GetChunk ( bit_stream_t p_bit_stream, byte_t * p_buffer, size_t i_buf_len ) : It is an analog of memcpy(), but taking a bit stream as first argument. p_buffer must be allocated and at least i_buf_len long. It is useful to copy data you want to keep track of.		GetChunk ( bit_stream_t p_bit_stream, byte_t * p_buffer, size_t i_buf_len ) : It is an analog of memcpy(), but taking a bit stream as first argument. p_buffer must be allocated and at least i_buf_len long. It is useful to copy data you want to keep track of.

	All these functions recreate a continuous elementary stream paradigm. When the bit buffer is empty, they take the following word in the current packet. When the packet is empty, it switches to the next data_packet_t, or if ~~unapplicable~~ to the next pes_packet_t (see p_bit_stream->pf_next_data_packet). All this is completely transparent.		All these functions recreate a continuous elementary stream paradigm. When the bit buffer is empty, they take the following word in the current packet. When the packet is empty, it switches to the next data_packet_t, or if inapplicable to the next pes_packet_t (see p_bit_stream->pf_next_data_packet). All this is completely transparent.

	===Packet changes and alignment issues===		===Packet changes and alignment issues===
Line 77:		Line 77:
	==The MPEG video decoder==		==The MPEG video decoder==

	VLC media player provides an MPEG-1, and an MPEG-2 Main Profile @ Main Level decoder. It has been natively written for VLC, and is quite mature. Its status is a bit special, since it is ~~splitted~~ between two ~~logicial~~ entities : video parser and video decoder. The initial goal is to separate bit stream parsing functions from highly parallelizable mathematical algorithms. In theory, there can be one video parser thread (and only one, otherwise we would have race conditions reading the bit stream), along with a pool of video decoder threads, which do IDCT and motion compensation on several blocks at once.		VLC media player provides an MPEG-1, and an MPEG-2 Main Profile @ Main Level decoder. It has been natively written for VLC, and is quite mature. Its status is a bit special, since it is split between two logical entities : video parser and video decoder. The initial goal is to separate bit stream parsing functions from highly parallelizable mathematical algorithms. In theory, there can be one video parser thread (and only one, otherwise we would have race conditions reading the bit stream), along with a pool of video decoder threads, which do IDCT and motion compensation on several blocks at once.

	It doesn't (and won't) support MPEG-4 or DivX decoding. It is not an encoder. It should support the whole MPEG-2 MP@ML specification, though some features are still left untested, like Differential Motion Vectors. Please bear in mind before complaining that the input elementary stream must be valid (for instance this is not the case when you directly read a DVD multi-angle .vob file).		It doesn't (and won't) support MPEG-4 or DivX decoding. It is not an encoder. It should support the whole MPEG-2 MP@ML specification, though some features are still left untested, like Differential Motion Vectors. Please bear in mind before complaining that the input elementary stream must be valid (for instance this is not the case when you directly read a DVD multi-angle .vob file).
Line 84:		Line 84:
	Motion compensation plug-ins		Motion compensation plug-ins

	Motion compensation (i.e. copy of regions from a reference picture) is very platform-~~dependant~~ (for instance with MMX or AltiVec versions), so we moved it to the plugins/motion directory. It is more convenient for the video decoder, and resulting plug-ins may be used by other video decoders (MPEG-4 ?). A motion plugin must define 6 functions, coming straight from the specification : vdec_MotionFieldField420, vdec_MotionField16x8420, vdec_MotionFieldDMV420, vdec_MotionFrameFrame420, vdec_MotionFrameField420, vdec_MotionFrameDMV420. The equivalent 4:2:2 and 4:4:4 functions are unused, since these formats are forbidden in MP@ML (it would only take longer compilation time).		Motion compensation (i.e. copy of regions from a reference picture) is very platform-dependent (for instance with MMX or AltiVec versions), so we moved it to the plugins/motion directory. It is more convenient for the video decoder, and resulting plug-ins may be used by other video decoders (MPEG-4 ?). A motion plugin must define 6 functions, coming straight from the specification : vdec_MotionFieldField420, vdec_MotionField16x8420, vdec_MotionFieldDMV420, vdec_MotionFrameFrame420, vdec_MotionFrameField420, vdec_MotionFrameDMV420. The equivalent 4:2:2 and 4:4:4 functions are unused, since these formats are forbidden in MP@ML (it would only take longer compilation time).

	Look at the C version of the algorithms if you want more information. Note also that the DMV algorithm is untested and is probably buggy.		Look at the C version of the algorithms if you want more information. Note also that the DMV algorithm is untested and is probably buggy.
Line 106:		Line 106:
	The MPEG video decoder of VLC can take advantage of several processors if necessary. The idea is to launch a pool of decoders, which will do IDCT/motion compensation on several macroblocks at once.		The MPEG video decoder of VLC can take advantage of several processors if necessary. The idea is to launch a pool of decoders, which will do IDCT/motion compensation on several macroblocks at once.

	The functions managing the pool are in {{VLCSourceFile\|src/video_decoder/vpar_pool.c}}. Its use on non-SMP machines is not ~~recommanded~~, since it is actually slower than the monothread version. Even on SMP machines sometimes...		The functions managing the pool are in {{VLCSourceFile\|src/video_decoder/vpar_pool.c}}. Its use on non-SMP machines is not recommended, since it is actually slower than the monothread version. Even on SMP machines sometimes...

	{{Hacker_Guide}}		{{Hacker_Guide}}

	[[Category:Pages to check]]		[[Category:Pages to check]]

DoesItReallyMatter: Link with Template:VLCSourceFile

2019-02-15T06:38:41Z

Link with Template:VLCSourceFile

← Older revision		Revision as of 06:38, 15 February 2019
Line 9:		Line 9:
	==Decoder configuration==		==Decoder configuration==

	The input thread spawns the appropriate decoder modules from src/input/decoder.c. The CreateDecoder( input_thread_t p_input,es_format_t fmt, int i_object_type ) function creates a p_dec variable of type i_object_type(in our case VLC_OBJECT_DECODER), then calls module_Need( vlc_object_t p_this, const char psz_capability,const char psz_name, bool b_strict ) with psz_capability="decoder" and *psz_name="$codec". module_Need builds a list of the possible modules for this capability, using a score. As an example, in the a52 module, we can read:		The input thread spawns the appropriate decoder modules from {{VLCSourceFile\|src/input/decoder.c}}. The CreateDecoder( input_thread_t p_input,es_format_t fmt, int i_object_type ) function creates a p_dec variable of type i_object_type(in our case VLC_OBJECT_DECODER), then calls module_Need( vlc_object_t p_this, const char psz_capability,const char psz_name, bool b_strict ) with psz_capability="decoder" and *psz_name="$codec". module_Need builds a list of the possible modules for this capability, using a score. As an example, in the a52 module, we can read:

	set_capability( "decoder", 100 );		set_capability( "decoder", 100 );
Line 18:		Line 18:
	==Packet structures==		==Packet structures==

	The input module provides an advanced API for delivering stream data to the decoders. First let's have a look at the packet structures. They are defined in include/input_ext-dec.h.		The input module provides an advanced API for delivering stream data to the decoders. First let's have a look at the packet structures. They are defined in {{VLCSourceFile\|include/input_ext-dec.h}}.

	data_packet_t contains a pointer to the physical location of data. Decoders should only start to read them at p_payload_start until p_payload_end. Thereafter, it will switch to the next packet, p_next if it is not NULL. If the b_discard_payload flag is up, the content of the packet is messed up and it should be discarded.		data_packet_t contains a pointer to the physical location of data. Decoders should only start to read them at p_payload_start until p_payload_end. Thereafter, it will switch to the next packet, p_next if it is not NULL. If the b_discard_payload flag is up, the content of the packet is messed up and it should be discarded.
Line 57:		Line 57:
	We have to study the conjunction of two problems. First, a data_packet_t can have an even number of bytes, for instance 177, so the last word will be truncated. Second, many CPU (sparc, alpha...) can only read words aligned on a word boundary (that is, 32 bits for a 32-bit word). So packet changes are a lot more complicated than you can imagine, because we have to read truncated words and get aligned.		We have to study the conjunction of two problems. First, a data_packet_t can have an even number of bytes, for instance 177, so the last word will be truncated. Second, many CPU (sparc, alpha...) can only read words aligned on a word boundary (that is, 32 bits for a 32-bit word). So packet changes are a lot more complicated than you can imagine, because we have to read truncated words and get aligned.

	For instance GetBits() will call UnalignedGetBits() from src/input/input_ext-dec.c. Basically it will read byte after byte until the stream gets realigned. UnalignedShowBits() is a bit more complicated and may require a temporary packet (p_bit_stream->showbits_data).		For instance GetBits() will call UnalignedGetBits() from {{VLCSourceFile\|src/input/input_ext-dec.c}}. Basically it will read byte after byte until the stream gets realigned. UnalignedShowBits() is a bit more complicated and may require a temporary packet (p_bit_stream->showbits_data).

	To use the bit stream, you have to call p_decoder_config->pf_init_bit_stream( bit_stream_t * p_bit_stream, decoder_fifo_t * p_fifo ) to set up all variables. You will probably need to regularly fetch specific information from the packet, for instance the PTS. If p_bit_stream->pf_bit_stream_callback is not NULL, it will be called on a packet change. See src/video_parser/video_parser.c for an example. The second argument indicates whether it is just a new data_packet_t or also a new pes_packet_t. You can store your own structure in p_bit_stream->p_callback_arg.		To use the bit stream, you have to call p_decoder_config->pf_init_bit_stream( bit_stream_t * p_bit_stream, decoder_fifo_t * p_fifo ) to set up all variables. You will probably need to regularly fetch specific information from the packet, for instance the PTS. If p_bit_stream->pf_bit_stream_callback is not NULL, it will be called on a packet change. See {{VLCSourceFile\|src/video_parser/video_parser.c}} for an example. The second argument indicates whether it is just a new data_packet_t or also a new pes_packet_t. You can store your own structure in p_bit_stream->p_callback_arg.

	===Warning===		===Warning===
Line 71:		Line 71:
	===Limitations in the current design===		===Limitations in the current design===

	To add a new decoder, you'll still have to add the stream type as there's still a hard-wired piece of code in src/input/input_programs.c .		To add a new decoder, you'll still have to add the stream type as there's still a hard-wired piece of code in {{VLCSourceFile\|src/input/input_programs.c}} .

	The MPEG audio decoder is native, but doesn't support layer 3 decoding [too much trouble], the AC3 decoder is a port from Aaron Holtzman's libac3 (the original libac3 isn't reentrant), and the SPU decoder is native. You may want to have a look at BitstreamCallback in the AC3 decoder. In that case we have to jump the first 3 bytes of a PES packet, which are not part of the elementary stream. The video decoder is a bit special and will be described in the following section.		The MPEG audio decoder is native, but doesn't support layer 3 decoding [too much trouble], the AC3 decoder is a port from Aaron Holtzman's libac3 (the original libac3 isn't reentrant), and the SPU decoder is native. You may want to have a look at BitstreamCallback in the AC3 decoder. In that case we have to jump the first 3 bytes of a PES packet, which are not part of the elementary stream. The video decoder is a bit special and will be described in the following section.
Line 81:		Line 81:
	It doesn't (and won't) support MPEG-4 or DivX decoding. It is not an encoder. It should support the whole MPEG-2 MP@ML specification, though some features are still left untested, like Differential Motion Vectors. Please bear in mind before complaining that the input elementary stream must be valid (for instance this is not the case when you directly read a DVD multi-angle .vob file).		It doesn't (and won't) support MPEG-4 or DivX decoding. It is not an encoder. It should support the whole MPEG-2 MP@ML specification, though some features are still left untested, like Differential Motion Vectors. Please bear in mind before complaining that the input elementary stream must be valid (for instance this is not the case when you directly read a DVD multi-angle .vob file).

	The most interesting file is vpar_synchro.c, it is really worth the shot. It explains the whole frame dropping algorithm. In a nutshell, if the machine is powerful enough, we decoder all IPBs, otherwise we decode all IPs and Bs if we have enough time (this is based on on-the-fly decoding time statistics). Another interesting file is vpar_blocks.c, which describes all block (including coefficients and motion vectors) parsing algorithms. Look at the bottom of the file, we indeed generate one optimized function for every common picture type, and one slow generic function. There are also several levels of optimization (which makes compilation slower but certain types of files faster decoded) called VPAR_OPTIM_LEVEL, level 0 means no optimization, level 1 means optimizations for MPEG-1 and MPEG-2 frame pictures, level 2 means optimizations for MPEG-1 and MPEG-2 field and frame pictures.		The most interesting file is {{VLCSourceFile\|src/vpar_synchro.c}}, it is really worth the shot. It explains the whole frame dropping algorithm. In a nutshell, if the machine is powerful enough, we decoder all IPBs, otherwise we decode all IPs and Bs if we have enough time (this is based on on-the-fly decoding time statistics). Another interesting file is vpar_blocks.c, which describes all block (including coefficients and motion vectors) parsing algorithms. Look at the bottom of the file, we indeed generate one optimized function for every common picture type, and one slow generic function. There are also several levels of optimization (which makes compilation slower but certain types of files faster decoded) called VPAR_OPTIM_LEVEL, level 0 means no optimization, level 1 means optimizations for MPEG-1 and MPEG-2 frame pictures, level 2 means optimizations for MPEG-1 and MPEG-2 field and frame pictures.
	Motion compensation plug-ins		Motion compensation plug-ins

Line 106:		Line 106:
	The MPEG video decoder of VLC can take advantage of several processors if necessary. The idea is to launch a pool of decoders, which will do IDCT/motion compensation on several macroblocks at once.		The MPEG video decoder of VLC can take advantage of several processors if necessary. The idea is to launch a pool of decoders, which will do IDCT/motion compensation on several macroblocks at once.

	The functions managing the pool are in src/video_decoder/vpar_pool.c. Its use on non-SMP machines is not recommanded, since it is actually slower than the monothread version. Even on SMP machines sometimes...		The functions managing the pool are in {{VLCSourceFile\|src/video_decoder/vpar_pool.c}}. Its use on non-SMP machines is not recommanded, since it is actually slower than the monothread version. Even on SMP machines sometimes...

	{{Hacker_Guide}}		{{Hacker_Guide}}

	[[Category:Pages to check]]		[[Category:Pages to check]]

DoesItReallyMatter: /* Limitations in the current design */ Fix typo

2019-02-15T06:24:16Z

Limitations in the current design: Fix typo

← Older revision		Revision as of 06:24, 15 February 2019
Line 71:		Line 71:
	===Limitations in the current design===		===Limitations in the current design===

	To add a new decoder, you'll still have to add the stream type as there's still a a hard-wired piece of code in src/input/input_programs.c .		To add a new decoder, you'll still have to add the stream type as there's still a hard-wired piece of code in src/input/input_programs.c .

	The MPEG audio decoder is native, but doesn't support layer 3 decoding [too much trouble], the AC3 decoder is a port from Aaron Holtzman's libac3 (the original libac3 isn't reentrant), and the SPU decoder is native. You may want to have a look at BitstreamCallback in the AC3 decoder. In that case we have to jump the first 3 bytes of a PES packet, which are not part of the elementary stream. The video decoder is a bit special and will be described in the following section.		The MPEG audio decoder is native, but doesn't support layer 3 decoding [too much trouble], the AC3 decoder is a port from Aaron Holtzman's libac3 (the original libac3 isn't reentrant), and the SPU decoder is native. You may want to have a look at BitstreamCallback in the AC3 decoder. In that case we have to jump the first 3 bytes of a PES packet, which are not part of the elementary stream. The video decoder is a bit special and will be described in the following section.

DoesItReallyMatter: Added the page to :Category:Pages to check, note was on the talk page

2019-01-21T11:35:36Z

Added the page to Category:Pages to check, note was on the talk page

← Older revision		Revision as of 11:35, 21 January 2019
Line 109:		Line 109:

	{{Hacker_Guide}}		{{Hacker_Guide}}

			[[Category:Pages to check]]

Corbax: /* Symmetrical Multiprocessing */

2013-11-14T17:24:21Z

Symmetrical Multiprocessing

← Older revision		Revision as of 17:24, 14 November 2013
Line 108:		Line 108:
	The functions managing the pool are in src/video_decoder/vpar_pool.c. Its use on non-SMP machines is not recommanded, since it is actually slower than the monothread version. Even on SMP machines sometimes...		The functions managing the pool are in src/video_decoder/vpar_pool.c. Its use on non-SMP machines is not recommanded, since it is actually slower than the monothread version. Even on SMP machines sometimes...

	~~[[Category:Coding]]~~
	{{Hacker_Guide}}		{{Hacker_Guide}}

Corbax at 16:19, 30 October 2013

2013-10-30T16:19:49Z

← Older revision		Revision as of 16:19, 30 October 2013
Line 109:		Line 109:

	[[Category:Coding]]		[[Category:Coding]]
	{{~~Documentation~~}}		{{Hacker_Guide}}

Corbax: /* Symmetrical Multiprocessing */

2013-10-01T14:46:12Z

Symmetrical Multiprocessing

← Older revision		Revision as of 14:46, 1 October 2013
Line 108:		Line 108:
	The functions managing the pool are in src/video_decoder/vpar_pool.c. Its use on non-SMP machines is not recommanded, since it is actually slower than the monothread version. Even on SMP machines sometimes...		The functions managing the pool are in src/video_decoder/vpar_pool.c. Its use on non-SMP machines is not recommanded, since it is actually slower than the monothread version. Even on SMP machines sometimes...

			[[Category:Coding]]
	{{Documentation}}		{{Documentation}}
	~~[[Category:Coding]]~~