Difference between revisions of "Hacker Guide/Core"

From VideoLAN Wiki
Jump to navigation Jump to search
 
(29 intermediate revisions by 9 users not shown)
Line 1: Line 1:
 +
{{Back to|Hacker Guide}}
 
== Introduction to libVLCcore  ==
 
== Introduction to libVLCcore  ==
  
The core of {{VLC}} is called [[LibVLCcore|'''libVLCcore''']].<br>
+
The core of {{VLC}} is called '''[[libVLCcore]]'''.<br>
It manages the threads, the modules (codecs, demuxers, etc...), the modules' layers, the clocks, the playlist and all low-level control in VLC.<br>For example, it is responsible for the synchronization management between all the audio and video tracks.  
+
It manages the threads, the modules (codecs, demuxers, etc...), the modules' layers, the clocks, the playlist and all low-level control in VLC.<br>For example, it is responsible for the synchronization management between all the audio, video and subtitle tracks.  
  
On top of libVLCcore, there is [[LibVLC|'''libVLC''']] that allow external application builders to access to all features of the core.<br> Modules are linked with libvlccore, to interact with the core.  
+
On top of libVLCcore, there is '''[[libVLC]]''' that allow external application builders to access to all features of the core.<br> Modules are linked with libvlccore, to interact with the core.  
  
'''Modules are built against libvlccore. External applications are built against [[LibVLC|libVLC]].'''
+
'''Modules are built against libvlccore. External applications are built against [[libVLC]].'''
  
 
== VLC Pipeline and Modularity ==
 
== VLC Pipeline and Modularity ==
  
One of the main concept in VLC is "'''''modularity'''''".  
+
One of the main concepts in VLC is "'''''modularity'''''".  
  
VLC is, in fact, a complete [http://en.wikipedia.org/wiki/Multimedia_framework multimedia framework] (like DirectShow or GStreamer) where you can load and plug-in many modules dynamically, depending on the necessity.  
+
VLC is, in fact, a complete [[wikipedia:multimedia framework|multimedia framework]] (like DirectShow or GStreamer) where you can load and plug-in many modules dynamically, depending on the necessity.  
  
The core framework is used to do the "wiring" and the media processing, from input (files, network streams) to output (audio or video, on ascreen or a network). It uses modules to do most of the work at every stage (various muxers, demuxers, decoders, filters and outputs).<br> Even the interfaces are plugins for libVLC.  
+
The core framework is used to do the "wiring" and the media processing, from input (files, network streams) to output (audio or video, on a screen or a network). It uses modules to do most of the work at every stage (various muxers, demuxers, decoders, filters and outputs).<br> Even the interfaces are plugins for libVLC.  
  
 
=== The modules in VLC  ===
 
=== The modules in VLC  ===
  
So, VLC uses '''modules''' to do most of the work, at every stage of the pipeline. Modules are loaded according at runtime depending on the necessity (See VLC Modules Loading).
+
So, VLC uses '''modules''' to do most of the work, at every stage of the pipeline. Modules are loaded accordingly at runtime depending on the necessity. (See [[Documentation:VLC_Modules_Loading|VLC Modules loading mechanism]]).
  
 
Every module offers different features that will best suit a particular use-case or a particular environment.<br>
 
Every module offers different features that will best suit a particular use-case or a particular environment.<br>
Besides, most portability of VLC result in the writing of an audio_output/video_output/interface module to support a new platform (eg. BeOS or MacOS X).  
+
Besides, most portability of VLC results from writing [[{{#rel2abs:../Audio_Output}}|audio_output]]/[[{{#rel2abs:../Video Output}}|video_output]]/[[{{#rel2abs:../Interfaces}}|interface]] modules specific to the platform.
  
Plugin modules are loaded and unloaded dynamically by functions in src/modules.c and include/vlc_modules*.h. The API for writing modules will be discussed in a following chapter.  
+
'''plugins''' modules are loaded and unloaded dynamically by functions in {{VLCSourceFile|src/modules/modules.c}} .<br>
 +
Modules can also be built directly into the application which uses libVLC, for instance when VLC is on an operating system that does not have support for dynamically loadable code. They are then called '''builtins'''.
  
Modules can also be built directly into the application which uses libVLC, for instance on an operating system that does not have support for dynamically loadable code. Modules statically built into the application are called builtins.
+
In the [[{{#rel2abs:../VLC source tree}}|source code]], modules are usually located inside the [[{{#rel2abs:../Modules source tree}}|modules/ subdirectory]].
  
In the source code, modules are usually located inside the modules/ subdirectory.
+
== Thread management  ==
  
== Threads &amp; Synchronisation<br>  ==
+
VLC is heavily multi-threaded.
  
=== Thread management  ===
+
The single-threaded approach would have introduced too much complexity because decoder preemptibility and scheduling would then be a mastermind (for instance decoders and outputs have to be separated, otherwise it cannot be warrantied that a frame will be played at the exact presentation time).<br>
 +
Multi-process approach wasn't chosen either, because multi-process decoders usually imply more overhead (problems of shared memory) and communication between processes is harder.
  
VLC is heavily multi-threaded. We chose against a single-thread approach because decoder preemptibility and scheduling would be a mastermind (for instance decoders and outputs have to be separated, otherwise it cannot be warrantied that a frame will be played at the exact presentation time), and we currently have no plan to support a single-threaded client. Multi-process decoders usually imply more overhead (problems of shared memory) and communication between processes is harder.  
+
VLC's threading structure is modeled after POSIX threads (''pthread''). However, for portability reasons, VLC does not use <code>pthread_*</code> functions directly, but a similar custom set of APIs.
  
Our threading structure is modeled on pthreads. However, for portability reasons, we don't call pthread_* functions directly, but use a similar wrapper, made of vlc_thread_create, vlc_thread_exit, vlc_thread_join, vlc_mutex_init, vlc_mutex_lock, vlc_mutex_unlock, vlc_mutex_destroy, vlc_cond_init, vlc_cond_signal, vlc_cond_broadcast, vlc_cond_wait, vlc_cond_destroy, and structures vlc_thread_t, vlc_mutex_t, and vlc_cond_t.
+
=== Threads (vlc_thread_t) ===
 +
; <code>vlc_clone()</code>
 +
: Creates a thread.
 +
; <code>vlc_join()</code>
 +
: Waits for a thread to terminate and releases its resources
  
=== Synchronization  ===
+
=== Mutual exclusion (vlc_mutex_t) ===
 +
; <code>vlc_mutex_init()</code>
 +
: Creates a non-recursive mutex.
 +
; <code>vlc_mutex_init_recursive()</code>
 +
: Creates a recursive mutex (discouraged).
 +
; <code>vlc_mutex_lock()</code>
 +
: Locks a mutex, waiting if required.
 +
; <code>vlc_mutex_trylock()</code>
 +
: Locks a mutex if it is not already locked, or returns an error otherwise.
 +
; <code>vlc_mutex_unlock()</code>
 +
: Unlocks a mutex.
 +
; <code>vlc_mutex_destroy()</code>
 +
: Destroys a mutex.
 +
 
 +
=== Condition variable (vlc_cond_t) ===
 +
; <code>vlc_cond_init()</code>
 +
: Creates a condition variable using the monotonic clock (<code>mdate()</code>) for timeouts.
 +
; <code>vlc_cond_init_daytime()</code>
 +
: Creates a condition variable using the realtime clock / wall clock for timeouts.
 +
; <code>vlc_cond_signal()</code>
 +
: Signals one thread waiting on a condition variable.
 +
; <code>vlc_cond_broadcast()</code>
 +
: Signals all threads waiting on a condition variable.
 +
; <code>vlc_cond_wait()</code>
 +
: Waits for a condition variable to be signaled (can also wake up spuriously).
 +
; <code>vlc_cond_timedwait()</code>
 +
: Waits for a condition variable to be signaled up to a certain timeout (can also wake up spuriously).
 +
; <code>vlc_cond_destroy()</code>
 +
: Destroys a condition variable.
 +
 
 +
=== Misc ===
 +
VLC also has abstractions for slim read/write locks, spin locks, thread-specific variables, similar to POSIX threads.
 +
 
 +
=== Atomic variables ===
 +
Atomic variables are small <code>(sizeof(void *))</code> values that can be manipulated from multiple threads without locking. See <code>include/vlc_atomic.h</code> for the list of supported operations.
 +
 
 +
== Synchronization  ==
  
 
Another key feature of VLC is that decoding and playing are asynchronous: decoding is done by a decoder thread, playing is done by audio_output or video_output thread. The design goal is to ensure that an audio or video frame is played exactly at the right time, without blocking any of the decoder threads. This leads to a complex communication structure between the interface, the input, the decoders and the outputs.  
 
Another key feature of VLC is that decoding and playing are asynchronous: decoding is done by a decoder thread, playing is done by audio_output or video_output thread. The design goal is to ensure that an audio or video frame is played exactly at the right time, without blocking any of the decoder threads. This leads to a complex communication structure between the interface, the input, the decoders and the outputs.  
  
Having several input and video_output threads reading multiple files at the same time is permitted, despite the fact that the current interface doesn't allow any way to do it [this is subject to change in the near future]. Anyway the client has been written from the ground up with this in mind. This also implies that a non-reentrant library (including in particular liba52) cannot be used without using a global lock.  
+
Having several input and video_output threads reading multiple files at the same time is permitted, despite the fact that the current interface doesn't allow any way to do it (this is subject to change in the near future). Anyway the client has been written from the ground up with this in mind. This also implies that a non-reentrant library (including in particular liba52<sup>[[{{TALKPAGENAME}}|See talk page]]</sup>) cannot be used without using a global lock.  
  
Presentation Time Stamps located in the system layer of the stream are passed to the decoders, and all resulting samples are dated accordingly. The output layers are supposed to play them at the right time. Dates are converted to microseconds&nbsp;; an absolute date is the number of microseconds since Epoch (Jan 1st, 1970). The mtime_t type is a signed 64-bit integer.  
+
Presentation Time Stamps located in the system layer of the stream are passed to the decoders, and all resulting samples are dated accordingly. The output layers are supposed to play them at the right time. Dates are converted to microseconds, an absolute date is the number of microseconds since Epoch (Jan 1st, 1970). The <code>mtime_t</code> type is a signed 64-bit integer.  
  
The current date can be retrieved with mdate(). The execution of a thread can be suspended until a certain date via mwait ( mtime_t date ). You can sleep for a fixed number of microseconds with msleep ( mtime_t delay ).  
+
The current date can be retrieved with <code>mdate()</code>. The execution of a thread can be suspended until a certain date via <code>mwait ( mtime_t date )</code>. You can sleep for a fixed number of microseconds with <code>msleep ( mtime_t delay )</code>.  
  
==== Warning  ====
+
=== Warning  ===
  
Please remember to wake up slightly before the presentation date, if some particular treatment needs to be done (e.g. a chroma transformation). For instance in modules/codec/libmpeg2.c, track of the average decoding times is kept to ensure pictures are not decoded too late.  
+
Please remember to wake up slightly before the presentation date, if some particular treatment needs to be done (e.g. a chroma transformation). For instance in <code>modules/codec/libmpeg2.c</code>, track of the average decoding times is kept to ensure pictures are not decoded too late.
  
 
== Core Source code details  ==
 
== Core Source code details  ==
  
All the [[LibVLCcore|libVLCcore]] source files are located in the src/ directory and its subdirectories:  
+
All the [[libVLCcore]] source files are located in the {{VLCSourceFolder|src}} directory and its subdirectories:  
 +
 
 +
; [[{{#rel2abs:../Audio Output}}|<code>audio_output/</code>]]
 +
: Initializes the audio mixer, ie. finds the right playing frequency, and then resamples audio frames received from the decoder(s).
 +
; <code>config/</code>
 +
: Load the configuration from command line and configuration file, provides functions for the modules to read and write to configuration
 +
; <code>control/</code>
 +
: Functions to control the behaviour of [[libVLCcore]], like Play/Pause, volume management, fullscreen, log verbosity, etc.
 +
; <code>extras/</code>
 +
: Mostly platform-specific code
 +
; [[{{#rel2abs:../Input}}|<code>input/</code>]]
 +
: Opens an input module, reads packets, parses them and passes reconstituted elementary streams to the decoder(s).
 +
; [[{{#rel2abs:../Interfaces}}|<code>interface/</code>]]
 +
: Contains code for user interaction such as key presses and device ejection.
 +
; <code>misc/</code>
 +
: Miscellaneous utilities used in other parts of libvlc, such as the thread system, the message queue, CPU detection, the object lookup system, or platform-specific code.
 +
; <code>modules/</code>
 +
: Modules management
 +
; <code>network/</code>
 +
: Network interface (socket management, network errors, etc.)
 +
; <code>osd/</code>
 +
: On Screen Display manipulation
 +
; <code>playlist/</code>
 +
: Manages playlist interaction such as stop, play, next, or random playback.
 +
; <code>stream_output/</code>
 +
: Functions to stream audio and video to the network
 +
; <code>test/</code>
 +
: libVLC needs to be tested, and not only by users :)
 +
; <code>text/</code>
 +
: Charset stuff
 +
; [[{{#rel2abs:../Video Output}}|<code>video_output/</code>]]
 +
: Initializes the video display, gets all pictures and subpictures (ie. subtitles) from the decoder(s), optionally converts them to another format (such as YUV to RGB), and displays them.
  
*[[Documentation:Hacker's Guide/Audio Output|audio_output/]]: initializes the audio mixer, ie. finds the right playing frequency, and then resamples audio frames received from the decoder(s).
+
{{Hacker_Guide}}
*config/: load the configuration from command line and configuration file, provides functions for the modules to read and write to configuration
 
*control/: functions to control the behaviour of [[LibVLCcore|libVLCcore]], like Play/Pause, volume management, fullscreen, log verbosity, etc.
 
*extras/: mostly platform-specific code
 
*[[Documentation:Hacker's Guide/Input|input/]]: opens an input module, reads packets, parses them and passes reconstituted elementary streams to the decoder(s).*modules/: module management
 
*[[Documentation:Hacker's Guide/Interfaces|interface/]]: contains code for user interaction such as key presses and device ejection.
 
*misc/: miscellaneous utilities used in other parts of libvlc, such as the thread system, the message queue, CPU detection, the object lookup system, or platform-specific code.
 
*modules/: modules management
 
*network/: network interface (socket management, network errors, etc.)
 
*osd/: On Screen Display manipulation
 
*playlist/: manages playlist interaction such as stop, play, next, or random playback.
 
*stream_output/: functions to stream audio and video to the network
 
*test/: libVLC needs to be tested, and not only by users&nbsp;:)
 
*text/: charset stuff
 
*[[Documentation:Hacker's Guide/Video Output|video_output/]]: initializes the video display, gets all pictures and subpictures (ie. subtitles) from the decoder(s), optionally converts them to another format (such as YUV to RGB), and displays them.
 

Latest revision as of 21:33, 30 May 2019

← Back to Hacker Guide

Introduction to libVLCcore

The core of VLC media player is called libVLCcore.
It manages the threads, the modules (codecs, demuxers, etc...), the modules' layers, the clocks, the playlist and all low-level control in VLC.
For example, it is responsible for the synchronization management between all the audio, video and subtitle tracks.

On top of libVLCcore, there is libVLC that allow external application builders to access to all features of the core.
Modules are linked with libvlccore, to interact with the core.

Modules are built against libvlccore. External applications are built against libVLC.

VLC Pipeline and Modularity

One of the main concepts in VLC is "modularity".

VLC is, in fact, a complete multimedia framework (like DirectShow or GStreamer) where you can load and plug-in many modules dynamically, depending on the necessity.

The core framework is used to do the "wiring" and the media processing, from input (files, network streams) to output (audio or video, on a screen or a network). It uses modules to do most of the work at every stage (various muxers, demuxers, decoders, filters and outputs).
Even the interfaces are plugins for libVLC.

The modules in VLC

So, VLC uses modules to do most of the work, at every stage of the pipeline. Modules are loaded accordingly at runtime depending on the necessity. (See VLC Modules loading mechanism).

Every module offers different features that will best suit a particular use-case or a particular environment.
Besides, most portability of VLC results from writing audio_output/video_output/interface modules specific to the platform.

plugins modules are loaded and unloaded dynamically by functions in src/modules/modules.c .
Modules can also be built directly into the application which uses libVLC, for instance when VLC is on an operating system that does not have support for dynamically loadable code. They are then called builtins.

In the source code, modules are usually located inside the modules/ subdirectory.

Thread management

VLC is heavily multi-threaded.

The single-threaded approach would have introduced too much complexity because decoder preemptibility and scheduling would then be a mastermind (for instance decoders and outputs have to be separated, otherwise it cannot be warrantied that a frame will be played at the exact presentation time).
Multi-process approach wasn't chosen either, because multi-process decoders usually imply more overhead (problems of shared memory) and communication between processes is harder.

VLC's threading structure is modeled after POSIX threads (pthread). However, for portability reasons, VLC does not use pthread_* functions directly, but a similar custom set of APIs.

Threads (vlc_thread_t)

vlc_clone()
Creates a thread.
vlc_join()
Waits for a thread to terminate and releases its resources

Mutual exclusion (vlc_mutex_t)

vlc_mutex_init()
Creates a non-recursive mutex.
vlc_mutex_init_recursive()
Creates a recursive mutex (discouraged).
vlc_mutex_lock()
Locks a mutex, waiting if required.
vlc_mutex_trylock()
Locks a mutex if it is not already locked, or returns an error otherwise.
vlc_mutex_unlock()
Unlocks a mutex.
vlc_mutex_destroy()
Destroys a mutex.

Condition variable (vlc_cond_t)

vlc_cond_init()
Creates a condition variable using the monotonic clock (mdate()) for timeouts.
vlc_cond_init_daytime()
Creates a condition variable using the realtime clock / wall clock for timeouts.
vlc_cond_signal()
Signals one thread waiting on a condition variable.
vlc_cond_broadcast()
Signals all threads waiting on a condition variable.
vlc_cond_wait()
Waits for a condition variable to be signaled (can also wake up spuriously).
vlc_cond_timedwait()
Waits for a condition variable to be signaled up to a certain timeout (can also wake up spuriously).
vlc_cond_destroy()
Destroys a condition variable.

Misc

VLC also has abstractions for slim read/write locks, spin locks, thread-specific variables, similar to POSIX threads.

Atomic variables

Atomic variables are small (sizeof(void *)) values that can be manipulated from multiple threads without locking. See include/vlc_atomic.h for the list of supported operations.

Synchronization

Another key feature of VLC is that decoding and playing are asynchronous: decoding is done by a decoder thread, playing is done by audio_output or video_output thread. The design goal is to ensure that an audio or video frame is played exactly at the right time, without blocking any of the decoder threads. This leads to a complex communication structure between the interface, the input, the decoders and the outputs.

Having several input and video_output threads reading multiple files at the same time is permitted, despite the fact that the current interface doesn't allow any way to do it (this is subject to change in the near future). Anyway the client has been written from the ground up with this in mind. This also implies that a non-reentrant library (including in particular liba52See talk page) cannot be used without using a global lock.

Presentation Time Stamps located in the system layer of the stream are passed to the decoders, and all resulting samples are dated accordingly. The output layers are supposed to play them at the right time. Dates are converted to microseconds, an absolute date is the number of microseconds since Epoch (Jan 1st, 1970). The mtime_t type is a signed 64-bit integer.

The current date can be retrieved with mdate(). The execution of a thread can be suspended until a certain date via mwait ( mtime_t date ). You can sleep for a fixed number of microseconds with msleep ( mtime_t delay ).

Warning

Please remember to wake up slightly before the presentation date, if some particular treatment needs to be done (e.g. a chroma transformation). For instance in modules/codec/libmpeg2.c, track of the average decoding times is kept to ensure pictures are not decoded too late.

Core Source code details

All the libVLCcore source files are located in the src directory and its subdirectories:

audio_output/
Initializes the audio mixer, ie. finds the right playing frequency, and then resamples audio frames received from the decoder(s).
config/
Load the configuration from command line and configuration file, provides functions for the modules to read and write to configuration
control/
Functions to control the behaviour of libVLCcore, like Play/Pause, volume management, fullscreen, log verbosity, etc.
extras/
Mostly platform-specific code
input/
Opens an input module, reads packets, parses them and passes reconstituted elementary streams to the decoder(s).
interface/
Contains code for user interaction such as key presses and device ejection.
misc/
Miscellaneous utilities used in other parts of libvlc, such as the thread system, the message queue, CPU detection, the object lookup system, or platform-specific code.
modules/
Modules management
network/
Network interface (socket management, network errors, etc.)
osd/
On Screen Display manipulation
playlist/
Manages playlist interaction such as stop, play, next, or random playback.
stream_output/
Functions to stream audio and video to the network
test/
libVLC needs to be tested, and not only by users :)
text/
Charset stuff
video_output/
Initializes the video display, gets all pictures and subpictures (ie. subtitles) from the decoder(s), optionally converts them to another format (such as YUV to RGB), and displays them.
This page is part of official VLC media player Documentation (User GuideStreaming HowToHacker GuideModules)
Please read the Documentation Editing Guidelines before you edit the documentation
Permission is granted to copy, distribute and/or modify this document under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version.