Difference between revisions of "Hacker Guide/Core"
| Line 1: | Line 1: | ||
| − | == Introduction to libVLCcore == | + | == Introduction to libVLCcore == |
| − | |||
| − | It manages the threads, the modules (codecs, demuxers, etc...), the modules' layers, the clocks, the playlist and all low-level control in VLC. For example, it is responsible for the synchronization management between all the audio and video tracks. | + | The core of VLC media player is called [[LibVLCcore]].<br>It manages the threads, the modules (codecs, demuxers, etc...), the modules' layers, the clocks, the playlist and all low-level control in VLC. For example, it is responsible for the synchronization management between all the audio and video tracks. |
| − | On top of libvlccore, there is [[ | + | On top of libvlccore, there is [[LibVLC]] that allow external application builders to access to all features of the core.<br> Modules are linked with libvlccore, to interact with the core. |
| + | '''Modules are built against libvlccore. External applications are built against [[LibVLC]].'''<br> | ||
| − | Modules | + | == Modules == |
| − | + | One of the main concept in VLC is "modularity". | |
| + | VLC is, in fact, a complete [http://en.wikipedia.org/wiki/Multimedia_framework multimedia framework] (like DirectShow or GStreamer) where you can load and plug-in many modules dynamically, depending on the necessity. | ||
| − | + | <br> The core framework is used to do the media processing, from input (files, network streams) to output (audio or video, on ascreen or a network), going through various muxers, demuxers, decoders and filters.<br> Even the interfaces are plugins for LibVLC. It is up to the developer to choose which module will be loaded.<br> | |
| − | |||
| − | VLC | + | === ''The module chain in VLC'' === |
| + | <br> Modules are located in the modules/ subdirectory and are loaded at runtime. Every module may offer different features that will best suit a particular file or a particular environment. Besides, most portability works result in the writing of an audio_output/video_output/interface module to support a new platform (eg. BeOS or MacOS X). | ||
| − | The | + | Plugin modules are loaded and unloaded dynamically by functions in src/modules.c and include/vlc_modules*.h. The API for writing modules will be discussed in a following chapter. |
| − | + | Modules can also be built directly into the application which uses LibVLC, for instance on an operating system that does not have support for dynamically loadable code. Modules statically built into the application are called builtins. | |
| + | == Threads == | ||
| − | + | === Thread management === | |
| − | + | VLC is heavily multi-threaded. We chose against a single-thread approach because decoder preemptibility and scheduling would be a mastermind (for instance decoders and outputs have to be separated, otherwise it cannot be warrantied that a frame will be played at the exact presentation time), and we currently have no plan to support a single-threaded client. Multi-process decoders usually imply more overhead (problems of shared memory) and communication between processes is harder. | |
| + | Our threading structure is modeled on pthreads. However, for portability reasons, we don't call pthread_* functions directly, but use a similar wrapper, made of vlc_thread_create, vlc_thread_exit, vlc_thread_join, vlc_mutex_init, vlc_mutex_lock, vlc_mutex_unlock, vlc_mutex_destroy, vlc_cond_init, vlc_cond_signal, vlc_cond_broadcast, vlc_cond_wait, vlc_cond_destroy, and structures vlc_thread_t, vlc_mutex_t, and vlc_cond_t. | ||
| − | + | === Synchronization === | |
| − | + | Another key feature of VLC is that decoding and playing are asynchronous: decoding is done by a decoder thread, playing is done by audio_output or video_output thread. The design goal is to ensure that an audio or video frame is played exactly at the right time, without blocking any of the decoder threads. This leads to a complex communication structure between the interface, the input, the decoders and the outputs. | |
| − | + | Having several input and video_output threads reading multiple files at the same time is permitted, despite the fact that the current interface doesn't allow any way to do it [this is subject to change in the near future]. Anyway the client has been written from the ground up with this in mind. This also implies that a non-reentrant library (including in particular liba52) cannot be used without using a global lock. | |
| − | + | Presentation Time Stamps located in the system layer of the stream are passed to the decoders, and all resulting samples are dated accordingly. The output layers are supposed to play them at the right time. Dates are converted to microseconds ; an absolute date is the number of microseconds since Epoch (Jan 1st, 1970). The mtime_t type is a signed 64-bit integer. | |
| − | |||
| − | + | The current date can be retrieved with mdate(). The execution of a thread can be suspended until a certain date via mwait ( mtime_t date ). You can sleep for a fixed number of microseconds with msleep ( mtime_t delay ). | |
| − | + | ==== Warning ==== | |
| − | + | Please remember to wake up slightly before the presentation date, if some particular treatment needs to be done (e.g. a chroma transformation). For instance in modules/codec/libmpeg2.c, track of the average decoding times is kept to ensure pictures are not decoded too late. | |
| − | + | == Core Source code details == | |
| − | + | All the [[LibVLCcore]] source files are located in the src/ directory and its subdirectories: | |
| − | + | *[[Documentation:Hacker's Guide/Audio Output|audio_output/]]: initializes the audio mixer, ie. finds the right playing frequency, and then resamples audio frames received from the decoder(s). | |
| − | + | *config/: load the configuration from command line and configuration file, provides functions for the modules to read and write to configuration | |
| − | + | *control/: functions to control the behaviour of [[LibVLCcore]], like Play/Pause, volume management, fullscreen, log verbosity, etc. | |
| − | + | *extras/: mostly platform-specific code | |
| − | + | *[[Documentation:Hacker's Guide/Input|input/]]: opens an input module, reads packets, parses them and passes reconstituted elementary streams to the decoder(s).*modules/: module management | |
| − | + | *[[Documentation:Hacker's Guide/Interfaces|interface/]]: contains code for user interaction such as key presses and device ejection. | |
| − | + | *misc/: miscellaneous utilities used in other parts of libvlc, such as the thread system, the message queue, CPU detection, the object lookup system, or platform-specific code. | |
| − | + | *modules/: modules management | |
| − | + | *network/: network interface (socket management, network errors, etc.) | |
| − | + | *osd/: On Screen Display manipulation | |
| − | + | *playlist/: manages playlist interaction such as stop, play, next, or random playback. | |
| − | *[[Documentation:Hacker's Guide/Audio Output| audio_output/]]: initializes the audio mixer, ie. finds the right playing frequency, and then resamples audio frames received from the decoder(s). | + | *stream_output/: functions to stream audio and video to the network |
| − | *config/: load the configuration from command line and configuration file, provides functions for the modules to read and write to configuration | + | *test/: LibVLC needs to be tested, and not only by users :) |
| − | *control/: functions to control the behaviour of [[ | + | *text/: charset stuff |
| − | *extras/: mostly platform-specific code | + | *[[Documentation:Hacker's Guide/Video Output|video_output/]]: initializes the video display, gets all pictures and subpictures (ie. subtitles) from the decoder(s), optionally converts them to another format (such as YUV to RGB), and displays them. |
| − | *[[Documentation:Hacker' | ||
| − | *[[Documentation:Hacker's Guide/Interfaces| interface/]]: contains code for user interaction such as key presses and device ejection. | ||
| − | *misc/: miscellaneous utilities used in other parts of libvlc, such as the thread system, the message queue, CPU detection, the object lookup system, or platform-specific code. | ||
| − | *modules/: modules management | ||
| − | *network/: network interface (socket management, network errors, etc.) | ||
| − | *osd/: On Screen Display manipulation | ||
| − | *playlist/: manages playlist interaction such as stop, play, next, or random playback. | ||
| − | *stream_output/: functions to stream audio and video to the network | ||
| − | *test/: LibVLC needs to be tested, and not only by users :) | ||
| − | *text/: charset stuff | ||
| − | *[[Documentation:Hacker's Guide/Video Output| video_output/]]: initializes the video display, gets all pictures and subpictures (ie. subtitles) from the decoder(s), optionally converts them to another format (such as YUV to RGB), and displays them. | ||
Revision as of 13:47, 29 October 2010
Contents
Introduction to libVLCcore
The core of VLC media player is called LibVLCcore.
It manages the threads, the modules (codecs, demuxers, etc...), the modules' layers, the clocks, the playlist and all low-level control in VLC. For example, it is responsible for the synchronization management between all the audio and video tracks.
On top of libvlccore, there is LibVLC that allow external application builders to access to all features of the core.
Modules are linked with libvlccore, to interact with the core.
Modules are built against libvlccore. External applications are built against LibVLC.
Modules
One of the main concept in VLC is "modularity".
VLC is, in fact, a complete multimedia framework (like DirectShow or GStreamer) where you can load and plug-in many modules dynamically, depending on the necessity.
The core framework is used to do the media processing, from input (files, network streams) to output (audio or video, on ascreen or a network), going through various muxers, demuxers, decoders and filters.
Even the interfaces are plugins for LibVLC. It is up to the developer to choose which module will be loaded.
The module chain in VLC
Modules are located in the modules/ subdirectory and are loaded at runtime. Every module may offer different features that will best suit a particular file or a particular environment. Besides, most portability works result in the writing of an audio_output/video_output/interface module to support a new platform (eg. BeOS or MacOS X).
Plugin modules are loaded and unloaded dynamically by functions in src/modules.c and include/vlc_modules*.h. The API for writing modules will be discussed in a following chapter.
Modules can also be built directly into the application which uses LibVLC, for instance on an operating system that does not have support for dynamically loadable code. Modules statically built into the application are called builtins.
Threads
Thread management
VLC is heavily multi-threaded. We chose against a single-thread approach because decoder preemptibility and scheduling would be a mastermind (for instance decoders and outputs have to be separated, otherwise it cannot be warrantied that a frame will be played at the exact presentation time), and we currently have no plan to support a single-threaded client. Multi-process decoders usually imply more overhead (problems of shared memory) and communication between processes is harder.
Our threading structure is modeled on pthreads. However, for portability reasons, we don't call pthread_* functions directly, but use a similar wrapper, made of vlc_thread_create, vlc_thread_exit, vlc_thread_join, vlc_mutex_init, vlc_mutex_lock, vlc_mutex_unlock, vlc_mutex_destroy, vlc_cond_init, vlc_cond_signal, vlc_cond_broadcast, vlc_cond_wait, vlc_cond_destroy, and structures vlc_thread_t, vlc_mutex_t, and vlc_cond_t.
Synchronization
Another key feature of VLC is that decoding and playing are asynchronous: decoding is done by a decoder thread, playing is done by audio_output or video_output thread. The design goal is to ensure that an audio or video frame is played exactly at the right time, without blocking any of the decoder threads. This leads to a complex communication structure between the interface, the input, the decoders and the outputs.
Having several input and video_output threads reading multiple files at the same time is permitted, despite the fact that the current interface doesn't allow any way to do it [this is subject to change in the near future]. Anyway the client has been written from the ground up with this in mind. This also implies that a non-reentrant library (including in particular liba52) cannot be used without using a global lock.
Presentation Time Stamps located in the system layer of the stream are passed to the decoders, and all resulting samples are dated accordingly. The output layers are supposed to play them at the right time. Dates are converted to microseconds ; an absolute date is the number of microseconds since Epoch (Jan 1st, 1970). The mtime_t type is a signed 64-bit integer.
The current date can be retrieved with mdate(). The execution of a thread can be suspended until a certain date via mwait ( mtime_t date ). You can sleep for a fixed number of microseconds with msleep ( mtime_t delay ).
Warning
Please remember to wake up slightly before the presentation date, if some particular treatment needs to be done (e.g. a chroma transformation). For instance in modules/codec/libmpeg2.c, track of the average decoding times is kept to ensure pictures are not decoded too late.
Core Source code details
All the LibVLCcore source files are located in the src/ directory and its subdirectories:
- audio_output/: initializes the audio mixer, ie. finds the right playing frequency, and then resamples audio frames received from the decoder(s).
- config/: load the configuration from command line and configuration file, provides functions for the modules to read and write to configuration
- control/: functions to control the behaviour of LibVLCcore, like Play/Pause, volume management, fullscreen, log verbosity, etc.
- extras/: mostly platform-specific code
- input/: opens an input module, reads packets, parses them and passes reconstituted elementary streams to the decoder(s).*modules/: module management
- interface/: contains code for user interaction such as key presses and device ejection.
- misc/: miscellaneous utilities used in other parts of libvlc, such as the thread system, the message queue, CPU detection, the object lookup system, or platform-specific code.
- modules/: modules management
- network/: network interface (socket management, network errors, etc.)
- osd/: On Screen Display manipulation
- playlist/: manages playlist interaction such as stop, play, next, or random playback.
- stream_output/: functions to stream audio and video to the network
- test/: LibVLC needs to be tested, and not only by users :)
- text/: charset stuff
- video_output/: initializes the video display, gets all pictures and subpictures (ie. subtitles) from the decoder(s), optionally converts them to another format (such as YUV to RGB), and displays them.
