Hacker Guide/Core

Introduction to libVLCcore

The core of VLC media player is called libVLCcore.
It manages the threads, the modules (codecs, demuxers, etc...), the modules' layers, the clocks, the playlist and all low-level control in VLC.
For example, it is responsible for the synchronization management between all the audio and video tracks.

On top of libVLCcore, there is libVLC that allow external application builders to access to all features of the core.
Modules are linked with libvlccore, to interact with the core.

Modules are built against libvlccore. External applications are built against libVLC.

VLC Pipeline and Modularity

One of the main concept in VLC is "modularity".

VLC is, in fact, a complete multimedia framework (like DirectShow or GStreamer) where you can load and plug-in many modules dynamically, depending on the necessity.

The core framework is used to do the "wiring" and the media processing, from input (files, network streams) to output (audio or video, on ascreen or a network). It uses modules to do most of the work at every stage (various muxers, demuxers, decoders, filters and outputs).
Even the interfaces are plugins for libVLC.

The modules in VLC

So, VLC uses modules to do most of the work, at every stage of the pipeline. Modules are loaded according at runtime depending on the necessity (See VLC Modules Loading).

Every module offers different features that will best suit a particular use-case or a particular environment.
Besides, most portability of VLC result in writing audio_output/video_output/interface modules specific to the platform.

plugins modules are loaded and unloaded dynamically by functions in src/modules.c .
Modules can also be built directly into the application which uses libVLC, for instance when VLC is on operating system that does not have support for dynamically loadable code. They are then called builtins.

In the source code, modules are usually located inside the modules/ subdirectory.

Threads & Synchronisation

Thread management

VLC is heavily multi-threaded.

The single-threaded approach would have introduced too much complexity because decoder preemptibility and scheduling would then be a mastermind (for instance decoders and outputs have to be separated, otherwise it cannot be warrantied that a frame will be played at the exact presentation time).
Multi-process approach wasn't chosen either, because multi-process decoders usually imply more overhead (problems of shared memory) and communication between processes is harder.

VLC's threading structure is modeled on pthreads.
However, for portability reasons, we don't call pthread_* functions directly, but use a similar wrapper, made of vlc_thread_create, vlc_thread_join, vlc_mutex_init, vlc_mutex_lock, vlc_mutex_unlock, vlc_mutex_destroy, vlc_cond_init, vlc_cond_signal, vlc_cond_broadcast, vlc_cond_wait, vlc_cond_destroy, etc... and structures vlc_thread_t, vlc_mutex_t, and vlc_cond_t.

Synchronization

Another key feature of VLC is that decoding and playing are asynchronous: decoding is done by a decoder thread, playing is done by audio_output or video_output thread. The design goal is to ensure that an audio or video frame is played exactly at the right time, without blocking any of the decoder threads. This leads to a complex communication structure between the interface, the input, the decoders and the outputs.

Having several input and video_output threads reading multiple files at the same time is permitted, despite the fact that the current interface doesn't allow any way to do it [this is subject to change in the near future]. Anyway the client has been written from the ground up with this in mind. This also implies that a non-reentrant library (including in particular liba52) cannot be used without using a global lock.

Presentation Time Stamps located in the system layer of the stream are passed to the decoders, and all resulting samples are dated accordingly. The output layers are supposed to play them at the right time. Dates are converted to microseconds ; an absolute date is the number of microseconds since Epoch (Jan 1st, 1970). The mtime_t type is a signed 64-bit integer.

The current date can be retrieved with mdate(). The execution of a thread can be suspended until a certain date via mwait ( mtime_t date ). You can sleep for a fixed number of microseconds with msleep ( mtime_t delay ).

Warning

Please remember to wake up slightly before the presentation date, if some particular treatment needs to be done (e.g. a chroma transformation). For instance in modules/codec/libmpeg2.c, track of the average decoding times is kept to ensure pictures are not decoded too late.

Core Source code details

All the libVLCcore source files are located in the src/ directory and its subdirectories:

• audio_output/: initializes the audio mixer, ie. finds the right playing frequency, and then resamples audio frames received from the decoder(s).
• config/: load the configuration from command line and configuration file, provides functions for the modules to read and write to configuration
• control/: functions to control the behaviour of libVLCcore, like Play/Pause, volume management, fullscreen, log verbosity, etc.
• extras/: mostly platform-specific code
• input/: opens an input module, reads packets, parses them and passes reconstituted elementary streams to the decoder(s).*modules/: module management
• interface/: contains code for user interaction such as key presses and device ejection.
• misc/: miscellaneous utilities used in other parts of libvlc, such as the thread system, the message queue, CPU detection, the object lookup system, or platform-specific code.
• modules/: modules management
• network/: network interface (socket management, network errors, etc.)
• osd/: On Screen Display manipulation
• playlist/: manages playlist interaction such as stop, play, next, or random playback.
• stream_output/: functions to stream audio and video to the network
• test/: libVLC needs to be tested, and not only by users :)
• text/: charset stuff
• video_output/: initializes the video display, gets all pictures and subpictures (ie. subtitles) from the decoder(s), optionally converts them to another format (such as YUV to RGB), and displays them.