SoC 2020

This page summarizes the current list of ideas for VideoLAN project, for Google Summer Of Code 2020.

The list is not exhaustive, and is not limitative. Feel free to apply with your own idea!

Introduction & Information

This wiki page covers the attempt by the VideoLAN project to act as a mentoring organization for Google Summer of Code, in order to improve VLC, VLMC and dav1d (or the libVLC engine) and their dependencies.

We have projects in C, C++, ASM, JS, Go, Obj-C, GPU Shaders, C#, Java/Kotlin and Swift.

Summer of Code

Google Summer of Code is a way for university students to have a paid internship by Google to work on open source projects and become top developers!

VideoLAN was a Google Summer of Code mentoring organization in 2007, 2008, 2009, 2010, 2011, 2013, 2016, 2017, 2018 and 2019.

This page lists some ideas for Summer of Code projects on dav1d, VLMC, libVLC and VLC media player.

VLC & libVLC

VLC media player is a cross-platform multimedia player, encoder and streamer application. It is one of the most successful open-source projects without any professional structure underneath.

VLC media player is downloaded at an approximate monthly rate of 25 millions from the main website and that's not including third-party distributions (Linux)!

You can find more information on VLC on Wikipedia or on this wiki.

The engine of VLC is libVLC, and is used by VLMC.

dav1d

dav1d is the reference AV1 decoder that is shipped inside VLC, Chrome and Firefox.

It's a very low-level codec, mostly written in C and assembly.

VideoLAN Movie Creator

VLMC is a cross-platform non-linear video editing software based on libVLC technology. It was started as a final year student project at the French IT school EPITECH.

VLMC currently awaits a transition to the current libvlc API and it's actual 1.0 release!

Summer of Code rules

If selected and developed, SoC projects for VLMC and VLC will be included in later releases.

All projects are covered by the GPL (v2+) or LGPL (v2.1+) licenses depending on the module.

How to Start

Find an idea

First, you need to find an idea.

This current page gives you a list of ideas. Those ideas are NOT exhaustive: you can bring your own idea! Some of the best ideas we've ever had were custom ideas!

Submit your idea

You need to submit your idea on the Google Summer of Code platform.

You should do so, very quickly, even before having finished compilation and the next steps, so that we can give you early feedback.

Compile libVLC or VLMC

This may sound trivial, but it's harder than many expect. See https://wiki.videolan.org/Category:Building/ for more informations.

You should come on IRC to get help to compile.

Provide a small patch

To demonstrate your skills, share a small patch with us. This will let you become familiar with Git, in case you don't know it already and our process on merging patches.

Let's get in touch

If you have a doubt, are not sure about anything or want clarification, please get in touch with us.

We have 3 major communication channels:

• Our mailing-lists to discuss patches and further development related topics;
• Furthermore, we have our web forums for VLC-related end-user support - a VLMC section will be created once the product is published.
• Finally, there is our IRC channel #videolan (for libVLC) and #vlmc on the Freenode network. It's open to any kind of discussion. Usage issues, questions how to compile VLC/VLMC, getting to know the fellow developers, etc.

Ideas for VLC & libVLC

Advanced Audio Filters

Project Description: We are looking for a skilled audiophile that knows a lot about audio theory and practice to work on new audio filters for VLC.

Tasks to do:

• SRS WoW like or other 3D effects;
• channels mixing, notably upmixing, like Prologic-II;
• tracks mixing, and transitions;
• scriptable new audio filters in lua and enable users to create whatever audio filtering function they want in a Lua script;
• LADSPA or other libraries integration.

Requirements: This project needs some good audio knowledge and good C experience.

Qualification task: Port any audio filter from MPlayer

Proposed mentor: unidan

VLC Qt interface work

Project Description: The VLC interface is quite outdated on Linux and Windows. It has a lot of features, but some are not properly exposed.

We are currently reworking the interface, but we need help.

Scope of the tasks to do:

The idea is to get closer to what the WinRT/UWP interface looks like (search on google image to get an idea).

The WinRT interface is simpler, more user friendly, and has a better "media center" feel into it.

It would require integration with the media library and with the current interface.

Qml is the preferred way of improving the current UI.

Requirements: This project requires Qt/C++ knowledge, and qml would be a nice plus.

Proposed mentor: Pierre

VLC.js

Project Description: We're currently porting VLC to the web, using WebAssembly. This is a current work in progress, so we need a lot of help to improve it

This requires working close to the toolchain, compilation and the web browsers.

Scope of the tasks to do:

There are lots of things to do (not all of them need to be done by the same person:

• Improving compilation of the VLC core to Wasm,
• Improving compilation of VLC 3rd party libraries,
• Fixing the VLC core to support the Web platform,
• Implement WebCodec inside VLC,
• Improve audio pipeline inside VLC,
• Improve emscripten :)

Requirements: This project requires C knowledge, and compilation.

Proposed mentor: JB

VLC macOS interface redesign

Project Description: The VLC interface is quite outdated on macOS and we are currently in the process of re-writing it to give it a modern feel, but also to integrate recent additions to libvlc regarding playback control and library management.

This project for the summer is to rework heavily this interface to make it beautiful and useful again.

Scope of the tasks to do:

There is a full design already done and tested. The major hurdle is to actually implement it the way we want it to be. The iOS/tvTV interface is simpler, more user friendly, and has a better "media center" feel into it, which influenced what we want to achieve on the Mac. Note that the objective is to use AppKit. UIKit (aka "Marzipan") will not be part of this project.

Iterating from the current UI and closely collaborating with the team currently working on it is a requirement.

Requirements: This project requires Obj-C knowledge, a thorough understanding of OOP and proven previous Mac development experience. You cannot use swift for this project.

Proposed mentor: David Fuhrmann, Felix Paul Kühne

GPU shader filters

Project Description: VLC is using OpenGL and Direct3D11 on modern machines. It is now possible to do all kinds of video processing that was done in the CPU directly in the GPU.

This project for the summer is to port some of the existing CPU filters to the GPU using shaders.

Scope of the tasks to do:

The first task will be to identify all the video filters in VLC and identify the ones that could be done as shaders. Then implement them. Either in OpenGL shaders, Direct3D11 shaders, both and maybe Vulkan.

Requirements: This project requires OpenGL or Direct3D11 shaders knowledge as well as knowledge on video filters in general.

Proposed mentor: Steve Lhomme, unidan

VLC iOS new UI

Project Description: We're currently in the process of rewriting and updating the entire UI for VLC iOS

There is a lot of components that need refactoring and need to get an updated UI. Like for example the network navigation section.

Further more tests should be written for these components to ensure that we don't break features on old or new devices.

The project for this summer is to write these components in Swift with updated UI and make sure the app becomes and stays stable.

Tasks to do:

• Get an overview of the current App and components that need an update
• Refactor and give the appropriate components a new look
• Writing tests for these components to ensure that functionality works on all supported devices
• Rewriting the first start screens

Requirements: This project requires Obj-C and Swift knowledge and ideally knowledge of writing tests for iOS but this can be learned.

Proposed mentor: Soomin Lee, Felix Paul Kühne

Open Source Fuzzing platform

The idea of this project is to develop a new web platform to support fuzzing or code analysis, like oss_fuzz, but open source.
That would allow to see the defects, to be able to act on them.

It should be adaptable for clang-analyser or other similar tools.

It can be based on gitlab, if needed, but should be developed in Go.

Requirements: This project requires Go knowledge and ideally knowledge of JS

Proposed mentor: Hugo Beauzée-Luyssen

Interactive movie support

The idea is to add support for interactive movies inside VLC using the matroska.org mkv format. An interactive movie is like a gamebook but for movies.

Tasks to do:

• Extend the MKV spec to add support for titles, menus, that will be used by interactive movies
• Improve a MKV muxing tool (ffmpeg, mkvtoolnix) to support the extended spec
• Add interactive movies support inside VLC's MKV module

Those will be done in C and C++

Proposed mentor: Thomas Guillem, Steve Lhomme

Develop a MPD server inside VLC

The idea is to add support for VLC to act as a MPD server.

Tasks to do:

• Study the MPD protocol
• Code a MPD server as a control module that will connect to the medialibrary

Those will be done in C and C++, and it requires familiarity with an MPD client

Proposed mentor: Thomas Guillem

Improve the web interface of VLC in Vue.js

The idea is to improve the new web interface of VLC, that is currently in Vue.JS. It requires significant improvements

• Review the current code
• Clean and improve the modularity
• Adapt to the new playlist code
• Connect to the new medialibrary
• Improve the look

Those will be done in C and C++, and it requires familiarity with an MPD client

Proposed mentor: David

Improve VLCKit Objective-C - Swift interoperability

Project description:

Currently VLCKit is not very Swift friendly. Indeed, it will be beneficial for VLCKit to have a better Swift interoperability interface than one generated by default.

Tasks to do:

• Study exposed VLCKit methods
• Improve Objective-C - Swift interop
• Make a test Swift project to test the interoperability

Requirements: This project requires Objective-C and Swift knowledge.

Proposed mentor: Felix Paul Kühne, Soomin Lee

Other ideas for VLC & libVLC

Those ideas are not detailed, but they are ideas that we could help to spring new ideas. We can help work with you to make those more detailed.

• Integrate Rust inside VLC, as a demuxer or a parser
• Improve Vulkan output for VLC, including HDR support
• Improve id3 tag and metadata handling in VLC
• Support NVDec decoding inside VLC
• Work on emscripten version of VLC, aka VLC.js
• Bridge module for GMI'C or other video filters
• Automated Testing Environment like ffmpeg Fate (port ?) for demuxing, non-hw decoding
• Integrate libavfilter in VLC
• Improve the libVLCSharp bindings for VLC in C#
• Provide setups for popular streaming services / sout templates (ui ?)

Ideas for VLMC

Port to Android

Project Description: VLMC is now working on the desktop. It would be nice to port it also to the mobile world, starting by Android.

This project would require to adapt to smaller screens, and simplify the UI to fit the Android workflow.

Tasks to do:

1. Fix the VLMC build for Android. As libVLC works fine on Android, this is more focused on compiling the Qt part for Android.
2. Split more parts of the UI in components that are reusable on mobile.
3. Change the components to fit the small resolutions of the Android devices.
4. Rewrite some UI components in QML.
5. Write a simpler timeline widget for Mobile workflows.

Requirements: This project requires C++/Qt/qml knowledge, and access to Android devices.

Proposed mentors: Geoffrey/Hugo

Port to iOS

Project Description: VLMC is now working on the desktop. It would be nice to port it also to the mobile world, also on iOS.

This project would require to adapt to smaller screens, and simplify the UI to fit the iOS workflow.

Tasks to do:

1. Fix the VLMC build for iOS. As libVLC works fine on iOS, this is more focused on compiling the Qt part for iOS.
2. Split more parts of the UI in components that are reusable on mobile.
3. Change the components to fit the small resolutions of the iOS devices.
4. Rewrite some UI components in QML.
5. Write a simpler timeline widget for Mobile workflows.

Requirements: This project requires C++/Qt/qml knowledge, and access to iOS devices. Previous iOS development experience is preferred.

Proposed mentors: Felix/Hugo

Media Workflow rework

Project Description: VLMC is now working on the desktop, but the media workflow is quite slow and is missing some common use cases.

This should be fixed in MLT framework backend for libVLC and is the continuation from the work from the previous Summer of Code.

Tasks to do:

1. Understand the MLT libVLC backend codebase,
2. Clean the backend and port it to a cleaner code state in C++,
3. Benchmark the code and make it more efficient,
4. Work to merge this backend upstream,
5. Implement all the missing features in libVLC.

Requirements: This project requires C/C++ knowledge, and understanding of multimedia concepts is a good positive point.

Proposed mentors: Hugo

Workflow audio filters

Project Description: VLMC is now working on the desktop, but we need to have a good setup of audio filters to make it usable for most people.

Tasks to do:

1. Understand the VLMC and MLT codebase interactions,
2. Add audio filters on the MLT framework backend using the LADSPA/LV2 library or a similar open source library,
3. Code the interface corresponding to those filters, and their options,
4. Prepare visualizations, and notably for soundwave filters,
5. Integrate those inside the timeline UI,
6. Optimize.

Requirements: This project requires C++/Qt/qml knowledge, and enough understanding of audio.

Proposed mentors: J-B

Workflow video filters

Project Description: VLMC is now working on the desktop, but we need to have a good setup of video filters to make it usable for most people.

Tasks to do:

1. Understand the VLMC and MLT codebase interactions,
2. Add video filters on the MLT framework backend using the movit library or a similar open source library,
3. Code the interface corresponding to those filters, and their options,
4. Add transitions between clips
5. Code the interface to manage those transitions,
6. Optimize the code.

Requirements: This project requires C/C++ knowledge.

Proposed mentors: Felix

Improve multiplatform support

VLMC is aiming at being fully cross platform, but drifted a bit from that goal. While most VLMC's code is cross platform, and can be easily built targeting platforms, some dependencies might prove more difficult to build or compose with VLMC.

Tasks to do:

1. Fix MLT cross compilation for Windows (This will most likely require patching directly inside MLT, all contributions must be sent upstream).
2. Fix medialibrary code & build process on macOS (and potentially iOS). The medialibrary misses some platform specific code to be able to list all connected drives (hard drives & USB removable storage, mostly) along with filesystem browsing
3. Update the documentation along the way
4. Create jenkins build configuration to have nightly builds on jenkins.videolan.org & nightlies.videolan.org for all supported platforms

Requirements: This project requires buildsystems/toolchains knowledge, basic system programming on macOS & Windows

Proposed mentors: Hugo

Remote UI

We would like to have a way to use VLMC from a web browser. You can easily imagine having a nice, shiny & simple UI for minimal movie edition, which would go hand in hand with the cloud storage feature.

Tasks to do: This task aims toward the uncoupling of the rendering backend & UI, as the renderer will run server side, while the UI runs on the client side.

The idea is to be able to have a UI interacting with the renderer without having to be in the same process, or even machine.

Requirements: This project requires C++ and JS knowledge.

Proposed mentor: jb, chouquette, fkuehne

Ideas for dav1d

dav1d ARM NEON optimization

Improving the performance of the AV1 decoder is very important for VLC and the whole ecosystem.

It requires to:

• Profiling to identify hotspots of the new libdav1d video decoder on arm64 across common test files
• Analysis of potential benefits of NEON SIMD for these hotspots
• Write arm64 NEON optimization using directly written assembly code

Requirements: This project requires C and ASM knowledge, as well as system programming skills

Contact 'j-b'

dav1d GPU Compute Shaders

Improving the performance of the AV1 decoder is very important for VLC and the whole ecosystem.

This project requires to port one of the filter, like SGR or Wiener to one of the Shader languages. iPhones or Xbox One would be a good target.

This is a tricky project, but is doable during the summer

Requirements: This project requires C and GPU Shaders knowledge, as well as system programming skills

Contact 'j-b'

dav1d statistics extractions

The dav1d AV1 decoder is a new high performance AV1 decoder by VideoLAN.

Current open source tools for AV1 analysis use instrumentation in the reference decoder libaom to extract decode-time metadata for display and reporting, but support for sophisticated analysis is lacking.

To speed development of AV1 tools like the rav1e, it would be helpful to add similar decoder metadata extraction APIs to the dav1d decoder so that rapid testing of encoder algorithms is easier. This includes the ability to quickly produce statistics, visualizations and other reporting that can be used for tuning encoder parameters or guiding development. Advanced ideas include adding similar encoder metadata API to rav1e that add encode-time visualizations.

Requirements: This project requires C knowledge.

Contact 'unlord'

Ideas for VLC dependencies

libmicrodns refactoring

Our current mDNS discoverer is working, but is not so respectful of the RFC. Possible improvements include:

• Device TTL support
• Device removal detection
• Better request pacing
• Delegate socket interactions to the caller
• Unit testing
• Fuzzing

Requirements: This project require C knowledge, as well as system programming skills

Proposed mentor: chouquette, tguillem

rav1e, the AV1 encoder in Rust

The rav1e project aims to create a new AV1 encoder implementation in Rust.

To support this we need better AV1 analysis tools that give detailed information about AV1 streams including:

• bit rate across the frame
• frame allocation within a scene
• display of frame data (block partitioning, mode decisions, motion vectors, etc)
• objective quality over time
• average rate (for streaming)

Requirements: This project requires Rust knowledge.

Contact 'unlord'