SoC 2010 Phonon Capture API

This project is part of Google Summer of Code 2010. Student: Casian Andrei Mentor: Hugo Beauzee-Luyssen

Phonon High-level Capture API

Abstract

The goal of this project is to enable Phonon applications to access and display input devices, like webcams. The Phonon code-base is well designed and flexible, and apparently no major modifications are needed to the current API. The ease of use for the Phonon application developer is a priority. Features implemented in Phonon will be implemented in the Phonon-VLC back-end in parallel. Various device classes should be supported.

Information

The exam session lasts here most of June, so I won't be able to work at full capacity during this time (I hope I can achieve 50%-75% of the July work time).

If extra time remains, I will work on improving the overall quality of Phonon.

Timeline

The following timeline may be a bit optimistic, but I will try to respect it. It will be revised during the summer if neccessary. By revised I mean adding smaller tasks, not removing tasks or delaying.

Date	Status	Description
May 10 - May 24	Completed	Setup development environment, test things, play with the code
May 24 - June 15	V4L ok	Setup MediaSource to handle capture devices, implement in Phonon-VLC
June 5 - June 15	V4L ok	Implement API to get information about capture devices
June 15 - June 25	Waiting	Link video and audio capture devices in AVCapture
June 25 - July 12	Waiting	Test, improve, simplify the classes implemented previously
July 12 - July 25	Completed	Improve Phonon-VLC - document code
July 12 - July 25	Not started	Improve Phonon-VLC - implement some lacking API
July 25 - August 16	Cancelled	Implement the capture API in other backends
July 25 - August 16	Not started	Improve Phonon-VLC or capture

Repositories

There is one repo for Phonon and one for the Phonon-VLC backend.

• http://gitorious.org/~skelet/phonon/soc-phonon-capture-api
• http://gitorious.org/~skelet/phonon/soc-phonon-vlc-capture-api

Status report

May 31

One week has passed and I still determining what I can use from Phonon::Experimental. There is capture-api code in there, and I need to see if it works or not and how it works. I can't start writing code around without knowing what is already done. Unfortunately the stupid school keeps throwing stuff at me (until June 2) and progress is slow. However, I want to get over this study and test phase by the start of next week. Now I'm putting together a couple of primitive apps to test things about the existing capture-api code.

June 6

During this week I've investigated more into what exists regarding capture in Phonon::Experimental. I've created another executable test in phonon/experimental/tests, that retrieves the lists of devices and their properties for audio output, capture, and video capture. I've implemented a couple of simple methods to do this in Phonon::Experimental::GlobalConfig. It appears that the devices and properties are working (they look the same as in the KDE config center). I need to find a way to test this with a more complicated device configuration, perhaps without PulseAudio. After that, I will have to implement getting the capture device list for Phonon-VLC.

Next week I plan to work with the MediaObject / MediaSource classes, to try to make them to use VLC to capture audio.

June 14

Unfortunately much of this week I was bogged down in trying to learn for an exam, but I also managed to implement things in Phonon-VLC. Not as much as I hoped, but now the lists of available v4l capture devices appear to be working (with one small device id issue). I should try to step up work this week, by disregarding (a bit) another exam. The small device id's issue will be fixed and I can start working on the fun part.

June 22

The primitive testing application using Phonon with Phonon-VLC shows up images from a webcam. But the device list is not easy to use and improvements have to be made. Additional improvements are needed in the other areas of the code. Some cleanup in Phonon::Experimental regarding VideoCaptureDevice should also be done.

I will start working on these various improvements this week.

June 28

The improvements have been made. V4L2 devices appear to be working properly with Phonon and Phonon-VLC. The device list now shows a normal device name, and a simple device description, not something like /dev/video0. There are #ifdef's disabling video capture or audio capture or both.

Audio capture is completely untested, but the code is the same as for video, so it should normally work. I need to look for libaudiodevices or something to grab a device list not related to V4L.

I will also start working on the Experimental::AVCapture class. At a first glance, there is not much to do there.

July 12

I have documented most of the Phonon-VLC code with Doxygen. I've gained good understanding about how the whole Phonon system works. Previously I was planning to work on AVCapture, but I found out that I had insufficient knowledge about Phonon and Phonon-VLC to do it. That is why I did the documentation stuff.

Now I need to decide on what to work next and start working on it. Options are:

• Test audio capture / audio data output, fix any issues
• Work on the AVCapture class and implement AVCaptureInterface in Phonon-VLC
• Try to fix / implement disabled / incomplete features for Phonon-VLC
• Fix the Phonon tests
• Implement more device categories, other than V4L, for other platforms

The initial timeline had to be modified to adapt to the current situation.

By tommorow I should have decided what to do next, and start working.