SoC x264 2009

x264 has loads of possibilities for SoC 2009 projects.

This is part of the VideoLAN candidature for Google Summer of Code 2009.

• Mentor (and author of this page): Dark Shikari
• Possible backup mentor: pengvado

Introduction to x264

x264 is probably the most efficient, compression-wise, open source video encoder there is. It is quite competitive with commercial encoders, outclassing a large number of them.

While not actually part of VLC or ffmpeg (it has its own codebase), it is a major library used by both, licensed under the GPL, in addition to being a standalone encoder. As the only major open-source H.264 encoder, x264 has a near-complete monopoly on H.264 encoding in the consumer world, along with being used by many major corporations, including Facebook and Google. Some companies, such as Avail Media, have in the past offered bounties on improvements to the encoder.

x264 project ideas

This is not at all an exhaustive list; this is just a few I thought up with. I'm willing to mentor any reasonable project on x264 to the best of my ability. I'm being pretty conservative here, so I'm picking projects that are probably not at all too ambitious for a good student. If anything, I might be underestimating the amount of work that can be done, so feel free to propose something else if you're feeling creative.

-- Dark Shikari

Size key

Depends heavily on the skill and willingness to work of the student. An extremely dedicated and talented student might be able to implement MBAFF in a summer, but it is certainly not fair to expect such a thing from most students.

• Very Large: Probably too large to completed in one summer.
• Large: Probably the right size for a full-summer project.
• Medium: Probably too small. Could be combined with another project, of course.
• Small: A small project, but definitely useful, and could be part of a larger project.

Skills needed

These are required for all listed projects and probably anything not listed, too.

• Basic C programming.
• Basic understanding of video encoding, or at least willingness to do the appropriate reading up on the topic before the summer begins.
• Confidence in the ability to learn the basics of following and similar topics (though not all projects will require such information):

• Discrete cosine transform and similar frequency transforms
• Motion estimation and compensation
• Quantization and entropy encoding

A PDF with a chapter that can serve as a primer to video compression can be found here. It also has some more specific chapters on MPEG-4 Part 2 and Part 10.

Projects

Fast inter refinement

Size: Medium to large.

Description: Improve heuristics and decision-making for inter refinement to improve efficiency on non-insane encoding settings. This would involve various early termination heuristics along with methods of deciding which partition modes need to be searched while performing minimal actual searching on those partition modes. This would be similar to, but a vastly more in-depth analysis of what was proposed in the "Fast-Ref-Search" patch.

Difficulty: Medium

Fast intra refinement

Size: Small to medium

Description: Similar to above, but covering intra modes instead. Would probably involve considerable statistical analysis of intra mode data, along with creative solutions for improved RDO refinement. We already have some ideas on this one, but haven't implemented any of them.

Difficulty: Medium

RDO B-frame decision

Size: Medium to large

Description: x264's biggest weakness is its B-frame decision algorithm, which can often be extremely subtopimal, with OPSNR losses as high as 1db in some cases. Improving this would drastically increase the effectiveness of the encoder.

Difficulty: Medium-high

4:4:4 and 4:2:2 color support

Size: Medium to large

Description: x264 doesn't support any color spaces other than YV12. This would solve this problem by adding the ability to use YUY2 and YV24 color spaces. This might be useful for some animation footage, or graphics; plus its been requested often.

Difficulty: Medium

Film grain modeling

Size: Medium to large

An integral part of the standard... but supported by basically nothing despite its potential usefulness. This would involve implementing FGM in both x264 and some sort of decoder, preferably ffmpeg. Some work has already been done in this category, so you won't be starting from nothing.

Difficulty: Medium

Other possible projects

Anything here (and not here) can potentially be picked from at the request of a student.

• Assembly optimizations of any sort

• Extra skills: Assembly coding
• Difficulty: Medium
• Examples:

• Port cacheline split to the motion compensation code for increased speed (this could further be used to improve ffh264's decoding).
• Assembly-optimize some things that haven't been already.
• Port some MMX assembly to SSE where it seems useful.
• Play around with potential SSE4 optimizations.

• Psychovisual optimizations for mode decision and quantization (e.g. QNS)

• Could also include work on adaptive quantization, a huge benefit for x264 quality-wise.
• Extra skills: Creativity and perhaps some understanding of DCT/Fourier math.
• Difficulty: Medium-high
• Examples:

• SSIM-QNS optimization?
• Adaptive deadzone?
• Adaptive lambda?

• Implementing MBAFF or PicAFF (potentially too difficult for a SoC project, however)

• Difficulty: Very high

• Fast RD optimization using heuristics

• Extra skills: Reading lots of IEEE papers
• Difficulty: Medium

• Motion search improvements

• Difficulty: Medium

• More RDO

• Difficulty: Medium

• Anything else reasonable, honestly. There's all sorts of ideas floating around.

Qualification tasks

This year, the qualification tasks will be actual, useful features for x264--small projects that should take a few weeks. We're willing to give all the technical help you need, but of course we won't write the code for you. "Passing" a qualification task is at the mentor's discretion. Note these are designed to be difficult and help lead you into your main project. If you can't do the qualification task for the project, you surely cannot do the project either!

Optimization

If you're interested in the optimization task, the qualification task is to speed up x264 on x86 (32 or 64-bit) by 2% on "normal settings" without changing the output. This is much harder than it sounds.

ARM Support

If you're interested in the ARM task, your qualification task will be to:

• Fix the unaligned access bug in the bitstream writer.
• Write NEON SIMD assembly for at least a few of the simpler significant DSP functions (SAD, SATD, etc).

4:4:4 and 4:2:2 support

If you're interested in working on this project, your task is to produce an x264-encoded bitstream in 4:4:4 or 4:2:2 format. It does not actually have to be remotely viewable (that is, you don't have to implement any of the code to handle motion compensation, deblocking, or anything else involving 4:4:4/4:2:2 chroma data), but the bitstream has to be written correctly (correct syntax elements). The patch you write for this will be the starting point for your main project.

Contact info

If you are interested, drop by #videolan, #x264, or #x264dev on Freenode.

You should also contact the admin jb.