X264 TODO

This page contains an incomplete list of things available in x264 for you to do. It's organized into sections covering various parts of x264.

Sequential elimination (SEA), used for exhaustive search, might be more generally applicable to algorithms like UMH, by letting us skip a lot of SADs. The downside is we won't be able to use SAD_X4 anymore.
(T)ESA is currently wrong for motion searches done on weightp duplicates. This effect is miniscule, but it still should be fixed.
Hierarchical motion estimation might be a useful way to catch very long motion vectors without the cost of UMH or ESA. It might also help regularize motion.
Somehow take into account the effect of motion vector decision on future blocks.
- Hierarchical motion estimation
- Approximations from lookahead MVs
- Iterative ME (as per Snow)
- Trellis motion estimation

Make the early terminations smarter. Currently they're just hacks -- some statistical analysis might be useful.
SAD (subme 1) i8x8 vs i4x4 decision is a bit bad. Can it be improved without significant speed loss?

Can we find more ways to skip more motion searches in multiref?
On extremely fast encoding settings, fast skip is actually kind of slow. But anything dumber (e.g. SAD) is completely useless. Is there some better balance that can be achieved here?
Chroma-aware mode decision for B-frames?
See the TODOs for deblock-aware RD in common/deblock.c.

Psy-RD is a hack. It works, but it's a hack. If you apply QNS with Psy-RD as the metric, it goes way overboard and gives terrible results. This means that Psy-RD only works because normal mode decision is limited in the way it can modify the image to better suit the metric. Is there a way to make it better?
Should RD be linear at all? Perhaps we should weight more heavily against low quality blocks and also try to ignore minuscule distortion that viewers can't see.
Psy-trellis (and maybe psy-RD?) are too strong at very high QPs.
Psy-trellis should be merged with Psy-RD. There are patches for this, but they probably won't be committed until psy-RD itself is fixed.
RD should take into account local variance.
Lambda should be varied on a per-DCT-block basis instead of a per-macroblock basis.
Lambda should be picked independent of quantizer (i.e. with greater precision).
Classic problem: a block is mostly high complexity but has a small area of low complexity. How do we judge whether that area is important? Good example: sharp text on background with film grain; grain gets blurred out because of the text.
- If we think it's important all the time, we ruin the quality of many clips that rely on raising complexity on edges (Touhou).

Lookahead should be multithreaded, either by splitting the frame (sliced threads) or running multiple frame analysis calls at once.
Temporal MV predictors in lookahead? There's a patch for these somewhere, but they biased heavily in favor of B-frames, likely by improving the motion search.
Should lookahead use variable lambda based on quantizer (esp. due to adaptive quant)? If so, should it take into account estimated ratecontrol quantizer, too? If so, how?

CAVLC "trellis" is a hack. It works, but it's a hack. Make it better. See the TODOs in encoder/rdo.c.
There's room for something between trellis and deadzone in terms of complexity. libvpx has a good example -- it biases towards zero-runs in its "medium speed" quantizer. This can't be SIMD'd easily, but is still vastly faster than trellis. A nonlinear quantizer (be more likely to round up larger coefficients) might also be useful.
Floyd-Steinberg for quantization? Try pushing quantization error to nearby DCT coefficients. Should this go from high to low or low to high?
Energy-preserving quantizer -- maintain L1 (or maybe L2? I'm not sure) energy. Should we maintain it in the spatial domain (post-iDCT) or residual domain? Probably the former.

Finish adaptive MBAFF. Talk to horlicks about this one.
Make slice-max-mbs/slice-max-size work with interlacing. Pretty much requires a good portion of adaptive MBAFF.

Make weightp work with interlacing. Preferably abuse reference duplication to make it useful for MBAFF.
Make weightp work with chroma. Talk to DylanZA about getting his current patch for this one.
Finish K-means decision for weightp. Talk to DylanZA about getting his current patch for this one.
Add explicit weighting for B-frames, too. This helps in nonlinear fades, among other cases.

VBV might be able to utilize the ability to re-encode a row of the frame for improved accuracy.
- Maybe re-encode everything in case of an underflow that row-reencoding can't fix? This might be better than underflowing.
Current per-frame VBV is a hack. It only adapts per row and is O(N^2), where N is the number of rows. An O(N) solution would be able to react more often and thus be more accurate.
Make the frame size predictors better. They currently are kind of crappy.
Ratecontrol code as a whole is a bit of a mess. It could be improved. There's a lot of cruft left over that is probably not needed now, like qcomp.
2-pass VBV is actually more likely to underflow than 1-pass because it doesn't adapt as aggressively and trusts first pass data a lot. This trust is often misplaced if the first pass was a fast one. This should be improved.

Finish audio support. Talk to Kovensky about this one.
Add more filters.
- Deinterlacers (YADIF).
- Denoisers (HQDN3D?).
- IVTC, decomb?

Contents