H.264 intra mode decision for reducing complexity using directional masks and neighboring modes

Abstract

The computational complexity for the RDO-based intra prediction in H.264 is very high due to many coding modes. There are 9 modes for 4×4 luma blocks, 4 modes for 16×16 luma blocks and 4 modes for 8×8 chroma blocks, respectively, so that all possible combinations of coding modes per MB are 592. We propose an intra mode decision algorithm for reducing complexity using directional masks and neighboring blocks' modes. The proposed algorithm has a very simple structure and reduces the number of mode combinations into 132 at the most. Simulation results show that the proposed algorithm reduces the encoding time up to 70% with negligible PSNR loss and bit-rate increase compared with exhaustive testing of H.264.