Fast coding unit decision and mode selection for intra‐frame coding in high‐efficiency video coding

High-efficiency video coding (HEVC) is a new video coding compression standard. As the successor to H.264/AVC, it provides better performance and supports higher resolution. However, the encoding complexity increases drastically. One of the major reasons is that the coding unit (CU) in HEVC is multi-sized and adjustable rather than fixed as in H.264. In addition, the number of prediction modes used in intra-frame coding is expanded from 9 to 35. The authors analysed the statistical correlations of CU depth to the deviation of pixels in the largest coding unit (LCU) and rate-distortion cost (RDcost). Accordingly, a fast CU decision method is proposed, which contains two steps: first, the depth to begin searching is determined according to the deviation of the LCU and then splitting the current CU further is decided according to RDcost. For intra-prediction, we also propose a fast mode selection method to reduce complexity. This method can quickly determine the modes for rate-distortion optimisation when the combination of most probable modes reveals the pattern direction. Software simulations show that the proposed methods reduce encoding time by more than 50% with an average of 1.4% increase of BD-rate compared to reference software HM12.0.