Rate‐Distortion Theory

Abstract Rate‐distortion theory is the branch of information theory that treats compressing the data produced by an information source down to a specified encoding rate that is strictly less than the source's entropy. This necessarily entails some lossiness, or distortion, between the original source data and the best approximation thereto that can be produced on the basis of the encoder's output bits. Rate‐distortion theory was introduced in the seminal works written in 1948 and 1959 by C. E. Shannon , the founder of information theory. We describe Shannon's contribution and then trace its subsequent development worldwide. Heavier than usual emphasis is placed on the concept of “matching” a channel to a…