Expensive HDMI cables make no difference - the absolute proof

Comprehensive, proper testing answers the question once and for all

22 Jul 2015

Original HDMI article

This is the original article that we published, when we first tested HDMI cables. When we did our testing in 2012 we used a paused video for video capture, which some people said was cheating and didn't prove anything. While our new testing proves the same thing, we've left this article up, as it goes into more detail about the science of HDMI.

Whether or not to buy an expensive HDMI cable is one of the most contentious subjects on the internet. On pro the side of buying them are home cinema magazines and audiophile websites, which sing the praises of more expensive cables, heralding how they can do remarkable things, such as boosting flesh tones.

On the other side, you have the sceptics that say HDMI uses digital, so all cables are equal. In fact, James Randi has even extended his $1 million prize for anyone that can prove paranormal activity to include anyone that can prove that expensive HDMI cables make a difference.

We've always been sceptical about the benefits of more expensive cables, but the problem was always proving it in a scientific, repeatable way that removes any argument. Now, we've found the way to do that and can once and for all prove beyond a doubt if expensive cables make any difference.

The only way to answer the question of whether or not expensive cables really make a difference is to test image quality objectively. Subjective testing using a group of people is inherently flawed, as the average person will almost always pick a side when given two options to choose from – even if they are identical. HDMI cables are no exception to this rule.

Sony PlayStation 3

Using identical kit is no guarantee of identical results and subjective testing is inherently flawed

There are other problems, too. Using identical TVs, Blu-ray players and discs still can’t create a level playing field, because of minor differences between the colour casts of each set. Even identical devices have the potential to affect the results of a blind test. When we compared two PlayStation 3 consoles, we noticed a distinct difference in the colour reproduction of a Blu-ray disc - independent of the TV or HDMI cable we used. Unless you colour calibrate each device first, a blind test can’t produce reliable results.

To objectively test HDMI cables we first bought the Digital Foundry TrueHD capture card. This high-quality card allowed us to capture the HDMI output from a device in a RAW uncompressed format with no error correction, using 24-bit RGB. This eliminates any colour cast or error correction from the equation, as we're dealing with raw frames captured.

Next, we used a PC to output the Blu-ray version of Sintel, an open source film produced by the Blender Foundation. The choice of a PC may seem a little strange, and some people would argue that we should have used a Blu-ray player instead. However, we're not interested in the argument of which device produces the best-quality video, but rather if HDMI cables produce different results. To this end, the PC gave us one important advantage: we could screenshot a frame from the film on the PC and compare this to a frame captured on the TrueHD card. If the two were the same, the cable had made absolutely no difference; if the two differed, the cable was affecting quality.

To ensure that we managed to get the same frames at the PC and TrueHD ends, we paused the film and jumped to chapter two. It may seem strange to pause the film, as we weren't capturing motion, but as far as the PC's graphics card, cable and TrueHD card are concerned, pausing has no effect. The PC still has to output at the same rate, but rather than outputting 24 different frames per second for motion, it's outputting 24 identical frames per second.

As an HDMI-certified cable allows for a maximum of 1-bit error in a billion (roughly one pixel incorrect per second), we captured 24 frames of the paused Sintel (one second's worth) and saved them as uncompressed bitmap files that we could then compare to original frame we'd screen-grabbed on the PC. As the TrueHD doesn't apply any correction to the source, a single pixel's difference would be highlighted.

Sintel

Sintel is an open source Blu-ray film, which we used to test if HDMI cables make any difference at all

Compare and contrast

Once we had our captured frames, we needed an objective way of comparing the captured frames to the original frame we'd captured on the PC. For this we decided to use two methods. First, we used the ImageMagick Compare command. This takes two images and creates a third picture, highlighting any pixels that are different in red. Next, we created a MD5 hash of each file we captured. An MD5 hash is a unique fingerprint of file: if two files have the same MD5 hash, they're physically and scientifically identical, no argument.

By also creating an MD5 hash of the ImageMagick-created compare files, we could prove that they're all identical, too. In essence, then, we ensured that our captured frames were identical and that our comparison frames were identical.

HDMI tested: demonstration of a one-pixel error

ImageMagick's Compare command can detect a single pixel error and highlight it.

In both of our verification tests, a single shade of colour in a single pixel will be enough to trigger a difference. To test this level of accuracy, we edited a screenshot in Photoshop, changing the red colour value of a single pixel by one point before running our comparison tools. The red highlight in the comparison image and different MD5 hashes indicate a difference in the two images.

This method of testing not only determines the number of errors transmitted by a particular cable, it will also indicate if colours change at all when switching between cables - an idea frequently used to explain why certain cables are “better” than others in high-end home cinema reviews. Basically, if the cable does anything at all, the captured frame will be different from the frame screencaptured on the PC.

We looked at a range of cables, from the bargain basement cables that come bundled with Blu-ray players and TVs, to incredibly expensive high-end ones with huge feature lists covering the packaging. There was a vast difference in terms of price, ranging from £5 to an eye-watering £150, which could easily confuse anyone looking for a new cable.

HDMI cables table

The list of cables that we used for our testing

Absolutely no difference

After methodically testing each cable, our comprehensive results revealed the truth. We couldn’t find a single error in any of the frames of video we captured using our tests, regardless of the cable we used. Even allowing for a single error per billion bits, none of our captured frames contained as much as a single pixel out of order by a single colour shade.

HDMI tested: ImageMagick compare

This image is the ImageMagick Compare-created image. If there were any errors, the pixels would be highlighted in red

The MD5 hash values backed up our findings – the captured frames were identical to the original ones. As verification, the MD5 hash of the ImageMagick Compare files were also identical. For reference, here are the MD5 hash values generated.

HDMI hash comparison

The MD5 hash proves that the captured files are all identical

We had to force an error by reversing a uni-directional cable, which generated several incorrect pixels. Because our capture card doesn’t apply any error correction, these pixels simply failed to render.

Comparing our captured images to one another also proves that individual cables have no impact on colour reproduction. Because each capture was identical, with no differentiation between the colours of individual pixels, there can be no question that the final picture displayed on a TV is unaffected by your choice of HDMI cable. With no active circuitry inside them, HDMI cables don’t know what format the data it transmits will get re-encoded into when it reaches its destination, so there’s no way it could adjust colour values, improve skin tone or create better black response.

More expensive cables may have been engineered to eliminate timing jitter or reduce binary errors, but we didn't get any errors with our Lindy 10m cable, which costs less than the Atlas 1m cable. What's clear from our tests is that we got no errors on any of the cables, and more expensive cables did not improve image quality at all. Given our results, we'd definitely buy cheaper HDMI-certified cables.

Read more

In-Depth