Expensive HDMI cables make no difference - the absolute proof
Comprehensive, proper testing answers the question once and for all
To objectively test HDMI cables we first bought the Digital Foundry TrueHD capture card. This high-quality card allowed us to capture the HDMI output from a device in a RAW uncompressed format with no error correction, using 24-bit RGB. This eliminates any colour cast or error correction from the equation, as we're dealing with raw frames captured.
Next, we used a PC to output the Blu-ray version of Sintel, an open source film produced by the Blender Foundation. The choice of a PC may seem a little strange, and some people would argue that we should have used a Blu-ray player instead. However, we're not interested in the argument of which device produces the best-quality video, but rather if HDMI cables produce different results. To this end, the PC gave us one important advantage: we could screenshot a frame from the film on the PC and compare this to a frame captured on the TrueHD card. If the two were the same, the cable had made absolutely no difference; if the two differed, the cable was affecting quality.
To ensure that we managed to get the same frames at the PC and TrueHD ends, we paused the film and jumped to chapter two. It may seem strange to pause the film, as we weren't capturing motion, but as far as the PC's graphics card, cable and TrueHD card are concerned, pausing has no effect. The PC still has to output at the same rate, but rather than outputting 24 different frames per second for motion, it's outputting 24 identical frames per second.
As an HDMI-certified cable allows for a maximum of 1-bit error in a billion (roughly one pixel incorrect per second), we captured 24 frames of the paused Sintel (one second's worth) and saved them as uncompressed bitmap files that we could then compare to original frame we'd screen-grabbed on the PC. As the TrueHD doesn't apply any correction to the source, a single pixel's difference would be highlighted.
Sintel is an open source Blu-ray film, which we used to test if HDMI cables make any difference at all
Compare and contrast
Once we had our captured frames, we needed an objective way of comparing the captured frames to the original frame we'd captured on the PC. For this we decided to use two methods. First, we used the ImageMagick Compare command. This takes two images and creates a third picture, highlighting any pixels that are different in red. Next, we created a MD5 hash of each file we captured. An MD5 hash is a unique fingerprint of file: if two files have the same MD5 hash, they're physically and scientifically identical, no argument.
By also creating an MD5 hash of the ImageMagick-created compare files, we could prove that they're all identical, too. In essence, then, we ensured that our captured frames were identical and that our comparison frames were identical.
ImageMagick's Compare command can detect a single pixel error and highlight it.
In both of our verification tests, a single shade of colour in a single pixel will be enough to trigger a difference. To test this level of accuracy, we edited a screenshot in Photoshop, changing the red colour value of a single pixel by one point before running our comparison tools. The red highlight in the comparison image and different MD5 hashes indicate a difference in the two images.