Converting hex dumps to binary data

In my last article, I told the story of me repairing my display's EDID. One step was taking the hex dump of the EDID data and converting it to a binary file. Today, I want to show two ways of doing this.

If you need a reasonable portable way of doing it, without having to install additional software, I recommend using Perl. Despite internet rumors, Perl is not dead at all. On the contrary, it is still a very capable tool that has manipulating text as a strong point. It is also part of the base of most Linux or BSD systems, so you can almost always expect it to just be there. I even have a Perl binary on my mobile phone, though it is an rather old version. In the odd case that you have no Perl on your system, it is most likely easy to install (on Windows, I use scoop.sh for this).

#!/usr/bin/perl
use v5.32;
use warnings;

while (<<>>) {
    while (/\s*([0-9a-f]{2})/ig) {
	print chr(hex($1));
    }
}

This script, as you can see, is quite short. It starts with a prologue, mixing the usual shebang together with two pragma imports. These do not do much for me in a finished script, but are invaluable during development - I am bound to screw something up sooner or later, like everyone else.

The while loop uses a short notation that reads either from STDIN or from all files passed as program arguments (exactly like cat(1) and many other Unix binaries work). The inner loop iterates over matches of two consecutive hexadecimal characters, ignoring case. For each match, we print the corresponding character. That is all!

To use this script, pipe the hex dump into it and pipe the script's STDOUT into a file. You should now have the binary file you need. If you pipe your binary file into a tool like xxd(1), you should get an output matching the original input (formatting not withstanding).

Even easier than this script would be just using xxd itself. While its primary function is creating hex dumps, it also has the ability to work in reverse. On the other hand, you are more likely not to have xxd on a computer than having no Perl.

You need the following parameters:

xxd -r -p

These parameters activate reverse mode and also make xxd work with plain hex dumps (i.e. just the pure data, no offset or ASCII columns). Pay attention that you do not try to combine the two switches: xxd does only work with them separated! To quote the man page:

Note that a "lazy" parser is used which does not check for more than
the first option letter, unless the option is followed by a parameter

When called like this, it works just like our Perl script: It reads hex dumps from STDIN and writes binary data to STDOUT. You can therefore use it just like the script before, pipe the EDID hex dump into it and pipe the STDOUT into a file.

As promised in the previous article, converting hex dumps to raw binary files is not hard and can be done with existing tools or tiny self-written scripts. Knowing how to do this is a useful skill that has many uses - even for those that never have problems with their EDID.