Categories
arrays c# hex

How do you convert a byte array to a hexadecimal string, and vice versa?

1557

How can you convert a byte array to a hexadecimal string, and vice versa?

1

  • 10

    The accepted answer below appear to allocate a horrible amount of strings in the string to bytes conversion. I’m wondering how this impacts performance

    Mar 6, 2009 at 16:41

1609

You can use Convert.ToHexString starting with .NET 5.
There’s also a method for the reverse operation: Convert.FromHexString.


For older versions of .NET you can either use:

public static string ByteArrayToString(byte[] ba)
{
  StringBuilder hex = new StringBuilder(ba.Length * 2);
  foreach (byte b in ba)
    hex.AppendFormat("{0:x2}", b);
  return hex.ToString();
}

or:

public static string ByteArrayToString(byte[] ba)
{
  return BitConverter.ToString(ba).Replace("-","");
}

There are even more variants of doing it, for example here.

The reverse conversion would go like this:

public static byte[] StringToByteArray(String hex)
{
  int NumberChars = hex.Length;
  byte[] bytes = new byte[NumberChars / 2];
  for (int i = 0; i < NumberChars; i += 2)
    bytes[i / 2] = Convert.ToByte(hex.Substring(i, 2), 16);
  return bytes;
}

Using Substring is the best option in combination with Convert.ToByte. See this answer for more information. If you need better performance, you must avoid Convert.ToByte before you can drop SubString.

19

  • 30

    You’re using SubString. Doesn’t this loop allocate a horrible amount of string objects?

    Mar 6, 2009 at 16:36

  • 33

    Honestly – until it tears down performance dramatically, I would tend to ignore this and trust the Runtime and the GC to take care of it.

    – Tomalak

    Mar 6, 2009 at 17:11

  • 94

    Because a byte is two nibbles, any hex string that validly represents a byte array must have an even character count. A 0 should not be added anywhere – to add one would be making an assumption about invalid data that is potentially dangerous. If anything, the StringToByteArray method should throw a FormatException if the hex string contains an odd number of characters.

    Mar 9, 2010 at 19:01

  • 8

    @00jt You must make an assumption that F == 0F. Either it is the same as 0F, or the input was clipped and F is actually the start of something you have not received. It is up to your context to make those assumptions, but I believe a general purpose function should reject odd characters as invalid instead of making that assumption for the calling code.

    Jan 28, 2013 at 15:35

  • 12

    @DavidBoike The question had NOTHING to do with “how to handle possibly clipped stream values” Its talking about a String. String myValue = 10.ToString(“X”); myValue is “A” not “0A”. Now go read that string back into bytes, oops you broke it.

    – 00jt

    Jan 30, 2013 at 19:25


531

Performance Analysis

Note: new leader as of 2015-08-20.

I ran each of the various conversion methods through some crude Stopwatch performance testing, a run with a random sentence (n=61, 1000 iterations) and a run with a Project Gutenburg text (n=1,238,957, 150 iterations). Here are the results, roughly from fastest to slowest. All measurements are in ticks (10,000 ticks = 1 ms) and all relative notes are compared to the [slowest] StringBuilder implementation. For the code used, see below or the test framework repo where I now maintain the code for running this.

Disclaimer

WARNING: Do not rely on these stats for anything concrete; they are simply a sample run of sample data. If you really need top-notch performance, please test these methods in an environment representative of your production needs with data representative of what you will use.

Results

Lookup tables have taken the lead over byte manipulation. Basically, there is some form of precomputing what any given nibble or byte will be in hex. Then, as you rip through the data, you simply look up the next portion to see what hex string it would be. That value is then added to the resulting string output in some fashion. For a long time byte manipulation, potentially harder to read by some developers, was the top-performing approach.

Your best bet is still going to be finding some representative data and trying it out in a production-like environment. If you have different memory constraints, you may prefer a method with fewer allocations to one that would be faster but consume more memory.

Testing Code

Feel free to play with the testing code I used. A version is included here but feel free to clone the repo and add your own methods. Please submit a pull request if you find anything interesting or want to help improve the testing framework it uses.

  1. Add the new static method (Func<byte[], string>) to /Tests/ConvertByteArrayToHexString/Test.cs.
  2. Add that method’s name to the TestCandidates return value in that same class.
  3. Make sure you are running the input version you want, sentence or text, by toggling the comments in GenerateTestInput in that same class.
  4. Hit F5 and wait for the output (an HTML dump is also generated in the /bin folder).
static string ByteArrayToHexStringViaStringJoinArrayConvertAll(byte[] bytes) {
    return string.Join(string.Empty, Array.ConvertAll(bytes, b => b.ToString("X2")));
}
static string ByteArrayToHexStringViaStringConcatArrayConvertAll(byte[] bytes) {
    return string.Concat(Array.ConvertAll(bytes, b => b.ToString("X2")));
}
static string ByteArrayToHexStringViaBitConverter(byte[] bytes) {
    string hex = BitConverter.ToString(bytes);
    return hex.Replace("-", "");
}
static string ByteArrayToHexStringViaStringBuilderAggregateByteToString(byte[] bytes) {
    return bytes.Aggregate(new StringBuilder(bytes.Length * 2), (sb, b) => sb.Append(b.ToString("X2"))).ToString();
}
static string ByteArrayToHexStringViaStringBuilderForEachByteToString(byte[] bytes) {
    StringBuilder hex = new StringBuilder(bytes.Length * 2);
    foreach (byte b in bytes)
        hex.Append(b.ToString("X2"));
    return hex.ToString();
}
static string ByteArrayToHexStringViaStringBuilderAggregateAppendFormat(byte[] bytes) {
    return bytes.Aggregate(new StringBuilder(bytes.Length * 2), (sb, b) => sb.AppendFormat("{0:X2}", b)).ToString();
}
static string ByteArrayToHexStringViaStringBuilderForEachAppendFormat(byte[] bytes) {
    StringBuilder hex = new StringBuilder(bytes.Length * 2);
    foreach (byte b in bytes)
        hex.AppendFormat("{0:X2}", b);
    return hex.ToString();
}
static string ByteArrayToHexViaByteManipulation(byte[] bytes) {
    char[] c = new char[bytes.Length * 2];
    byte b;
    for (int i = 0; i < bytes.Length; i++) {
        b = ((byte)(bytes[i] >> 4));
        c[i * 2] = (char)(b > 9 ? b + 0x37 : b + 0x30);
        b = ((byte)(bytes[i] & 0xF));
        c[i * 2 + 1] = (char)(b > 9 ? b + 0x37 : b + 0x30);
    }
    return new string(c);
}
static string ByteArrayToHexViaByteManipulation2(byte[] bytes) {
    char[] c = new char[bytes.Length * 2];
    int b;
    for (int i = 0; i < bytes.Length; i++) {
        b = bytes[i] >> 4;
        c[i * 2] = (char)(55 + b + (((b - 10) >> 31) & -7));
        b = bytes[i] & 0xF;
        c[i * 2 + 1] = (char)(55 + b + (((b - 10) >> 31) & -7));
    }
    return new string(c);
}
static string ByteArrayToHexViaSoapHexBinary(byte[] bytes) {
    SoapHexBinary soapHexBinary = new SoapHexBinary(bytes);
    return soapHexBinary.ToString();
}
static string ByteArrayToHexViaLookupAndShift(byte[] bytes) {
    StringBuilder result = new StringBuilder(bytes.Length * 2);
    string hexAlphabet = "0123456789ABCDEF";
    foreach (byte b in bytes) {
        result.Append(hexAlphabet[(int)(b >> 4)]);
        result.Append(hexAlphabet[(int)(b & 0xF)]);
    }
    return result.ToString();
}
static readonly uint* _lookup32UnsafeP = (uint*)GCHandle.Alloc(_Lookup32, GCHandleType.Pinned).AddrOfPinnedObject();
static string ByteArrayToHexViaLookup32UnsafeDirect(byte[] bytes) {
    var lookupP = _lookup32UnsafeP;
    var result = new string((char)0, bytes.Length * 2);
    fixed (byte* bytesP = bytes)
    fixed (char* resultP = result) {
        uint* resultP2 = (uint*)resultP;
        for (int i = 0; i < bytes.Length; i++) {
            resultP2[i] = lookupP[bytesP[i]];
        }
    }
    return result;
}
static uint[] _Lookup32 = Enumerable.Range(0, 255).Select(i => {
    string s = i.ToString("X2");
    return ((uint)s[0]) + ((uint)s[1] << 16);
}).ToArray();
static string ByteArrayToHexViaLookupPerByte(byte[] bytes) {
    var result = new char[bytes.Length * 2];
    for (int i = 0; i < bytes.Length; i++)
    {
        var val = _Lookup32[bytes[i]];
        result[2*i] = (char)val;
        result[2*i + 1] = (char) (val >> 16);
    }
    return new string(result);
}
static string ByteArrayToHexViaLookup(byte[] bytes) {
    string[] hexStringTable = new string[] {
        "00", "01", "02", "03", "04", "05", "06", "07", "08", "09", "0A", "0B", "0C", "0D", "0E", "0F",
        "10", "11", "12", "13", "14", "15", "16", "17", "18", "19", "1A", "1B", "1C", "1D", "1E", "1F",
        "20", "21", "22", "23", "24", "25", "26", "27", "28", "29", "2A", "2B", "2C", "2D", "2E", "2F",
        "30", "31", "32", "33", "34", "35", "36", "37", "38", "39", "3A", "3B", "3C", "3D", "3E", "3F",
        "40", "41", "42", "43", "44", "45", "46", "47", "48", "49", "4A", "4B", "4C", "4D", "4E", "4F",
        "50", "51", "52", "53", "54", "55", "56", "57", "58", "59", "5A", "5B", "5C", "5D", "5E", "5F",
        "60", "61", "62", "63", "64", "65", "66", "67", "68", "69", "6A", "6B", "6C", "6D", "6E", "6F",
        "70", "71", "72", "73", "74", "75", "76", "77", "78", "79", "7A", "7B", "7C", "7D", "7E", "7F",
        "80", "81", "82", "83", "84", "85", "86", "87", "88", "89", "8A", "8B", "8C", "8D", "8E", "8F",
        "90", "91", "92", "93", "94", "95", "96", "97", "98", "99", "9A", "9B", "9C", "9D", "9E", "9F",
        "A0", "A1", "A2", "A3", "A4", "A5", "A6", "A7", "A8", "A9", "AA", "AB", "AC", "AD", "AE", "AF",
        "B0", "B1", "B2", "B3", "B4", "B5", "B6", "B7", "B8", "B9", "BA", "BB", "BC", "BD", "BE", "BF",
        "C0", "C1", "C2", "C3", "C4", "C5", "C6", "C7", "C8", "C9", "CA", "CB", "CC", "CD", "CE", "CF",
        "D0", "D1", "D2", "D3", "D4", "D5", "D6", "D7", "D8", "D9", "DA", "DB", "DC", "DD", "DE", "DF",
        "E0", "E1", "E2", "E3", "E4", "E5", "E6", "E7", "E8", "E9", "EA", "EB", "EC", "ED", "EE", "EF",
        "F0", "F1", "F2", "F3", "F4", "F5", "F6", "F7", "F8", "F9", "FA", "FB", "FC", "FD", "FE", "FF",
    };
    StringBuilder result = new StringBuilder(bytes.Length * 2);
    foreach (byte b in bytes) {
        result.Append(hexStringTable[b]);
    }
    return result.ToString();
}

Update (2010-01-13)

Added Waleed’s answer to analysis. Quite fast.

Update (2011-10-05)

Added string.Concat Array.ConvertAll variant for completeness (requires .NET 4.0). On par with string.Join version.

Update (2012-02-05)

Test repo includes more variants such as StringBuilder.Append(b.ToString("X2")). None upset the results any. foreach is faster than {IEnumerable}.Aggregate, for instance, but BitConverter still wins.

Update (2012-04-03)

Added Mykroft’s SoapHexBinary answer to analysis, which took over third place.

Update (2013-01-15)

Added CodesInChaos’s byte manipulation answer, which took over first place (by a large margin on large blocks of text).

Update (2013-05-23)

Added Nathan Moinvaziri’s lookup answer and the variant from Brian Lambert’s blog. Both rather fast, but not taking the lead on the test machine I used (AMD Phenom 9750).

Update (2014-07-31)

Added @CodesInChaos’s new byte-based lookup answer. It appears to have taken the lead on both the sentence tests and the full-text tests.

Update (2015-08-20)

Added airbreather’s optimizations and unsafe variant to this answer’s repo. If you want to play in the unsafe game, you can get some huge performance gains over any of the prior top winners on both short strings and large texts.

21

  • 7

    Despite making the code available for you to do the very thing you requested on your own, I updated the testing code to include Waleed answer. All grumpiness aside, it is much faster.

    – patridge

    Jan 13, 2010 at 16:29

  • 2

    @CodesInChaos Done. And it won in my tests by quite a bit as well. I don’t pretend to fully understand either of the top methods yet, but they are easily hidden from direct interaction.

    – patridge

    Jan 15, 2013 at 18:01

  • 6

    This answer has no intention of answering the question of what is “natural” or commonplace. The goal is to give people some basic performance benchmarks since, when you need to do these conversion, you tend to do them a lot. If someone needs raw speed, they just run the benchmarks with some appropriate test data in their desired computing environment. Then, tuck that method away into an extension method where you never look its implementation again (e.g., bytes.ToHexStringAtLudicrousSpeed()).

    – patridge

    Apr 8, 2013 at 20:37

  • 2

    Just produced a high performance lookup table based implementation. Its safe variant is about 30% faster than the current leader on my CPU. The unsafe variants are even faster. stackoverflow.com/a/24343727/445517

    Jun 21, 2014 at 17:12

  • 2

    @Goodies I’ve discovered that the simple Convert.ToBase64String() is VERY fast (faster than Lookup by byte (via CodesInChaos) ) in my testing – so if anyone doesn’t care about the output being hexadecimal, that’s a quick one-line replacement.

    Aug 10, 2018 at 9:44

260

There’s a class called SoapHexBinary that does exactly what you want.

using System.Runtime.Remoting.Metadata.W3cXsd2001;

public static byte[] GetStringToBytes(string value)
{
    SoapHexBinary shb = SoapHexBinary.Parse(value);
    return shb.Value;
}

public static string GetBytesToString(byte[] value)
{
    SoapHexBinary shb = new SoapHexBinary(value);
    return shb.ToString();
}

8

  • 38

    SoapHexBinary is available from .NET 1.0 and is in mscorlib. Despite it’s funny namespace, it does exactly what the question asked.

    Jun 28, 2011 at 6:48

  • 4

    Great find! Note that you will need to pad odd strings with a leading 0 for GetStringToBytes, like the other solution.

    Oct 31, 2011 at 17:10

  • Have you seen the implementation thought? The accepted answer has a better one IMHO.

    – mfloryan

    Jan 26, 2012 at 13:42

  • 6

    Interesting to see the Mono implementation here: github.com/mono/mono/blob/master/mcs/class/corlib/…

    – Jeremy

    Apr 29, 2012 at 4:40

  • 10

    SoapHexBinary is not supported in .NET Core/ .NET Standard…

    – juFo

    Mar 11, 2020 at 9:12