Categories
capturing-group regex regex-group

What is a non-capturing group in regular expressions?

2236

How are non-capturing groups, i.e., (?:), used in regular expressions and what are they good for?

1

2812

Let me try to explain this with an example.

Consider the following text:

http://stackoverflow.com/
https://stackoverflow.com/questions/tagged/regex

Now, if I apply the regex below over it…

(https?|ftp)://([^/\r\n]+)(/[^\r\n]*)?

… I would get the following result:

Match "http://stackoverflow.com/"
     Group 1: "http"
     Group 2: "stackoverflow.com"
     Group 3: "https://stackoverflow.com/"

Match "https://stackoverflow.com/questions/tagged/regex"
     Group 1: "https"
     Group 2: "stackoverflow.com"
     Group 3: "https://stackoverflow.com/questions/tagged/regex"

But I don’t care about the protocol — I just want the host and path of the URL. So, I change the regex to include the non-capturing group (?:).

(?:https?|ftp)://([^/\r\n]+)(/[^\r\n]*)?

Now, my result looks like this:

Match "http://stackoverflow.com/"
     Group 1: "stackoverflow.com"
     Group 2: "https://stackoverflow.com/"

Match "https://stackoverflow.com/questions/tagged/regex"
     Group 1: "stackoverflow.com"
     Group 2: "https://stackoverflow.com/questions/tagged/regex"

See? The first group has not been captured. The parser uses it to match the text, but ignores it later, in the final result.


EDIT:

As requested, let me try to explain groups too.

Well, groups serve many purposes. They can help you to extract exact information from a bigger match (which can also be named), they let you rematch a previous matched group, and can be used for substitutions. Let’s try some examples, shall we?

Imagine you have some kind of XML or HTML (be aware that regex may not be the best tool for the job, but it is nice as an example). You want to parse the tags, so you could do something like this (I have added spaces to make it easier to understand):

   \<(?<TAG>.+?)\> [^<]*? \</\k<TAG>\>
or
   \<(.+?)\> [^<]*? \</\1\>

The first regex has a named group (TAG), while the second one uses a common group. Both regexes do the same thing: they use the value from the first group (the name of the tag) to match the closing tag. The difference is that the first one uses the name to match the value, and the second one uses the group index (which starts at 1).

Let’s try some substitutions now. Consider the following text:

Lorem ipsum dolor sit amet consectetuer feugiat fames malesuada pretium egestas.

Now, let’s use this dumb regex over it:

\b(\S)(\S)(\S)(\S*)\b

This regex matches words with at least 3 characters, and uses groups to separate the first three letters. The result is this:

Match "Lorem"
     Group 1: "L"
     Group 2: "o"
     Group 3: "r"
     Group 4: "em"
Match "ipsum"
     Group 1: "i"
     Group 2: "p"
     Group 3: "s"
     Group 4: "um"
...

Match "consectetuer"
     Group 1: "c"
     Group 2: "o"
     Group 3: "n"
     Group 4: "sectetuer"
...

So, if we apply the substitution string:

$1_$3$2_$4

… over it, we are trying to use the first group, add an underscore, use the third group, then the second group, add another underscore, and then the fourth group. The resulting string would be like the one below.

L_ro_em i_sp_um d_lo_or s_ti_ a_em_t c_no_sectetuer f_ue_giat f_ma_es m_la_esuada p_er_tium e_eg_stas.

You can use named groups for substitutions too, using ${name}.

To play around with regexes, I recommend http://regex101.com/, which offers a good amount of details on how the regex works; it also offers a few regex engines to choose from.

10

  • 3

    @ajsie: Traditional (capturing) groups are most useful if you’re performing a replacement operation on the results. Here’s an example where I’m grabbing comma-separated last & first names and then reversing their order (thanks to named groups)… regexhero.net/tester/?id=16892996-64d4-4f10-860a-24f28dad7e30

    Aug 19, 2010 at 15:43

  • 5

    Might also point out that non-capturing groups are uniquely useful when using regex as split delimiters: “Alice and Bob”-split”\s+(?:and|or)\s+”

    – Yevgeniy

    May 7, 2014 at 18:06

  • 10

    It would be interesting to have the difference between non-capturing groups (?:), and lookahead and lookbehind assertions (?=, ?!) explained. I just started learning about regular expressions, but from what I understand, non-capturing groups are used for matching and “return” what they match, but that “return value” is not “stored” for back-referencing. Lookahead and lookbehind assertions on the other hand are not only not “stored”, they are also not part of a match, they just assert that something would match, but their “match” value is ignored, if I’m not mistaken… (Am I roughly right?)

    – Christian

    May 11, 2014 at 20:40

  • 7

    [] is a set; [123] matches any char inside the set once; [^123] matches anything NOT inside the set once; [^/\r\n]+ matches one or more chars that are different from /, \r, \n.

    Jun 5, 2014 at 20:18

  • 1

    Very good explanation, thanks. Just a small callout. In the first result snippet in the answer, group 1 should be “https” not “http.”

    – rudyg123

    Aug 15, 2018 at 3:18

230

You can use capturing groups to organize and parse an expression. A non-capturing group has the first benefit, but doesn’t have the overhead of the second. You can still say a non-capturing group is optional, for example.

Say you want to match numeric text, but some numbers could be written as 1st, 2nd, 3rd, 4th,… If you want to capture the numeric part, but not the (optional) suffix you can use a non-capturing group.

([0-9]+)(?:st|nd|rd|th)?

That will match numbers in the form 1, 2, 3… or in the form 1st, 2nd, 3rd,… but it will only capture the numeric part.

2

  • 2

    Without non-capturing group, I could do: ([0-9]+)(st|nd|rd|th)?? With \1 I have the number, no ?: needed. BTW what is the ? at the end?

    – Timo

    Mar 17, 2021 at 7:43

  • In this case, ? at the end means the capture group is optional.

    Dec 9, 2021 at 21:49

142

?: is used when you want to group an expression, but you do not want to save it as a matched/captured portion of the string.

An example would be something to match an IP address:

/(?:\d{1,3}\.){3}\d{1,3}/

Note that I don’t care about saving the first 3 octets, but the (?:...) grouping allows me to shorten the regex without incurring the overhead of capturing and storing a match.

2

  • 10

    For inexperienced readers: This would match an IP address, but also invalid IP addresses. An expression to validate an IP address would be much more complex. So, don’t use this to validate an IP address.

    Aug 27, 2020 at 19:23

  • Just to add, this is saying you have 1 to 3 digits followed by a “.”, exactly three times, followed by another 1 to 3 digits.

    – Amber

    Feb 23 at 18:34