This chapter is from the book
This chapter is from the book
This chapter is from the book
9.4. Regular Expressions
Regular expressions specify string patterns. Use them whenever you need to locate strings that match a particular pattern. For example, suppose you want to find hyperlinks in an HTML file. You need to look for strings of the pattern <a href="...">. But wait—there may be extra spaces, or the URL may be enclosed in single quotes. Regular expressions give you a precise syntax for specifying what sequences of characters are legal matches.
In the following sections, you will see the regular expression syntax used by the Java API, and how to put regular expressions to work.
9.4.1. The Regular Expression Syntax
In a regular expression, a character denotes itself unless it is one of the reserved characters
. * + ? { | ( ) [ \ ^ $
For example, the regular expression Java only matches the string Java.
The symbol . matches any single character. For example, .a.a matches Java and data.
The * symbol indicates that the preceding constructs may be repeated 0 or more times; for a +, it is 1 or more times. A suffix of ? indicates that a construct is optional (0 or 1 times). For example, be+s? matches be, bee, and bees. You can specify other multiplicities with { } (see Table 9.4).
A | denotes an alternative: .(oo|ee)f matches beef or woof. Note the parentheses—without them, .oo|eef would be the alternative between .oo and eef. Parentheses are also used for grouping—see Section 9.4.4.
A character class is a set of character alternatives enclosed in brackets, such as [Jj], [0-9], [A-Za-z], or [^0-9]. Inside a character class, the - denotes a range (all characters whose Unicode values fall between the two bounds). However, a - that is the first or last character in a character class denotes itself. A ^ as the first character in a character class denotes the complement (all characters except those specified).
Table 9.4 contains a number of predefined character classes such as \d (digits). There are many more with the \p prefix, such as \p{Sc} (Unicode currency symbols)—see Table 9.5.
The characters ^ and $ match the beginning and end of input.
If you need to have a literal . * + ? { | ( ) [ \ ^ $, precede it by a backslash. Inside a character class, you only need to escape [ and \, provided you are careful about the positions of ] - ^. For example, []^-] is a class containing all three of them.
Instead of using backslashes, you can surround a string with \Q and \E. For example, \(\$0\.99\) and \Q($0.99)\E both match the string ($0.99).
Table 9.4: Regular Expression Syntax
Expression
Description
Example
Characters
c, not one of . * + ? { | ( ) [ \ ^ $
The character c.
J
.
Any character except line terminators, or any character if the DOTALL flag is set.
\X
Any Unicode “extended grapheme cluster”, which is perceived as a character or symbol
\x{p}
The Unicode code point with hex code p.
\x{1D546}
\uhhhh, \xhh, \0o, \0oo, \0ooo
The UTF-16 code unit with the given hex or octal value.
\uFEFF
\a, \e, \f, \n, \r, \t
Alert (\x{7}), escape (\x{1B}), form feed (\x{B}), newline (\x{A}), carriage return (\x{D}), tab (\x{9}).
\n
\cc, where c is in [A-Z] or one of @ [ \ ] ^ _ ?
The control character corresponding to the character c.
\cH is a backspace (\x{8}).
\c, where c is not in [A-Za-z0-9]
The character c.
\\
\Q ... \E
Everything between the start and the end of the quotation.
\Q(...)\E matches the string (...).
Character Classes
[C1C2...], where Ci are characters, ranges c-d, or character classes
Any of the characters represented by C1, C2,...
[0-9+-]
[^...]
Complement of a character class.
[^\d\s]
[...&&...]
Intersection of character classes.
[\p{L}&&[^A-Za-z]]
\p{...}, \P{...}
A predefined character class (see Table 9.5); its complement.
\p{L} matches a Unicode letter, and so does \pL—you can omit braces around a single letter.
\d, \D
Digits ([0-9], or \p{Digit} when the UNICODE_CHARACTER_CLASS flag is set); the complement.
\d+ is a sequence of digits.
\w, \W
Word characters ([a-zA-Z0-9_], or Unicode word characters when the UNICODE_CHARACTER_CLASS flag is set); the complement.
\s, \S
Spaces ([\n\r\t\f\x{B}], or \p{IsWhite_Space} when the UNICODE_CHARACTER_CLASS flag is set); the complement.
\s*,\s* is a comma surrounded by optional white space.
\h, \v, \H, \V
Horizontal whitespace, vertical whitespace, their complements.
Sequences and Alternatives
XY
Any string from X, followed by any string from Y.
[1-9][0-9]* is a positive number without leading zero.
X|Y
Any string from X or Y.
http|ftp
Grouping (see Section 9.4.4)
(X)
Captures the match of X.
'([^']*)' captures the quoted text.
\n
The nth group.
(['"]).*\1 matches 'Fred' or "Fred" but not "Fred'.
(?<name>X)
Captures the match of X with the given name.
'(?<id>[A-Za-z0-9]+)' captures the match with name id.
\k<name>
The group with the given name.
\k<id> matches the group with name id.
(?:X)
Use parentheses without capturing X.
In (?:http|ftp)://(.*), the match after :// is \1.
(?f1f2...:X), (?f1...-fk...:X), with fi in [dimsuUx]
Matches, but does not capture, X with the given flags (see Section 9.4.7) on or off (after -).
(?i:jpe?g) is a case-insensitive match.
Other (?...)
See the Pattern API documentation.
Quantifiers
X?
Optional X.
\+? is an optional + sign.
X*, X+
0 or more X, 1 or more X.
[1-9][0-9]+ is an integer ≥ 10.
X{n}, X{n,}, X{m,n}
n times X, at least n times X, between m and n times X.
[0-7]{1,3} are one to three octal digits.
Q?, where Q is a quantified expression
Reluctant quantifier, attempting the shortest match before trying longer matches.
.*(<.+?>).* matches the shortest sequence enclosed in angle brackets.
Q+, where Q is a quantified expression
Possessive quantifier, taking the longest match without backtracking.
'[^']*+' matches strings enclosed in single quotes and fails quickly on strings without a closing quote.
Boundary Matches
^ $
Beginning, end of input (or beginning, end of line in multiline mode).
^Java$ matches the input or line Java.
\A \Z \z
Beginning of input, end of input, absolute end of input (unchanged in multiline mode).
\b \B
Word boundary, nonword boundary.
\bJava\b matches the word Java.
\b{g}
Grapheme cluster boundary
Useful with split to decompose a string into grapheme clusters
\R
A Unicode line break.
\G
The end of the previous match.
Table 9.5: Predefined Character Classes \p{...}
Name
Description
posixClass
posixClass is one of Lower, Upper, Alpha, Digit, Alnum, Punct, Graph, Print, Cntrl, XDigit, Space, Blank, ASCII, interpreted as POSIX or Unicode class, depending on the UNICODE_CHARACTER_CLASS flag.
IsScript, sc=Script, script=Script
A script accepted by Character.UnicodeScript.forName.
InBlock, blk=Block, block=Block
A block accepted by Character.UnicodeBlock.forName.
Category, InCategory, gc=Category, general_category=Category
A one- or two-letter name for a Unicode general category.
IsProperty
Property is one of Alphabetic, Ideographic, Letter, Lowercase, Uppercase, Titlecase, Punctuation, Control, White_Space, Digit, Hex_Digit, Join_Control, Noncharacter_Code_Point, Assigned.
javaMethod
Invokes the method Character.isMethod (must not be deprecated).
9.4.2. Testing a Match
Generally, there are two ways to use a regular expression: Either you want to test whether a string matches the expression, or you want to find one or more matches of the expression in a string.
The static matches method tests whether an entire string matches a regular expression:
String regex = "[+-]?\\d+";
CharSequence input = ...;
if (Pattern.matches(regex, input)) {
// input matches the regular expression
...
}
If you need to use the same regular expression many times, it is more efficient to compile it. Then, create a Matcher for each input:
Pattern pattern = Pattern.compile(regex); Matcher matcher = pattern.matcher(input); if (matcher.matches()) ...
If the match succeeds, you can retrieve the location of matched groups—see Section 9.4.4.
To test whether a string contains a match, use the find method instead:
if (matcher.find()) {
// A substring of input matches the regular expression
...
}
The match and find methods mutate the state of the Matcher object. If you just want to find out whether a given Matcher has found a match, call the hasMatch method instead.
You can turn the pattern into a predicate. This is particularly useful with the filter method of a stream:
Pattern digits = Pattern.compile("[0-9]+");
List<String> strings = List.of("December", "31st", "1999");
List<String> matchingStrings = strings.stream()
.filter(digits.asMatchPredicate())
.toList(); // ["1999"]
The result contains all strings that match the regular expression.
Use the asPredicate method to test whether a string contains a match:
List<String> sringsContainingMatch = strings.stream()
.filter(digits.asPredicate())
.toList(); // ["31st", "1999"]
9.4.3. Finding All Matches
In this section, we consider a common use case for regular expressions—finding all matches in an input. Use this loop:
String input = ...;
Matcher matcher = pattern.matcher(input);
while (matcher.find()) {
String match = matcher.group();
int matchStart = matcher.start();
int matchEnd = matcher.end();
...
}
In this way, you can process each match in turn. As shown in the code fragment, you can get the matched string as well as its position in the input string.
More elegantly, you can call the results method to get a Stream<MatchResult>. The MatchResult interface has methods group, start, and end, just like Matcher. (In fact, the Matcher class implements this interface.) Here is how you get a list of all matches:
List<String> matches = pattern.matcher(input)
.results()
.map(MatchResult::group)
.toList();
If you have the data in a file, then you can use the Scanner.findAll method to get a Stream<MatchResult>, without first having to read the contents into a string. You can pass a Pattern or a pattern string:
var in = new Scanner(path);
Stream<String> words = in.findAll("\\pL+")
.map(MatchResult::group);
9.4.4. Groups
It is common to use groups for extracting components of a match. For example, suppose you have a line item in the invoice with item name, quantity, and unit price such as
Blackwell Toaster USD29.95
Here is a regular expression with groups for each component:
(\p{Alnum}+(\s+\p{Alnum}+)*)\s+([A-Z]{3})([0-9.]*)
After matching, you can extract the nth group from the matcher as
String contents = matcher.group(n);
Groups are ordered by their opening parenthesis, starting at 1. (Group 0 is the entire input.) In this example, here is how to take the input apart:
Matcher matcher = pattern.matcher(input);
if (matcher.matches()) {
item = matcher.group(1);
currency = matcher.group(3);
price = matcher.group(4);
}
We aren’t interested in group 2; it only arose from the parentheses that were required for the repetition. For greater clarity, you can denote that group as “non-capturing”. Then it doesn’t show up as a group in the matcher.
(\p{Alnum}+(?:\s+\p{Alnum}+)*)\s+([A-Z]{3})([0-9.]*)
Or, even better, use named groups:
(?<item>\p{Alnum}+(\s+\p{Alnum}+)*)\s+(?<currency>[A-Z]{3})(?<price>[0-9.]*)
Then, you can retrieve the groups by name:
item = matcher.group("item");
With the start and end methods of the Matcher and MatchResult classes, you can get the group positions in the input:
int itemStart = matcher.start("item");
int itemEnd = matcher.end("item");
The namedGroups method yields a Map<String, Integer from group names to numbers.
9.4.5. Splitting along Delimiters
Sometimes, you want to break an input along matched delimiters and keep everything else. The Pattern.split method automates this task. You obtain an array of strings, with the delimiters removed:
String input = ...;
Pattern commas = Pattern.compile("\\s*,\\s*");
String[] tokens = commas.split(input);
// "1, 2, 3" turns into ["1", "2", "3"]
If there are many tokens, you can fetch them lazily:
Stream<String> tokens = commas.splitAsStream(input);
To also collect the delimiters, use the splitWithDelimiters method:
tokens = commas.splitWithDelimiters(input, -1); // ["1", ", ", "2", " , ", "3", ",", "4"]
If the second argument is a positive number n, the separator pattern is applied at most n - 1 times. and the last element is the remaining string. Otherwise, the pattern is applied as often as possible. With a limit of zero, trailing empty strings are discarded.
If you don’t care about precompiling the pattern or lazy fetching, you can just use the split and splitWithDelimiter methods of the String class:
tokens = input.split("\\s*,\\s*");
If the input is in a file, use a scanner:
var in = new Scanner(path);
in.useDelimiter("\\s*,\\s*");
Stream<String> tokens = in.tokens();
9.4.6. Replacing Matches
If you want to replace all matches of a regular expression with a string, call replaceAll on the matcher:
Matcher matcher = commas.matcher(input);
String result = matcher.replaceAll(",");
// Normalizes the commas
Or, if you don’t care about precompiling, use the replaceAll method of the String class.
String result = input.replaceAll("\\s*,\\s*", ",");
The replacement string can contain group numbers $n or names ${name}. They are replaced with the contents of the corresponding captured group.
String result = "3:45".replaceAll(
"(\\d{1,2}):(?<minutes>\\d{2})",
"$1 hours and ${minutes} minutes");
// Sets result to "3 hours and 45 minutes"
You can use \ to escape $ and \ in the replacement string, or you can call the Matcher.quoteReplacement convenience method:
matcher.replaceAll(Matcher.quoteReplacement(str))
If you want to carry out a more complex operation than splicing in group matches, then you can provide a replacement function instead of a replacement string. The function accepts a MatchResult and yields a string. For example, here we replace all words with at least four letters with their uppercase version:
String result = Pattern.compile("\\pL{4,}")
.matcher("Mary had a little lamb")
.replaceAll(m -> m.group().toUpperCase());
// Yields "MARY had a LITTLE LAMB"
The replaceFirst method replaces only the first occurrence of the pattern.
9.4.7. Flags
Several flags change the behavior of regular expressions. You can specify them when you compile the pattern:
Pattern pattern = Pattern.compile(regex,
Pattern.CASE_INSENSITIVE | Pattern.UNICODE_CHARACTER_CLASS);
Or you can specify them inside the pattern:
String regex = "(?iU:expression)";
Here are the flags:
Pattern.CASE_INSENSITIVEori: Match characters independently of the letter case. By default, this flag takes only US ASCII characters into account.Pattern.UNICODE_CASEoru: When used in combination withCASE_INSENSITIVE, use Unicode letter case for matching.Pattern.UNICODE_CHARACTER_CLASSorU: Select Unicode character classes instead of POSIX. ImpliesUNICODE_CASE.Pattern.MULTILINEorm: Make^and$match the beginning and end of a line, not the entire input.Pattern.UNIX_LINESord: Only'\n'is a line terminator when matching^and$in multiline mode.Pattern.DOTALLors: Make the.symbol match all characters, including line terminators.Pattern.COMMENTSorx: Whitespace and comments (from#to the end of a line) are ignored.Pattern.LITERAL: The pattern is taking literally and must be matched exactly, except possibly for letter case.Pattern.CANON_EQ: Take canonical equivalence of Unicode characters into account. For example, u followed by ¨ (diaeresis) matches ü.
The last two flags cannot be specified inside a regular expression.