Examples
Finding Barcodes
A bigstring of text contains barcodes that need to be replaced with something else. Every barcode has a different number, but barcodes have the same pattern. An example barcode is the string ##167891546325##. We can’t match an exact sequence as every barcode is different. However we can use a regular expression to match the pattern.
replaceAll(listToCheck, “##” + “[0-9]{12}” + “##”, “barcode discontinued”)
The regular expression “##” + “[0-9]{12}” + “##” matches the pattern of:
- two hashes
- followed by 12 numbers
- followed by two hashes.
Finally, Jep expression replaceAll() replaces any matches to the text barcode discontinued.
Finding File Names
| Regexp | Purpose |
|---|---|
"inputFile_[0-9]{3}\\.txt" | Match file names with the format: inputFile_NNN.txt e.g. inputFile_034.txt where N are digits. |
"inputFile_[0-9]{3}\\.txt" | Match file names with the format: inputFile_NNN.txt eg. inputFile_034.txt where N are digits. |
"inputFile_" + $fileSeq + "\\.txt" | Match file names with the format: inputFile_$fileSeq.txt e.g. inputFile_034.txt where $fileSeq="034" |
Regular Expression Constructs
The following table shows the different characters and what they match.
Finding Characters
| Construct | Matches |
|---|---|
| x | The character x |
| \\\\ | The backslash character |
| \\/ | The forward slash character |
| \\\" | The double quote character |
| \\0n | The character with octal value 0n (0 <= n <= 7) |
| \\0nn | The character with octal value 0nn (0 <= n <= 7) |
| \\0mnn | The character with octal value 0mnn (0 <= m <= 3, 0 <= n <= 7) |
| \\xhh | The character with hexadecimal value 0xhh |
| \uhhhh | The character with hexadecimal value 0xhhhh |
| \\t | The tab character ('\u0009') |
| \\n | The newline (line feed) character ('\u000A') |
| \\r | The carriage-return character ('\u000D') |
| \\f | The form-feed character ('\u000C') |
| \\a | The alert (bell) character ('\u0007') |
| \\e | The escape character ('\u001B') |
| \\cx | The control character corresponding to x |
Finding Character Classes
| Construct | Matches |
|---|---|
| [abc] | a, b, or c (simple class) |
| [^abc] | Any character except a, b, or c (negation) |
| [a-zA-Z] | a through z or A through Z, inclusive (range) |
| [a-d[m-p]] | a through d, or m through p: [a-dm-p] (union) |
| [a-z&&[def]] | d, e, or f (intersection) |
| [a-z&&[^bc]] | a through z, except for b and c: [ad-z] (subtraction) |
| [a-z&&[^m-p]] | a through z, and not m through p: [a-lq-z](subtraction) |
Finding Predefined Character Classes
| Construct | Matches |
|---|---|
. | Any character |
| \\d | A digit: [0-9] |
| \\D | A non-digit: [^0-9] |
| \\s | A whitespace character: [ \\t\\n\\x0B\\f\\r] |
| \\S | A non-whitespace character: [^\\s] |
| \\w | A word character: [a-zA-Z_0-9] |
| \\W | A non-word character: [^\\w] |
POSIX Character Classes (US-ASCII only)
| Construct | Matches |
|---|---|
| \\p{Lower} | A lower-case alphabetic character: [a-z] |
| \\p{Upper} | An upper-case alphabetic character:[A-Z] |
| \\p{ASCII} | All ASCII:[\\x00-\\x7F] |
| \\p{Alpha} | An alphabetic character:[\\p{Lower}\\p{Upper}] |
| \\p{Digit} | A decimal digit: [0-9] |
| \\p{Alnum} | An alphanumeric character:[\\p{Alpha}\\p{Digit}] |
| \\p{Punct} | Punctuation: One of !"#$%&'()*+,-./:;<=>?@[\]^_`{|}~ |
| \\p{Graph} | A visible character: [\\p{Alnum}\\p{Punct}] |
| \\p{Print} | A printable character: [\\p{Graph}] |
| \\p{Blank} | A space or a tab: [ \\t] |
\\p{Cntrl} | A control character: [\\x00-\\x1F\\x7F] |
| \\p{XDigit} | A hexadecimal digit: [0-9a-fA-F] |
| \\p{Space} | A whitespace character: [ \\t\\n\\x0B\\f\\r] |
Back References
| Construct | Matches |
|---|---|
| $n | Whatever the nth capturing group matched |
Classes for Unicode Blocks and Categories
| Construct | Matches |
|---|---|
| \\p{InGreek} | A character in the Greek block (simple block) |
| \\p{Lu} | An uppercase letter (simple category) |
\\p{Sc} | A currency symbol |
| \\P{InGreek} | Any character except one in the Greek block (negation) |
| [\\p{L}&&[^\\p{Lu}]] | Any letter except an uppercase letter (subtraction) |
Boundaries
| Construct | Matches |
|---|---|
| ^ | The beginning of a line |
| $ | The end of a line |
| \\b | A word boundary |
| \\B | A non-word boundary |
| \\A | The beginning of the input |
| \\G | The end of the previous match |
| \\Z | The end of the input but for the final terminator, if any |
| \\z | The end of the input |
Quantifiers
| Construct | Matches | ||
|---|---|---|---|
| Greedy | Reluctant | Possesive | |
| X? | X?? | X?+ | X, once or not at all |
| X* | X*? | X*+ | X, zero or more times |
| X+ | X+? | X++ | X, one or more times |
| X{n} | X{n}? | X{n}+ | X, exactly n times |
| X{n,} | X{n,}? | X{n,}+ | X, at least n times |
| X{n,m} | X{n,m}? | X{n,m}+ | X, at least n but not more than m times |
Logical Operators
| Construct | Matches |
|---|---|
| XY | X followed by Y |
| X|Y | Either X or Y |
| (X) | X, as a capturing group |
Special Constructs (Non-capturing)
| Construct | Matches |
|---|---|
| (?:X) | X, as a non-capturing group |
| (?idmsux-idmsux) | Nothing, but turns match flags on - off |
| (?idmsux-idmsux:X) | X, as a non-capturing group with the given flags on - off |
| (?=X) | X, via zero-width positive lookahead |
| (?!X) | X, via zero-width negative lookahead |
| (?<=X) | X, via zero-width positive lookbehind |
| (?<!X) | X, via zero-width negative lookbehind |
| (?>X) | X, as an independent, non-capturing group |
Quotation
| Construct | Matches |
|---|---|
| \\ | Nothing, but quotes the following character |
| \\Q | Nothing, but quotes all characters until \\E |
| \\E | Nothing, but ends quoting started by \\Q |
Examples of Regular Expressions as Arguments in Jep Expressions
if(!matches(value,"[\\p{Alnum}\\p{Space}]+"),error('Please only enter alpha numeric characters') |
This expression checks whether any characters are not (!) alphanumeric characters or spaces. If any are not, matches() will evaluate to 1, and the error message will be presented. The + is a ‘greedy quantifier’ (see table above) which means one or more times. As such, ‘value’ can contain as many alphanumeric characters or spaces as it likes, so long as it does not contain anything else.
$xml = replaceAll($xml, "<sc_tower_torqueing/type><id.*?<name>", "<type>"), $xml = replaceAll($xml, "</name></sc_tower_torqueing/type>", "</type>"), |
In line 1, the pattern being matched is the text “sc_tower_torqueing/type><id.” followed by any character, how ever many times it appears followed by the text “<name>” to finish. If the pattern is matched it will change any occurrence to the text “<type>”. The * means ‘wildcard’ (any character) whilst the ? means however many times it appears.
The second line is not a regular expression at all, it simply matches the exact text "</name></sc_tower_torqueing/type> and changes any occurrence to “</type>". It is not necessary to match a pattern if the exact text is known.
Backslashes and Escaping
In PhixFlow, some characters have special meanings, for example text may contain quotation marks (“) but PhixFlow reads quotation marks as marking the end of a string. If we want to use regular expressions to match a pattern that includes quotation marks, we need to tell PhixFlow to ignore its special meaning at treat it like a normal character.
In PhixFlow, to escape a character you must use two backslashes: \\, e.g. \\n for a new line. To match a backslash in the data you are matching, you will need four backslashes: \\\\ in your regular expression as the backslash is itself the escape character. See table above for details.
For example:
replaceAll(JSON Code,"[^\\p{Space}0-9A-Za-z!:\\\"%&\\[*()\\],-/_\\\\{}\\.]","") |
This expression is a useful example because some characters are ‘escaped’ and some are not. This expression checks text to make sure it does not contain any unusual characters by defining a list of acceptable characters. If it finds any unusual characters, it replaces them with nothing.
The opening “ in red are special characters, telling PhixFlow “this is the start and the end of the string to read”. The [ in blue are also special characters containing the string. The ^ is a special character, this means NOT ie match anything that is NOT in the following list.
The list itself is \\p{Space}0-9A-Za-z!:\"%&\\[*()\\],-/_\\\\{}\\ which is a list of acceptable character types. Notice how [ and ] are included, but are escaped as they are preceded by \\. \ is also included and is escaped with three \\\.
This expression permits spaces, number 0-9, uppercase or lowercase alphabetic characters, ! : “ % & [ * ( ) ] - / _ \ { } .
See Escaping Characters for more on special characters and how to escape them.
Note: A double backslash \\ can also make an alphanumeric character a construct (eg \\n means newline - see table for list of constructs). If you try escape an alphabetic character that is not a construct when escaped, you will get an error.
Line terminators
A line terminator is a one- or two-character sequence that marks the end of a line of the input character sequence. The following are recognized as line terminators:
A newline (line feed) character ('\\n'),
A carriage-return character followed immediately by a newline character ("\\r\\n"),
A standalone carriage-return character ('\\r'),
A next-line character ('\\u0085'),
A line-separator character ('\\u2028'), or
A paragraph-separator character ('\\u2029).
Groups and capturing
Capturing groups are numbered by counting their opening parentheses from left to right. In the expression ((A)(B(C))), for example, there are four such groups:
1 | ((A)(B(C))) |
|---|---|
2 | (A) |
3 | (B(C)) |
4 | (C) |
Group zero always stands for the entire expression.
During a match, each sub-sequence of the input sequence that matches a group is saved. The captured sub-sequence may be used later in the expression, via a back reference. In PhixFlow, a back reference has the format $n, where n is the number of the capturing group. See examples for replaceFirst.
The captured input associated with a group is always the sub-sequence that the group most recently matched. If a group is evaluated a second time because of a quantifier then its previously captured value, if any, will be retained if the second evaluation fails. Matching the string "aba" against the expression (a(b)?)+, for example, leaves group two set to "b". All captured input is discarded at the beginning of each match.
Groups beginning with (? are pure, non-capturing groups that do not capture text and do not count towards the group total.
are used to match patterns within strings or arrays as arguments within jep expressions, such as replaceAll() , replaceFirst() and matches() .
For example, imagine a bigstring of text contains barcodes that need to be replaced with something else. Every barcode has a different number, but barcodes have the same pattern. An example barcode is the string ##167891546325##. We can’t match an exact sequence as every barcode is different, but a regular expression can be used to match the pattern.
replaceAll(listToCheck, “##” + “[0-9]{12}” + “##”, “barcode discontinued”) |
The regular expression “##” + “[0-9]{12}” + “##” matches the pattern of two hashes, followed by 12 numbers, followed by two hashes, and the jep expression replaceAll() will replace any matches to the text “barcode discontinued” instead.
When entering a regular expression in a PhixFlow field, remember that it must be contained inside "quotes", except when using PhixFlow variables that contain strings, or functions that return strings.
Examples of Regular Expressions
"inputFile_[0-9]{3}\\.txt"
will match data - in this case file names - that have the format inputFile_NNN.txt, where N are digits. E.g. inputFile_034.txt.
"inputFile_" + today() + "\\.txt"
will match data - in this case file names - that have the format inputFile_YYYYMMDD.txt, where YYYYMMDD is the current date. E.g. inputFile_20130828.txt.
"inputFile_" + $fileSeq + "\\.txt"
will match data - in this case file names - that have the format inputFile_$fileSeq.txt. E.g. if $fileSeq="034", this will match inputFile_034.txt.
Regular expression constructs, and what they match
Construct | Matches |
|---|---|
Characters | |
x | The character x |
\\\\ | The backslash character |
\\/ | The forward slash character |
\\\" | The double quote character |
\\0n | The character with octal value 0n (0 <= n <= 7) |
\\0nn | The character with octal value 0nn (0 <= n <= 7) |
\\0mnn | The character with octal value 0mnn (0 <= m <= 3, 0 <= n <= 7) |
\\xhh | The character with hexadecimal value 0xhh |
\uhhhh | The character with hexadecimal value 0xhhhh |
\\t | The tab character ('\u0009') |
\\n | The newline (line feed) character ('\u000A') |
\\r | The carriage-return character ('\u000D') |
\\f | The form-feed character ('\u000C') |
\\a | The alert (bell) character ('\u0007') |
\\e | The escape character ('\u001B') |
\\cx | The control character corresponding to x |
Character classes | |
[abc] | a, b, or c (simple class) |
[^abc] | Any character except a, b, or c (negation) |
[a-zA-Z] | a through z or A through Z, inclusive (range) |
[a-d[m-p]] | a through d, or m through p: [a-dm-p] (union) |
[a-z&&[def]] | d, e, or f (intersection) |
[a-z&&[^bc]] | a through z, except for b and c: [ad-z] (subtraction) |
[a-z&&[^m-p]] | a through z, and not m through p: [a-lq-z](subtraction) |
Predefined character classes | |
. | Any character (may or may not match line terminators) |
\\d | A digit: [0-9] |
\\D | A non-digit: [^0-9] |
\\s | A whitespace character: [ \\t\\n\\x0B\\f\\r] |
\\S | A non-whitespace character: [^\\s] |
\\w | A word character: [a-zA-Z_0-9] |
\\W | A non-word character: [^\\w] |
POSIX character classes (US-ASCII only) | |
\\p{Lower} | A lower-case alphabetic character: [a-z] |
\\p{Upper} | An upper-case alphabetic character:[A-Z] |
\\p{ASCII} | All ASCII:[\\x00-\\x7F] |
\\p{Alpha} | An alphabetic character:[\\p{Lower}\\p{Upper}] |
\\p{Digit} | A decimal digit: [0-9] |
\\p{Alnum} | An alphanumeric character:[\\p{Alpha}\\p{Digit}] |
\\p{Punct} | Punctuation: One of !"#$%&'()*+,-./:;<=>?@[\]^_`{|}~ |
\\p{Graph} | A visible character: [\\p{Alnum}\\p{Punct}] |
\\p{Print} | A printable character: [\\p{Graph}] |
\\p{Blank} | A space or a tab: [ \\t] |
\\p{Cntrl} | A control character: [\\x00-\\x1F\\x7F] |
\\p{XDigit} | A hexadecimal digit: [0-9a-fA-F] |
\\p{Space} | A whitespace character: [ \\t\\n\\x0B\\f\\r] |
Classes for Unicode blocks and categories | |
\\p{InGreek} | A character in the Greek block (simple block) |
\\p{Lu} | An uppercase letter (simple category) |
\\p{Sc} | A currency symbol |
\\P{InGreek} | Any character except one in the Greek block (negation) |
[\\p{L}&&[^\\p{Lu}]] | Any letter except an uppercase letter (subtraction) |
Boundary matchers | |
^ | The beginning of a line |
$ | The end of a line |
\\b | A word boundary |
\\B | A non-word boundary |
\\A | The beginning of the input |
\\G | The end of the previous match |
\\Z | The end of the input but for the final terminator, if any |
\\z | The end of the input |
Greedy quantifiers | |
X? | X, once or not at all |
X* | X, zero or more times |
X+ | X, one or more times |
X{n} | X, exactly n times |
X{n,} | X, at least n times |
X{n,m} | X, at least n but not more than m times |
Reluctant quantifiers | |
X?? | X, once or not at all |
X*? | X, zero or more times |
X+? | X, one or more times |
X{n}? | X, exactly n times |
X{n,}? | X, at least n times |
X{n,m}? | X, at least n but not more than m times |
Possessive quantifiers | |
X?+ | X, once or not at all |
X*+ | X, zero or more times |
X++ | X, one or more times |
X{n}+ | X, exactly n times |
X{n,}+ | X, at least n times |
X{n,m}+ | X, at least n but not more than m times |
Logical operators | |
XY | X followed by Y |
X|Y | Either X or Y |
(X) | X, as a capturing group |
Back references | |
$n | Whatever the nth capturing group matched |
Quotation | |
\\ | Nothing, but quotes the following character |
\\Q | Nothing, but quotes all characters until \\E |
\\E | Nothing, but ends quoting started by \\Q |
Special constructs (non-capturing) | |
(?:X) | X, as a non-capturing group |
(?idmsux-idmsux) | Nothing, but turns match flags on - off |
(?idmsux-idmsux:X) | X, as a non-capturing group with the given flags on - off |
(?=X) | X, via zero-width positive lookahead |
(?!X) | X, via zero-width negative lookahead |
(?<=X) | X, via zero-width positive lookbehind |
(?<!X) | X, via zero-width negative lookbehind |
(?>X) | X, as an independent, non-capturing group |
Examples of Regular Expressions as Arguments in Jep Expressions
if(!matches(value,"[\\p{Alnum}\\p{Space}]+"),error('Please only enter alpha numeric characters') |
This expression checks whether any characters are not (!) alphanumeric characters or spaces. If any are not, matches() will evaluate to 1, and the error message will be presented. The + is a ‘greedy quantifier’ (see table above) which means one or more times. As such, ‘value’ can contain as many alphanumeric characters or spaces as it likes, so long as it does not contain anything else.
$xml = replaceAll($xml, "<sc_tower_torqueing/type><id.*?<name>", "<type>"), $xml = replaceAll($xml, "</name></sc_tower_torqueing/type>", "</type>"), |
In line 1, the pattern being matched is the text “sc_tower_torqueing/type><id.” followed by any character, how ever many times it appears followed by the text “<name>” to finish. If the pattern is matched it will change any occurrence to the text “<type>”. The * means ‘wildcard’ (any character) whilst the ? means however many times it appears.
The second line is not a regular expression at all, it simply matches the exact text "</name></sc_tower_torqueing/type> and changes any occurrence to “</type>". It is not necessary to match a pattern if the exact text is known.
Backslashes and Escaping
In PhixFlow, some characters have special meanings, for example text may contain quotation marks (“) but PhixFlow reads quotation marks as marking the end of a string. If we want to use regular expressions to match a pattern that includes quotation marks, we need to tell PhixFlow to ignore its special meaning at treat it like a normal character.
In PhixFlow, to escape a character you must use two backslashes: \\, e.g. \\n for a new line. To match a backslash in the data you are matching, you will need four backslashes: \\\\ in your regular expression as the backslash is itself the escape character. See table above for details.
For example:
replaceAll(JSON Code,"[^\\p{Space}0-9A-Za-z!:\\\"%&\\[*()\\],-/_\\\\{}\\.]","") |
This expression is a useful example because some characters are ‘escaped’ and some are not. This expression checks text to make sure it does not contain any unusual characters by defining a list of acceptable characters. If it finds any unusual characters, it replaces them with nothing.
The opening “ in red are special characters, telling PhixFlow “this is the start and the end of the string to read”. The [ in blue are also special characters containing the string. The ^ is a special character, this means NOT ie match anything that is NOT in the following list.
The list itself is \\p{Space}0-9A-Za-z!:\"%&\\[*()\\],-/_\\\\{}\\ which is a list of acceptable character types. Notice how [ and ] are included, but are escaped as they are preceded by \\. \ is also included and is escaped with three \\\.
This expression permits spaces, number 0-9, uppercase or lowercase alphabetic characters, ! : “ % & [ * ( ) ] - / _ \ { } .
See Escaping Characters for more on special characters and how to escape them.
Note: A double backslash \\ can also make an alphanumeric character a construct (eg \\n means newline - see table for list of constructs). If you try escape an alphabetic character that is not a construct when escaped, you will get an error.
Line terminators
A line terminator is a one- or two-character sequence that marks the end of a line of the input character sequence. The following are recognized as line terminators:
A newline (line feed) character ('\\n'),
A carriage-return character followed immediately by a newline character ("\\r\\n"),
A standalone carriage-return character ('\\r'),
A next-line character ('\\u0085'),
A line-separator character ('\\u2028'), or
A paragraph-separator character ('\\u2029).
Groups and capturing
Capturing groups are numbered by counting their opening parentheses from left to right. In the expression ((A)(B(C))), for example, there are four such groups:
1 | ((A)(B(C))) |
|---|---|
2 | (A) |
3 | (B(C)) |
4 | (C) |
Group zero always stands for the entire expression.
During a match, each sub-sequence of the input sequence that matches a group is saved. The captured sub-sequence may be used later in the expression, via a back reference. In PhixFlow, a back reference has the format $n, where n is the number of the capturing group. See examples for replaceFirst.
The captured input associated with a group is always the sub-sequence that the group most recently matched. If a group is evaluated a second time because of a quantifier then its previously captured value, if any, will be retained if the second evaluation fails. Matching the string "aba" against the expression (a(b)?)+, for example, leaves group two set to "b". All captured input is discarded at the beginning of each match.
Groups beginning with (? are pure, non-capturing groups that do not capture text and do not count towards the group total.