Regular expression front end User manual

Obviously, I don’t need to say much about regular expressions. This is where Jeffrey Friedl’s introduction to Mastering Regular Expressions comes in.

“If there were a list of great inventions in computer software, I believe there would be no more than twenty, and in that list, of course, there would be packet-based networks, the Web, Lisp, hashing, UNIX, compiler technology, relational models, object-oriented, XML. Regular expressions should definitely not be left out.

Regular expressions are a panacea for many practical tasks, improving development efficiency and program quality a hundredfold, and the key role they play in bioinformatics and human genome mapping is well known. Mr. Jiang Tao, the founder of CSDN, experienced the power of this tool in his early years when he was developing professional software products and was always impressed.”

So there’s no reason not to understand regular expressions, or even master them and use them.

This article starts with regular grammar, and then explains the principle of regular expression matching step by step with concrete examples. The code instances are in JS, PHP, Python, and Java (because there are some matching patterns that are not supported in JS, you need to use other languages). The content includes initial and advanced skills, suitable for beginners to learn and advance. This article tries to be simple and easy to understand. At the same time, in order to be comprehensive, it involves a lot of knowledge, a total of 12K words, a long space, please read patiently, if you have reading difficulties, please contact me in time.

A regular expression is a literal template made up of ordinary and special characters, also called metacharacters or qualifiers. Here’s a simple regular expression to match consecutive numbers:

/[0-9]+//\d+/

“\d” is the metacharacter, and “+” is the qualifier.

You can see that regular expressions are strictly case-sensitive.

[…]. Matches one of the characters in brackets. For example, [xyz] matches the characters x, y, or z. If the brackets contain metacharacters, the metacharacters are degraded to ordinary characters and no longer function as metacharacters. For example, [+.?] Matches plus, dot, or question mark.

| means or, says one of the both. Such as: a | b match a or b characters.

\ is an escape character, usually \ * +? | {[()]} ^ $. # and blank These characters are need to escape.

To test this, write a regular expression that matches a mobile phone number like this:

(\ + 86)? 1\d{10}

(1) “\ + 86” matching “+ 86” text, followed by metacharacters question mark, said can be 1 or 0 times matching, combined said “(\ + 86)?” Matches +86 or “”.

② The common character “1” matches the text “1”.

③ The metacaracter “\d” matches the digits 0 to 9, and the interval quantifier “{10}” means 10 matches, which together means that “\d{10}” matches 10 consecutive digits.

Above, the matching results are as follows:

1. Chinese characters: ^ [\ u4e00 – \ u9fa5] {0} $
2. Email: ^\w+([-+.]\w+)*@\w+([-.]\w+)*\.\w+([-.]\w+)*$
3. URL: ^https? ://([\w-]+.) +[\w-]+(/[\w-./?%&=]*)? $
4. Mobile phone number: ^1\ D {10}$
5. Id number: ^ (\ d {15} | \ d {and} \ | d (X)) $
6. [1-9]\d{5}(? ! \d) (zip code is 6 digits)

Pay attention to

• EREs is supported in js.
• When using BREs (basic regular expressions), you must use the following symbols (? , +, |, {and}, (,)) with escape characters \.
• Regular expressions of the form [[: XXXX :]] are built into PHP and are not supported in JS.

1. Grep supports regular expressions such as BREs, EREs, and PREs

   Grep without any arguments means “BREs” is used

   Grep followed by “-e” indicates that “EREs” is used.

   Grep followed by a “-p” parameter indicates that “PREs” is to be used

2. Egrep supports EREs and PREs regular expressions

   If egrep is not followed by any arguments, then “EREs” is used.

   The egrep command followed by the “-p” parameter means to use “PREs”

3. Sed supports BREs and EREs

   The sed directive defaults to “BREs”

   The sed command followed by the “-r” parameter indicates that “EREs” is to be used

4. Awk supports EREs and uses “EREs” by default.

By default, all qualifiers are in greedy mode, meaning to capture as many characters as possible; And after the qualifier you add? , is a non-greedy mode, which means to capture as few characters as possible. As follows:

Var STR = "aaab"; var STR = "aaab"; var STR = "aaab"; var STR = "aaab"; var STR = "aaab"; var STR = "aaab"; /; // Non-greedy mode console.log(str.match(reg1)); //["aaa"] capture all aconsole.log(str.match(regs)); //["a"], only the first a is captured because of non-greed mode

In fact, the non-greed pattern works very well, especially when matching HTML tags. For example, to match a paired DIV, scheme 1 May match many div tag pairs, while scheme 2 will match only one div tag pair.

var str = "<div class='v1'><div class='v2'>test</div><input type='text'/></div>"; var reg1 = /<div.*<\/div>/; Var reg2 = /<div.*? <\/div>/; // Match console.log(str.match(reg1)); //"<div class='v1'><div class='v2'>test</div><input type='text'/></div>"console.log(str.match(reg2)); //"<div class='v1'><div class='v2'>test</div>"

The grouping of regex is mainly achieved by parentheses, which enclose subexpressions as a grouping, and which can be followed by a qualifier indicating the number of repetitions. As follows, the ABC wrapped in parentheses is a grouping:

/(abc)+/.test("abc123") == true

So what’s the use of groups? In general, the grouping is a convenient representation of the number of repetitions, but it is also used for capture, as you can see below.

Non-trapping grouping, usually with parentheses and the words “? Composed of subexpressions, non-capturing groups do not create backreferences, as if there were no parentheses. As follows:

var color = "#808080"; var output = color.replace(/#(? :\d+)/,"$1"+"~~"); console.log(RegExp.$1); //""console.log(output); / / $1 ~ ~

Above, (? :\d+) represents a non-capturing group, and since the group does not capture anything, RegExp.$1 refers to the empty string.

Also, since the $1 backreference doesn’t exist, it ends up being replaced as a normal string.

In fact, there is no difference in search efficiency between capture-free and capture-free groups; neither is faster than the other.

Curing group, also called atomic group.

Grammar: (? >…).

As mentioned above, when we use the non-greedy pattern, we may do a lot of backtracking during the matching process, and the more backtracking, the less efficient the regular expression will run. The solidified grouping is designed to reduce the number of backtracking.

In fact, curing grouping (? >…). A match of is the same as a normal match and does not change the result. The only difference is that by the end of a solidified group match, the text it matches has solidified into a single unit and can only be retained or abandoned as a whole. Untried alternate states in parentheses subexpresses are abandoned, so a backtrace can never select any of them (and therefore cannot take part in a backtrace). Let’s look at an example to better understand the solidification grouping.

If you want to deal with a batch of data, the original format is 123.456, because the floating point number display problem, part of the data format will become 123.456000000789 this, now requires only to retain the decimal point after 2~3, but the last one can not be 0, then how to write this regular?

Var STR = "123.456000000789"; str = str.replace(/(\.\d\d[1-9]?) \d*/,"$1"); / / 123.456

The above re, for the “123.456” format of data, will be processed for nothing. To improve efficiency, we change the last “*” to a “+”. As follows:

Var STR = "123.456"; str = str.replace(/(\.\d\d[1-9]?) \d+/,"$1"); / / 123.45

At this point, “\ d \ d [1-9]?” Subexpression, matches “45” instead of “456”, because the end of the regular expression uses “+”, which means at least one digit must be matched at the end, so the end subexpression “\d+” matches “6”. Obviously “123.45” is not the match we expect, so what should we do? Could you let “[1-9]?” Once a match is made, no backtracking is done, and we use the fixed grouping we mentioned above.

“(\ \ d \ d (? > [1-9])? )\d+ “is a solidified grouping of the above regularization. Since the string “123.456” does not satisfy the re of the solidified group, the match will fail as expected.

Let’s analyze the regularization of the solidified group (\.\d\d(? > [1-9])? )\d+ why does not match the string “123.456”?

It is obvious that there are only two matching results for the above curing grouping.

Case ①: If [1-9] fails to match, is the re returned? The state of reserve left behind. Then match out of the cured group and proceed to [\d+]. When control leaves the cured group, there is no standby state to give up (since no standby state was created in the cured group at all).

Case 2: If [1-9] matching is successful, after the matching is separated from the curing group,? The saved standby state still exists, but is discarded because it belongs to a cured group that has ended.

For the string “123.456”, [1-9] can match successfully, so it matches case ②. Let’s revert to the execution scene of case 2.

1. The state of the match: the match has reached the “6” position, the match will continue to move forward; = = >
2. The subexpression \d+ finds no match, and the regex engine attempts a traceback; = = >
3. Is there a standby state for backtracking? = = >
4. “?” The saved standby state belongs to the cured group that has ended, so it will be abandoned; = = >
5. In this case, the “6” matched by the solidified group cannot be used for backtracking of the regular engine. = = >
6. Attempts to backtrack failed; = = >
7. The regular match fails.==>
8. The text “123.456” is not matched by the regular expression, which is expected.

The corresponding flow chart is as follows:

Unfortunately, the solidified grouping syntax is not supported in javaScript, Java, and Python, but it is available in PHP and. NET. A PHP version of the solidified grouping form of regular expressions is provided below to try out.

$STR = "123.456"; echo preg_replace("/(\.\d\d(? > [1-9])? )\d+/","\\1",$str); // Cure grouping

Not only that, PHP provides syntax that takes precedence over quantifiers. As follows:

$STR = "123.456"; echo preg_replace("/(\.\d\d[1-9]? +)\d+/","\\1",$str); // Take precedence over quantifiers

While Java does not support a solidified grouping syntax, Java does provide a quantifier first syntax that also avoids regular backtracking. As follows:

String STR = "123.456"; System.out.println(str.replaceAll("(\\.\\d\\d[1-9]? +)\\d+", "$1")); / / 123.456

It’s worth noting that the replaceAll method in Java needs to escape the backslash.

The thousands separator is, as the name suggests, the comma in a number. Refer to the western custom of adding a symbol to the number to avoid the fact that the number is too long to see its value intuitively. Therefore, add a comma every three digits to the number, which is the thousands separator.

So how do you convert a string of numbers into thousands?

var str = "1234567890"; (+str).toLocaleString(); / / "1234567890"

As above, toLocaleString() returns a “localized” string form of the current object.

• If the object is of type Number, the value is returned as a symbol-split string.
• If the object is of type Array, each item in the Array is converted to a string, and then the strings are concatenated with the specified delimiter and returned.

The toLocaleString method is special and localized. For Chinese, the default delimiter is the English comma. So you can use it to just convert a number to a string in the form of a thousand separator. If internationalization is considered, the above approach may fail.

Let’s try to deal with that by looking around.

function thousand(str){ return str.replace(/(? ! (^)? =([0-9]{3})+$)/g,','); }console.log(thousand(str)); / / "1234567890" the console. The log (thousand (" 123456 ")); / / "123456" the console. The log (thousand (" 1234567879876543210 ")); / / "1234567879876543210"

The re used above is divided into two pieces. ! ^) and (? =([0-9]{3})+$).

1. [0-9]{3} indicates three consecutive digits.
2. “([0-9]{3})+” indicates at least one or more consecutive occurrences of three digits.
3. “([0-9]{3})+$” represents consecutive positive integer multiples of three until the end of the string.
4. then(? = ([0-9] {3}) + $)Matches a zero-width position, and from that position to the end of the string, has a positive integer multiple of 3 in the middle.
5. The regular expression uses the global match g, which means that after a match is reached, it continues to match until no match is found.
6. Replacing this with a comma is essentially adding a comma every three digits.
7. Of course, the string “123456”, which happens to have a positive multiple of 3, cannot be preceded by a comma. Then using(? ! ^)Specifies that the replacement position cannot be the starting position.

The thousand separator example shows the power of looking around in one step.

ES6 extends re with two more modifiers (which other languages may not support):

• The y (sticky) modifier, similar to g, is also a global match, and the next match starts at the next position where the last match was successful. The difference is that the G modifier only needs to have a match at the remaining position, while the Y modifier ensures that the match must start at the first remaining position.

var s = "abc_ab_a"; var r1 = /[a-z]+/g; var r2 = /[a-z]+/y; console.log(r1.exec(s),r1.lastIndex); // ["abc", index: 0, input: "abc_ab_a"] 3console.log(r2.exec(s),r2.lastIndex); // ["abc", index: 0, input: "abc_ab_a"] 3console.log(r1.exec(s),r1.lastIndex); // ["ab", index: 4, input: "abc_ab_a"] 6console.log(r2.exec(s),r2.lastIndex); // null 0

As shown above, since the second match starts at the subscript 3, the corresponding string is “_”, and the regular object R2, which uses the y modifier, needs to start at the first remaining position, the match fails and returns null.

The sticky property of the regular object, indicating whether the y modifier is set. We’ll talk about that later.

• The u modifier provides support for adding 4 bytecode points to regular expressions. For example, the “𝌆” character is a 4-byte character. Using the regular match directly would have failed, while using the u modifier would have waited for the correct result.

Var s = "𝌆"; console.log(/^.$/.test(s)); //falseconsole.log(/^.$/u.test(s)); //true

• String.fromcodepoint () : Returns the corresponding character from a Unicode code point
• From character String. The prototype. CodePointAt () : returns the corresponding code
• String.prototype.at() : Returns the character at the given position in the String

For the unicode character set in JS, please refer to Ruan Yifeng’s detailed explanation of Unicode and JavaScript.

The above is an ES6 extension to regular. On the other hand, in terms of methods, the methods associated with regular expressions in javaScript are:

There are seven jS-related methods from RegExp and String objects. First, let’s look at the js regular class RegExp.

The compile method is used to change and recompile regular expressions during execution.

Compile (pattern[, flags])

Refer to the RegExp constructor above for parameter description. The usage is as follows:

var reg = new RegExp("abc", "gi"); var reg2 = reg.compile("new abc", "g"); console.log(reg); // /new abc/gconsole.log(reg2); // undefined

As you can see, the compile method changes the original regular expression object and recompiles it, and its return value is null.

The exec method is used to detect a string match to the regular expression, returning an array of results if a matching text is found, or null otherwise.

Grammar: exec (string)

The array returned by the exec method contains two additional properties, index and input. And the array has the following characteristics:

• The 0th item represents the text captured by the regular expression
• Items 1 to N indicate the first to n backreferences, pointing to the text captured by the first to n groups in sequence. You can use RegExp.$+ “Number 1 to N” to obtain the text in the group in sequence
• Index indicates the initial position of the matched string
• Input represents the string being retrieved

Exec behaves the same with or without the global “g” in the regular expression. But regular expression objects behave a little differently. Let’s elaborate on how regular expression objects behave differently.

Assume that the regular expression object is reg, the detected character is string, and the returned value of reg.exec(string) is array.

If reg contains the global identifier “g”, then the reg.lastIndex property represents the position after the end of the matched string in the original string, where the next match begins. LastIndex == array. Index (where the match started) + array[0].length(the length of the match string). As follows:

var reg = /([a-z]+)/gi, string = "World Internet Conference"; var array = reg.exec(string); console.log(array); //["World", "World", index: 0, input: "World Internet Conference"]console.log(RegExp.$1); //Worldconsole.log(reg.lastIndex); Array. Index + array[0].length

As the search continues, the value of array.index increments, that is, the value of reg.lastIndex increments synchronously. Therefore, we can also iterate through all the matching text in the string by calling the exec method repeatedly. Until the exec method matches no more text, it returns null and resets the reg.lastIndex property to 0.

Following the example above, let’s go ahead and execute the code to see if the above is correct, as follows:

array = reg.exec(string); console.log(array); //["Internet", "Internet", index: 6, input: "World Internet Conference"]console.log(reg.lastIndex); //14array = reg.exec(string); console.log(array); //["Conference", "Conference", index: 15, input: "World Internet Conference"]console.log(reg.lastIndex); //25array = reg.exec(string); console.log(array); //nullconsole.log(reg.lastIndex); / / 0

In the above code, with repeated calls to the exec method, the reg.lastIndex property is eventually reset to 0.

Issue review

In explaining the test method, we left a question. If the regular expression has the global flag g, the result of the test method above will be affected by reg.lastIndex, as well as the exec method. Since reg.lastIndex is not always zero and determines where the next match begins, you must manually reset the lastIndex property to 0 if you want to start retrieving new strings after a match has been made in a string. Avoid errors such as:

var reg = /[0-9]+/g, str1 = "123abc", str2 = "123456"; reg.exec(str1); console.log(reg.lastIndex); //3var array = reg.exec(str2); console.log(array); //["456", index: 3, input: "123456"]

The correct result should be “123456”, so it is recommended to add “reg.lastIndex = 0;” before executing the exec method the second time. .

If reg does not contain the global identifier “g”, then the result of the exec method (array) will be exactly the same as the result of the string.match(reg) method.

The new PATTERN attribute in H5 specifies the pattern used to validate input fields, and the pattern matching of pattern supports the way regular expressions are written. The default Pattern attribute is all-match, that is, it matches all text regardless of whether the regular expression contains the “^”, “$” metacaracters.

Note: Pattern applies to the following input types: text, search, URL, telephone, email, and password. To cancel form validation, add the novalidate attribute to the form tag.

In the initial stage of learning regularity, focus on understanding ① greedy and non-greedy patterns, ② grouping, ③ capture and non-capture grouping, ④ named grouping, ⑤ fixed grouping, and realize the subtlety of design. And the advanced stage, mainly lies in the skilled use of ⑥ zero-width assertion (or look around) to solve problems, and familiar with the principle of regular matching.

In fact, regularity is not very powerful in javaScript. JavaScript only supports (1) greedy and non-greedy modes, (2) grouping, (3) trapping and non-trapping grouping, and (6) sequential looking around in zero-width assertions. If you are a little more familiar with the seven regulars-related methods in JS (compile, test, exec, match, search, replace, split), you will be able to handle text or strings with ease.

Regular expressions, which are gifted at text processing, are so powerful that in many cases they are the only solution. Re is not limited to JS, it is supported by popular editors (such as Sublime, Atom) and ides (such as WebStorm, IntelliJ IDEA). You can even, at any time in any language, try to use re to solve problems that might not have been solved before, but can now be solved easily.

Regular data in other languages:

• Python Regular expression Guide
• Java regular expressions

That’s all for this article. If you have any questions or good ideas, please feel free to leave a comment below.

Louis

Introduction to this article: this article lasts for two months, a total of 12K words, in order to restore the use of regular front scene in a concise and comprehensive, collected a large number of regular information, and eliminate many redundant words, code words are not easy, like please click 👍 or collection, I will continue to keep updated.

This paper links: louiszhai. Making. IO / 2016/06/13 /…

Refer to the article

• “Mastering Regular Expressions” by Jeffrey Friedl [third edition]
• Comparison of Linux shell regular expressions (BREs,EREs,PREs)
• Regular Expressions capture groups/Non-capture Groups Introduction _ Regular expressions _ Home of scripting
• Regular expression (a) – metacharacter – inverse – bloggers park
• Regular expression detail – Guisu, program life. He who does not advance loses ground. – Blog channel – CSDN.NET
• Regular expression – fixed grouping – Taek – blog garden
• Regular expressions: Greedy and non-greedy patterns in detail (overview) _ Regular expressions _ Home of scripts
• JAVASCRIPT regular expression learning — > Basics and zero-width assertions — Feather in the Wind — blog channel — CSDN.NET
• Unicode and JavaScript in Detail – ruan Yifeng’s web log