LuaSearch - Navigate Lua Module Documentation


NAME

rex - regular expressions (Lua rexlib)


OVERVIEW

Lrexlib is a regular expression library for Lua 5.1. The makefiles provided build it into shared libraries called rex_posix.so and rex_pcre.so, which can be used with require or loadlib.

The library provides POSIX and PCRE regular expression matching:


INTRODUCTION

Lrexlib provides bindings of the two principal regular expression library interfaces POSIX (http://www.opengroup.org/onlinepubs/009695399/basedefs/xbd_chap09.html) and PCRE (http://www.pcre.org/pcre.txt) to Lua (http://www.lua.org) 5.1.

Lrexlib builds into shared libraries called by default rex_posix.so and rex_pcre.so, which can be used with require.


NOTES

  1. Most functions and methods in Lrexlib have mandatory and optional arguments. There are no dependencies between arguments in Lrexlib's functions and methods. Any optional argument can be supplied as nil (or omitted if it is trailing one), the library will then use the default value for that argument.

  2. This document uses the following syntax for optional arguments: they are bracketed separately, and commas are left outside brackets, e.g.:
      MyFunc (arg1, arg2, [arg3], [arg4])

  3. Throughout this document, the identifier rex is used in place of either rex_posix or rex_pcre, that are the default namespaces for the corresponding libraries.

  4. All functions receiving a regular expression pattern as an argument will generate an error if that pattern is found invalid by the used POSIX (http://www.opengroup.org/onlinepubs/009695399/basedefs/xbd_chap09.html) / PCRE (http://www.pcre.org/pcre.txt) library.


REFERENCE - Functions

rex.match

  rex.match (subj, patt, [init], [cf], [ef], [lo])

The function searches for the first match of the regexp patt in the string subj, starting from offset init, subject to flags cf and ef.

PCRE: A locale lo may be specified.

subj
subject string n/a

patt
regular expression pattern. Type: string. Default: n/a.

init
start offset in the subject (can be negative). Type: number. Default: 1.

cf
compilation flags (bitwise OR). Type: number. Default: 0.

ef
execution flags (bitwise OR). Type: number. Default: 0.

lo
[PCRE] locale. Type: string. Default: nil.

Returns on success: 1. All substring matches (``captures''), in the order they appear in the pattern. false is returned for sub-patterns that did not participate in the match. If the pattern specified no captures then the whole matched substring is returned.

Returns on failure: 1. nil. 2. The return value of the underlying pcre_exec/regexec call (a number).

rex.find

  rex.find (subj, patt, [init], [cf], [ef], [lo])

The function searches for the first match of the regexp patt in the string subj, starting from offset init, subject to flags cf and ef.

PCRE: A locale lo may be specified.

subj
subject. Type: string. Default: n/a.

patt
regular expression pattern. Type: string. Default: n/a.

init
start offset in the subject (can be negative). Type: number. Default: 1.

cf
compilation flags (bitwise OR). Type: number. Default: 0.

ef
execution flags (bitwise OR). Type: number. Default: 0.

lo
[PCRE] locale. Type: string. Default: nil.

Returns on success: 1. The start point of the match (a number). 2. The end point of the match (a number). 3. All substring matches (``captures''), in the order they appear in the pattern. false is returned for sub-patterns that did not participate in the match.

Returns on failure: 1. nil. 2. The return value of the underlying pcre_exec/regexec call (a number).

rex.gmatch

  rex.gmatch (subj, patt, [cf], [ef], [lo])

The function returns an iterator for repeated matching of the pattern patt in the string subj, subject to flags cf and ef.

PCRE: A locale lo may be specified.

subj
subject. Type: string. Default: n/a.

patt
regular expression pattern. Type: string. Default: n/a.

cf
compilation flags (bitwise OR). Type: number. Default: 0.

ef
execution flags (bitwise OR). Type: number. Default: 0.

lo
[PCRE] locale. Type: string. Default: nil.

The iterator function is called by Lua. On every iteration (that is, on every match), it returns all captures in the order they appear in the pattern (or the entire match if the pattern specified no captures). The iteration will continue till the subject fails to match.

rex.gsub

  rex.gsub (subj, patt, repl, [n], [cf], [ef], [lo])

The function searches for all matches of the pattern patt in the string subj and substitutes the found matches according to the parameter repl (see details below).

PCRE: A locale lo may be specified.

subj
subject. Type: string. Default: n/a.

patt
regular expression pattern. Type: string. Default: n/a.

repl
substitution source. Type: string, function or table. Default: n/a.

n
maximum number of matches to search for; unlimited if not supplied. Type: number. Default: nil.

cf
compilation flags (bitwise OR). Type: number. Default: 0.

ef
execution flags (bitwise OR). Type: number. Default: 0.

lo
[PCRE] locale. Type: string. Default: nil.

Returns: 1. The subject string with the substitutions made. 2. Number of matches found.

The parameter repl can be either a string, a function or a table. The function behaves differently depending on the repl type:

  1. If repl is a string then it is treated as a template for substitution, where the %X occurences in repl are handled in a special way, depending on the value of the character X:
  2. If repl is a function then it gets called on each match with the submatches passed as parameters (if there are no submatches then the entire match is passed as the only parameter). The substitution string is derived depending on the first return value of function repl:

    Though gsub is in general consistent with the API and behavior of Lua's string.gsub, it has one extension with regards to string.gsub behavior:

  3. If repl is a table then the first submatch (or the entire match if there are no submatches) is used as the key and the value stored in repl under that key is used for substitution depending on its type.


reg.split

  rex.split (subj, sep, [cf], [ef], [lo])

This function is used for splitting a subject string subj into parts (sections). The sep parameter is a regular expression pattern representing separators between the sections.

The function returns an iterator for repeated matching of the pattern sep in the string subj, subject to flags cf and ef.

PCRE: A locale lo may be specified.

subj
subject. Type: string. Default: n/a.

sep
separator (regular expression pattern). Type: string. Default: n/a.

cf
compilation flags (bitwise OR). Type: number. Default: 0.

ef
execution flags (bitwise OR). Type: number. Default: 0.

lo
[PCRE] locale. Type: string. Default: nil.

On every iteration pass, the iterator returns: 1. A subject section (can be an empty string), followed by 2. All captures in the order they appear in the sep pattern (or the entire match if the sep pattern specified no captures). If there is no match (this can occur only in the last iteration), then nothing is returned after the subject section.

The iteration will continue till the end of the subject. Unlike rex.gmatch, there will always be at least one iteration pass, even if there's no matches in the subject.

rex.plainfind

  rex.plainfind (subj, patt, [init], [ci])

The function searches for the first match of the string patt in the subject subj, starting from offset init.

The string patt is not regular expression, all its characters stand for themselves.

Both strings subj and patt can have embedded zeros.

The flag ci specifies case-insensitive search (current locale is used).

This function uses neither PCRE nor POSIX regex library.

subj
subject. Type: string. Default: n/a.

patt
text to find. Type: string. Default: n/a.

init
start offset in the subject (can be negative). Type: number. Default: 1.

ci
case insensitive search. Type: boolean. Default: false.

Returns on success: 1. The start point of the match (a number). 2. The end point of the match (a number).

Returns on failure: 1. nil

rex.new

  rex.new (patt, [cf], [lo])

The functions compiles regular expression patt into a regular expression object whose internal representation is correspondent to the library used (PCRE or POSIX regex). The returned result then can be used by the methods r:tfind, r:exec and r:dfa_exec. Regular expression objects are automatically garbage collected.

PCRE: A locale lo may be specified.

patt
regular expression pattern. Type: string. Default: n/a.

cf
compilation flags (bitwise OR). Type: number. Default: 0.

lo
[PCRE] locale. Type: string. Default: nil.

Returns: 1. Compiled regular expression (a userdata).

rex.flags

  rex.flags ([tb])

This function returns a table containing numeric values of the constants defined by the used regex library (either PCRE or POSIX). Those constants are keyed by their names (strings). If the table argument tb is supplied then it is used as the output table, else a new table is created.

The constants contained in the returned table can then be used in most functions and methods where compilation flags or execution flags can be specified. They can also be used for comparing with return codes of some functions and methods for determining the reason of failure. For details, see PCRE (http://www.pcre.org/pcre.txt) and POSIX (http://www.opengroup.org/onlinepubs/009695399/basedefs/xbd_chap09.html) documentation.

Returns: 1. A table filled with the results.

rex.config

[PCRE only. See pcre_config in the PCRE (http://www.pcre.org/pcre.txt) docs.]

  rex.config ([tb])

This function returns a table containing the values of the configuration parameters used at PCRE library build-time. Those parameters (numbers) are keyed by their names (strings). If the table argument tb is supplied then it is used as the output table, else a new table is created.

tb
a table for placing results into. Type: table. Default: nil.

Returns: 1. A table filled with the results.

rex.version

[PCRE only. See pcre_version in the PCRE (http://www.pcre.org/pcre.txt) docs.]

  rex.version ()

This function returns a string containing the version of the used PCRE library and its release date.


REFERENCE - Methods

r:tfind

  r:tfind (subj, [init], [ef])

The method searches for the first match of the compiled regexp r in the string subj, starting from offset init, subject to execution flags ef.

r
regex object produced by rex.new. Type: userdata. Default: n/a.

subj
subject. Type: string. Default: n/a.

init
start offset in the subject (can be negative). Type: number. Default: 1.

ef
execution flags (bitwise OR). Type: number. Default: 0.

Returns on success: 1. The start point of the match (a number). 2. The end point of the match (a number). 3. Substring matches (``captures'' in Lua terminology) are returned as a third result, in a table. This table contains false in the positions where the corresponding sub-pattern did not participate in the match. PCRE: if named subpatterns are used then the table also contains substring matches keyed by their correspondent subpattern names (strings).

Returns on failure: 1. nil. 2. The return value of the underlying pcre_exec / regexec call (a number).

Notes: If named subpatterns (see PCRE http://www.pcre.org/pcre.txt docs) are used then the returned table also contains substring matches keyed by their correspondent subpattern names (strings).

r:exec

  r:exec (subj, [init], [ef])

The method searches for the first match of the compiled regexp r in the string subj, starting from offset init, subject to execution flags ef.

r
regex object produced by rex.new. Type: userdata. Default: n/a.

subj
subject. Type: string. Default: n/a.

init
start offset in the subject (can be negative). Type: number. Default: 1.

ef
execution flags (bitwise OR). Type: number. Default: 0.

Returns on success: 1. The start point of the first match (a number). 2. The end point of the first match (a number). 3. The offsets of substring matches (``captures'' in Lua terminology) are returned as a third result, in a table. This table contains false in the positions where the corresponding sub-pattern did not participate in the match. PCRE: if named subpatterns are used then the table also contains substring matches keyed by their correspondent subpattern names (strings).

Returns on failure: 1. nil. 2. The return value of the underlying pcre_exec / regexec call (a number).

Example: If the whole match is at offsets 10,20 and substring matches are at offsets 12,14 and 16,19 then the function returns the following: 10, 20, { 12,14,16,19 }.

r:dfa_exec

[PCRE 6.0 and later. See pcre_dfa_exec in the PCRE (http://www.pcre.org/pcre.txt) docs.]

  r:dfa_exec (subj, [init], [ef], [ovecsize], [wscount])

The method matches a compiled regular expression r against a given subject string subj, using a DFA matching algorithm.

r
regex object produced by rex.new. Type: userdata. Default: n/a.

subj
subject. Type: string. Default: n/a.

init
start offset in the subject (can be negative). Default: number. Default: 1.

ef
execution flags (bitwise OR). Type: number. Default: 0.

ovecsize
size of the array for result offsets. Type: number. Default: 100.

wscount
number of elements in the working space array. Type: number. Default: 50.

Returns on success: 1. The start point of the matches found (a number). 2. A table containing the end points of the matches found, the longer matches first. 3. The return value of the underlying pcre_dfa_exec call (a number).

Returns on failure: 1. nil 2. The return value of the underlying pcre_dfa_exec call (a number).

Example: If there are 3 matches found starting at offset 10 and ending at offsets 15, 20 and 25 then the function returns the following: 10, { 25,20,15 }, 3.


Incompatibilities with the Previous Version

The following changes are incompatible with Lrexlib version 1.19:

1. Lua 5.1 is required

2. Functions rex.newPCRE and rex.newPOSIX renamed to rex.new

3. Functions rex.flagsPCRE and flagsPOSIX renamed to rex.flags

4. Function rex.versionPCRE renamed to rex.version

5. Method r:match renamed to r:tfind

6. Method r:gmatch removed (similar functionality is provided by function rex.gmatch)

7. Methods r:tfind and r:exec: 2 values are returned on failure

8. Method r:exec: the returned table may additionally contain named subpatterns (PCRE only)


VERSION

This is version 2.0.


CREDITS

by Reuben Thomas (rrt _ at _ sc3d.org) and Shmuel Zeigerman (shmuz _ at _ actcom.co.il) [maintainer]

Thanks to Thatcher Ulrich for bug and warning fixes. Thanks to Nick Gammon for adding support for PCRE named subpatterns.


CONTACT

Please report bugs and make suggestions to the maintainer, or use the LuaForge trackers and mailing lists.


LICENSE

Lrexlib is copyright Reuben Thomas 2000-2007 and copyright Shmuel Zeigerman 2004-2007, and is released under the MIT license, like Lua (see http://www.lua.org/copyright.html for the full license; it's basically the same as the BSD license). There is no warranty.