Which Regexp syntax is used ?

[buhtr]

Hello, I am considering to replace RE2 library with PIRE in a high load application which works with a stream. I am stuck with a regexp syntax, could you clarify which one do I need to use?

[dprokoptsev]

Pls clarify your question. Regexp syntax for pire is pretty standard, although a little restrictive in range specifications and perlish (?:.*)-like things.
You might want to play with examples/pigrep to find out which regexps match which strings.

[buhtr]

If it correct to say there are several standards, they are much the same but have some differences in supported constructions, for example, POSIX BRE/ERE, Perlre.

Currently, we allow our users to use Perl syntax for regexp in our application but we have advised them not to use some "heavy expressions". Mainly, these regexps are applied for searching patterns in a stream and we try to avoid backtracking and similar stuff.

As I guessed from you comment your have removed the ability of backtracking, am I right ? I think it would be great to have info in docs which regexp constructions from Perlre (or something else?) can be used or just which of them we cannot use.

Thus, If we move to PIRE I will need just disallow backtracking and some ranges? Could you point where I can read about ranges limitations? Ranges are not widely used in our current regexps but I would like to understand which of them need to be corrected if we replace RE2 lib with your lib.

I will definitely try example which you have suggested but I need to understand the whole scope of limitation.

[dprokoptsev]

Range limitations are mostly syntactic (e.g. pire disallows unescaped dashes, even in ranges like [a-z0-9-] where it's prettly clear that the last dash means a dash literally).

Concerning backtracking and stuff, backtracking things are essentially hacks allowing perlre (whose regexp-match routine has exponential complexity) to strip off certain branches of match tree, thus giving match routines a performance boost. Pire, utilizing linear-complex match algorithm, does not need any, so you may just write arbitrary complex regexp and be sure that if it manages to compile into a Scanner, it won't impose any performance penalties, no matter how heavy and complex the regexp was.

I'm not sure you were asking about this, but one of major features pire lacks compared to perl is ability to match previously captured fragment of input text (smth like /([A-Z]+)_\1/). In fact, a set of strings matched by this "regexp" exceeds beyond regular language family and thus is beyond pire's scope.

Back to the question, allowed syntactic elements are:

• pipe for alternatives;
• Klene star, plus, question mark or braces for various kinds of repetition;
• parentheses for groupings;
• character ranges in brackets (smth like [A-Za-z0-9@#$%]).

Nevertheless, these elements should allow you to express an arbitrary complex regexp.

[buhtr]

Thanks for your answer, now I've got the point.
One more question. How stable is it? Do you use it in your production environment?

[dprokoptsev]

Sure. It has been used in Yandex production environment for several years.