Re: re.sub(r'((?<=\A)|(?<=,))(?=,|\Z)', 'NA', ',1,,,two,3,,,') how does it work?
[email protected] (Stefan Ram)
| Newsgroups | gmane.comp.python.general |
|---|---|
| Organization | Stefan Ram |
| Message-ID | <[email protected]> |
Veek M <[email protected]> wrote or quoted: >If it's matching the empty string '' then why don't we get NA,NA1 etc >for ,1 | How Regex Engines Handle Lookarounds and Empty Matches | | In regular expressions, lookarounds - including lookaheads (?= . . . ) | and lookbehinds (?<= . . . ) - are /zero-width assertions/. They act | as /conditional checks/. They inspect the string to see if a pattern | exists, but they do not "consume" (wipe out or move past) any charac- | ters. | | Understanding how a regex engine processes these assertions requires | looking at two distinct phases: /evaluation within a single match | attempt/, and /progression through the string/. | | 1. The Fixed-Position Rule (Chaining Lookarounds) | | When a regex engine tests a pattern at a specific position in a | string, the internal pointer stays completely still until the entire | pattern either succeeds or fails. | | If you chain multiple lookarounds together, they all evaluate | from the exact same spatial slot, one after the other. | | Example: Chaining Lookaheads | | Consider the pattern "(?=x)(?=x)" applied to the string "xy". | | 1. The engine starts at Position 0 (the empty space right before | "x"). | | 2. First "(?=x)" check: Looks ahead from Position 0, sees "x", and | returns "True". The pointer does not move. | | 3. Second "(?=x)" check: Looks ahead from the same Position 0, sees | "x" again, and returns "True". The pointer does not move. | | Both assertions pass at Position 0. The engine declares a match of | length zero at that position. If one's using a substitution function | like Python's "re.sub(r'(?=x)(?=x)?, 'z', ?xy')", it inserts "z" at | that empty slot, resulting in "zxy". | | 2. The Forced-Advance Rule (Preventing Infinite Loops) | | If lookarounds don't move the pointer, why doesn't a global "search | and replace" loop forever, inserting infinitely many "z" at Position | 0? | | Regex engines implement a universal safety mechanism: /The Forced- | Advance Rule/. | | Standard Match If a regex matches actual characters (like "[^,]+"), | the pointer naturally moves past those characters for the next search | cycle. | | Zero-Width Match If a regex matches an empty string (zero width), | the engine applies the safety rule. After completing the match and | performing any substitution, /it forces the pointer forward by exactly | one position/ before attempting the next match. | | Edge Cases & Common Logic Traps | | Edge Case A: Contradictory Lookarounds | | What happens if you look ahead for two different characters at the | same time? | | (?=x)(?=y) | | Result This will never match anything. | | Why At any given empty slot in a string, the single character | immediately following that slot cannot be both "x" and "y" simul- | taneously. | | Edge Case B: Conflicting Lookahead and Lookbehind | | You can combine lookaheads and lookbehinds to pinpoint exact | boundaries. | | (?<=x)(?=y) | | Result This successfully matches the empty space between an "x" and | a "y" (such as inside the string "xy"). | | Why At Position 1 (between "x" and "y"), the lookbehind looks back- | ward and sees "x" ("True"), while the lookahead looks forward and sees | "y" ("True"). | | Edge Case C: Quantifiers on Zero-Width Matches | | What happens if you tell a lookaround to repeat using a quantifier | like "*" (zero or more times)? | | (?=x)* | | Result This creates an immediate zero-width match at every single | position in the string, even where "x" does not exist. | | Why At a position where "x" is absent, "(?=x)" fails. However, | because "*" means "zero or more times," matching it zero times is | considered a total success. The engine registers an empty match, | forces the pointer forward by one via the safety rule, and repeats | this at every slot. Lines marked with "| " come from my editing, where I start by writing prompts for the chatbot and then edit the generated texts and format them for USENET.