I'm confused with the following three patterns, would someone explain it in more detail?
## IPython with Python 2.7.3 In [62]: re.findall(r'[a-z]*',"f233op") Out[62]: ['f', '', '', '', 'op', ''] ## why does the last '' come out? In [63]: re.findall(r'([a-z])*',"f233op") Out[63]: ['f', '', '', '', 'p', ''] ## why does the character 'o' get lost? In [64]: re.findall(r'([a-z]*)',"f233op") Out[64]: ['f', '', '', '', 'op', ''] ## what's the different than line 63 above? 
re.findall(r'[a-z]*',"f233op") This pattern is matching zero-or-more instances of lower case alphabet characters. The ZERO-or-more part is key here, since a match of nothing, starting from every index position in the string, is just as valid as a match of f or op. The last empty string returned is the match starting from the end of the string (the position between p and $ (end of string).
re.findall(r'([a-z])*',"f233op") Now you are matching character groups, consisting of a single lower-case alphabet character. The o is no longer returned because this is a greedy search, and the last valid matched group will be returned. So if you changed the string to f233op12fre, the final e would be returned, but no the preceding f or r. Likewise, if you take out the p in your string, you still see that o is returned as a valid match.
Conversely, if you tried to make this regex non-greedy by adding a ? (eg. ([a-z])*?), the returned set of matches would all be empty strings, since a valid match of nothing has a higher precedence of a valid match of something.
re.findall(r'([a-z]*)',"f233op") Nothing is different in the matched characters, but now you are returning character groups instead of raw matches. The output of this regex query will be the same as your first example, but you'll notice that if you add an additional matching group, you will suddenly see the results of each match attempt grouped into tuples:
IN : re.findall(r'([a-z]*)([0-9]*)',"f233op") OUT: [('f', '233'), ('op', ''), ('', '')] Contrast this with the same pattern, minus the parenthesis (groups), and you'll see why they are important:
IN : re.findall(r'[a-z]*[0-9]*',"f233op") OUT: ['f233', 'op', ''] It can be useful to plug regex patterns like these into regex diagram generators like Regexplained to see how the pattern matching logic works. For example, as an explanation as to why your regex is always returning empty character string matches, take a look at the difference between the patterns [a-z]* and [a-z]+.
Don't forget to check the Python docs for the re library if you get stuck, they actually give a pretty stellar explanation for the standard regex syntax.
re.findall(r'([a-z]*)([0-9]*)',"f233op"). re.findall(...) returns character group(s), and more than one group are presented in the pattern, the result will be a list of tuples. When the engine goes over f233op with the pattern ([a-z]*)([0-9]*), it first hits f ( with pattern ([a-z]*) ), then it meets 233( with pattern ([0-9]*), since it's greedy, matches digits as many as possible), afterwards, f and 233 are grouped together as a tuple -> ('f', '233'). Likewise, ([a-z]*) matches op, ([0-9]*) matches ''. Do I express the matching logic correct so far?('op', '') is the result of the match that starts between 3 and o, while ('', '') is the result of the match that starts from between p and $ (end of line). It helps to think of regex matches as always starting and ending between characters (including ^ and $). To illustrate, look at the result of re.findall('[a-z]*?', 'abc'). It is four empty strings.You get the final '' because [a-z]* is matching the empty string at the end.
The character 'o' is missing because you have told re.findall to match groups, and each group has a single character. Put another way, you’re doing the equivalent of
m = re.match(r'([a-z])*', 'op') m.group(1) which will return 'p', because that’s the last thing captured by the parens (capture group 1).
Again, you’re matching groups, but this time multi-character groups.
o go? Why not ['f', '', '', '', 'o', 'p', '']?
([a-z])* against a string, then the capture value for group 1 will be the last instance that matched. i.e. matching against 'foobar', I will get 'r'.
[a-z]*? would return a set of empty strings, since a valid match of nothing has higher precedence than a valid match of something.Your surprising results are related to the Regular Expression Quantifier *.
Consider:
[a-z]* 
Vs:
[a-z]+ 
Consider as another example that I think is more illustrative of what you are seeing:
>>> re.findall(r'[a-z]*', '123456789') ['', '', '', '', '', '', '', '', '', ''] There are no characters in the set [a-z] in the string 123456789. Yet, since * means 'zero or more', all character positions 'match' by not matching any characters at that position.
For example, assume you just wanted to test if there were any letters in a string, and you use a regex like so:
>>> re.search(r'[a-z]*', '1234') <_sre.SRE_Match object at 0x1069b6988> # a 'match' is returned, but this is # probably not what was intended Now consider:
>>> re.findall(r'[a-z]*', '123abc789') ['', '', '', 'abc', '', '', '', ''] Vs:
>>> re.findall(r'([a-z])*', '123abc789') ['', '', '', 'c', '', '', '', ''] The first pattern is [a-z]*. The part [a-z] is a character class matching a single character in the set a-z unless modified; the addition of * quantifier will greedily match as many characters as possible if more than zero -- hence the match of 'abc' but will also allow zero characters to be a match (or a character outside the character set to match the position since 0 is a match).
The addition of a grouping in ([a-z])* effectively reduces the match in the quantified set back to a single character and the last character matched in the set is returned.
If you want the effect of grouping (say in a more complex pattern) use a non capturing group:
>>> re.findall(r'(?:[a-z])*', '123abc789') ['', '', '', 'abc', '', '', '', ''] re.findall(r'([a-z])*', '123abc789') for example, the pattern '([a-z])*', it consists of a lower-case character in the group, and * tells that search the group(1 character group) as many as possible. So when it goes over abc, it first fetches a, it matches the pattern, but it greedily finds that b also matches the pattern, then it drops a, and then, it fetches the new character c as the replacement of b. it goes on 7 afterwards, but it only gets a match of '' since 7 is not a char.
re.findall(r'([a-z])', '123abc789') for a third variant.([a-z]), it consists of EXACT one-lowercase-alphabetic character group, so when it goes through the string 123abc789, any non-lowercase-alphabetic character is dropped (such as 1,2,3,7,8,9), and the first match is found when it hits a, no greedy is specified, it will not try any further search, so a is returned as a part of list. Likewise, b and c are appended to the result list --> ['a', 'b', 'c']In line 63 you're finding all instances of a group (indicated with the parens) of characters of length 1. The * isn't doing much for you here (just causing you to match zero length groups).
In the other examples having the * next to the [a-z], you match adjacent characters of any length.
EDIT
Playing around with this tool may help.
'o' in 63. And it's characters of any length > 0 (which is where the empty strings come from).
'o' part, but I think @alastair has that one covered in his comment on his response. I realized my bug confusing * with + (for >=0 and >0 matches, respectively) and fixed that.
*to+and possibly you wont get the empty ones (i'm not a python dev tho)