Using end of word mark with unicode in regular expressions in Python

Question

The following matches in Idle, but does not match when run in a method in a module file:

import re re.search('\\bשלום\\b','שלום עולם',re.UNICODE)

while the following matches in both cases:

import re re.search('שלום','שלום עולם',re.UNICODE)

(Notice that stackoverflow erroneously switches the first and second items in the line above as this is a right to left language)

How can I make the first code match inside a py file?

Update: What I should have written for the first segment is that it matches in Idle, but does not match when run in eclipse console with PyDev.

The first re.search() doesn't work for me in IDLE or a module. — Lee
– Lee, Commented Jun 15, 2010 at 15:30
Did you try re.LOCALE instead of UNICODE? I'd install locale-he but am kind of afraid I'd never get it switched back. Off-topic: some say google translate goes too far apt-get install user-he; hebrew-settings becomes apt-get install user-en; english-settings which is impressive, but wrong ;) — msw
– msw, Commented Jun 15, 2010 at 15:37
@Lee, nor does the first for me, although the second does. I mention this as we are probably both in a non-he locale and soooo many things depend upon it. Oddly, it got the paste order correct. — msw
– msw, Commented Jun 15, 2010 at 15:46
Thanks for checking guys. I don't think I have anything set specifically to Hebrew, just to unicode. Idle\Options\Configure IDLE\General\Default Source Encoding is set to UTF-8 and in C:\Python26\Lib\site.py I have encoding = "utf-8" instead of encoding = "ascii". Also make sure you're usign a unicode supporting font such as Courier or Courier New — Jonathan Livni
– Jonathan Livni, Commented Jun 17, 2010 at 14:07

tzot · Accepted Answer · 2010-07-15 14:29:24Z

2

Seems to work for me when I'm using unicode strings:

# -*- coding: utf-8 -*- import re match = re.search(u'\\bשלום\\b', u'שלום עולם', re.U)

See it in action: http://codepad.org/xWz5cZj5

edited Jul 15, 2010 at 14:29

tzot

96.6k30 gold badges151 silver badges210 bronze badges

answered Jun 15, 2010 at 18:55

Kobi

139k41 gold badges259 silver badges302 bronze badges

Sign up to request clarification or add additional context in comments.

5 Comments

tzot Over a year ago

Is the # coding=utf-8 notation the same as # -*- coding: utf-8 -*-? I'm asking because it's the first time I see it like this. If not, please correct it.

Kobi Over a year ago

@ΤΖΩΤΖΙΟΥ - sorry to disappoint you, but I don't know. :| I don't know any Python, in fact, and learned every bit from Google and the documentations. I did that odd thing because I want to learn Python (one day), and I know Hebrew.

tzot Over a year ago

No disappointment here, don't worry; you did fine for someone not knowing Python :) It was possible that there was an alternative notation that I didn't know. I corrected it for you.

Kobi Over a year ago

@ΤΖΩΤΖΙΟΥ - No problem. The code I posted doesn't work without it, but I guess codepad.org isn't an accurate representation. Thanks!

John Machin Over a year ago

@ΤΖΩΤΖΙΟΥ: See docs.python.org/reference/…

Collectives™ on Stack Overflow

Using end of word mark with unicode in regular expressions in Python

1 Answer 1

5 Comments

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

5 Comments

Related