Curious to see alternate approach on a search/replace via regex

Discussion in 'Python' started by rh, Feb 6, 2013.

  1. rh

    rh Guest

    I am curious to know if others would have done this differently. And if so
    how so?

    This converts a url to a more easily managed filename, stripping the
    http protocol off.

    This:

    http://alongnameofasite1234567.com/q?sports=run&a=1&b=1

    becomes this:

    alongnameofasite1234567_com_q_sports_run_a_1_b_1


    def u2f(u):
    nx = re.compile(r'https?://(.+)$')
    u = nx.search(u).group(1)
    ux = re.compile(r'([-:./?&=]+)')
    return ux.sub('_', u)

    One alternate is to not do the compile step. There must also be a way to
    do it all at once. i.e. remove the protocol and replace the chars.
    rh, Feb 6, 2013
    #1
    1. Advertising

  2. rh

    Roy Smith Guest

    In article <>,
    rh <> wrote:

    > I am curious to know if others would have done this differently. And if so
    > how so?
    >
    > This converts a url to a more easily managed filename, stripping the
    > http protocol off.


    I would have used the urlparse module.

    http://docs.python.org/2/library/urlparse.html
    Roy Smith, Feb 6, 2013
    #2
    1. Advertising

  3. rh

    Nick Mellor Guest

    Hi RH,

    translate methods might be faster (and a little easier to read) for your use case. Just precompute and re-use the translation table punct_flatten.

    Note that the translate method has changed somewhat for Python 3 due to the separation of text from bytes. The is a Python 3 version.

    from urllib.parse import urlparse

    flattened_chars = "./&=?"
    punct_flatten = str.maketrans(flattened_chars, '_' * len(flattened_chars))
    parts = urlparse('http://alongnameofasite1234567.com/q?sports=run&a=1&b=1')
    unflattened = parts.netloc + parts.path + parts.query
    flattened = unflattened.translate(punct_flatten)
    print (flattened)

    Cheers,

    Nick

    On Thursday, 7 February 2013 08:41:05 UTC+11, rh wrote:
    > I am curious to know if others would have done this differently. And if so
    >
    > how so?
    >
    >
    >
    > This converts a url to a more easily managed filename, stripping the
    >
    > http protocol off.
    >
    >
    >
    > This:
    >
    >
    >
    > http://alongnameofasite1234567.com/q?sports=run&a=1&b=1
    >
    >
    >
    > becomes this:
    >
    >
    >
    > alongnameofasite1234567_com_q_sports_run_a_1_b_1
    >
    >
    >
    >
    >
    > def u2f(u):
    >
    > nx = re.compile(r'https?://(.+)$')
    >
    > u = nx.search(u).group(1)
    >
    > ux = re.compile(r'([-:./?&=]+)')
    >
    > return ux.sub('_', u)
    >
    >
    >
    > One alternate is to not do the compile step. There must also be a way to
    >
    > do it all at once. i.e. remove the protocol and replace the chars.
    Nick Mellor, Feb 7, 2013
    #3
  4. rh

    Nick Mellor Guest

    Hi RH,

    translate methods might be faster (and a little easier to read) for your use case. Just precompute and re-use the translation table punct_flatten.

    Note that the translate method has changed somewhat for Python 3 due to the separation of text from bytes. The is a Python 3 version.

    from urllib.parse import urlparse

    flattened_chars = "./&=?"
    punct_flatten = str.maketrans(flattened_chars, '_' * len(flattened_chars))
    parts = urlparse('http://alongnameofasite1234567.com/q?sports=run&a=1&b=1')
    unflattened = parts.netloc + parts.path + parts.query
    flattened = unflattened.translate(punct_flatten)
    print (flattened)

    Cheers,

    Nick

    On Thursday, 7 February 2013 08:41:05 UTC+11, rh wrote:
    > I am curious to know if others would have done this differently. And if so
    >
    > how so?
    >
    >
    >
    > This converts a url to a more easily managed filename, stripping the
    >
    > http protocol off.
    >
    >
    >
    > This:
    >
    >
    >
    > http://alongnameofasite1234567.com/q?sports=run&a=1&b=1
    >
    >
    >
    > becomes this:
    >
    >
    >
    > alongnameofasite1234567_com_q_sports_run_a_1_b_1
    >
    >
    >
    >
    >
    > def u2f(u):
    >
    > nx = re.compile(r'https?://(.+)$')
    >
    > u = nx.search(u).group(1)
    >
    > ux = re.compile(r'([-:./?&=]+)')
    >
    > return ux.sub('_', u)
    >
    >
    >
    > One alternate is to not do the compile step. There must also be a way to
    >
    > do it all at once. i.e. remove the protocol and replace the chars.
    Nick Mellor, Feb 7, 2013
    #4
  5. rh

    rh Guest

    On Thu, 7 Feb 2013 04:53:22 -0800 (PST)
    Nick Mellor <> wrote:

    > Hi RH,
    >
    > translate methods might be faster (and a little easier to read) for
    > your use case. Just precompute and re-use the translation table
    > punct_flatten.
    >
    > Note that the translate method has changed somewhat for Python 3 due
    > to the separation of text from bytes. The is a Python 3 version.
    >
    > from urllib.parse import urlparse
    >
    > flattened_chars = "./&=?"
    > punct_flatten = str.maketrans(flattened_chars, '_' * len
    > (flattened_chars)) parts = urlparse
    > ('http://alongnameofasite1234567.com/q?sports=run&a=1&b=1')
    > unflattened = parts.netloc + parts.path + parts.query flattened =
    > unflattened.translate(punct_flatten) print (flattened)


    I like the idea of using a library but since I'm learning python I wanted
    to try out the regex stuff. I haven't looked but I'd think that urllib might
    (should?) have a builtin so that one wouldn't have to specify the
    flattened_chars list. I'm sure there's a name for those chars but I don't know
    it. Maybe just punctuation??

    Also my version converts the ? into _ but urllib sees that as the query
    separator and removes it. Just point this out for completeness sake.

    This would mimic what I did:
    unflattened = parts.netloc + parts.path + '_' + parts.query

    >
    > Cheers,
    >
    > Nick
    >
    > On Thursday, 7 February 2013 08:41:05 UTC+11, rh wrote:
    > > I am curious to know if others would have done this differently.
    > > And if so
    > >
    > > how so?
    > >
    > >
    > >
    > > This converts a url to a more easily managed filename, stripping the
    > >
    > > http protocol off.
    > >
    > >
    > >
    > > This:
    > >
    > >
    > >
    > > http://alongnameofasite1234567.com/q?sports=run&a=1&b=1
    > >
    > >
    > >
    > > becomes this:
    > >
    > >
    > >
    > > alongnameofasite1234567_com_q_sports_run_a_1_b_1
    > >
    > >
    > >
    > >
    > >
    > > def u2f(u):
    > >
    > > nx = re.compile(r'https?://(.+)$')
    > >
    > > u = nx.search(u).group(1)
    > >
    > > ux = re.compile(r'([-:./?&=]+)')
    > >
    > > return ux.sub('_', u)
    > >
    > >
    > >
    > > One alternate is to not do the compile step. There must also be a
    > > way to
    > >
    > > do it all at once. i.e. remove the protocol and replace the chars.



    --
    rh, Feb 8, 2013
    #5
  6. rh

    Nick Mellor Guest

    Hi RH,

    It's essential to know about regex, of course, but often there's a better, easier-to-read way to do things in Python.

    One of Python's aims is clarity and ease of reading.

    Regex is complex, potentially inefficient and hard to read (as well as being the only reasonable way to do things sometimes.)

    Best,

    Nick

    On Friday, 8 February 2013 16:47:03 UTC+11, rh wrote:
    > On Thu, 7 Feb 2013 04:53:22 -0800 (PST)
    >
    > Nick Mellor <> wrote:
    >
    >
    >
    > > Hi RH,

    >
    > >

    >
    > > translate methods might be faster (and a little easier to read) for

    >
    > > your use case. Just precompute and re-use the translation table

    >
    > > punct_flatten.

    >
    > >

    >
    > > Note that the translate method has changed somewhat for Python 3 due

    >
    > > to the separation of text from bytes. The is a Python 3 version.

    >
    > >

    >
    > > from urllib.parse import urlparse

    >
    > >

    >
    > > flattened_chars = "./&=?"

    >
    > > punct_flatten = str.maketrans(flattened_chars, '_' * len

    >
    > > (flattened_chars)) parts = urlparse

    >
    > > ('http://alongnameofasite1234567.com/q?sports=run&a=1&b=1')

    >
    > > unflattened = parts.netloc + parts.path + parts.query flattened =

    >
    > > unflattened.translate(punct_flatten) print (flattened)

    >
    >
    >
    > I like the idea of using a library but since I'm learning python I wanted
    >
    > to try out the regex stuff. I haven't looked but I'd think that urllib might
    >
    > (should?) have a builtin so that one wouldn't have to specify the
    >
    > flattened_chars list. I'm sure there's a name for those chars but I don't know
    >
    > it. Maybe just punctuation??
    >
    >
    >
    > Also my version converts the ? into _ but urllib sees that as the query
    >
    > separator and removes it. Just point this out for completeness sake.
    >
    >
    >
    > This would mimic what I did:
    >
    > unflattened = parts.netloc + parts.path + '_' + parts.query
    >
    >
    >
    > >

    >
    > > Cheers,

    >
    > >

    >
    > > Nick

    >
    > >

    >
    > > On Thursday, 7 February 2013 08:41:05 UTC+11, rh wrote:

    >
    > > > I am curious to know if others would have done this differently.

    >
    > > > And if so

    >
    > > >

    >
    > > > how so?

    >
    > > >

    >
    > > >

    >
    > > >

    >
    > > > This converts a url to a more easily managed filename, stripping the

    >
    > > >

    >
    > > > http protocol off.

    >
    > > >

    >
    > > >

    >
    > > >

    >
    > > > This:

    >
    > > >

    >
    > > >

    >
    > > >

    >
    > > > http://alongnameofasite1234567.com/q?sports=run&a=1&b=1

    >
    > > >

    >
    > > >

    >
    > > >

    >
    > > > becomes this:

    >
    > > >

    >
    > > >

    >
    > > >

    >
    > > > alongnameofasite1234567_com_q_sports_run_a_1_b_1

    >
    > > >

    >
    > > >

    >
    > > >

    >
    > > >

    >
    > > >

    >
    > > > def u2f(u):

    >
    > > >

    >
    > > > nx = re.compile(r'https?://(.+)$')

    >
    > > >

    >
    > > > u = nx.search(u).group(1)

    >
    > > >

    >
    > > > ux = re.compile(r'([-:./?&=]+)')

    >
    > > >

    >
    > > > return ux.sub('_', u)

    >
    > > >

    >
    > > >

    >
    > > >

    >
    > > > One alternate is to not do the compile step. There must also be a

    >
    > > > way to

    >
    > > >

    >
    > > > do it all at once. i.e. remove the protocol and replace the chars.

    >
    >
    >
    >
    >
    > --
    Nick Mellor, Feb 8, 2013
    #6
  7. rh

    Nick Mellor Guest

    Hi RH,

    It's essential to know about regex, of course, but often there's a better, easier-to-read way to do things in Python.

    One of Python's aims is clarity and ease of reading.

    Regex is complex, potentially inefficient and hard to read (as well as being the only reasonable way to do things sometimes.)

    Best,

    Nick

    On Friday, 8 February 2013 16:47:03 UTC+11, rh wrote:
    > On Thu, 7 Feb 2013 04:53:22 -0800 (PST)
    >
    > Nick Mellor <> wrote:
    >
    >
    >
    > > Hi RH,

    >
    > >

    >
    > > translate methods might be faster (and a little easier to read) for

    >
    > > your use case. Just precompute and re-use the translation table

    >
    > > punct_flatten.

    >
    > >

    >
    > > Note that the translate method has changed somewhat for Python 3 due

    >
    > > to the separation of text from bytes. The is a Python 3 version.

    >
    > >

    >
    > > from urllib.parse import urlparse

    >
    > >

    >
    > > flattened_chars = "./&=?"

    >
    > > punct_flatten = str.maketrans(flattened_chars, '_' * len

    >
    > > (flattened_chars)) parts = urlparse

    >
    > > ('http://alongnameofasite1234567.com/q?sports=run&a=1&b=1')

    >
    > > unflattened = parts.netloc + parts.path + parts.query flattened =

    >
    > > unflattened.translate(punct_flatten) print (flattened)

    >
    >
    >
    > I like the idea of using a library but since I'm learning python I wanted
    >
    > to try out the regex stuff. I haven't looked but I'd think that urllib might
    >
    > (should?) have a builtin so that one wouldn't have to specify the
    >
    > flattened_chars list. I'm sure there's a name for those chars but I don't know
    >
    > it. Maybe just punctuation??
    >
    >
    >
    > Also my version converts the ? into _ but urllib sees that as the query
    >
    > separator and removes it. Just point this out for completeness sake.
    >
    >
    >
    > This would mimic what I did:
    >
    > unflattened = parts.netloc + parts.path + '_' + parts.query
    >
    >
    >
    > >

    >
    > > Cheers,

    >
    > >

    >
    > > Nick

    >
    > >

    >
    > > On Thursday, 7 February 2013 08:41:05 UTC+11, rh wrote:

    >
    > > > I am curious to know if others would have done this differently.

    >
    > > > And if so

    >
    > > >

    >
    > > > how so?

    >
    > > >

    >
    > > >

    >
    > > >

    >
    > > > This converts a url to a more easily managed filename, stripping the

    >
    > > >

    >
    > > > http protocol off.

    >
    > > >

    >
    > > >

    >
    > > >

    >
    > > > This:

    >
    > > >

    >
    > > >

    >
    > > >

    >
    > > > http://alongnameofasite1234567.com/q?sports=run&a=1&b=1

    >
    > > >

    >
    > > >

    >
    > > >

    >
    > > > becomes this:

    >
    > > >

    >
    > > >

    >
    > > >

    >
    > > > alongnameofasite1234567_com_q_sports_run_a_1_b_1

    >
    > > >

    >
    > > >

    >
    > > >

    >
    > > >

    >
    > > >

    >
    > > > def u2f(u):

    >
    > > >

    >
    > > > nx = re.compile(r'https?://(.+)$')

    >
    > > >

    >
    > > > u = nx.search(u).group(1)

    >
    > > >

    >
    > > > ux = re.compile(r'([-:./?&=]+)')

    >
    > > >

    >
    > > > return ux.sub('_', u)

    >
    > > >

    >
    > > >

    >
    > > >

    >
    > > > One alternate is to not do the compile step. There must also be a

    >
    > > > way to

    >
    > > >

    >
    > > > do it all at once. i.e. remove the protocol and replace the chars.

    >
    >
    >
    >
    >
    > --
    Nick Mellor, Feb 8, 2013
    #7
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Demian Brecht
    Replies:
    18
    Views:
    219
  2. Demian Brecht
    Replies:
    0
    Views:
    126
    Demian Brecht
    Feb 6, 2013
  3. MRAB
    Replies:
    0
    Views:
    107
  4. Peter Otten
    Replies:
    0
    Views:
    85
    Peter Otten
    Feb 7, 2013
  5. Demian Brecht
    Replies:
    0
    Views:
    84
    Demian Brecht
    Feb 7, 2013
Loading...

Share This Page