Help with regular expressions

Discussion in 'Python' started by dmbkiwi, Aug 26, 2003.

  1. dmbkiwi

    dmbkiwi Guest

    I have a problem. I have written a python based theme for a linux app
    called superkaramba, which is effectively an engine for desktop applets
    that utilises python as its theming language. The script basically parses
    weather websites, and displays the info in a visually appealing way.

    A couple of other people have contributed code to this project,
    particularly relating to the parsing of the websites. Unfortunately, it
    is not parsing one particular part of the website properly. This is
    because it is expecting the data to be in a certain form, and occasionally
    it is in a different form. Unfortunately this causes the entire script to
    fail to run.

    Unfortunately, I know very little about regular expressions and can't get
    hold of the person who wrote this part of the script. The other issue I am
    struggling with is that there are no error messages as to what's going
    wrong, which makes it more difficult to code round the issue.

    The issue comes down to a couple of lines in the html for the web page.
    The following lines parse correctly:

    <TD ALIGN=LEFT VALIGN=TOP CLASS=obsInfo1>Wind:</TD>
    <TD ALIGN=LEFT VALIGN=TOP CLASS=obsInfo2>From the Northeast at 6&nbsp;mph</TD>

    these don't:

    <TD ALIGN=LEFT VALIGN=TOP CLASS=obsInfo1>Wind:</TD>
    <TD ALIGN=LEFT VALIGN=TOP CLASS=obsInfo2>calm&nbsp;</TD>

    the relevant portion of the python script is as follows:

    print '==================================================='
    p_current = r'''(?isx) # Ignore case, Dot matches all, Verbose
    wxicons/52/(?P<icon>\d*?)\.gif # Icon
    ..*?obsTempTextA>(?P<temp>\d*?)&deg; # Temp
    ..*?obsTextA>(?P<sky>.*?)</b> # Sky
    ..*?Feels\sLike<br>(?P<heat>.*?)&deg; # Heat
    ..*?UV\sIndex:.*?Info2>(?P<uv>.*?)&nbsp
    ..*?Dew\sPoint:.*?Info2>(?P<dew>.*?)&deg;
    ..*?Humidity:.*?Info2>(?P<hum>\d+)
    ..*?Visibility:.+?Info2>(?P<vis>.*?)</td>
    ..*?Pressure:.+?Info2>(?P<baro>.*?)\sinches\sand\s(?P<change>.*?)</td>
    ..*?Wind:.+?Info2>(?P<wind>.*?)\sat\s(?P<speed>\d*?)&nbsp;
    '''


    match = re.search(p_current, data1)
    if match:
    now.icon(match.group('icon'))
    now.temperature(match.group('temp'), 'F')
    now.relative_heat(match.group('heat'), 'F')
    now.sky(match.group('sky'))
    now.uv(match.group('uv'))
    now.dewpoint(match.group('dew'), 'F')
    now.humidity(match.group('dew'))
    now.visibility(match.group('vis'))
    now.pressure(match.group('baro'), 'inHg')
    now.pressure_change(match.group('change'))
    mywind = match.group('wind')
    now.wind(mywind.replace('From the ', ''))
    now.wind_speed(match.group('speed'), 'mph')

    Obviously the issue is that the regular expression expects "at", and in
    the second line of the html that doesn't parse, there is no at.

    The question I have, is how do I go about fixing this. What I want is to
    test to see if the line does or doesn't contain an "at", and if not,
    change it to contain an "at". I'm just not sure how to code the RE in
    python to do this.

    Any help would be appreciated.

    Matt
     
    dmbkiwi, Aug 26, 2003
    #1
    1. Advertising

  2. dmbkiwi enlightened us with:
    > A couple of other people have contributed code to this project,
    > particularly relating to the parsing of the websites.
    > Unfortunately, it is not parsing one particular part of the website
    > properly. This is because it is expecting the data to be in a
    > certain form, and occasionally it is in a different form.
    > Unfortunately this causes the entire script to fail to run.


    You seem to expect old HTML. Why not use XHTML only ('tidy' can
    convert between them) and use a regular XML parser? Much, much, much
    easier! And you won't have to be afraid of messing up your regular
    expressions ;-)

    Sybren
    --
    The problem with the world is stupidity. Not saying there should be a
    capital punishment for stupidity, but why don't we just take the
    safety labels off of everything and let the problem solve itself?
     
    Sybren Stuvel, Aug 26, 2003
    #2
    1. Advertising

  3. dmbkiwi

    dmbkiwi Guest

    On Tue, 26 Aug 2003 08:47:33 +0000, Sybren Stuvel wrote:

    > dmbkiwi enlightened us with:
    >> A couple of other people have contributed code to this project,
    >> particularly relating to the parsing of the websites.
    >> Unfortunately, it is not parsing one particular part of the website
    >> properly. This is because it is expecting the data to be in a
    >> certain form, and occasionally it is in a different form.
    >> Unfortunately this causes the entire script to fail to run.

    >
    > You seem to expect old HTML. Why not use XHTML only ('tidy' can
    > convert between them) and use a regular XML parser? Much, much, much
    > easier! And you won't have to be afraid of messing up your regular
    > expressions ;-)
    >
    > Sybren


    XML would be nice, but unfortunately I have no choice as to the markup
    language used by the site. It's a website on the world wide web, not a
    site overwhich I have any control. My regular expressions are at the
    mercy of the developers of that site.

    Any other suggestions?

    Matt
     
    dmbkiwi, Aug 26, 2003
    #3
  4. dmbkiwi

    John J. Lee Guest

    dmbkiwi <> writes:
    > On Tue, 26 Aug 2003 08:47:33 +0000, Sybren Stuvel wrote:

    [...]
    > > You seem to expect old HTML. Why not use XHTML only ('tidy' can
    > > convert between them) and use a regular XML parser? Much, much, much
    > > easier! And you won't have to be afraid of messing up your regular
    > > expressions ;-)
    > >
    > > Sybren

    >
    > XML would be nice, but unfortunately I have no choice as to the markup
    > language used by the site. It's a website on the world wide web, not a
    > site overwhich I have any control. My regular expressions are at the
    > mercy of the developers of that site.


    You misunderstand. HTMLTidy (or its descendant, tidylib) reads ugly,
    non-conformant HTML and spits out clean, conformant XHTML (or HTML).

    uTidylib is a ctypes wrapper of tidylib.

    import tidy
    from cStringIO import StringIO
    tidydoc = tidy.parseString(html)
    s = StringIO()
    tidydoc.write(s)
    tidied_html = s.getvalue()


    mxTidy is a wrapper of a shared-library-ized HTMLTidy.

    from mx.Tidy import tidy
    tidied_html = tidy(html)[2]


    John
     
    John J. Lee, Aug 27, 2003
    #4
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Chris R. Timmons

    Re: Help with regular expressions.

    Chris R. Timmons, Jul 4, 2003, in forum: ASP .Net
    Replies:
    0
    Views:
    404
    Chris R. Timmons
    Jul 4, 2003
  2. David Waz...

    Re: Help with regular expressions.

    David Waz..., Jul 4, 2003, in forum: ASP .Net
    Replies:
    0
    Views:
    389
    David Waz...
    Jul 4, 2003
  3. Stephajn Craig

    Regular Expressions....HELP!

    Stephajn Craig, Jul 16, 2003, in forum: ASP .Net
    Replies:
    1
    Views:
    3,038
    Cowboy \(Gregory A. Beamer\)
    Jul 16, 2003
  4. Jay Douglas
    Replies:
    0
    Views:
    610
    Jay Douglas
    Aug 15, 2003
  5. Noman Shapiro
    Replies:
    0
    Views:
    235
    Noman Shapiro
    Jul 17, 2013
Loading...

Share This Page