Problem with imaplib (weird result if mailbox contains a %)

[Antoon Pardon]

This little program gives IMO a strange result.

import imaplib

user = "cpapen"

cyr = imaplib.IMAP4("imap.vub.ac.be")
cyr.login("cyrus", "cOn-A1r")
rc, lst = cyr.list('""', "user/%s/*" % user)
for el in lst:
print "%r" % (el,)

And the result is:

'(\\HasNoChildren) "/" "user/cpapen/Out"'
'(\\HasNoChildren) "/" "user/cpapen/Punten"'
'(\\HasNoChildren) "/" "user/cpapen/Spam"'
'(\\HasNoChildren) "/" "user/cpapen/agoog to be"'
'(\\HasNoChildren) "/" "user/cpapen/artistiek &- kunst"'
'(\\HasNoChildren) "/" "user/cpapen/copains et copinnes =x="'
'(\\HasNoChildren) "/" "user/cpapen/cp &- writing"'
'(\\HasNoChildren) "/" "user/cpapen/examen"'
'(\\HasNoChildren) "/" "user/cpapen/important info (pass)"'
'(\\HasNoChildren) "/" "user/cpapen/lesmateriaal"'
'(\\HasNoChildren) "/" "user/cpapen/love &- flesh for fantasy"'
'(\\HasNoChildren) "/" "user/cpapen/media"'
'(\\HasNoChildren) "/" "user/cpapen/music &- beats"'
('(\\HasNoChildren) "/" {25}', 'user/cpapen/newsletters %')
''
'(\\HasNoChildren) "/" "user/cpapen/organisatie &- structuur"'
'(\\HasNoChildren) "/" "user/cpapen/sociale wetenschappen"'
'(\\HasNoChildren) "/" "user/cpapen/the closest ones to me [x]"'
'(\\HasNoChildren) "/" "user/cpapen/vubrations"'
'(\\HasNoChildren) "/" "user/cpapen/wm2addressbook"'
'(\\HasNoChildren) "/" "user/cpapen/wm2prefs"'
'(\\HasNoChildren) "/" "user/cpapen/wm2signature"'


What I have a problem with is the 14th and 15th line.
All other entries are strings but the 14th is a tuple.
and the 15th is an empty string. As far as I can tell
every time a "%" is in the mailbox name I get this kind of
result.

I'm using python 2.3.3 and the imap sytem is Cyrus.

Can someone explain what is going one?

Is this a bug?

If it is, is it fixed in later versions?

Whether or not it is a bug, can I rely on the mailbox
being the last item in the tuple in these cases?

 

[Leo Kislov]

This little program gives IMO a strange result.

import imaplib

user = "cpapen"

cyr = imaplib.IMAP4("imap.vub.ac.be")
cyr.login("cyrus", "cOn-A1r")
rc, lst = cyr.list('""', "user/%s/*" % user)
for el in lst:
print "%r" % (el,)

And the result is:

'(\\HasNoChildren) "/" "user/cpapen/Out"'
'(\\HasNoChildren) "/" "user/cpapen/Punten"'
'(\\HasNoChildren) "/" "user/cpapen/Spam"'
'(\\HasNoChildren) "/" "user/cpapen/agoog to be"'
'(\\HasNoChildren) "/" "user/cpapen/artistiek &- kunst"'
'(\\HasNoChildren) "/" "user/cpapen/copains et copinnes =x="'
'(\\HasNoChildren) "/" "user/cpapen/cp &- writing"'
'(\\HasNoChildren) "/" "user/cpapen/examen"'
'(\\HasNoChildren) "/" "user/cpapen/important info (pass)"'
'(\\HasNoChildren) "/" "user/cpapen/lesmateriaal"'
'(\\HasNoChildren) "/" "user/cpapen/love &- flesh for fantasy"'
'(\\HasNoChildren) "/" "user/cpapen/media"'
'(\\HasNoChildren) "/" "user/cpapen/music &- beats"'
('(\\HasNoChildren) "/" {25}', 'user/cpapen/newsletters %')
''
'(\\HasNoChildren) "/" "user/cpapen/organisatie &- structuur"'
'(\\HasNoChildren) "/" "user/cpapen/sociale wetenschappen"'
'(\\HasNoChildren) "/" "user/cpapen/the closest ones to me [x]"'
'(\\HasNoChildren) "/" "user/cpapen/vubrations"'
'(\\HasNoChildren) "/" "user/cpapen/wm2addressbook"'
'(\\HasNoChildren) "/" "user/cpapen/wm2prefs"'
'(\\HasNoChildren) "/" "user/cpapen/wm2signature"'


What I have a problem with is the 14th and 15th line.
All other entries are strings but the 14th is a tuple.
and the 15th is an empty string. As far as I can tell
every time a "%" is in the mailbox name I get this kind of
result.

I'm using python 2.3.3 and the imap sytem is Cyrus.

Can someone explain what is going one?

Is this a bug?

Empty string seems to be a bug. But tuple is by design, read the docs
and imap rfc. The protocol is convoluted in the first place, and so is
python interface.

If it is, is it fixed in later versions?

Why don't you try to pull imaplib.py from later versions? I don't think
it changed that much so it should be compatible with python 2.3

Whether or not it is a bug, can I rely on the mailbox
being the last item in the tuple in these cases?

Yes (at least for list command)

-- Leo

 

[Antoon Pardon]

This little program gives IMO a strange result.

import imaplib

user = "cpapen"

cyr = imaplib.IMAP4("imap.vub.ac.be")
cyr.login("cyrus", "cOn-A1r")
rc, lst = cyr.list('""', "user/%s/*" % user)
for el in lst:
print "%r" % (el,)

And the result is:

'(\\HasNoChildren) "/" "user/cpapen/Out"'
'(\\HasNoChildren) "/" "user/cpapen/Punten"'
'(\\HasNoChildren) "/" "user/cpapen/Spam"'
'(\\HasNoChildren) "/" "user/cpapen/agoog to be"'
'(\\HasNoChildren) "/" "user/cpapen/artistiek &- kunst"'
'(\\HasNoChildren) "/" "user/cpapen/copains et copinnes =x="'
'(\\HasNoChildren) "/" "user/cpapen/cp &- writing"'
'(\\HasNoChildren) "/" "user/cpapen/examen"'
'(\\HasNoChildren) "/" "user/cpapen/important info (pass)"'
'(\\HasNoChildren) "/" "user/cpapen/lesmateriaal"'
'(\\HasNoChildren) "/" "user/cpapen/love &- flesh for fantasy"'
'(\\HasNoChildren) "/" "user/cpapen/media"'
'(\\HasNoChildren) "/" "user/cpapen/music &- beats"'
('(\\HasNoChildren) "/" {25}', 'user/cpapen/newsletters %')
''
'(\\HasNoChildren) "/" "user/cpapen/organisatie &- structuur"'
'(\\HasNoChildren) "/" "user/cpapen/sociale wetenschappen"'
'(\\HasNoChildren) "/" "user/cpapen/the closest ones to me [x]"'
'(\\HasNoChildren) "/" "user/cpapen/vubrations"'
'(\\HasNoChildren) "/" "user/cpapen/wm2addressbook"'
'(\\HasNoChildren) "/" "user/cpapen/wm2prefs"'
'(\\HasNoChildren) "/" "user/cpapen/wm2signature"'


What I have a problem with is the 14th and 15th line.
All other entries are strings but the 14th is a tuple.
and the 15th is an empty string. As far as I can tell
every time a "%" is in the mailbox name I get this kind of
result.

I'm using python 2.3.3 and the imap sytem is Cyrus.

Can someone explain what is going one?

Is this a bug?

Empty string seems to be a bug. But tuple is by design, read the docs
and imap rfc. The protocol is convoluted in the first place, and so is
python interface.

Are there more docs than at http://www.python.org/doc/. I don't find
those very helpfull in explaining this.

I also took a look at rfc 2060 and to be honest I don't find anything
there to explain this difference. I only took a closer look at section
7.2.2. So maybe I should look somewehere else but after reading section
7.2.2. I don't understand why the list method returned a tuple for this
mailbox instead of the following string:

'(\\HasNoChildren) "/" "user/cpapen/newsletters %"'

Why don't you try to pull imaplib.py from later versions? I don't think
it changed that much so it should be compatible with python 2.3

I could take my hands on a 2.4 version and the result was the same.

Yes (at least for list command)

Well that is at least comforting.

 

[Leo Kislov]

This little program gives IMO a strange result.

import imaplib

user = "cpapen"

cyr = imaplib.IMAP4("imap.vub.ac.be")
cyr.login("cyrus", "cOn-A1r")
rc, lst = cyr.list('""', "user/%s/*" % user)
for el in lst:
print "%r" % (el,)

And the result is:

'(\\HasNoChildren) "/" "user/cpapen/Out"'
'(\\HasNoChildren) "/" "user/cpapen/Punten"'
'(\\HasNoChildren) "/" "user/cpapen/Spam"'
'(\\HasNoChildren) "/" "user/cpapen/agoog to be"'
'(\\HasNoChildren) "/" "user/cpapen/artistiek &- kunst"'
'(\\HasNoChildren) "/" "user/cpapen/copains et copinnes =x="'
'(\\HasNoChildren) "/" "user/cpapen/cp &- writing"'
'(\\HasNoChildren) "/" "user/cpapen/examen"'
'(\\HasNoChildren) "/" "user/cpapen/important info (pass)"'
'(\\HasNoChildren) "/" "user/cpapen/lesmateriaal"'
'(\\HasNoChildren) "/" "user/cpapen/love &- flesh for fantasy"'
'(\\HasNoChildren) "/" "user/cpapen/media"'
'(\\HasNoChildren) "/" "user/cpapen/music &- beats"'
('(\\HasNoChildren) "/" {25}', 'user/cpapen/newsletters %')
''
'(\\HasNoChildren) "/" "user/cpapen/organisatie &- structuur"'
'(\\HasNoChildren) "/" "user/cpapen/sociale wetenschappen"'
'(\\HasNoChildren) "/" "user/cpapen/the closest ones to me [x]"'
'(\\HasNoChildren) "/" "user/cpapen/vubrations"'
'(\\HasNoChildren) "/" "user/cpapen/wm2addressbook"'
'(\\HasNoChildren) "/" "user/cpapen/wm2prefs"'
'(\\HasNoChildren) "/" "user/cpapen/wm2signature"'


What I have a problem with is the 14th and 15th line.
All other entries are strings but the 14th is a tuple.
and the 15th is an empty string. As far as I can tell
every time a "%" is in the mailbox name I get this kind of
result.

I'm using python 2.3.3 and the imap sytem is Cyrus.

Can someone explain what is going one?

Is this a bug?

Empty string seems to be a bug. But tuple is by design, read the docs
and imap rfc. The protocol is convoluted in the first place, and so is
python interface.

Are there more docs than at http://www.python.org/doc/. I don't find
those very helpfull in explaining this.

I also took a look at rfc 2060 and to be honest I don't find anything
there to explain this difference. I only took a closer look at section
7.2.2. So maybe I should look somewehere else but after reading section
7.2.2. I don't understand why the list method returned a tuple for this
mailbox instead of the following string:

'(\\HasNoChildren) "/" "user/cpapen/newsletters %"'

This is described in section 4.3. imaplib is too close to the protocol.
It should interpret response for each command separately. For example
list method could return list of tuples like:

("\\HasNoChildren", "/", "user/cpapen/newsletters %")

Without this abstraction level in imaplib you have to build it
yourself.

I could take my hands on a 2.4 version and the result was the same.

I was talking only about empty string response. Is it still there?
Anyway, this issue requires investigation. That could also be a bug in
the server.

-- Leo

 

[Antoon Pardon]

This is described in section 4.3. imaplib is too close to the protocol.
It should interpret response for each command separately. For example
list method could return list of tuples like:

("\\HasNoChildren", "/", "user/cpapen/newsletters %")

Without this abstraction level in imaplib you have to build it
yourself.

Ah yes, I'm beginning to understand now, what is going on.

I also did a imap session by hand using telnet. That helped.

I was talking only about empty string response. Is it still there?
Anyway, this issue requires investigation. That could also be a bug in
the server.

The empty line is still there using 2.4.
This is part of the session I did by hand. It doesn't seem to be a
server problem.

$ telnet machine.domain imap
Trying xxx.xxx.xxx.xxx...
Connected to machine.domain
Escape character is '^]'.
* OK maxi Cyrus IMAP4 v2.2.13 server ready
0001 LOGIN ... ....
0001 OK User logged in
0002 LIST "" user/cpapen/*
* LIST (\HasNoChildren) "/" "user/cpapen/Out"
....
* LIST (\HasNoChildren) "/" "user/cpapen/music &- beats"
* LIST (\HasNoChildren) "/" {25}
user/cpapen/newsletters %
* LIST (\HasNoChildren) "/" "user/cpapen/organisatie &- structuur"
* LIST (\HasNoChildren) "/" "user/cpapen/sociale wetenschappen"
....


Many thanks for your contribution.

 

[Donn Cave]

$ telnet machine.domain imap
Trying xxx.xxx.xxx.xxx...
Connected to machine.domain
Escape character is '^]'.
* OK maxi Cyrus IMAP4 v2.2.13 server ready
0001 LOGIN ... ....
0001 OK User logged in
0002 LIST "" user/cpapen/*
* LIST (\HasNoChildren) "/" "user/cpapen/Out"
...
* LIST (\HasNoChildren) "/" "user/cpapen/music &- beats"
* LIST (\HasNoChildren) "/" {25}
user/cpapen/newsletters %
* LIST (\HasNoChildren) "/" "user/cpapen/organisatie &- structuur"
* LIST (\HasNoChildren) "/" "user/cpapen/sociale wetenschappen"

{5}\r\nhello\r\n is an IMAP "literal". It's unlucky that
Cyrus uses this for some LIST responses and not others, since
that will be a surprise to clients that use ad hoc parsing -
like imaplib users.

Donn Cave, (e-mail address removed)