Q on naming nested packages/modules

Discussion in 'Python' started by kj, Sep 1, 2009.

  1. kj

    kj Guest

    I'm having a hard time getting the hang of Python's package/module
    scheme. I'd like to find out what's considered best practice when
    dealing with the scenario illustrated below.

    The quick description of the problem is: how can I have two nested
    modules, spam.ham and spam.ham.eggs?

    Suppose I have a module (I'm not even sure this is the right word)
    called spam.ham, so I start out with the following file structure:

    spam/
    |-- ham.py
    `-- __init__.py

    With this arrangement, the line

    import spam.ham

    ....in client code works as expected.

    But now suppose that I want to factor out some code in spam/ham.py
    to a helper module. (The reason behind factoring out this new
    module is to "declutter" spam/ham.py, and improve its readibility.)
    My instinct (from my Perl past) is to put this factored-out code
    in a file spam/ham/eggs.py, i.e. to create the "nested" module
    spam.ham.eggs, which requires expanding the tree as follows

    spam/
    |-- ham/
    | |-- eggs.py
    | `-- __init__.py
    |-- ham.py
    `-- __init__.py

    ....and adding the following spam/ham.py to

    # spam/ham.py
    from . import eggs

    This doesn't work so well, because now spam/ham.py is not read.
    It seems that adding the spam/ham directory, or maybe adding the
    file spam/ham/__init__.py, causes spam/ham.py to be overlooked.


    Clearly, I'm not playing this game right...

    What is considered "best practice" for the use case sketched above?
    Should I, e.g. rename the directory spam/ham something like spam/ham_
    and refer to the helper module as spam.ham_.eggs? Or is some other
    convention preferred?

    I consulted PEP 8, but besides recommending "short, all-lowercase
    names" for modules, it gives little guidance on the situation
    described above.

    TIA!

    kynn
     
    kj, Sep 1, 2009
    #1
    1. Advertising

  2. On Tue, Sep 1, 2009 at 11:58 AM, kj<> wrote:
    >
    >
    >
    > I'm having a hard time getting the hang of Python's package/module
    > scheme.  I'd like to find out what's considered best practice when
    > dealing with the scenario illustrated below.
    >
    > The quick description of the problem is: how can I have two nested
    > modules, spam.ham and spam.ham.eggs?
    >
    > Suppose I have a module (I'm not even sure this is the right word)
    > called spam.ham, so I start out with the following file structure:
    >
    >  spam/
    >  |-- ham.py
    >  `-- __init__.py
    >
    > With this arrangement, the line
    >
    > import spam.ham
    >
    > ...in client code works as expected.
    >
    > But now suppose that I want to factor out some code in spam/ham.py
    > to a helper module.  (The reason behind factoring out this new
    > module is to "declutter" spam/ham.py, and improve its readibility.)
    > My instinct (from my Perl past) is to put this factored-out code
    > in a file spam/ham/eggs.py, i.e. to create the "nested" module
    > spam.ham.eggs, which requires expanding the tree as follows
    >
    >  spam/
    >  |-- ham/
    >  |   |-- eggs.py
    >  |   `-- __init__.py
    >  |-- ham.py
    >  `-- __init__.py
    >
    > ...and adding the following spam/ham.py to
    >
    > # spam/ham.py
    > from . import eggs
    >
    > This doesn't work so well, because now spam/ham.py is not read.
    > It seems that adding the spam/ham directory, or maybe adding the
    > file spam/ham/__init__.py, causes spam/ham.py to be overlooked.
    >
    >
    > Clearly, I'm not playing this game right...
    >
    > What is considered "best practice" for the use case sketched above?
    > Should I, e.g. rename the directory spam/ham something like spam/ham_
    > and refer to the helper module as spam.ham_.eggs?  Or is some other
    > convention preferred?
    >
    > I consulted PEP 8, but besides recommending "short, all-lowercase
    > names" for modules, it gives little guidance on the situation
    > described above.
    >


    Take everything in ham.py and stick it in ham/__init__.py instead.
    This will give you the behavior you're looking for (don't import
    spam.ham.__init__. Everything in __init__.py is loaded into spam.ham)
    > TIA!
    >
    > kynn
    > --
    > http://mail.python.org/mailman/listinfo/python-list
    >
     
    Benjamin Kaplan, Sep 1, 2009
    #2
    1. Advertising

  3. kj

    kj Guest

    In <h7jga8$ijj$> kj <> writes:

    >I'm having a hard time getting the hang of Python's package/module
    >scheme. I'd like to find out what's considered best practice when
    >dealing with the scenario illustrated below.


    >The quick description of the problem is: how can I have two nested
    >modules, spam.ham and spam.ham.eggs?


    Following up my own post...

    From inspecting the directory structure of some of the standard
    Python modules I infer the following rules:

    1. the source for "leaf" modules lives in files named after them
    (e.g. if x.y.z is a "leaf" module, its source code is in x/y/z.py)

    2. the source for "non-leaf" modules lives in files named __init__.py
    (e.g. if x.y is a "non-leaf" module, its source code lives in
    the file x/y/__init__.py)

    In the examples above, the module x.y is a "non-leaf" module because
    there is a module x.y.z.

    I.e. the "leaf"-ness of a module depends solely on whether other
    modules deeper in the hierarchy are present.

    An implication of all this is that if now I wanted to create a new
    module x.y.z.w, this means that the previously "leaf"-module x.y.z
    would become "non-leaf". In other words, I'd have to:

    1. create the new directory x/y/z
    2. *rename* the file x/y/z.py to x/y/z/__init__.py
    3. create the file x/y/z/w.py to hold the source for the new x.y.z.w
    module

    Is the above correct? (BTW, to my Perl-pickled brain, step 2 above
    is the one that causes most distress... But I think I can cope.)

    kynn
     
    kj, Sep 1, 2009
    #3
  4. kj

    Carl Banks Guest

    On Sep 1, 8:58 am, kj <> wrote:
    > I'm having a hard time getting the hang of Python's package/module
    > scheme.  I'd like to find out what's considered best practice when
    > dealing with the scenario illustrated below.
    >
    > The quick description of the problem is: how can I have two nested
    > modules, spam.ham and spam.ham.eggs?
    >
    > Suppose I have a module (I'm not even sure this is the right word)
    > called spam.ham, so I start out with the following file structure:
    >
    >   spam/
    >   |-- ham.py
    >   `-- __init__.py
    >
    > With this arrangement, the line
    >
    > import spam.ham
    >
    > ...in client code works as expected.
    >
    > But now suppose that I want to factor out some code in spam/ham.py
    > to a helper module.  (The reason behind factoring out this new
    > module is to "declutter" spam/ham.py, and improve its readibility.)
    > My instinct (from my Perl past) is to put this factored-out code
    > in a file spam/ham/eggs.py, i.e. to create the "nested" module
    > spam.ham.eggs, which requires expanding the tree as follows
    >
    >   spam/
    >   |-- ham/
    >   |   |-- eggs.py
    >   |   `-- __init__.py
    >   |-- ham.py
    >   `-- __init__.py
    >
    > ...and adding the following spam/ham.py to
    >
    > # spam/ham.py
    > from . import eggs
    >
    > This doesn't work so well, because now spam/ham.py is not read.
    > It seems that adding the spam/ham directory, or maybe adding the
    > file spam/ham/__init__.py, causes spam/ham.py to be overlooked.


    There's a hint here.

    You can move the contents of ham.py into ham/__init__.py, and it'll
    work the way you want, maybe with only a minor change or two.


    > Clearly, I'm not playing this game right...
    >
    > What is considered "best practice" for the use case sketched above?
    > Should I, e.g. rename the directory spam/ham something like spam/ham_
    > and refer to the helper module as spam.ham_.eggs?  Or is some other
    > convention preferred?


    The way you were intending is often the approach people use. I doubt
    there are any detailed consensus recommendations about this in the
    Python community.


    > I consulted PEP 8, but besides recommending "short, all-lowercase
    > names" for modules, it gives little guidance on the situation
    > described above.


    Of course it doesn't, PEP 8 is a style guide, not a project
    organization guide.


    Carl Banks
     
    Carl Banks, Sep 1, 2009
    #4
  5. kj

    Ethan Furman Guest

    kj wrote:
    > In <h7jga8$ijj$> kj <> writes:
    >
    >
    >>I'm having a hard time getting the hang of Python's package/module
    >>scheme. I'd like to find out what's considered best practice when
    >>dealing with the scenario illustrated below.

    >
    >
    >>The quick description of the problem is: how can I have two nested
    >>modules, spam.ham and spam.ham.eggs?

    >
    >
    > Following up my own post...
    >
    >>From inspecting the directory structure of some of the standard

    > Python modules I infer the following rules:
    >
    > 1. the source for "leaf" modules lives in files named after them
    > (e.g. if x.y.z is a "leaf" module, its source code is in x/y/z.py)
    >
    > 2. the source for "non-leaf" modules lives in files named __init__.py
    > (e.g. if x.y is a "non-leaf" module, its source code lives in
    > the file x/y/__init__.py)
    >
    > In the examples above, the module x.y is a "non-leaf" module because
    > there is a module x.y.z.
    >
    > I.e. the "leaf"-ness of a module depends solely on whether other
    > modules deeper in the hierarchy are present.
    >
    > An implication of all this is that if now I wanted to create a new
    > module x.y.z.w, this means that the previously "leaf"-module x.y.z
    > would become "non-leaf". In other words, I'd have to:
    >
    > 1. create the new directory x/y/z
    > 2. *rename* the file x/y/z.py to x/y/z/__init__.py
    > 3. create the file x/y/z/w.py to hold the source for the new x.y.z.w
    > module
    >
    > Is the above correct? (BTW, to my Perl-pickled brain, step 2 above
    > is the one that causes most distress... But I think I can cope.)
    >
    > kynn


    Looking at the layout of the (most excellent!-) xlrd package, the bulk
    of the code is in the __init__.py file, and other supporting code is the
    same directory, accessed in __init__ with normal imports.

    I also am unclear on when it's best to have supporting files in the same
    directory versus a subdirectory... perhaps it is that flat is better
    than nested?

    ~Ethan~
     
    Ethan Furman, Sep 1, 2009
    #5
  6. kj

    Terry Reedy Guest

    kj wrote:
    >
    >
    > But now suppose that I want to factor out some code in spam/ham.py
    > to a helper module. (The reason behind factoring out this new
    > module is to "declutter" spam/ham.py, and improve its readibility.)
    > My instinct (from my Perl past) is to put this factored-out code
    > in a file spam/ham/eggs.py, i.e. to create the "nested" module
    > spam.ham.eggs, which requires expanding the tree as follows
    >
    > spam/
    > |-- ham/
    > | |-- eggs.py
    > | `-- __init__.py
    > |-- ham.py
    > `-- __init__.py
    >
    > ...and adding the following spam/ham.py to
    >
    > # spam/ham.py
    > from . import eggs
    >
    > This doesn't work so well, because now spam/ham.py is not read.
    > It seems that adding the spam/ham directory, or maybe adding the
    > file spam/ham/__init__.py, causes spam/ham.py to be overlooked.
    >
    >
    > Clearly, I'm not playing this game right...


    One way is the __init__.py answer you have already. Another is to put
    eggs.py (or _eggs.py to indicate that it is private) in spam and skip
    the ham directory.

    spam/
    __init__.py
    ham.py
    _eggs.py

    tjr
     
    Terry Reedy, Sep 1, 2009
    #6
  7. > An implication of all this is that if now I wanted to create a new
    > module x.y.z.w, this means that the previously "leaf"-module x.y.z
    > would become "non-leaf". In other words, I'd have to:
    >
    > 1. create the new directory x/y/z
    > 2. *rename* the file x/y/z.py to x/y/z/__init__.py
    > 3. create the file x/y/z/w.py to hold the source for the new x.y.z.w
    > module


    With regard to point 2 -- would it be possible to just move z.py into
    x/y/z, and put 'from z import *' into x/y/z/__init__.py, for the same
    effect? Or is that not a good idea?

    On Tue, 01 Sep 2009 09:53:08 -0700, kj <> wrote:

    > In <h7jga8$ijj$> kj <> writes:
    >
    >> I'm having a hard time getting the hang of Python's package/module
    >> scheme. I'd like to find out what's considered best practice when
    >> dealing with the scenario illustrated below.

    >
    >> The quick description of the problem is: how can I have two nested
    >> modules, spam.ham and spam.ham.eggs?

    >
    > Following up my own post...
    >
    >> From inspecting the directory structure of some of the standard

    > Python modules I infer the following rules:
    >
    > 1. the source for "leaf" modules lives in files named after them
    > (e.g. if x.y.z is a "leaf" module, its source code is in x/y/z.py)
    >
    > 2. the source for "non-leaf" modules lives in files named __init__.py
    > (e.g. if x.y is a "non-leaf" module, its source code lives in
    > the file x/y/__init__.py)
    >
    > In the examples above, the module x.y is a "non-leaf" module because
    > there is a module x.y.z.
    >
    > I.e. the "leaf"-ness of a module depends solely on whether other
    > modules deeper in the hierarchy are present.
    >
    > An implication of all this is that if now I wanted to create a new
    > module x.y.z.w, this means that the previously "leaf"-module x.y.z
    > would become "non-leaf". In other words, I'd have to:
    >
    > 1. create the new directory x/y/z
    > 2. *rename* the file x/y/z.py to x/y/z/__init__.py
    > 3. create the file x/y/z/w.py to hold the source for the new x.y.z.w
    > module
    >
    > Is the above correct? (BTW, to my Perl-pickled brain, step 2 above
    > is the one that causes most distress... But I think I can cope.)
    >
    > kynn




    --
    Rami Chowdhury
    "Never attribute to malice that which can be attributed to stupidity" --
    Hanlon's Razor
    408-597-7068 (US) / 07875-841-046 (UK) / 0189-245544 (BD)
     
    Rami Chowdhury, Sep 1, 2009
    #7
  8. kj

    kj Guest

    In <> "Rami Chowdhury" <> writes:

    >> An implication of all this is that if now I wanted to create a new
    >> module x.y.z.w, this means that the previously "leaf"-module x.y.z
    >> would become "non-leaf". In other words, I'd have to:
    >>
    >> 1. create the new directory x/y/z
    >> 2. *rename* the file x/y/z.py to x/y/z/__init__.py
    >> 3. create the file x/y/z/w.py to hold the source for the new x.y.z.w
    >> module


    >With regard to point 2 -- would it be possible to just move z.py into
    >x/y/z, and put 'from z import *' into x/y/z/__init__.py, for the same
    >effect? Or is that not a good idea?


    I think in this case what you would need is something like 'from
    ...z import *', but this does not work either; one gets the error
    "SyntaxError: 'import *' not allowed with 'from .'".

    A third solution would be to change step 2 above to this:

    2. create the *symlink* x/y/z/__init__.py --> ../z.py

    This certainly would make things clearer-looking to me. Having
    all these files called the same thing, __init__.py, but having
    completely different meaning, makes my head hurt... In contrast,
    the __init__.py *symlinks* would be there just to keep python happy;
    the real content is where I'd expect it.

    I ran a simple test of this idea and it works, but there may be
    some subtle reasons for not doing it... Or maybe it's not a good
    idea simply because it is not the Python way, and I'd agree that
    this would be reason enough to avoid it.

    kynn
     
    kj, Sep 1, 2009
    #8
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Paul  Smith
    Replies:
    0
    Views:
    735
    Paul Smith
    Nov 18, 2003
  2. Harman
    Replies:
    1
    Views:
    2,519
    Moiristo
    Jul 28, 2006
  3. David Lyon
    Replies:
    19
    Views:
    589
    David Lyon
    Apr 24, 2009
  4. Aljosa Mohorovic

    naming packages for pypi

    Aljosa Mohorovic, Jun 26, 2009, in forum: Python
    Replies:
    3
    Views:
    385
    Aljosa Mohorovic
    Jun 26, 2009
  5. Ian
    Replies:
    3
    Views:
    174
Loading...

Share This Page