Best way to handle changing list of class imports

Discussion in 'Python' started by Scott Grant, Oct 10, 2009.

  1. Scott Grant

    Scott Grant Guest

    Hi there,

    I'd like to set up a framework in which I can add or remove new
    classes of a given expected subclass to my package, and have the
    system load that set at runtime and be able to use them. In essence,
    if I have a class X, and subclasses A, B, and C that derive from X,
    what's the most pythonic way to allow me to add or remove A, B, or C
    from the package before runtime, to iterate over all known imported
    classes that inherit from X, while minimizing the modifications to my
    existing program.

    Is the best way to set up a directory in my package for these classes,
    and to define the list of active imports in the __all__ list in
    __init__.py? Or is there a better way?

    For detail, I'm writing a system that students will be able to submit
    game strategies that will compete against each other in a framework
    that passes them game state and expects moves in return. It would be
    great to be able to add or remove these player strategies as new ones
    come in, but I don't want to add a bunch of overhead importing each
    one specifically in the game manager itself.

    Thanks for your time,
    Scott
     
    Scott Grant, Oct 10, 2009
    #1
    1. Advertising

  2. Scott Grant schrieb:
    > Hi there,
    >
    > I'd like to set up a framework in which I can add or remove new
    > classes of a given expected subclass to my package, and have the
    > system load that set at runtime and be able to use them. In essence,
    > if I have a class X, and subclasses A, B, and C that derive from X,
    > what's the most pythonic way to allow me to add or remove A, B, or C
    > from the package before runtime, to iterate over all known imported
    > classes that inherit from X, while minimizing the modifications to my
    > existing program.
    >
    > Is the best way to set up a directory in my package for these classes,
    > and to define the list of active imports in the __all__ list in
    > __init__.py? Or is there a better way?
    >
    > For detail, I'm writing a system that students will be able to submit
    > game strategies that will compete against each other in a framework
    > that passes them game state and expects moves in return. It would be
    > great to be able to add or remove these player strategies as new ones
    > come in, but I don't want to add a bunch of overhead importing each
    > one specifically in the game manager itself.


    I think you re-think your approach and kind of invert it. Instead of
    viewing your system as game-strategies plugged into the framework, make
    a game-strategy *use* your framework.

    Thus what your students deliver must be an executable script that sets
    up the game, and then passes the own strategy into it.

    Diez
     
    Diez B. Roggisch, Oct 10, 2009
    #2
    1. Advertising

  3. Scott Grant

    Scott Grant Guest

    On Oct 10, 2:42 pm, "Diez B. Roggisch" <> wrote:
    > Scott Grant schrieb:
    >
    >
    >
    > > Hi there,

    >
    > > I'd like to set up a framework in which I can add or remove new
    > > classes of a given expected subclass to my package, and have the
    > > system load that set at runtime and be able to use them.  In essence,
    > > if I have a class X, and subclasses A, B, and C that derive from X,
    > > what's the most pythonic way to allow me to add or remove A, B, or C
    > > from the package before runtime, to iterate over all known imported
    > > classes that inherit from X, while minimizing the modifications to my
    > > existing program.

    >
    > > Is the best way to set up a directory in my package for these classes,
    > > and to define the list of active imports in the __all__ list in
    > > __init__.py?  Or is there a better way?

    >
    > > For detail, I'm writing a system that students will be able to submit
    > > game strategies that will compete against each other in a framework
    > > that passes them game state and expects moves in return.  It would be
    > > great to be able to add or remove these player strategies as new ones
    > > come in, but I don't want to add a bunch of overhead importing each
    > > one specifically in the game manager itself.

    >
    > I think you re-think your approach and kind of invert it. Instead of
    > viewing your system as game-strategies plugged into the framework, make
    > a game-strategy *use* your framework.
    >
    > Thus what your students deliver must be an executable script that sets
    > up the game, and then passes the own strategy into it.
    >
    > Diez


    Part of the benefit of the original approach (and the main reason I
    want to use it that way) is that in multi-player or competitive games,
    these strategies should be able to compete against each other. It it
    was a rock-paper-scissors system, I'd like strategy A to compete
    against strategy B, and with large sets of strategies, to allow my
    system to set up a crosstable with the minimal amount of hardcoding.
    I'd like to know which strategy is best by setting up a competition
    against all other submissions.
     
    Scott Grant, Oct 10, 2009
    #3
  4. Scott Grant

    Dave Angel Guest

    Scott Grant wrote:
    > On Oct 10, 2:42 pm, "Diez B. Roggisch" <> wrote:
    >
    >> Scott Grant schrieb:
    >>
    >>
    >>
    >>
    >>> Hi there,
    >>>
    >>> I'd like to set up a framework in which I can add or remove new
    >>> classes of a given expected subclass to my package, and have the
    >>> system load that set at runtime and be able to use them. In essence,
    >>> if I have a class X, and subclasses A, B, and C that derive from X,
    >>> what's the most pythonic way to allow me to add or remove A, B, or C
    >>> from the package before runtime, to iterate over all known imported
    >>> classes that inherit from X, while minimizing the modifications to my
    >>> existing program.
    >>>
    >>> Is the best way to set up a directory in my package for these classes,
    >>> and to define the list of active imports in the __all__ list in
    >>> __init__.py? Or is there a better way?
    >>>
    >>> For detail, I'm writing a system that students will be able to submit
    >>> game strategies that will compete against each other in a framework
    >>> that passes them game state and expects moves in return. It would be
    >>> great to be able to add or remove these player strategies as new ones
    >>> come in, but I don't want to add a bunch of overhead importing each
    >>> one specifically in the game manager itself.
    >>>

    >> I think you re-think your approach and kind of invert it. Instead of
    >> viewing your system as game-strategies plugged into the framework, make
    >> a game-strategy *use* your framework.
    >>
    >> Thus what your students deliver must be an executable script that sets
    >> up the game, and then passes the own strategy into it.
    >>
    >> Diez
    >>

    >
    > Part of the benefit of the original approach (and the main reason I
    > want to use it that way) is that in multi-player or competitive games,
    > these strategies should be able to compete against each other. It it
    > was a rock-paper-scissors system, I'd like strategy A to compete
    > against strategy B, and with large sets of strategies, to allow my
    > system to set up a crosstable with the minimal amount of hardcoding.
    > I'd like to know which strategy is best by setting up a competition
    > against all other submissions.
    >
    >

    I understand your goal to have several (or all) of the students'
    strategy files present, and competing. But I have to ask whether you
    wouldn't be better off launching each student in a separate process (or
    separate machine, even), and competing over sockets or somesuch. Two
    reasons come to mind: 1) students have a way of working *around* a
    problem, and sabotage (intended or not) is a definite possibility. 2)
    scalability to larger numbers of strategies.

    But I'll assume you've already considered and dismissed this kind of
    approach. So essentially you're building an extensible framework, where
    parts of it are "discovered" at load time, and automatically merged in.

    For that, I'd suggest reserving a directory at a known location, doing
    an os.path.dirname() on that directory, and building a list of module
    names. Then use __import__() to load them, and build a list of module
    objects, and a list of classes in those modules. Suggest classname to
    be simply formed by uppercasing the first letter of the module file.

    DaveA
     
    Dave Angel, Oct 10, 2009
    #4
  5. Scott Grant

    Scott Grant Guest

    On Oct 10, 6:44 pm, Dave Angel <> wrote:
    > Scott Grant wrote:
    > > On Oct 10, 2:42 pm, "Diez B. Roggisch" <> wrote:

    >
    > >> Scott Grant schrieb:

    >
    > >>> Hi there,

    >
    > >>> I'd like to set up a framework in which I can add or remove new
    > >>> classes of a given expected subclass to my package, and have the
    > >>> system load that set at runtime and be able to use them.  In essence,
    > >>> if I have a class X, and subclasses A, B, and C that derive from X,
    > >>> what's the most pythonic way to allow me to add or remove A, B, or C
    > >>> from the package before runtime, to iterate over all known imported
    > >>> classes that inherit from X, while minimizing the modifications to my
    > >>> existing program.

    >
    > >>> Is the best way to set up a directory in my package for these classes,
    > >>> and to define the list of active imports in the __all__ list in
    > >>> __init__.py?  Or is there a better way?

    >
    > >>> For detail, I'm writing a system that students will be able to submit
    > >>> game strategies that will compete against each other in a framework
    > >>> that passes them game state and expects moves in return.  It would be
    > >>> great to be able to add or remove these player strategies as new ones
    > >>> come in, but I don't want to add a bunch of overhead importing each
    > >>> one specifically in the game manager itself.

    >
    > >> I think you re-think your approach and kind of invert it. Instead of
    > >> viewing your system as game-strategies plugged into the framework, make
    > >> a game-strategy *use* your framework.

    >
    > >> Thus what your students deliver must be an executable script that sets
    > >> up the game, and then passes the own strategy into it.

    >
    > >> Diez

    >
    > > Part of the benefit of the original approach (and the main reason I
    > > want to use it that way) is that in multi-player or competitive games,
    > > these strategies should be able to compete against each other.  It it
    > > was a rock-paper-scissors system, I'd like strategy A to compete
    > > against strategy B, and with large sets of strategies, to allow my
    > > system to set up a crosstable with the minimal amount of hardcoding.
    > > I'd like to know which strategy is best by setting up a competition
    > > against all other submissions.

    >
    > I understand your goal to have several (or all) of the students'
    > strategy files present, and competing.  But I have to ask whether you
    > wouldn't be better off launching each student in a separate process (or
    > separate machine, even), and competing over sockets or somesuch.  Two
    > reasons come to mind:  1) students have a way of working *around* a
    > problem, and sabotage (intended or not) is a definite possibility.  2)
    > scalability to larger numbers of strategies.
    >
    > But I'll assume you've already considered and dismissed this kind of
    > approach.  So essentially you're building an extensible framework, where
    > parts of it are "discovered" at load time, and automatically merged in.


    That's it, in a nutshell. Instead of having some sort of scripting
    language to author these strategies, it would be swell to just use the
    native Python code.

    Back in the day, I wrote a chess client that competed over the telnet
    sessions at freechess.org, and that was a really nice way to test out
    the code. However, the drawback was that I obviously couldn't get a
    full crosstable of my code versus every other competitor on the
    network. In this case, I'd like to just set each opponent up against
    every other, run n times, and see what the overall results are. Doing
    this locally (and additionally, allowing the competitors to test their
    results locally for themselves) is a really exciting prospect.

    > For that, I'd suggest reserving a directory at a known location, doing
    > an os.path.dirname() on that directory, and building a list of module
    > names.  Then use __import__() to load them, and build a list of module
    > objects, and a list of classes in those modules.  Suggest classname to
    > be simply formed by uppercasing the first letter of the module file.


    Wonderful, I'll also take a look at this solution. Thanks!
     
    Scott Grant, Oct 11, 2009
    #5
  6. Scott Grant

    greg Guest

    Dave Angel wrote:

    > For that, I'd suggest reserving a directory at a known location, doing
    > an os.path.dirname() on that directory, and building a list of module
    > names. Then use __import__() to load them, and build a list of module
    > objects, and a list of classes in those modules. Suggest classname to
    > be simply formed by uppercasing the first letter of the module file.


    Another possibility would be to require each student's module
    to define a factory function with a standard name, such as
    'create_strategy', so you wouldn't have to deal with a different
    class name in each module.

    Also, an alternative to using __import__() would be to use
    execfile() to load and execute the student's code. Then you
    wouldn't have to put the directory containing the students'
    modules on the import path, and they wouldn't get put into
    sys.modules, which would help to isolate them from each other.
    (Otherwise a student could cheat by importing someone else's
    stragegy class and passing it off as their own!)

    However, each student's submission would more or less be
    restricted to a single file in that case, whereas using
    __import__() would allow it to be a package with submodules.
    Whether that's a serious disadvantage depends on how big and
    complicated you expect the submissions to be.

    --
    Greg
     
    greg, Oct 12, 2009
    #6
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Replies:
    6
    Views:
    301
    Graham Dumpleton
    Dec 31, 2007
  2. Albert
    Replies:
    4
    Views:
    11,044
    Albert
    Jul 10, 2008
  3. ManicQin
    Replies:
    0
    Views:
    299
    ManicQin
    Feb 9, 2009
  4. zildjohn01
    Replies:
    0
    Views:
    695
    zildjohn01
    Feb 22, 2011
  5. Victor Hooi
    Replies:
    1
    Views:
    131
    Devin Jeanpierre
    Nov 25, 2013
Loading...

Share This Page