Measuring bytes of packet sent from python application

Discussion in 'Python' started by Kangkook Jee, Jan 5, 2009.

  1. Kangkook Jee

    Kangkook Jee Guest

    Hi, all

    I'd like to measure number of bytes sent(or recv'd) from my python
    application. Does anyone have any idea how can I achieve this?

    I tried to do this by tracing some socket calls (send, sendto, sendAll)
    using 'metaclass' but I could find exactly place that I can put this in.


    My application runs some number of protocols (bittorrent, xmlrpc ..) in
    it and will be measured for a couple of hours.

    Thanks a lot for your help, in advance
    /Kangkook
    Kangkook Jee, Jan 5, 2009
    #1
    1. Advertising

  2. On Jan 5, 2:26 pm, Kangkook Jee <> wrote:
    > I'd like to measure number of bytes sent(or recv'd) from my python
    > application. Does anyone have any idea how can I achieve this?
    >
    > I tried to do this by tracing some socket calls (send, sendto, sendAll)
    >   using 'metaclass' but I could find exactly place that I can put this in.
    >
    > My application runs some number of protocols (bittorrent, xmlrpc ..) in
    > it and will be measured for a couple of hours.
    >


    A good universal tool on the Linux platform is tcpdump. It takes some
    learning, but is very useful for this kind of task. You can use a tool
    like ethereal to visualize the data that tcpdump gathers.
    Jonathan Gardner, Jan 5, 2009
    #2
    1. Advertising

  3. Kangkook Jee

    Kangkook Jee Guest

    Jonathan Gardner wrote:
    > On Jan 5, 2:26 pm, Kangkook Jee <> wrote:
    >> I'd like to measure number of bytes sent(or recv'd) from my python
    >> application. Does anyone have any idea how can I achieve this?
    >>
    >> I tried to do this by tracing some socket calls (send, sendto, sendAll)
    >> using 'metaclass' but I could find exactly place that I can put this in.
    >>
    >> My application runs some number of protocols (bittorrent, xmlrpc ..) in
    >> it and will be measured for a couple of hours.
    >>

    >
    > A good universal tool on the Linux platform is tcpdump. It takes some
    > learning, but is very useful for this kind of task. You can use a tool
    > like ethereal to visualize the data that tcpdump gathers.


    Thanks a lot Jonathan

    That seems like a good solution for my issue but how can I distinguish
    traffics from my application to others?

    I'm still struggling to solve it within python process since it looks
    cleaner but it doesn't seems to be easy at all.
    Kangkook Jee, Jan 5, 2009
    #3
  4. On Jan 5, 3:23 pm, Kangkook Jee <> wrote:
    > Jonathan Gardner wrote:
    > > A good universal tool on the Linux platform is tcpdump. It takes some
    > > learning, but is very useful for this kind of task. You can use a tool
    > > like ethereal to visualize the data that tcpdump gathers.

    >
    > That seems like a good solution for my issue but how can I distinguish
    > traffics from my application to others?
    >


    There are a variety of ways to distinguish traffic with tcpdump and
    ethereal. Usually you should have some idea of what ports or hosts
    your traffic is going to. If not, then you will have some indicator
    within the packets themselves. Note that tcpdump is remarkable since
    it can identify TCP sessions, and not just individual packets.

    > I'm still struggling to solve it within python process since it looks
    > cleaner but it doesn't seems to be easy at all.


    Replacing a lower-level component of a 3rd party library is difficult
    at best. If you can get it to work without rewriting the library,
    congratulations. At the very least, you'll probably have to do some
    low-level things within the libraries themselves.
    Jonathan Gardner, Jan 5, 2009
    #4
  5. Kangkook Jee

    James Mills Guest

    On Tue, Jan 6, 2009 at 8:26 AM, Kangkook Jee <> wrote:
    > I'd like to measure number of bytes sent(or recv'd) from my python
    > application. Does anyone have any idea how can I achieve this?
    >
    > I tried to do this by tracing some socket calls (send, sendto, sendAll)
    > using 'metaclass' but I could find exactly place that I can put this in.
    >
    >
    > My application runs some number of protocols (bittorrent, xmlrpc ..) in
    > it and will be measured for a couple of hours.
    >
    > Thanks a lot for your help, in advance


    Something like this perhaps ? Example code
    follows ... NB: This uses circuits (1)

    $ ./test.py smtp-int.vision6.com.au 25
    Trying smtp-int.vision6.com.au...
    Connected to smtp-int.vision6.com.au
    220 marvin.vision6.com.au ESMTP
    EHLO localhost
    250-marvin.vision6.com.au
    250-AUTH LOGIN CRAM-MD5 PLAIN
    250-AUTH=LOGIN CRAM-MD5 PLAIN
    250-PIPELINING
    250 8BITMIME
    QUIT
    221 marvin.vision6.com.au
    Connection closed
    Traffic
    In: 179 bytes
    Out: 20 bytes

    ---------------------------------------------------------
    http://codepad.org/V32fsqKt

    Note the Stats component on line 60.
    ---------------------------------------------------------

    cheers
    James
    James Mills, Jan 5, 2009
    #5
  6. On Mon, 05 Jan 2009 18:23:10 -0500, Kangkook Jee wrote:

    > Jonathan Gardner wrote:
    >> On Jan 5, 2:26 pm, Kangkook Jee <> wrote:
    >>> I'd like to measure number of bytes sent(or recv'd) from my python
    >>> application. Does anyone have any idea how can I achieve this?
    >>>
    >>> I tried to do this by tracing some socket calls (send, sendto,
    >>> sendAll)
    >>> using 'metaclass' but I could find exactly place that I can put this
    >>> in.
    >>>
    >>> My application runs some number of protocols (bittorrent, xmlrpc ..)
    >>> in it and will be measured for a couple of hours.
    >>>
    >>>

    >> A good universal tool on the Linux platform is tcpdump. It takes some
    >> learning, but is very useful for this kind of task. You can use a tool
    >> like ethereal to visualize the data that tcpdump gathers.

    >
    > Thanks a lot Jonathan
    >
    > That seems like a good solution for my issue but how can I distinguish
    > traffics from my application to others?
    >
    > I'm still struggling to solve it within python process since it looks
    > cleaner but it doesn't seems to be easy at all.



    A relatively straightforward, if not totally accurate, way of doing that
    is to create a layer between your application code and the network
    libraries, and have the layer measure the amount of data you send before
    it reaches the specific protocols being used.

    Naturally this won't give you a totally accurate count of network traffic
    sent from your application, since every protocol includes some overhead,
    but that might not matter for your use-case.



    --
    Steven
    Steven D'Aprano, Jan 6, 2009
    #6
  7. Kangkook Jee

    Roger Binns Guest

    -----BEGIN PGP SIGNED MESSAGE-----
    Hash: SHA1

    Kangkook Jee wrote:
    > That seems like a good solution for my issue but how can I distinguish
    > traffics from my application to others?


    I use nethogs on Ubuntu. If you use Intrepid, you can press 'm' to make
    it change amongst different displays (eg cumulative traffic,
    send/receive rates etc).

    Roger
    -----BEGIN PGP SIGNATURE-----
    Version: GnuPG v1.4.9 (GNU/Linux)

    iEYEARECAAYFAklizQYACgkQmOOfHg372QQc2gCgiMCeY1X9Qd/Fyh77SuDUNPqx
    7lkAn1ozstsf2542+OuFuBC54vtOaaUR
    =qSxN
    -----END PGP SIGNATURE-----
    Roger Binns, Jan 6, 2009
    #7
  8. On Jan 5, 6:08 pm, Grant Edwards <> wrote:
    > On 2009-01-05, Kangkook Jee <> wrote:
    > > I'm still struggling to solve it within python process since
    > > it looks cleaner but it doesn't seems to be easy at all.

    >
    > I don't why adding bunches of code to your app would be
    > "cleaner" than gathering data using external programs.
    >


    Those external programs may not be available on all the platforms he
    might want to run in.

    Or he may have some specific requirements that those tools can't
    satisfy.

    Software reuse is wonderful when it works. But it doesn't work all the
    time, even though it probably should.
    Jonathan Gardner, Jan 9, 2009
    #8
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Andy M
    Replies:
    1
    Views:
    473
    Rutger Smit
    Sep 6, 2004
  2. William Connery
    Replies:
    0
    Views:
    260
    William Connery
    Dec 5, 2006
  3. Li Han
    Replies:
    2
    Views:
    501
    bobicanprogram
    Feb 9, 2009
  4. Jeff
    Replies:
    3
    Views:
    333
    Frédéric H
    Jun 4, 2009
  5. Gelonida N
    Replies:
    4
    Views:
    868
    Gelonida N
    Sep 11, 2011
Loading...

Share This Page