Sorting and Ordering by date etc

Discussion in 'Perl Misc' started by ThePotPlants, May 31, 2004.

  1. ThePotPlants

    ThePotPlants Guest

    I have a nasty set of text data that I want to sort into order, and remove
    duplicates from.
    This would only have taken 5 seconds in SQL, but I have no idea how to do it
    in Perl.

    Data looks like so...

    -33.580333 162.601833 01/12/2003 00:01:09
    -33.579833 162.601667 01/12/2003 00:01:51
    -33.579167 162.601500 01/12/2003 00:03:09
    -33.578667 162.601333 01/12/2003 00:04:51
    -33.578667 162.601333 01/12/2003 00:05:09

    What would be the most common approach to acheive this? hash array?
    Can you insert each line of data into an array, and perform operations on
    specific parts of it?

    Any suggestions would be greatly appreciated.

    P
    ThePotPlants, May 31, 2004
    #1
    1. Advertising

  2. ThePotPlants

    ThePotPlants Guest

    Correction:

    > I have a nasty set of text data that I want to sort into DATE order, and

    remove duplicates.
    ThePotPlants, May 31, 2004
    #2
    1. Advertising

  3. Gunnar Hjalmarsson, May 31, 2004
    #3
  4. ThePotPlants

    ThePotPlants Guest

    "Gunnar Hjalmarsson" <> wrote in message
    news:...
    > ThePotPlants wrote:
    > > I have a nasty set of text data that I want to sort into order, and
    > > remove duplicates from.

    >
    > That were two FAQs.
    >
    > perldoc -q sort
    >
    > perldoc -q duplicate
    >
    > Note that you are expected to check out the FAQ before asking for help
    > here. Other useful tips about posting in this newsgroup can be found
    > at http://mail.augustmail.com/~tadmc/clpmisc/clpmisc_guidelines.html


    My apologies. Thanks for the pointer.
    ThePotPlants, May 31, 2004
    #4
  5. ThePotPlants

    Eric Bohlman Guest

    "ThePotPlants" <> wrote in
    news:47xuc.12357$:

    > I have a nasty set of text data that I want to sort into order, and
    > remove duplicates from.
    > This would only have taken 5 seconds in SQL, but I have no idea how to
    > do it in Perl.


    Others have already pointed you to Perl's reference material concerning
    duplicates and sorting. However, since you're obviously familiar with
    doing such things in SQL, you ought to know that there are several perl
    modules that will allow you to use SQL on arbitrary data, including text
    files and in-memory structures. Check out DBI (the engine-independent
    interface for working with SQL) and DBD::RAM, DBD::Anydata, and DBD::CSV
    ("drivers" for DBI that allow it to query arbitrary data).
    Eric Bohlman, May 31, 2004
    #5
  6. ThePotPlants wrote:
    >
    > I have a nasty set of text data that I want to sort into order, and remove
    > duplicates from.
    > This would only have taken 5 seconds in SQL, but I have no idea how to do it
    > in Perl.
    >
    > Data looks like so...
    >
    > -33.580333 162.601833 01/12/2003 00:01:09
    > -33.579833 162.601667 01/12/2003 00:01:51
    > -33.579167 162.601500 01/12/2003 00:03:09
    > -33.578667 162.601333 01/12/2003 00:04:51
    > -33.578667 162.601333 01/12/2003 00:05:09
    >
    > What would be the most common approach to acheive this? hash array?
    > Can you insert each line of data into an array, and perform operations on
    > specific parts of it?
    >
    > Any suggestions would be greatly appreciated.


    my @data = (
    "-33.580333 162.601833 01/12/2003 00:01:09\n",
    "-33.579833 162.601667 01/12/2003 00:01:51\n",
    "-33.579167 162.601500 01/12/2003 00:03:09\n",
    "-33.578667 162.601333 01/12/2003 00:04:51\n",
    "-33.578667 162.601333 01/12/2003 00:05:09\n",
    );

    my @sorted =
    map substr( $_, 17 ),
    sort
    map sprintf( '%s%s%s%s',
    ( m<(\d\d/\d\d)/(\d{4}) (\d\d:\d\d:\d\d)> )[ 1, 0, 2 ], $_ ),
    keys %{[ { map { $_ => 1 } @data } ]};

    print for @sorted;



    John
    --
    use Perl;
    program
    fulfillment
    John W. Krahn, May 31, 2004
    #6
  7. ThePotPlants

    ThePotPlants Guest

    "ThePotPlants" <> wrote in message
    news:47xuc.12357$...
    > I have a nasty set of text data that I want to sort into order, and remove
    > duplicates from.
    > This would only have taken 5 seconds in SQL, but I have no idea how to do

    it
    > in Perl.
    >
    > Data looks like so...
    >
    > -33.580333 162.601833 01/12/2003 00:01:09
    > -33.579833 162.601667 01/12/2003 00:01:51
    > -33.579167 162.601500 01/12/2003 00:03:09
    > -33.578667 162.601333 01/12/2003 00:04:51
    > -33.578667 162.601333 01/12/2003 00:05:09


    I have cheated.
    Imported into Access.
    Write SQL. Group by, and order output.
    Export to text file.

    I feel dirty...
    ThePotPlants, Jun 1, 2004
    #7
  8. ThePotPlants

    Uri Guttman Guest

    >>>>> "T" == ThePotPlants <> writes:

    T> I have cheated.
    T> Imported into Access.
    T> Write SQL. Group by, and order output.
    T> Export to text file.

    T> I feel dirty...

    please go wash your hands and branes. now!

    uri

    --
    Uri Guttman ------ -------- http://www.stemsystems.com
    --Perl Consulting, Stem Development, Systems Architecture, Design and Coding-
    Search or Offer Perl Jobs ---------------------------- http://jobs.perl.org
    Uri Guttman, Jun 1, 2004
    #8
  9. ThePotPlants <> wrote:

    > I have a nasty set of text data that I want to sort into order, and remove
    > duplicates from.



    Please check the Perl FAQ *before* posting to the Perl newsgroup!

    perldoc -q sort

    perldoc -q duplicate


    > This would only have taken 5 seconds in SQL, but I have no idea how to do it

    ^^
    > in Perl.



    Me either because you have left the "it" unspecified AFAICT.

    Maybe you want to sort by date?

    The Subject says sort by date, but your sample data are all
    the same date, so that isn't it.

    Maybe you want to sort by time?

    Nope, your sample data is already sorted by time.

    If you tell us what you want done, it will be much easier for us
    to help you get it done...


    > Data looks like so...
    >
    > -33.580333 162.601833 01/12/2003 00:01:09
    > -33.579833 162.601667 01/12/2003 00:01:51
    > -33.579167 162.601500 01/12/2003 00:03:09
    > -33.578667 162.601333 01/12/2003 00:04:51
    > -33.578667 162.601333 01/12/2003 00:05:09
    >
    > What would be the most common approach to acheive this?



    That depends on what your "this" is and you haven't shared that.

    What columns do you want to sort by?

    What columns do you want to uniqify?


    > Can you insert each line of data into an array, and perform operations on
    > specific parts of it?



    Sure. By using an "array slice" see perlsyn.pod.


    > Any suggestions would be greatly appreciated.



    Since you haven't specified a problem that we can solve, I'll make
    one up. :)

    Sort by time column, forget about finding duplicates.

    Adapting the Schwartzian Transform code given in the answer
    to one of your Frequently Asked Questions:

    -----------------------------------
    #!/usr/bin/perl
    use warnings;
    use strict;

    my @records = <DATA>;

    my @sorted = map { $_->[0] }
    sort { $a->[1] cmp $b->[1] }
    map { [ $_, (split)[3] ] } @records;

    print for @sorted;


    __DATA__
    -33.579167 162.601500 01/12/2003 00:03:09
    -33.579833 162.601667 01/12/2003 00:01:51
    -33.578667 162.601333 01/12/2003 00:04:51
    -33.578667 162.601333 01/12/2003 00:05:09
    -33.580333 162.601833 01/12/2003 00:01:09
    -----------------------------------


    --
    Tad McClellan SGML consulting
    Perl programming
    Fort Worth, Texas
    Tad McClellan, Jun 1, 2004
    #9
  10. ThePotPlants <> wrote:

    > Write SQL.



    Maybe someone could translate the SQL sorting into Perl sorting for you.

    They would need to _see_ the SQL for that of course... (hint)


    --
    Tad McClellan SGML consulting
    Perl programming
    Fort Worth, Texas
    Tad McClellan, Jun 1, 2004
    #10
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Craig Douglas

    Ordering FileInfo[] by date

    Craig Douglas, Jan 30, 2004, in forum: ASP .Net
    Replies:
    0
    Views:
    352
    Craig Douglas
    Jan 30, 2004
  2. Peter Grison

    Date, date date date....

    Peter Grison, May 28, 2004, in forum: Java
    Replies:
    10
    Views:
    3,243
    Michael Borgwardt
    May 30, 2004
  3. Kevin Walzer

    Re: PIL (etc etc etc) on OS X

    Kevin Walzer, Aug 1, 2008, in forum: Python
    Replies:
    4
    Views:
    383
    Fredrik Lundh
    Aug 13, 2008
  4. nbigaouette

    Z-Ordering (Morton ordering) question

    nbigaouette, Nov 5, 2009, in forum: C Programming
    Replies:
    2
    Views:
    2,127
  5. TomT

    Ordering by date ???

    TomT, Aug 2, 2004, in forum: ASP General
    Replies:
    0
    Views:
    89
Loading...

Share This Page