duplicates

Discussion in 'Perl Misc' started by julia_2683@hotmail.com, May 29, 2007.

  1. Guest

    I would like to find and delete duplicates to reduce the number of
    entries I have for each word. Need your help to achieve that.
    Input
    dog -> doggy
    dog -> dogs
    want -> wants
    want -> wanting
    want -> wanted
    eat -> eaten
    eat -> eating
    eat -> eated
    output
    dog -> doggy dogs
    want -> wants wanting wanted.

    Thanks
    , May 29, 2007
    #1
    1. Advertising

  2. Paul Lalli Guest

    On May 29, 1:33 pm, wrote:
    > I would like to find and delete duplicates to reduce the number of
    > entries I have for each word. Need your help to achieve that.
    > Input
    > dog -> doggy
    > dog -> dogs
    > want -> wants
    > want -> wanting
    > want -> wanted
    > eat -> eaten
    > eat -> eating
    > eat -> eated
    > output
    > dog -> doggy dogs
    > want -> wants wanting wanted.


    What have you tried so far? How did it not meet your expectations?
    >From where does this data come, and in what structure do you have it

    stored? Have you searched the FAQ?

    $ perldoc -q duplicate
    Found in /opt2/Perl5_8_4/lib/perl5/5.8.4/pod/perlfaq4.pod
    How can I remove duplicate elements from a list or array?


    Paul Lalli
    Paul Lalli, May 29, 2007
    #2
    1. Advertising

  3. wrote:
    > I would like to find and delete duplicates to reduce the number of
    > entries I have for each word. Need your help to achieve that.
    > Input
    > dog -> doggy
    > dog -> dogs
    > want -> wants
    > want -> wanting
    > want -> wanted
    > eat -> eaten
    > eat -> eating
    > eat -> eated
    > output
    > dog -> doggy dogs
    > want -> wants wanting wanted.
    >
    > Thanks
    >

    put into a hash
    if exists
    $hash{dog} .= " dogs";
    Billy N. Patton, May 29, 2007
    #3
  4. Guest

    On May 29, 10:33 am, wrote:
    > I would like to find and delete duplicates to reduce the number of
    > entries I have for each word. Need your help to achieve that.


    How about this:

    #!/usr/bin/perl

    use strict;
    use warnings;

    my %word;
    while (<DATA>) {
    chomp;
    my ($word1, $word2) = split (/ -> /, $_);
    push @{$word{$word1}}, $word2;
    }
    print($_," -> ",join(" ",@{$word{$_}}),"\n") for keys %word;

    __DATA__
    dog -> doggy
    dog -> dogs
    want -> wants
    want -> wanting
    want -> wanted
    eat -> eaten
    eat -> eating
    eat -> eated

    OUTPUT
    eat -> eaten eating eated
    want -> wants wanting wanted
    dog -> doggy dogs


    >> eat -> eated


    eated???


    --
    The best way to get a good answer is to ask a good question.
    David Filmer (http://DavidFilmer.com)
    , May 29, 2007
    #4
  5. On Tue, 29 May 2007 11:38:42 -0700, Jim Gibson
    <> wrote:

    >Here's a fish:


    What's of the elitist clpmisc attitude? ;-)


    Michele
    --
    {$_=pack'B8'x25,unpack'A8'x32,$a^=sub{pop^pop}->(map substr
    (($a||=join'',map--$|x$_,(unpack'w',unpack'u','G^<R<Y]*YB='
    ..'KYU;*EVH[.FHF2W+#"\Z*5TI/ER<Z`S(G.DZZ9OX0Z')=~/./g)x2,$_,
    256),7,249);s/[^\w,]/ /g;$ \=/^J/?$/:"\r";print,redo}#JAPH,
    Michele Dondi, May 29, 2007
    #5
  6. Brad Baxter Guest

    On May 29, 4:08 pm, Michele Dondi <> wrote:
    > On Tue, 29 May 2007 11:38:42 -0700, Jim Gibson
    >
    > <> wrote:
    > >Here's a fish:

    >
    > What's of the elitist clpmisc attitude? ;-)


    s/of/with/;
    -- elitist (alleged) English speaker

    --
    Brad
    Brad Baxter, May 29, 2007
    #6
  7. <> wrote:
    > On May 29, 10:33 am, wrote:


    >>> eat -> eated

    >
    > eated???



    I think that's when something is warmed up in England...


    --
    Tad McClellan SGML consulting
    Perl programming
    Fort Worth, Texas
    Tad McClellan, May 30, 2007
    #7
  8. Brad Baxter <> wrote:
    > On May 29, 4:08 pm, Michele Dondi <> wrote:
    >> On Tue, 29 May 2007 11:38:42 -0700, Jim Gibson
    >>
    >> <> wrote:
    >> >Here's a fish:

    >>
    >> What's of the elitist clpmisc attitude? ;-)

    >
    > s/of/with/;



    Due to the smiley, I think

    s/of/become of/;

    was the intent.


    > -- elitist (alleged) English speaker



    I only speak 'merican though, so I may be way off base...


    --
    Tad McClellan SGML consulting
    Perl programming
    Fort Worth, Texas
    Tad McClellan, May 30, 2007
    #8
  9. jeevs Guest

    On May 29, 11:28 pm, wrote:
    > On May 29, 10:33 am, wrote:
    >
    > > I would like to find and delete duplicates to reduce the number of
    > > entries I have for each word. Need your help to achieve that.

    >
    > How about this:
    >
    > #!/usr/bin/perl
    >
    > use strict;
    > use warnings;
    >
    > my %word;
    > while (<DATA>) {
    > chomp;
    > my ($word1, $word2) = split (/ -> /, $_);
    > push @{$word{$word1}}, $word2;
    > }
    > print($_," -> ",join(" ",@{$word{$_}}),"\n") for keys %word;
    >
    > __DATA__
    > dog -> doggy
    > dog -> dogs
    > want -> wants
    > want -> wanting
    > want -> wanted
    > eat -> eaten
    > eat -> eating
    > eat -> eated
    >
    > OUTPUT
    > eat -> eaten eating eated
    > want -> wants wanting wanted
    > dog -> doggy dogs
    >
    > >> eat -> eated

    >
    > eated???
    >
    > --
    > The best way to get a good answer is to ask a good question.
    > David Filmer (http://DavidFilmer.com)



    Hi David .. Sorry but i dint follow the syntax you used

    like you used

    push (@{$hash{$key}}, $value);

    How does this work.. I mean i can understand when one uses @{$arr_ref}
    to derefenciate an array ref.
    but what does @{$hash{$key}} stands for... IF you can point me to some
    data or link that will be fine too...
    jeevs, May 30, 2007
    #9
  10. Paul Lalli Guest

    On May 30, 2:55 am, jeevs <> wrote:
    > On May 29, 11:28 pm, wrote:
    > > my %word;
    > > while (<DATA>) {
    > > chomp;
    > > my ($word1, $word2) = split (/ -> /, $_);
    > > push @{$word{$word1}}, $word2;
    > > }
    > > print($_," -> ",join(" ",@{$word{$_}}),"\n") for keys %word;


    > Hi David .. Sorry but i dint follow the syntax you used
    >
    > like you used
    >
    > push (@{$hash{$key}}, $value);
    >
    > How does this work.. I mean i can understand when one uses @{$arr_ref}
    > to derefenciate an array ref.
    > but what does @{$hash{$key}} stands for...


    It's exactly the same thing - @{REF} dereferences the array reference
    REF. In @{$arr_ref}, $arr_ref is the array reference. In
    @{$hash{$key}}, $hash{$key} is the array reference. That is, the
    value of %hash whose key is $key is a reference to an array. Hashes
    and arrays can store *any* kind of scalar variable - integers, floats,
    strings, and references too.

    > IF you can point me to some
    > data or link that will be fine too...


    perldoc perllol
    perldoc perldsc

    Paul Lalli
    Paul Lalli, May 30, 2007
    #10
  11. jeevs <> wrote:

    > Sorry but i dint follow the syntax you used
    >
    > like you used
    >
    > push (@{$hash{$key}}, $value);
    >
    > How does this work.. I mean i can understand when one uses @{$arr_ref}
    > to derefenciate an array ref.
    > but what does @{$hash{$key}} stands for...



    It stands for the same thing, only the array ref is stored
    in $hash{$key} rather than in $arr_ref.


    > IF you can point me to some
    > data or link that will be fine too...



    perldoc perlreftut


    --
    Tad McClellan SGML consulting
    Perl programming
    Fort Worth, Texas
    Tad McClellan, May 30, 2007
    #11
  12. On 29 May 2007 23:55:16 -0700, jeevs <> wrote:

    >push (@{$hash{$key}}, $value);
    >
    >How does this work.. I mean i can understand when one uses @{$arr_ref}
    >to derefenciate an array ref.
    >but what does @{$hash{$key}} stands for... IF you can point me to some


    It's the same as

    my $arr_ref=$hash{$key};
    push @{$arr_ref}, $value;

    >data or link that will be fine too...


    perldoc perlref

    is the starting point.


    Michele
    --
    {$_=pack'B8'x25,unpack'A8'x32,$a^=sub{pop^pop}->(map substr
    (($a||=join'',map--$|x$_,(unpack'w',unpack'u','G^<R<Y]*YB='
    ..'KYU;*EVH[.FHF2W+#"\Z*5TI/ER<Z`S(G.DZZ9OX0Z')=~/./g)x2,$_,
    256),7,249);s/[^\w,]/ /g;$ \=/^J/?$/:"\r";print,redo}#JAPH,
    Michele Dondi, May 30, 2007
    #12
  13. On Tue, 29 May 2007 20:58:25 -0500, Tad McClellan
    <> wrote:

    >>> What's of the elitist clpmisc attitude? ;-)

    >>
    >> s/of/with/;

    >
    >
    >Due to the smiley, I think
    >
    > s/of/become of/;
    >
    >was the intent.


    Yep, that's what I meant.

    >> -- elitist (alleged) English speaker

    >
    >
    >I only speak 'merican though, so I may be way off base...


    Ah wid huv thought ye wir scattish. (Fae Alba, ken?)


    Michele
    --
    {$_=pack'B8'x25,unpack'A8'x32,$a^=sub{pop^pop}->(map substr
    (($a||=join'',map--$|x$_,(unpack'w',unpack'u','G^<R<Y]*YB='
    ..'KYU;*EVH[.FHF2W+#"\Z*5TI/ER<Z`S(G.DZZ9OX0Z')=~/./g)x2,$_,
    256),7,249);s/[^\w,]/ /g;$ \=/^J/?$/:"\r";print,redo}#JAPH,
    Michele Dondi, May 30, 2007
    #13
  14. Guest

    On May 29, 11:55 pm, jeevs <> wrote:
    > Hi David .. Sorry but i dint follow the syntax you used
    > like you used
    >
    > push (@{$hash{$key}}, $value);
    >
    > How does this work.. I mean i can understand when one uses @{$arr_ref}
    > to derefenciate an array ref.
    > but what does @{$hash{$key}} stands for...


    $hash{$key} has a value, which must (if defined) be a scalar. In this
    case, the scalar value is an array reference, so we are creating a
    hash of arrays (HoA), or, more specifically (and correctly), a hash of
    array references.

    perldoc perlreftut
    perldoc perlref


    --
    The best way to get a good answer is to ask a good question.
    David Filmer (http://DavidFilmer.com)
    , May 30, 2007
    #14
  15. Bart Lateur Guest

    wrote:

    >I would like to find and delete duplicates to reduce the number of
    >entries I have for each word. Need your help to achieve that.
    >Input
    >dog -> doggy
    >dog -> dogs
    >want -> wants
    >want -> wanting
    >want -> wanted
    >eat -> eaten
    >eat -> eating
    >eat -> eated
    >output
    >dog -> doggy dogs
    >want -> wants wanting wanted.


    What you appear to be after, is finding the stem of words. I believe the
    author of the search engine index module KinoSearch, Marvin Humphrey AKA
    creamygoodness, also supports a stemmer module on CPAN, which BTW gets
    used by the indexer -- but you can use it independently, too.

    Home page: http://www.rectangular.com/kinosearch/
    CPAN page: http://search.cpan.org/dist/KinoSearch/

    The basic stemmer module is Lingua::Stem::Snowball,
    http://search.cpan.org/perldoc?Lingua::Stem::Snowball

    --
    Bart.
    Bart Lateur, May 30, 2007
    #15
  16. On Wed, 30 May 2007 15:12:26 -0700, Jim Gibson
    <> wrote:

    >> >Here's a fish:

    >>
    >> What's of the elitist clpmisc attitude? ;-)

    >
    >I wasn't trying to be elitist. Some regulars on clpm frown on simply


    I know. In fact I was "complaining" you weren't. The rationale being
    that clpmisc regulars do not give fishes, period! ;-)

    >My apologies if I offended anybody.


    No, you didn't!


    Michele
    --
    {$_=pack'B8'x25,unpack'A8'x32,$a^=sub{pop^pop}->(map substr
    (($a||=join'',map--$|x$_,(unpack'w',unpack'u','G^<R<Y]*YB='
    ..'KYU;*EVH[.FHF2W+#"\Z*5TI/ER<Z`S(G.DZZ9OX0Z')=~/./g)x2,$_,
    256),7,249);s/[^\w,]/ /g;$ \=/^J/?$/:"\r";print,redo}#JAPH,
    Michele Dondi, May 30, 2007
    #16
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. William F. Robertson, Jr.

    Re: Removing duplicates from a DropdownList

    William F. Robertson, Jr., Aug 4, 2003, in forum: ASP .Net
    Replies:
    1
    Views:
    573
    brian richards
    Aug 4, 2003
  2. makthar
    Replies:
    0
    Views:
    359
    makthar
    Aug 4, 2003
  3. brian richards

    Re: Removing duplicates from a DropdownList

    brian richards, Aug 4, 2003, in forum: ASP .Net
    Replies:
    0
    Views:
    358
    brian richards
    Aug 4, 2003
  4. S. Justin Gengo

    Re: Removing duplicates from a DropdownList

    S. Justin Gengo, Aug 4, 2003, in forum: ASP .Net
    Replies:
    0
    Views:
    1,411
    S. Justin Gengo
    Aug 4, 2003
  5. brian richards

    Re: Removing duplicates from a DropdownList

    brian richards, Aug 4, 2003, in forum: ASP .Net
    Replies:
    1
    Views:
    8,725
    Marina
    Aug 4, 2003
Loading...

Share This Page