A good data structure to store INI files.

Discussion in 'Perl Misc' started by Marc Lucksch, Feb 10, 2009.

  1. Marc Lucksch

    Marc Lucksch Guest

    Maybe I'm an idiot here, but I can't figure this one out even if my life
    would depend on it:

    I have an file in the quite easy INI format, put I have no idea how to
    store it in a way where it can be easily accessed by other people and
    without any long explanations.

    Anyway here is the example file:

    > [Ship]
    > ids_name = 237033
    > ids_info = 66567
    > ids_info1 = 66567
    > ship_class = 1
    > nickname = li_elite
    > LODranges = 0, 75, 150, 1300
    > fuse = intermed_damage_smallship01, 0.000000, 400
    > fuse = intermed_damage_smallship02, 0.000000, 200
    > ;#Well its a lot more in one section, but this proves the point.
    >
    > [Ship]
    > ids_name = 237033
    > ids_info = 66567
    > ship_class = 1
    > nickname = li_elite2
    > LODranges = 0, 175, 150, 1300
    > fuse = intermed_damage_smallship01, 0.000000, 400
    > fuse = intermed_damage_smallship02, 0.000000, 200
    > fuse = intermed_damage_smallship03, 0.000000, 133
    > ;.... (about 250kb of this)


    The file is from the Microsoft game Freelancer, but it is unlike any
    normal ini files:

    - The section names are doubled, (there are multiple [Ship] entries)
    - Some of the keys are doubled as well (0 - 4 fuse entries for example)
    - Some of the values are seperated by ",". (That's not a big problem)

    The backstory of this whole thing is:

    In 1999 Microsoft/Digital Anvil made this game and it came with a lot of
    BINI files, which are just somewhat compressed INI files, looking like
    the one above. After half a year BINI file decompressors and compressors
    were written and I plan to write/wrote one as well for another project
    (plasma.sf.net) in Perl of course. The normal decompressors converted
    the file first and then read it, that's why tools based on this, like
    FL-Datastorm, take 10 Minutes to scan the game and my implementation
    takes about 2 seconds. (10 seconds when counting models)
    But why INI-files, It would seem abitrary to convert them to INI files,
    since it was a binary format before that look like it. However the game
    also accepts gladly uncompressed INI files.

    The game itself has many, many mods for it, since you just have to edit
    INI files mostly, and people have found a way to 'break' the BINI files
    in a way that only Freelancer can read them and not any other tools.
    This is a second reason for the parser, since I can't use most tools for
    my own server operator, since it _has_ to read those files. That is also
    the reason I can't just ship it with just a complete set of data files.

    Now while I want to put the parser on CPAN, I can't find any way to
    represent this type of data in an easy way... (While a little voice is
    constantly screaming "They are just INI files, ^*&*&%^%^*&$#$")

    So I tried the some of the many INI parsers on CPAN. (Since I also have
    to read normal ini files) But I have found none that can even parse that
    file correctly... (See below for examples)

    I would love for it to have an easy interface, not for me, but for
    others who might want to use it:

    > my $ini=read...("file.ini")
    > print $ini->{Ship}->{nickname}; #When there is one ship only, also
    > foreach my $ship(%{$ini->{Ship}) {
    > foreach my $fuse(
    > print $ship->{nickname}
    > }
    > }


    First idea was TIE, but that won't work with the duality of the thing.
    Overload might go, but that opens another can of worms with adding new
    sections and keys that need converting objects and other things.
    And overload+TIE is not good at all... (See man overload)
    And a pure Object based model is what Python and Java would do, not very
    perlish.

    So there I am, no idea, and I need help: First on the datastructure
    issue and second on a nice parser on CPAN that can read that file obove
    right. I don't want to write Yet::Another::INI::parser::Again for the
    fifth time.

    Marc "Maluku" Lucksch

    __DATA__ :)

    Config::INI::Reader of that file:
    > $VAR1 = {
    > 'Ship' => {
    > 'ship_class' => '1',
    > 'ids_info' => '66567',
    > 'ids_info1' => '66567',
    > 'nickname' => 'li_elite2',
    > 'fuse' => 'intermed_damage_smallship03, 0.000000, 133',
    > 'ids_name' => '237033',
    > 'LODranges' => '0, 175, 150, 1300'
    > }
    > };


    Result: One section missing, 4 fuses missing, fuse and LOD not splitted.

    Config::Format::INI

    > $VAR1 = {
    > 'Ship' => {
    > 'ship_class' => ['1'],
    > 'ids_info' => ['66567'],
    > 'nickname' => ['li_elite2'],
    > 'fuse' => [
    > 'intermed_damage_smallship03',
    > '0.000000',
    > '133'
    > ],
    > 'ids_name' => ['237033']
    > }
    > };


    Result: One section missing, 4 fuses missing, LODRanges and ids_info1
    not parsed (why?). The missing LODRanges confuse me a lot here.

    > $VAR1 = {
    > 'Ship' => {
    > 'ship_class' => ['1','1'],
    > 'ids_info' => ['66567','66567'],
    > 'ids_info1' => '66567',
    > 'nickname' => ['li_elite','li_elite2'],
    > 'fuse' => [
    > 'intermed_damage_smallship01, 0.000000, 400',
    > 'intermed_damage_smallship02, 0.000000, 200',
    > 'intermed_damage_smallship03, 0.000000, 133',
    > 'intermed_damage_smallship01, 0.000000, 400',
    > 'intermed_damage_smallship02, 0.000000, 200',
    > 'intermed_damage_smallship03, 0.000000, 133'
    > ],
    > 'ids_name' => ['237033','237033'],
    > 'LODranges' => [
    > '0, 75, 150, 1300',
    > '0, 175, 150, 1300'
    > ]
    > }
    > };


    Result: Well it's all there, but I can't work with it and value are not
    splitted by the comma

    And my one FlBini (Games::Freelancer::BINI, when I figure this out)..
    $VAR1 = bless( {
    > 'Iter' => 0,
    > 'Data' => [
    > 'Ship',
    > bless( {
    > 'Iter' => 0,
    > 'Data' => [
    > 'ids_name',
    > ['237033'],
    > 'ids_info',
    > ['66567'],
    > 'ids_info1',
    > ['66567'],
    > 'ship_class',
    > ['1'],
    > 'nickname',
    > ['li_elite'],
    > 'LODranges',
    > [
    > '0',
    > '75',
    > '150',
    > '1300'
    > ],
    > 'fuse',
    > [
    > 'intermed_damage_smallship01',
    > '0.000000',
    > '400'
    > ],
    > 'fuse',
    > [
    > 'intermed_damage_smallship02',
    > '0.000000',
    > '200'
    > ],
    > 'fuse',
    > [
    > 'intermed_damage_smallship03',
    > '0.000000',
    > '133'
    > ]
    > ]
    > }, 'INIArray' ),
    > 'Ship',
    > bless( {
    > 'Iter' => 0,
    > 'Data' => [
    > 'ids_name',
    > ['237033],
    > 'ids_info',
    > ['66567],
    > 'ship_class',
    > ['1'],
    > 'nickname',
    > ['li_elite2],
    > 'LODranges',
    > [
    > '0',
    > '175',
    > '150',
    > '1300'
    > ],
    > 'fuse',
    > [
    > 'intermed_damage_smallship01',
    > '0.000000',
    > '400'
    > ],
    > 'fuse',
    > [
    > 'intermed_damage_smallship02',
    > '0.000000',
    > '200'
    > ],
    > 'fuse',
    > [
    > 'intermed_damage_smallship03',
    > '0.000000',
    > '133'
    > ]
    > ]
    > }, 'INIArray' )
    > ]
    > }, 'INIArray' )


    Result: cubersome, no direct key access, therefore full scan is needed
    with a good reason. But all data is there.
     
    Marc Lucksch, Feb 10, 2009
    #1
    1. Advertising

  2. Marc Lucksch

    Ted Zlatanov Guest

    On Tue, 10 Feb 2009 18:34:41 +0100 Marc Lucksch <> wrote:

    ML> Now while I want to put the parser on CPAN, I can't find any way to
    ML> represent this type of data in an easy way... (While a little voice is
    ML> constantly screaming "They are just INI files, ^*&*&%^%^*&$#$")

    At least you're not thinking "oh, I know, I'll use XML" :)

    ML> So I tried the some of the many INI parsers on CPAN. (Since I also
    ML> have to read normal ini files) But I have found none that can even
    ML> parse that file correctly... (See below for examples)

    ML> I would love for it to have an easy interface, not for me, but for
    ML> others who might want to use it:
    ....
    ML> So there I am, no idea, and I need help: First on the datastructure
    ML> issue and second on a nice parser on CPAN that can read that file
    ML> obove right. I don't want to write Yet::Another::INI::parser::Again
    ML> for the fifth time.

    I don't know of any INI parsers that would do this for you, but I wrote
    the code below in 10 minutes and it seems to do the right thing. The
    only thing you need to configure is the list that initializes
    %comma_keys.

    You can write this in a stateful way but it's a waste of time if the
    data size is small and you don't expect the string "[Ship]" anywhere
    unexpected in the configuration.

    Ted

    #!/usr/bin/perl

    use warnings;
    use strict;
    use Data::Dumper;

    my $parsed = [];
    my %comma_keys = map { $_ => 1 } qw/fuse LODranges/;

    my $data = join '', grep { $_ !~ m/^;/ } <DATA>;

    foreach my $shipline (split '\[Ship\]', $data)
    {
    next unless $shipline;
    my $ship = {};
    push @$parsed, $ship;
    foreach my $line (split "\n", $shipline)
    {
    next unless $line;
    my ($key, $val) = split /\s*=\s*/, $line, 2;

    if (exists $comma_keys{$key})
    {
    $val = [ split /\s*,\s*/, $val ];
    }

    push @{$ship->{$key}}, $val;
    }

    foreach my $key (keys %$ship)
    {
    next if scalar @{$ship->{$key}} != 1;
    $ship->{$key} = $ship->{$key}->[0];
    }

    }

    print Dumper $parsed;
    __DATA__
    [Ship]
    ids_name = 237033
    ids_info = 66567
    ids_info1 = 66567
    ship_class = 1
    nickname = li_elite
    LODranges = 0, 75, 150, 1300
    fuse = intermed_damage_smallship01, 0.000000, 400
    fuse = intermed_damage_smallship02, 0.000000, 200
    ;#Well its a lot more in one section, but this proves the point.

    [Ship]
    ids_name = 237033
    ids_info = 66567
    ship_class = 1
    nickname = li_elite2
    LODranges = 0, 175, 150, 1300
    fuse = intermed_damage_smallship01, 0.000000, 400
    fuse = intermed_damage_smallship02, 0.000000, 200
    fuse = intermed_damage_smallship03, 0.000000, 133
    ;.... (about 250kb of this)
     
    Ted Zlatanov, Feb 10, 2009
    #2
    1. Advertising

  3. Marc Lucksch

    Marc Lucksch Guest

    Ted Zlatanov schrieb:
    > On Tue, 10 Feb 2009 18:34:41 +0100 Marc Lucksch <> wrote:
    >
    > ML> Now while I want to put the parser on CPAN, I can't find any way to
    > ML> represent this type of data in an easy way... (While a little voice is
    > ML> constantly screaming "They are just INI files, ^*&*&%^%^*&$#$")
    >
    > At least you're not thinking "oh, I know, I'll use XML" :)


    I'm quite happy Microsoft didn't use XML for this one when I look at
    ..docx files.

    > ML> So I tried the some of the many INI parsers on CPAN. (Since I also
    > ML> have to read normal ini files) But I have found none that can even
    > ML> parse that file correctly... (See below for examples)


    > I don't know of any INI parsers that would do this for you, but I wrote
    > the code below in 10 minutes and it seems to do the right thing.


    Well first off, thanks for the quick response and the nice script.

    Still leaves the problem on how the represent the data, the file is not
    only littered with 277 [Ship] entries, but also [collision group]'s, one
    [Pilot] and lots of [Simple]'s... The file I put was just an example. :(

    I want for example access the pilot this way: (Nice and easy for any
    user of this theoretical module)

    print $ini->{Pilot}->{nickname}; #Only one pilot.

    But the Ships this way:

    foreach (@{$ini->{Ships}}) {

    }
    # But with:
    print $ini->{Ships}->{nickname}; #Just print the first nickname of the
    first ship, first value.

    I can do that with overload, but how do I handle:

    $ini->{Ships}->{nickname}=[qw/li_elite li_fighter/]?:
    I should do this:
    [Ship]
    ....
    nickname=li_elite
    nickname=li_fighter
    ....
    I can't do that with overload, can I... I need to return a tied thing
    for that. But then I run into the tied/overload bug on
    print "$ini->{Ships}->{nickname}" #First entry, now tied, will return a
    tied object, BUT overload comes before tied, so it won't call '""' of
    the tied object, but from $ini->{Ships}, according to the overload manpage.

    + I want to keep the order or otherwise it will confuse diff a lot.


    The module should help people write scripts to easily extract and modify
    data in those files, without exporting them to .inis first.
    > The
    > only thing you need to configure is the list that initializes
    > %comma_keys.


    That might be a problem, since this is neither thats not the only ini
    file in there.

    There are shiparchs (this one), loadouts, equipments, universes,
    systems, bases, markets, asteroids, asteroid_fielf and a lot more files
    of those or similiar formats. (A quick search gave me 1000+ ini files)

    And I don't know the keys, lately someone discovered that you can add an
    spin = 0.1, 0, 0 to a planet in a system.ini file and the planet
    rotates. This was never seen before, maybe because they probably removed
    it in the final version. Who knows how many of those are still in there.

    But I need to save them all and store them into a database, to run
    queries on it. Well that is working nicely currently as well, but I was
    wondering for a better solution to my problem too.

    > You can write this in a stateful way but it's a waste of time if the
    > data size is small and you don't expect the string "[Ship]" anywhere
    > unexpected in the configuration.


    I don't think the parser is the big problem, mine is working nicely. I
    was just wondering if there is another one out there that can do it so I
    don't need to release yet another one.
    Since those files are ini files from Microsoft, who if not invented
    them, at least made them popular. (win.ini system.ini)..

    I just want to write the decoder/encoder to the binary format of those
    files (I heard they are being used in some other game, too)
     
    Marc Lucksch, Feb 10, 2009
    #3
  4. Marc Lucksch

    Guest

    On Tue, 10 Feb 2009 18:34:41 +0100, Marc Lucksch <> wrote:

    >Maybe I'm an idiot here, but I can't figure this one out even if my life
    >would depend on it:
    >
    >I have an file in the quite easy INI format, put I have no idea how to
    >store it in a way where it can be easily accessed by other people and
    >without any long explanations.
    >
    >Anyway here is the example file:
    >
    > > [Ship]
    > > ids_name = 237033
    > > ids_info = 66567
    > > ids_info1 = 66567
    > > ship_class = 1
    > > nickname = li_elite
    > > LODranges = 0, 75, 150, 1300
    > > fuse = intermed_damage_smallship01, 0.000000, 400
    > > fuse = intermed_damage_smallship02, 0.000000, 200
    > > ;#Well its a lot more in one section, but this proves the point.
    > >
    > > [Ship]
    > > ids_name = 237033
    > > ids_info = 66567
    > > ship_class = 1
    > > nickname = li_elite2
    > > LODranges = 0, 175, 150, 1300
    > > fuse = intermed_damage_smallship01, 0.000000, 400
    > > fuse = intermed_damage_smallship02, 0.000000, 200
    > > fuse = intermed_damage_smallship03, 0.000000, 133
    > > ;.... (about 250kb of this)

    >
    >The file is from the Microsoft game Freelancer, but it is unlike any
    >normal ini files:
    >
    >- The section names are doubled, (there are multiple [Ship] entries)
    >- Some of the keys are doubled as well (0 - 4 fuse entries for example)
    >- Some of the values are seperated by ",". (That's not a big problem)
    >

    [snip]
    >

    I don't know that Config::IniFiles will deal with duplicate sections and/or
    values correctly. My hunch is it won't. Because to write it out, ie: creating
    a new section, returns undef if it already exists.

    So, but as a check you could see the results if you parse the file with this:
    my @sections = $cfg->Sections();
    for my $sect_name (@sections)
    {
    my @values = $cfg->Parameters ($sect_name);
    for my $value (@Values)
    {
    ...
    }
    }

    Although I don't think it will parse as you want, you could easily write a regex
    parser to read in the file, then write it out using the Config::IniFiles module
    with its Group constructs. Might want to check out Groups and GroupMembers.

    Otherwise, I think win32 parses sequential (streaming) sections and values with a
    get next kind of thing.

    Maybe you should'nt use Perl. You know Perl isin't the be all and end all.

    -sln
     
    , Feb 10, 2009
    #4
  5. Marc Lucksch

    Marc Lucksch Guest

    Marc Lucksch schrieb:
    > Maybe I'm an idiot here, but I can't figure this one out even if my life
    > would depend on it:

    Maybe I posted the question wrong, it is not about the INI parsers, I
    just didn't want to release another one...

    Lets say I have this data structure, from an ini file, it cares for all
    the conditions that can happen (multiple sections, keys and values)

    my $ini={
    Ship=>[
    {#First Ship
    nickname=>[ #A key
    ['li_elite'] # value
    ],
    fuse=>[ #multikey with multivalue
    ['intermed_damage_smallship01','0.000000','400'],
    ['intermed_damage_smallship02','0.000000','200'],

    ], #....
    },{#Second Ship
    nickname=>[
    ['li_elite2']
    ],fuse=>[
    ['intermed_damage_smallship01','0.000000','400'],
    ['intermed_damage_smallship02','0.000000','200'],

    ], #....
    }, #A lot more 'Ships'
    ],Simple=>[
    #A lot of Simples
    ],ship=>[
    # 3 or 4 ships, sometime they are lowercase...
    # That's a problem with this data model,
    # Can't use Hash::Case::preserve either,
    # cause I would loose which ship's which.
    ],Pilot=>[
    {
    name=>[['pilot_corsair']],
    }#Just one pilot entry
    ],
    # lot more sections....
    }

    1. Problem:

    I can't save the order the sections or keys came in, nor the names of
    them so I can't write an unmodified file out again. I would have to tie
    that structure.
    I want to make is easily accessable:

    $ini->{Pilot}->{name} instead of:
    $ini->{Pilot}->[0]->{name}->[0]->[0].

    for (@$ini) {
    #in the order they came in, because a section can affect the
    #following one. As seen in weapon_equip.ini, where [LODsomething.]
    #before [Gun] affects the Gun LODs, I can't mix those up.
    }
    for (@{$ini->{Ships}}) {
    #Return ships in order of appearance.
    }

    But I would need overload for that, and I could save it into an array
    for the order and a hash for quick lookup.

    But I can't just return a hash with overload, because it wouldn't change
    my array on operations like

    delete $ini->{newvalue}; (Change in array missing.
    for (0 .. $#{$ini}) {
    delete $ini->[$_] if something($_); #Change in Hash missing.
    }
    Which one do I trust now, hash or array?

    or even

    delete $ini->{Pilot};
    $ini->{Pilot}=$otherpilot. #Order destroyed.

    So I would have to return a tied hash, which gives me a lot of trouble
    with overload again. (See perldoc overload, last section or so)


    .... No solution :( ...

    Either I need a better way to save it, or I'm missing something here?

    Marc "Maluku" Lucksch
     
    Marc Lucksch, Feb 10, 2009
    #5
  6. Marc Lucksch

    Guest

    On Tue, 10 Feb 2009 20:12:07 +0100, Marc Lucksch <> wrote:

    >Marc Lucksch schrieb:
    >> Maybe I'm an idiot here, but I can't figure this one out even if my life
    >> would depend on it:

    >Maybe I posted the question wrong, it is not about the INI parsers, I
    >just didn't want to release another one...
    >
    >Lets say I have this data structure, from an ini file, it cares for all
    >the conditions that can happen (multiple sections, keys and values)
    >
    >my $ini={
    > Ship=>[
    > {#First Ship
    > nickname=>[ #A key
    > ['li_elite'] # value
    > ],
    > fuse=>[ #multikey with multivalue
    > ['intermed_damage_smallship01','0.000000','400'],
    > ['intermed_damage_smallship02','0.000000','200'],
    >
    > ], #....
    > },{#Second Ship
    > nickname=>[
    > ['li_elite2']
    > ],fuse=>[
    > ['intermed_damage_smallship01','0.000000','400'],
    > ['intermed_damage_smallship02','0.000000','200'],
    >
    > ], #....
    > }, #A lot more 'Ships'
    > ],Simple=>[
    > #A lot of Simples
    > ],ship=>[

    ^^^^
    Whats this amother ship in the hash?? This won't work, you already have a
    ship.
    >
    >1. Problem:
    >
    >I can't save the order the sections or keys came in, nor the names of
    >them so I can't write an unmodified file out again. I would have to tie
    >that structure.
    >I want to make is easily accessable:
    >

    Utterly absurd, of course you can. If you want to save the order in
    which it exists in the ini file, dups an all, you have to break it up
    into compound arrays, this way dup's are irrelavent.

    Array's of references of references.
    Traversing the array, its up to you to know, based on the tag
    when to increment index+=2;

    There is no magic bullet, you have a crap structure in an ini file,
    deal with it.

    my @ini = (

    Ship=>[
    [#First Ship
    nickname=>[ #A key
    ['li_elite'] # value
    ],
    fuse=>[ #multikey with multivalue
    ['intermed_damage_smallship01','0.000000','400'],
    ['intermed_damage_smallship02','0.000000','200'],

    ], #....
    ],
    [#Second Ship
    nickname=>[
    ['li_elite2']
    ],fuse=>[
    ['intermed_damage_smallship01','0.000000','400'],
    ['intermed_damage_smallship02','0.000000','200'],

    ], #....
    ], #A lot more 'Ships'
    ],
    Simple=>[
    #A lot of Simples
    ],
    ship=>[
    # 3 or 4 ships, sometime they are lowercase...
    # That's a problem with this data model,
    # Can't use Hash::Case::preserve either,
    # cause I would loose which ship's which.
    ],
    Pilot=>[
    name=>[['pilot_corsair']],
    }#Just one pilot entry
    ],
    # lot more sections....
    );

    -sln
     
    , Feb 10, 2009
    #6
  7. Marc Lucksch

    Marc Lucksch Guest

    A problem with TIEHASH [was: A good data structure to store INI files.]

    A little addition why just TIE alone is also a no-go:

    Marc Lucksch schrieb:
    > $ini->{Pilot}->{name} instead of:
    > $ini->{Pilot}->[0]->{name}->[0]->[0].
    >
    > for (@$ini) {
    > #in the order they came in, because a section can affect the
    > #following one. As seen in weapon_equip.ini, where [LODsomething.]
    > #before [Gun] affects the Gun LODs, I can't mix those up.
    > }


    I could just say: Don't care about the shorter $ini->{Pilot}->{name}
    syntax, just let it be $ini->{Pilot}->[0]->{name}->[0]->[0].

    And just Tie the first hash into something like Tie::DxHash

    But I need the order of keys and sections while I write the file, or
    there the 'user' might need it (reasons above), there are two ways to do
    this (which I see:)

    FETCH returns an arrayref of all sections with that name, then you can
    do $ini->{Pilot}->[0]->{name}->[0]->[0].
    but:

    foreach my $key (%{$ini}) {
    # Now $key can be the same one over and over again.
    # Ship/Ship/Pilot/Ship/ship
    my $value=$ini->{$key}; #arrayref of all sections. But which one
    # did I want, the first? the seventh? I have to count the keys
    # myself.
    }
    Same with:

    while (my ($key,$value) = each(%$ini)) {
    }

    This is because FETCH gets called with the returnvalue of
    FIRSTKEY/NEXTKEY, there is no way to find out if it was values(), each()
    or $ini->{key}.
    And you can't just assume FIRSTKEY/NEXTKEY is followed by FETCH, because
    of keys().

    values() is the worst anyway: Data::Dumper would put it this way:
    $VAR1=[[],$VAR1->[0],$VAR1->[0],[name=>['foo']],$VAR1->[0],$VAR1->[0],....];


    You would need something like this, but it won't work.

    sub NEXTKEY {
    my $self=shift;
    $next=$self->get_next($_[0]); # Generate it,
    if (wantarray) { #Called by each()
    $value=$self->get($next)->[self->{count}->{$next}];
    return ($key,$value);
    }
    else { #Called from keys();
    return $next;
    }
    }

    Second way: FETCH keeps the keycounters inside, for stuff works, but
    that messes up $ini->{Pilot}->[7]->{name}->[0]->[0], because it look
    like this:
    $ini->{Pilot} for 1 .. 6;
    $ini->{Pilot}->{name}->[0]; #I would use the same tie thing for the keys
    #as well

    You would again need overload for this:

    $ini->{Pilot}->[1]->{name}
    $ini->{Pilot}->{name} (returns $ini->{Pilot}->[6]->{name}).

    But that again won't work because $ini->{Pilot}->[6]->{name} is again a
    tied object and there is that overload bug.

    Maybe someone can fix this, at least the wantarray for FISTKEY/NEXTKEY
    on each and values().

    Marc "Maluku" Lucksch
     
    Marc Lucksch, Feb 10, 2009
    #7
  8. Marc Lucksch

    Marc Lucksch Guest

    schrieb:
    > On Tue, 10 Feb 2009 20:12:07 +0100, Marc Lucksch <> wrote:
    >> 1. Problem:
    >>
    >> I can't save the order the sections or keys came in, nor the names of
    >> them so I can't write an unmodified file out again. I would have to tie
    >> that structure.
    >> I want to make is easily accessable:
    >>

    > Utterly absurd, of course you can. If you want to save the order in
    > which it exists in the ini file, dups an all, you have to break it up
    > into compound arrays, this way dup's are irrelavent.
    >
    > Array's of references of references.
    > Traversing the array, its up to you to know, based on the tag
    > when to increment index+=2;

    It certainly is the most stable (stable like in sort routines)
    structure, but see below for the problems with it.

    >
    > There is no magic bullet, you have a crap structure in an ini file,
    > deal with it.


    perls carefully designed datastructures of hash, arrays, ties defeated
    by a simple ini file?

    > my @ini = (
    >
    > Ship=>[
    > [#First Ship
    > nickname=>[ #A key
    > ['li_elite'] # value
    > ],
    > fuse=>[ #multikey with multivalue
    > ['intermed_damage_smallship01','0.000000','400'],
    > ['intermed_damage_smallship02','0.000000','200'],
    >
    > ], #....
    > ],
    > [#Second Ship
    > nickname=>[
    > ['li_elite2']
    > ],fuse=>[
    > ['intermed_damage_smallship01','0.000000','400'],
    > ['intermed_damage_smallship02','0.000000','200'],
    >
    > ], #....
    > ], #A lot more 'Ships'
    > ],
    > Simple=>[
    > #A lot of Simples
    > ],
    > ship=>[
    > # 3 or 4 ships, sometime they are lowercase...
    > # That's a problem with this data model,
    > # Can't use Hash::Case::preserve either,
    > # cause I would loose which ship's which.
    > ],
    > Pilot=>[
    > name=>[['pilot_corsair']],
    > }#Just one pilot entry
    > ],
    > # lot more sections....
    > );

    That's excatly like how I have it now (See last entry in the original
    post), but to access an SHIP section in @ini, I can't just go
    $ini->{Ship}->[1]->{nickname}, I have to do a full scan of the array and
    their keys and some files have about 1000 sections, and that is slow.

    Marc "Maluku" Lucksch
     
    Marc Lucksch, Feb 10, 2009
    #8
  9. Marc Lucksch

    Guest

    On Tue, 10 Feb 2009 19:42:57 GMT, wrote:

    >On Tue, 10 Feb 2009 20:12:07 +0100, Marc Lucksch <> wrote:
    >
    >>Marc Lucksch schrieb:
    >>> Maybe I'm an idiot here, but I can't figure this one out even if my life
    >>> would depend on it:

    >>Maybe I posted the question wrong, it is not about the INI parsers, I
    >>just didn't want to release another one...
    >>
    >>Lets say I have this data structure, from an ini file, it cares for all
    >>the conditions that can happen (multiple sections, keys and values)
    >>
    >>my $ini={
    >> Ship=>[
    >> {#First Ship
    >> nickname=>[ #A key
    >> ['li_elite'] # value
    >> ],
    >> fuse=>[ #multikey with multivalue
    >> ['intermed_damage_smallship01','0.000000','400'],
    >> ['intermed_damage_smallship02','0.000000','200'],
    >>
    >> ], #....
    >> },{#Second Ship
    >> nickname=>[
    >> ['li_elite2']
    >> ],fuse=>[
    >> ['intermed_damage_smallship01','0.000000','400'],
    >> ['intermed_damage_smallship02','0.000000','200'],
    >>
    >> ], #....
    >> }, #A lot more 'Ships'
    >> ],Simple=>[
    >> #A lot of Simples
    >> ],ship=>[

    > ^^^^
    >Whats this amother ship in the hash?? This won't work, you already have a
    >ship.
    >>
    >>1. Problem:
    >>
    >>I can't save the order the sections or keys came in, nor the names of
    >>them so I can't write an unmodified file out again. I would have to tie
    >>that structure.
    >>I want to make is easily accessable:
    >>

    >Utterly absurd, of course you can. If you want to save the order in
    >which it exists in the ini file, dups an all, you have to break it up
    >into compound arrays, this way dup's are irrelavent.
    >
    >Array's of references of references.
    >Traversing the array, its up to you to know, based on the tag
    >when to increment index+=2;
    >
    >There is no magic bullet, you have a crap structure in an ini file,
    >deal with it.
    >
    >my @ini = (
    >
    > Ship=>[
    > [#First Ship
    > nickname=>[ #A key
    > ['li_elite'] # value
    > ],
    > fuse=>[ #multikey with multivalue
    > ['intermed_damage_smallship01','0.000000','400'],
    > ['intermed_damage_smallship02','0.000000','200'],
    >
    > ], #....
    > ],
    > [#Second Ship
    > nickname=>[
    > ['li_elite2']
    > ],fuse=>[
    > ['intermed_damage_smallship01','0.000000','400'],
    > ['intermed_damage_smallship02','0.000000','200'],
    >
    > ], #....
    > ], #A lot more 'Ships'
    > ],
    > Simple=>[
    > #A lot of Simples
    > ],
    > ship=>[
    > # 3 or 4 ships, sometime they are lowercase...
    > # That's a problem with this data model,
    > # Can't use Hash::Case::preserve either,
    > # cause I would loose which ship's which.
    > ],
    > Pilot=>[
    > name=>[['pilot_corsair']],
    > }#Just one pilot entry
    > ],
    > # lot more sections....
    >);
    >
    >-sln


    The other alternative is where each section and key a hash key
    that is an array reference consisting of the order obtained, type and
    a value array reference (undef on sections). Sort the orer and print
    out the section/key/value.

    -sln
     
    , Feb 10, 2009
    #9
  10. Marc Lucksch

    Guest

    On Tue, 10 Feb 2009 19:53:51 GMT, wrote:

    >On Tue, 10 Feb 2009 19:42:57 GMT, wrote:
    >
    >>On Tue, 10 Feb 2009 20:12:07 +0100, Marc Lucksch <> wrote:
    >>
    >>>Marc Lucksch schrieb:
    >>>> Maybe I'm an idiot here, but I can't figure this one out even if my life
    >>>> would depend on it:
    >>>Maybe I posted the question wrong, it is not about the INI parsers, I
    >>>just didn't want to release another one...
    >>>
    >>>Lets say I have this data structure, from an ini file, it cares for all
    >>>the conditions that can happen (multiple sections, keys and values)
    >>>
    >>>my $ini={
    >>> Ship=>[
    >>> {#First Ship
    >>> nickname=>[ #A key
    >>> ['li_elite'] # value
    >>> ],
    >>> fuse=>[ #multikey with multivalue
    >>> ['intermed_damage_smallship01','0.000000','400'],
    >>> ['intermed_damage_smallship02','0.000000','200'],
    >>>
    >>> ], #....
    >>> },{#Second Ship
    >>> nickname=>[
    >>> ['li_elite2']
    >>> ],fuse=>[
    >>> ['intermed_damage_smallship01','0.000000','400'],
    >>> ['intermed_damage_smallship02','0.000000','200'],
    >>>
    >>> ], #....
    >>> }, #A lot more 'Ships'
    >>> ],Simple=>[
    >>> #A lot of Simples
    >>> ],ship=>[

    >> ^^^^
    >>Whats this amother ship in the hash?? This won't work, you already have a
    >>ship.
    >>>
    >>>1. Problem:
    >>>
    >>>I can't save the order the sections or keys came in, nor the names of
    >>>them so I can't write an unmodified file out again. I would have to tie
    >>>that structure.
    >>>I want to make is easily accessable:
    >>>

    >>Utterly absurd, of course you can. If you want to save the order in
    >>which it exists in the ini file, dups an all, you have to break it up
    >>into compound arrays, this way dup's are irrelavent.
    >>
    >>Array's of references of references.
    >>Traversing the array, its up to you to know, based on the tag
    >>when to increment index+=2;
    >>
    >>There is no magic bullet, you have a crap structure in an ini file,
    >>deal with it.
    >>
    >>my @ini = (
    >>
    >> Ship=>[
    >> [#First Ship
    >> nickname=>[ #A key
    >> ['li_elite'] # value
    >> ],
    >> fuse=>[ #multikey with multivalue
    >> ['intermed_damage_smallship01','0.000000','400'],
    >> ['intermed_damage_smallship02','0.000000','200'],
    >>
    >> ], #....
    >> ],
    >> [#Second Ship
    >> nickname=>[
    >> ['li_elite2']
    >> ],fuse=>[
    >> ['intermed_damage_smallship01','0.000000','400'],
    >> ['intermed_damage_smallship02','0.000000','200'],
    >>
    >> ], #....
    >> ], #A lot more 'Ships'
    >> ],
    >> Simple=>[
    >> #A lot of Simples
    >> ],
    >> ship=>[
    >> # 3 or 4 ships, sometime they are lowercase...
    >> # That's a problem with this data model,
    >> # Can't use Hash::Case::preserve either,
    >> # cause I would loose which ship's which.
    >> ],
    >> Pilot=>[
    >> name=>[['pilot_corsair']],
    >> }#Just one pilot entry
    >> ],
    >> # lot more sections....
    >>);
    >>
    >>-sln

    >
    >The other alternative is where each section and key a hash key
    >that is an array reference consisting of the order obtained, type and
    >a value array reference (undef on sections). Sort the orer and print
    >out the section/key/value.
    >
    >-sln


    %ini = {
    section=>[[order,name,key_index],..],
    key=>[[order,name,value_index],..],
    value=>[,,,,]
    }

    I think this will work.
    -sln
     
    , Feb 10, 2009
    #10
  11. Marc Lucksch

    Marc Lucksch Guest

    schrieb:
    > On Tue, 10 Feb 2009 19:53:51 GMT, wrote:
    >
    >> On Tue, 10 Feb 2009 19:42:57 GMT, wrote:
    >> The other alternative is where each section and key a hash key
    >> that is an array reference consisting of the order obtained, type and
    >> a value array reference (undef on sections). Sort the orer and print
    >> out the section/key/value.
    >>
    >> -sln

    >
    > %ini = {
    > section=>[[order,name,key_index],..],
    > key=>[[order,name,value_index],..],
    > value=>[,,,,]
    > }
    >
    > I think this will work.


    Well it is a good idea, but explaining that to potential users might be
    difficult.
    The other way includes a lot of objects, saving it as an array and an
    hash for lookup.

    while(my ($section,$data)=$ini->each()) {
    while (my ($key,$value)=$data->each()) {
    for my $entr (@$value) {
    ...
    }
    }
    }
    $ini->section("Ship")->key("fuse")->[0]; #First ship, first fuse

    $ini->section("Ship",7)->key("fuse",1)->[0];

    foreach my $section (@{$ini->sections("Ship"}) {
    $section->setkey("nickname",$section->key("nickname")."_v2");
    }

    This would work, but it is hard to explain to any users and will scare
    them (like POE does to me). Who wants to read 3 pages of POD just to
    change a single value. :(

    foreach my $section (@{$ini->{Ship}}) {
    $section->{nickname}.="_v2";
    }

    This is shorter but I can't find any way to do it.

    Well, I'm going to bed, gotta go to the dentist tomorrow, where there
    will be joy and nice things will happen to me. Happy thoughts, Happy
    thoughts. No pain. Happy thoughts

    Marc "Maluku" Lucksch
     
    Marc Lucksch, Feb 10, 2009
    #11
  12. Marc Lucksch

    Ted Zlatanov Guest

    On Tue, 10 Feb 2009 19:40:31 +0100 Marc Lucksch <> wrote:

    ML> Still leaves the problem on how the represent the data, the file is
    ML> not only littered with 277 [Ship] entries, but also [collision
    ML> group]'s, one [Pilot] and lots of [Simple]'s... The file I put was
    ML> just an example. :(

    Instead of splitting on [Ship], look for [\w+] and capture that word.

    ML> I want for example access the pilot this way: (Nice and easy for any
    ML> user of this theoretical module)

    ML> print $ini->{Pilot}->{nickname}; #Only one pilot.

    Look at how I flatten an array into the first element of the array if
    the array has a size of 1. Same idea at the top level.

    ML> + I want to keep the order or otherwise it will confuse diff a lot.

    The CPAN IxHash module will do ordered keys IIRC.

    ML> The module should help people write scripts to easily extract and
    ML> modify data in those files, without exporting them to .inis first.

    ML> That might be a problem, since this is neither thats not the only ini
    ML> file in there.

    ML> There are shiparchs (this one), loadouts, equipments, universes,
    ML> systems, bases, markets, asteroids, asteroid_fielf and a lot more
    ML> files of those or similiar formats. (A quick search gave me 1000+ ini
    ML> files)

    ML> And I don't know the keys, lately someone discovered that you can add
    ML> an spin = 0.1, 0, 0 to a planet in a system.ini file and the planet
    ML> rotates. This was never seen before, maybe because they probably
    ML> removed it in the final version. Who knows how many of those are still
    ML> in there.

    ML> But I need to save them all and store them into a database, to run
    ML> queries on it. Well that is working nicely currently as well, but I
    ML> was wondering for a better solution to my problem too.

    Sound like the format is changing quickly and it's not a standard INI
    format. You should keep your working module as it stands, instead of
    trying to squeeze existing modules into something they can't do. If the
    configuration is very flexible and you're feeling adventurous, consider
    writing a small data-based DSL to describe your configuration structure,
    and make the main parser the interpreter of that DSL. For example:

    my $syntax = [
    Ship => { multiple => 1, comma_keys => [qw/fuse LODranges/] ... },
    Pilot => { multiple => 0, comma_keys => [] ... }
    ];

    my $data = parse($file, $syntax);

    This approach works well for parsing structures that mutate quickly.
    Perl 6 grammars will actually make this sort of work easier, but in Perl
    5 you'd have to use Parse::RecDescent which IMO will not work well for
    you.

    Ted
     
    Ted Zlatanov, Feb 10, 2009
    #12
  13. Marc Lucksch

    Guest

    On Tue, 10 Feb 2009 21:34:36 +0100, Marc Lucksch <> wrote:

    > schrieb:
    >> On Tue, 10 Feb 2009 19:53:51 GMT, wrote:
    >>
    >>> On Tue, 10 Feb 2009 19:42:57 GMT, wrote:
    >>> The other alternative is where each section and key a hash key
    >>> that is an array reference consisting of the order obtained, type and
    >>> a value array reference (undef on sections). Sort the orer and print
    >>> out the section/key/value.
    >>>
    >>> -sln

    >>
    >> %ini = {
    >> section=>[[order,name,key_index],..],

    ^^^^
    order is not needed in this case
    [snip]
    >
    >Well it is a good idea, but explaining that to potential users might be
    >difficult.
    >The other way includes a lot of objects, saving it as an array and an
    >hash for lookup.
    >
    >while(my ($section,$data)=$ini->each()) {
    > while (my ($key,$value)=$data->each()) {
    > for my $entr (@$value) {
    > ...
    > }
    > }
    >}
    >$ini->section("Ship")->key("fuse")->[0]; #First ship, first fuse
    >
    >$ini->section("Ship",7)->key("fuse",1)->[0];
    >
    >foreach my $section (@{$ini->sections("Ship"}) {
    > $section->setkey("nickname",$section->key("nickname")."_v2");
    >}
    >
    >This would work, but it is hard to explain to any users and will scare
    >them (like POE does to me). Who wants to read 3 pages of POD just to
    >change a single value. :(
    >
    >foreach my $section (@{$ini->{Ship}}) {
    > $section->{nickname}.="_v2";
    >}
    >
    >This is shorter but I can't find any way to do it.
    >
    >Well, I'm going to bed, gotta go to the dentist tomorrow, where there
    >will be joy and nice things will happen to me. Happy thoughts, Happy
    >thoughts. No pain. Happy thoughts
    >
    >Marc "Maluku" Lucksch


    Excuse the below verbosity. There is one other way, a single linked list.
    Clean and simple. Below is a generalized double linked list schema based
    on a Node, with lots of maybe 'bloated' utility functions you could trim
    down.

    In your case the Node would contain a single point of Section/Key/Value
    but within a linked list, so order is clear and duplictes are no problem.
    Access is simple.

    See below, its easy to add or insert a node. Its easy to traverse the list.
    And the good part is you can rename the functions for the user AND you can
    even generate an ini file after the list is edited, ie: insert/remove, clear,
    add head tail, etc ...

    The Node structure for your ini usage would be of this form. You will have to
    adjust the exapmles before "Node Functions" in examples, for data hash change,
    not sure, check it out.

    I highly recommend this method.
    -sln

    *******************************************************

    sub new_node # modified (correct below)
    {
    my $node = {
    # Node Header
    prev => undef,
    next => undef,
    # Node Data
    data => { section => '',
    key => '',
    value => ''
    }
    };
    return $node;
    }

    So, traversal is something like this:
    # Traverse the list from the Head, print data
    #-
    print "\nTraverse list from Head -\n";
    $curnode = $ListHead;
    while (defined $curnode)
    {
    if (defined $curnode->{data}->{val}) {

    ### Use Data Dumper on $curnode->{data}->{val}
    }
    $curnode = $curnode->{next};
    }
    print "sleeping 2 seconds\n";
    sleep(2);



    ----------------------------
    ----------------------------
    ## nodes.pl
    ## -sln

    use strict;
    use warnings;

    ### Exercise List Creation and Destruction,
    ### the bare minimum.
    ### ---------------------------------------


    # Create a new List with 1,000,000 nodes
    #-
    my $ListHead = CreateList();
    my $ListTail = $ListHead;
    my $curnode = $ListTail;
    $curnode->{data}->{val} = 0;

    print "\nAdding 1,000,000 nodes\n";
    for (my $i=1; $i<1000000; $i++)
    {
    $curnode = CreateTail( $curnode );
    $curnode->{data}->{val} = $i;
    $ListTail = $curnode;
    }
    print "Done, sleeping 5 seconds\n";
    sleep(5);

    # Destroy it
    #-
    print "\nDestroying List\n";
    DestroyList( $ListHead );
    print "Done\n";


    # Create a different List with 1,500,000 nodes
    #-
    $ListHead = CreateList();
    $ListTail = $ListHead;
    $curnode = $ListTail;
    $curnode->{data}->{val} = 0;

    print "\nAdding 1,500,000 nodes\n";
    for (my $i=1; $i<1500000; $i++)
    {
    $curnode = CreateTail( $curnode );
    $curnode->{data}->{val} = $i;
    $ListTail = $curnode;
    }
    print "Done, sleeping 5 seconds\n";
    sleep(5);

    # Destroy it
    #-
    print "\nDestroying List\n";
    DestroyList( $ListHead );
    print "Done\n";


    # Traverse the list from the Head, print data
    #-
    print "\nTraverse list from Head -\n";
    $curnode = $ListHead;
    while (defined $curnode)
    {
    if (defined $curnode->{data}->{val}) {
    print "data is: val = $curnode->{data}->{val}\n";
    }
    $curnode = $curnode->{next};
    }
    print "sleeping 2 seconds\n";
    sleep(2);

    # Traverse the list from the Tail, print data
    #-
    print "\nTraverse list from Tail -\n";
    $curnode = $ListTail;
    while (defined $curnode)
    {
    if (defined $curnode->{data}->{val}) {
    print "data is: val = $curnode->{data}->{val}\n";
    }
    $curnode = $curnode->{prev};
    }

    ## =================================
    ## Node Functions
    ## =================================

    # Make a new node
    # Parameters- none
    # Return - ref to a new node
    # -
    sub new_node
    {
    my $node = {
    # Node Header
    prev => undef,
    next => undef,
    # Node Data
    data => {val => undef}
    };
    return $node;
    }

    # Create a new List
    # Parameters - None
    # Return - ref to a created List (ListHead)
    # -
    sub CreateList
    {
    return new_node();
    }

    # Destroy the List given any Node
    # Parameters - $Node = any node in the List
    # Return - None
    # Notes - Destructive on all List node's
    # -
    sub DestroyList
    {
    my $Node = shift;
    if (defined $Node)
    {
    TruncateListAfter( $Node );
    TruncateListBefore( $Node );
    undef (%{$Node});
    }
    }

    # Create a new List Head from the given Node
    # Parameters - $Node = find the head from this node, prepend new created head,
    # creates new List if undef
    # Return - ref to a created node (ListHead)
    # -
    sub CreateHead
    {
    my $Node = shift;
    if (defined $Node)
    {
    while (defined $Node->{prev}) {
    $Node = $Node->{prev};
    }
    }
    return CreateHeadAtNode( $Node );
    }

    # Create a new List Tail from the given Node
    # Parameters - $Node = find the tail from this node, append new created tail,
    # creates new List if undef
    # Return - ref to a created node (ListTail)
    # Notes - Destructive on truncated node's
    # -
    sub CreateTail
    {
    my $Node = shift;
    if (defined $Node)
    {
    while (defined $Node->{next}) {
    $Node = $Node->{next};
    }
    }
    return CreateTailAtNode( $Node );
    }

    # Create a new Head before Node, removes leading node's
    # Parameters - $Node = node to prepend new head,
    # creates new List if undef
    # Return - ref to a created node (ListHead)
    # -
    sub CreateHeadAtNode
    {
    my $Node = shift;
    my $ret = undef;
    if (defined $Node)
    {
    my $newnode = new_node();
    if (defined $Node->{prev}) {
    TruncateListBefore( $Node );
    }
    $newnode->{next} = $Node;
    $Node->{prev} = $newnode;
    $ret = $newnode;
    } else {
    $ret = new_node();
    }
    return $ret;
    }

    # Create a new Tail after Node, removes trailing node's
    # Parameters - $Node = node to append new tail,
    # creates new List if undef
    # Return - ref to a created node (ListTail)
    # Notes - Destructive on truncated node's
    # -
    sub CreateTailAtNode
    {
    my $Node = shift;
    my $ret = undef;
    if (defined $Node)
    {
    my $newnode = new_node();
    if (defined $Node->{next}) {
    TruncateListAfter( $Node );
    }
    $newnode->{prev} = $Node;
    $Node->{next} = $newnode;
    $ret = $newnode;
    } else {
    $ret = new_node();
    }
    return $ret;
    }

    # Add the passed in Node to the List as its new Head
    # Parameters - $Node = find the head from this node, prepend new head
    # - $NewNode = the new node
    # Return - ref to the added node (ListHead)
    # -
    sub AddHead
    {
    my ($Node, $NewNode) = @_;
    my $ret = undef;
    if (defined $Node && defined $NewNode)
    {
    while (defined $Node->{prev}) {
    $Node = $Node->{prev};
    }
    $ret = AddHeadAtNode( $Node, $NewNode );
    }
    return $ret;
    }

    # Add the passed in Node to the List as its new Tail
    # Parameters - $Node = find the tail from this node, append new tail
    # - $NewNode = the new node
    # Return - ref to the added node (ListTail)
    # -
    sub AddTail
    {
    my ($Node, $NewNode) = @_;
    my $ret = undef;
    if (defined $Node && defined $NewNode)
    {
    while (defined $Node->{next}) {
    $Node = $Node->{next};
    }
    $ret = AddTailAtNode( $Node, $NewNode );
    }
    return $ret;
    }

    # Add a new Head before Node, removes leading nodes
    # Parameters - $Node = node to prepend new head
    # - $NewNode = the new node
    # Return - ref to the added node (ListHead)
    # Notes - Destructive on truncated node's
    # -
    sub AddHeadAtNode
    {
    my ($Node, $NewNode) = @_;
    my $ret = undef;
    if (defined $Node && defined $NewNode)
    {
    my $newnode = $NewNode;
    if (defined $Node->{prev}) {
    TruncateListBefore( $Node );
    }
    $newnode->{next} = $Node;
    $Node->{prev} = $newnode;
    $ret = $newnode;
    }
    return $ret;
    }

    # Add a new Tail after Node, removes trailing nodes
    # Parameters - $Node = node to append new tail
    # - $NewNode = the new node
    # Return - ref to the added node (ListTail)
    # Notes - Destructive on truncated node's
    # -
    sub AddTailAtNode
    {
    my ($Node, $NewNode) = @_;
    my $ret = undef;
    if (defined $Node && defined $NewNode)
    {
    my $newnode = $NewNode;
    if (defined $Node->{next}) {
    TruncateListAfter( $Node );
    }
    $newnode->{prev} = $Node;
    $Node->{next} = $newnode;
    $ret = $newnode;
    }
    return $ret;
    }

    # Truncate a List after a Node (Node is intact)
    # Parameters - $Node = node to disconnect from
    # Return - None
    # Notes - Destructive on truncated node's
    # -
    sub TruncateListAfter
    {
    my $Node = shift;
    if (defined $Node)
    {
    my $curnode = $Node->{next};
    while (defined $curnode)
    {
    my $nextnode = $curnode->{next};
    undef (%{$curnode});
    $curnode = $nextnode;
    }
    $Node->{next} = undef;
    }
    }

    # Truncate a List before a Node (Node is intact)
    # Parameters - $Node = node to disconnect from
    # Return - None
    # Notes - Destructive on truncated node's
    # -
    sub TruncateListBefore
    {
    my $Node = shift;
    if (defined $Node)
    {
    my $curnode = $Node->{prev};
    while (defined $curnode)
    {
    my $prevnode = $curnode->{prev};
    undef (%{$curnode});
    $curnode = $prevnode;
    }
    $Node->{prev} = undef;
    }
    }

    # Create a new node after a node
    # Parameters - $Node = node to append to
    # Return - ref to a created node
    # -
    sub CreateNextNode
    {
    my $Node = shift;
    return InsertNextNode( $Node, new_node() );
    }

    # Create a new node before a node
    # Parameters - $Node = node to prepend to
    # Return - ref to a created node
    # -
    sub CreatePrevNode
    {
    my $Node = shift;
    return InsertPrevNode( $Node, new_node() );
    }


    # Insert a passed in node after a node
    # Parameters - $Node = node to append to
    # - $NewNode = the new node
    # Return - ref to the inserted node
    # -
    sub InsertNextNode
    {
    my ($Node, $NewNode) = @_;
    my $ret = undef;
    if (defined $Node && defined $NewNode)
    {
    my $newnode = $NewNode;
    my $nextnode = $Node->{next};

    if (defined $nextnode) {
    $nextnode->{prev} = $newnode;
    $newnode->{next} = $nextnode;
    }
    $newnode->{prev} = $Node;
    $Node->{next} = $newnode;
    $ret = $newnode;
    }
    return $ret;
    }

    # Insert a passed in node before a node
    # Parameters - $Node = node to prepend to
    # - $NewNode = the new node
    # Return - ref to the inserted node
    # -
    sub InsertPrevNode
    {
    my ($Node, $NewNode) = @_;
    my $ret = undef;
    if (defined $Node && defined $NewNode)
    {
    my $newnode = $NewNode;
    my $prevnode = $Node->{prev};

    if (defined $prevnode) {
    $prevnode->{next} = $newnode;
    $newnode->{prev} = $prevnode;
    }
    $newnode->{next} = $Node;
    $Node->{prev} = $newnode;
    $ret = $newnode;
    }
    return $ret;
    }

    # Remove node from list
    # Parameters - $Node = node ref
    # Return - ref to next node
    # Notes - Destructive on Node
    # -
    sub RemoveNodeRetNext
    {
    my $Node = shift;
    my $ret = undef;
    if (defined $Node)
    {
    my $nextnode = $Node->{next};
    my $prevnode = $Node->{prev};
    undef (%{$Node});

    if (defined $nextnode && defined $prevnode) {
    $nextnode->{prev} = $prevnode;
    $prevnode->{next} = $nextnode;
    $ret = $nextnode;
    } elsif (defined $nextnode) {
    $nextnode->{prev} = undef;
    $ret = $nextnode;
    } elsif (defined $prevnode) {
    $prevnode->{next} = undef;
    }
    }
    return $ret;
    }

    # Remove node from list
    # Parameters - $Node = node ref
    # Return - ref to prev node
    # Notes - Destructive on Node
    # -
    sub RemoveNodeRetPrev
    {
    my $Node = shift;
    my $ret = undef;
    if (defined $Node)
    {
    my $nextnode = $Node->{next};
    my $prevnode = $Node->{prev};
    undef (%{$Node});

    if (defined $nextnode && defined $prevnode) {
    $nextnode->{prev} = $prevnode;
    $prevnode->{next} = $nextnode;
    $ret = $prevnode;
    } elsif (defined $prevnode) {
    $prevnode->{next} = undef;
    $ret = $prevnode;
    } elsif (defined $nextnode) {
    $nextnode->{prev} = undef;
    }
    }
    return $ret;
    }
    __END__
     
    , Feb 10, 2009
    #13
  14. Marc Lucksch

    Guest

    On Tue, 10 Feb 2009 21:28:11 GMT, wrote:

    >On Tue, 10 Feb 2009 21:34:36 +0100, Marc Lucksch <> wrote:
    >
    >> schrieb:
    >>> On Tue, 10 Feb 2009 19:53:51 GMT, wrote:
    >>>
    >>>> On Tue, 10 Feb 2009 19:42:57 GMT, wrote:
    >>>> The other alternative is where each section and key a hash key
    >>>> that is an array reference consisting of the order obtained, type and
    >>>> a value array reference (undef on sections). Sort the orer and print
    >>>> out the section/key/value.
    >>>>
    >>>> -sln
    >>>

    [snip]
    >
    >I highly recommend this method.
    >-sln
    >
    >*******************************************************
    >
    >sub new_node # modified (correct below)
    >{
    > my $node = {
    > # Node Header
    > prev => undef,
    > next => undef,
    > # Node Data
    > data => { section => '',
    > key => '',
    > value => ''
    > }
    > };
    > return $node;
    >}
    >


    This would be better for you, get rid of 'data' altogether.
    You will have to adjust a couple of undef's for the 'data'
    hash in the clear methods in the core though.

    sub new_node # modified ( need to correct below)
    {
    my $node = {
    # Node Header
    prev => undef,
    next => undef,
    # Node Data
    section => '',
    key => '',
    value => ''
    };
    return $node;
    }
    To traverse:

    # Traverse the list from the Head, print data
    #-
    print "\nTraverse list from Head -\n";
    $curnode = $ListHead;
    while (defined $curnode)
    {
    if (defined $curnode->{section}) {
    print "section = ".$curnode->{section}."\n";
    print "key = ".$curnode->{key}."\n";
    print "value = ".$curnode->{value}."\n";
    }
    $curnode = $curnode->{next};
    }

    -sln
     
    , Feb 10, 2009
    #14
  15. Marc Lucksch

    Guest

    Re: A problem with TIEHASH [was: A good data structure to store INI files.]

    On Tue, 10 Feb 2009 20:45:25 +0100, Marc Lucksch <> wrote:

    >
    >
    >A little addition why just TIE alone is also a no-go:
    >

    [snip]

    Holy crap, a nightmare.
    -sln
     
    , Feb 10, 2009
    #15
  16. Marc Lucksch

    Guest

    On Tue, 10 Feb 2009 21:40:17 GMT, wrote:

    >On Tue, 10 Feb 2009 21:28:11 GMT, wrote:
    >
    >>On Tue, 10 Feb 2009 21:34:36 +0100, Marc Lucksch <> wrote:
    >>
    >>> schrieb:
    >>>> On Tue, 10 Feb 2009 19:53:51 GMT, wrote:
    >>>>
    >>>>> On Tue, 10 Feb 2009 19:42:57 GMT, wrote:
    >>>>> The other alternative is where each section and key a hash key
    >>>>> that is an array reference consisting of the order obtained, type and
    >>>>> a value array reference (undef on sections). Sort the orer and print
    >>>>> out the section/key/value.
    >>>>>
    >>>>> -sln
    >>>>

    >[snip]
    >>
    >>I highly recommend this method.
    >>-sln
    >>
    >>*******************************************************
    >>
    >>sub new_node # modified (correct below)
    >>{
    >> my $node = {
    >> # Node Header
    >> prev => undef,
    >> next => undef,
    >> # Node Data
    >> data => { section => '',
    >> key => '',
    >> value => ''
    >> }
    >> };
    >> return $node;
    >>}
    >>

    >
    >This would be better for you, get rid of 'data' altogether.
    >You will have to adjust a couple of undef's for the 'data'
    >hash in the clear methods in the core though.
    >
    >sub new_node # modified ( need to correct below)
    >{
    > my $node = {
    > # Node Header
    > prev => undef,
    > next => undef,
    > # Node Data
    > section => '',
    > key => '',
    > value => ''
    > };
    > return $node;
    >}
    >To traverse:
    >
    ># Traverse the list from the Head, print data
    >#-
    > print "\nTraverse list from Head -\n";
    > $curnode = $ListHead;
    > while (defined $curnode)
    > {
    > if (defined $curnode->{section}) {
    > print "section = ".$curnode->{section}."\n";
    > print "key = ".$curnode->{key}."\n";
    > print "value = ".$curnode->{value}."\n";
    > }
    > $curnode = $curnode->{next};
    > }
    >
    >-sln


    I am assuming you gave up and committed suicide.
    Node's are your only hope to manage this. Trust me.

    If you are between a rock and a hard place. I will actually
    do this for you in about 20 minutes. The entire thing, including
    user presentation.

    I will do it for fun, until I get employment.

    -sln
     
    , Feb 11, 2009
    #16
  17. Marc Lucksch

    Marc Lucksch Guest

    schrieb:
    > I am assuming you gave up and committed suicide.
    > Node's are your only hope to manage this. Trust me.


    No, I went to sleep like I said, sorry I couldn't answer.

    > If you are between a rock and a hard place. I will actually
    > do this for you in about 20 minutes. The entire thing, including
    > user presentation.


    But I had kind of an idea for a solution last night: By using
    Contextual::Return in a FETCH of a tied object, I can do what I want
    with the data. (Well actually it has grown to three tied objects) But it
    works like this:

    my $data=new Data::INI; (what I called it for now)
    my $ini=new Data::INI;
    $ini->{Ship}=$data; #Ambigious, carps about it.
    dump $ini->{Ship} #$data;
    dump $ini->{Ship}->[0] #$data;
    #or:
    push @{$ini->{Ship}},$data; #The same as above, no carp;
    @{$ini->{Ship}}=($data); #replace all


    $ini->{Ship}->{nickname}="li_eilte" #or $data->{nickname}="li_elite";
    push @{$ini->{Ship}->{fuse}},
    [qw/intermed_damage_smallship01 0.000000 400/];
    print $ini->{Ship}->[0]->{fuse}->[0]->[1] # prints 0.000000
    print $ini->{Ship}->{fuse}->[0]->[1] #The same
    $ini->{Ship}->{fuse}->[0]->[1]="1.0000" #Set
    $ini->{Ship}->{fuse}->[0]="1.0000" #Replace the first fuse.
    push @{$ini->{Ship}->{fuse}},
    [qw/intermed_damage_smallship01 0.000000 400/];


    while (my ($section,$data)=$ini->each()) {

    }
    foreach my $data(@{$ini}) {

    }
    #Dump it to .ini:
    print "$ini" #Very disturbing, but kinda cool.
    # [Ship]
    # nickname=li_elite
    # ....

    #This is quite a hack and not really finished, but it shows promise to
    be nice .

    #Object-like access will also be implemented, for those confuse by that
    one above:
    $ini->section("Ship",0)->key("nickname",0,0);
    $ini->get("Ship",0)->get("nickname",0)->[0];
    foreach

    > I will do it for fun, until I get employment.


    No need to, but thanks a lot for the offer.

    Coding this is the fun in it.

    Marc "Maluku" Lucksch
     
    Marc Lucksch, Feb 11, 2009
    #17
  18. Marc Lucksch

    Marc Lucksch Guest

    Ben Morrow schrieb:
    > Quoth Marc Lucksch <>:
    >> But I had kind of an idea for a solution last night: By using
    >> Contextual::Return in a FETCH of a tied object, I can do what I want
    >> with the data. (Well actually it has grown to three tied objects) But it
    >> works like this:

    >
    > You may want to look into objects with @{} and %{} overloading, rather
    > than trying to make tie do something it doesn't want to. Having a tied
    > variable with context-dependant return values is likely to be very
    > confusing.
    >
    > It seems like you want $ini->{Ship} to return something that can pretend
    > to be either a single value or an array of values. This is what @{}
    > overloading is for. Of course, the @{} overload will itself probably
    > need to return a tied array, but that's simple enough.


    Thanks for it, but I already do it this way (I don't see any other)

    My Data::INI looks like this:

    package Data::INI;
    require Data::INI::Entry;
    require Data::INI::Value;
    use Scalar::Util;
    use Contextual::Return;
    use Carp qw/confess cluck croak carp/;
    use strict;
    use warnings;
    our $VERSION = 0.1;

    use overload '%{}'=>\&gethash,'@{}'=>\&getarray,'""'=>\&serialize;

    sub new {
    my $class=shift;
    my $self={hash=>{},array=>[]};
    bless \$self,$class;
    }
    sub gethash {
    my %h;
    my $self = shift;
    tie %h, ref $self, $self;
    \%h;
    }
    sub getarray {
    my @a;
    my $self = shift;
    tie @a, ref $self, $self;
    \@a;
    }
    sub TIEARRAY {
    my $class = shift;
    my $self=shift;
    $self=[$self,0];
    bless \$self, $class;
    }
    sub TIEHASH {
    my $class = shift;
    my $self=shift;
    $self=[$self,1];
    bless \$self, $class;
    }
    [....]
    sub FETCH {
    my $self=shift;
    my $obj=${${$self}->[0]};
    if (${$self}->[1]) {
    my $key=shift;
    my $k=lc $key;
    unless ($obj->{hash}->{$k}) {
    my @a;
    tie @a,"Data::INI::Entry",${$self}->[0],$k;
    $obj->{hash}->{$k}=\@a;
    return $obj->{hash}->{$k};
    }
    if (@{$obj->{hash}->{$k}} > 1) {
    return $obj->{hash}->{$k};
    }
    if (@{$obj->{hash}->{$k}}) {
    return (
    ARRAYREF { $obj->{hash}->{$k} }
    DEFAULT { $obj->{hash}->{$k}->[0] }
    );
    }
    return (
    ARRAYREF { $obj->{hash}->{$k} }
    DEFAULT { undef }
    );
    }
    else {
    my $index=shift;
    my $rindex=$index;
    $index*=2;
    $index++;
    return undef unless $obj->{array}->[$index];
    return ${$obj->{array}->[$index]};
    }
    }
    [...]

    Well it's getting there.
     
    Marc Lucksch, Feb 11, 2009
    #18
  19. Marc Lucksch

    Marc Lucksch Guest

    Ben Morrow schrieb:
    > OK. It seems you understand how this works well enough. What I meant
    > though was that here...



    > ...you return your own object with @{} and %{} (or presumably "" if it's
    > a leaf) overloading. While it's true that C::R does that for you, and
    > provides a rather clean syntax, the fact the module starts with
    >
    > |[Evil stuff]
    >
    > and goes on to do other equally evil things would make me very wary of
    > it. It's not hard to create the object yourself.


    I thought of that first, but it is not so easy, as far as I understood
    man overload right, I can't do that so easily.

    Or maybe I'm reading it wrong:

    |man overload (bottom of page):
    | Relation between overloading and tie()ing is broken. Overloading is
    | triggered or not basing on the previous class of tie()d value.
    |
    | This happens because the presence of overloading is checked too early,
    | before any tie()d access is attempted. If the FETCH()ed class of the
    | tie()d value does not change, a simple workaround is to access the
    | value immediately after tie()ing, so that after this call the previous
    | class coincides with the current one.

    Marc "Maluku" Lucksch
     
    Marc Lucksch, Feb 11, 2009
    #19
  20. Marc Lucksch

    Brad Baxter Guest

    On Feb 11, 9:36 am, Marc Lucksch <> wrote:
    > Thanks for it, but I already do it this way (I don't see any other)
    >
    > My Data::INI looks like this:


    Whether or not this will serve your needs, I think one fairly
    straight-forward data structure that can faithfully represent
    typical ini file data is pairs of pairs. A "pairs" structure
    is an array of single-key hashes as defined in
    http://yaml.org/type/pairs.html

    In your case the typical ini data is enhanced by assuming that
    comma separated values should be loaded as arrays.

    The code below demonstrates what I mean.

    Brad

    #!/usr/local/bin/perl

    use strict;
    use warnings;
    use Data::Dumper;
    $Data::Dumper::Terse = 1;
    $Data::Dumper::Indent = 1;

    # http://search.cpan.org/dist/Data-Pairs/
    use Data::pairs;

    my $ini = Data::pairs->new();
    my $current;

    while( <DATA> ) {
    chomp;
    if( /^\[(\w+)]/ ) {
    my $section = $1;
    $current = Data::pairs->new();
    $ini->add( $section => $current );
    }
    elsif( /^(\w+)\s*=\s*(.*)/ ) {
    my( $key, $val ) = ( $1, $2 );
    $val = [ split /\s*,\s*/, $val ] if $val =~ /,/;
    $current->add( $key => $val );
    }
    else { # comments, blank lines, etc.
    my $this = $current? $current: $ini;
    $this->add( "" => $_ );
    }
    }

    # view internal structure
    print Dumper $ini;

    # rewrite the ini file data
    for( $ini->get_array() ) {
    my( $section, $pairs ) = each %$_;
    if( $section ) {
    print "[$section]\n";
    for( $pairs->get_array() ) {
    my( $key, $val ) = each %$_;
    $val = join( ', ', @$val ) if ref $val eq 'ARRAY';
    print $key? "$key = $val\n": "$val\n";
    }
    }
    else {
    print "$pairs\n"; # comments/blank lines at the top
    }
    }

    __DATA__

    [Ship]
    ids_name = 237033
    ids_info = 66567
    ids_info1 = 66567
    ship_class = 1
    nickname = li_elite
    LODranges = 0, 75, 150, 1300
    fuse = intermed_damage_smallship01, 0.000000, 400
    fuse = intermed_damage_smallship02, 0.000000, 200
    ;#Well its a lot more in one section, but this proves the point.

    [Ship]
    ids_name = 237033
    ids_info = 66567
    ship_class = 1
    nickname = li_elite2
    LODranges = 0, 175, 150, 1300
    fuse = intermed_damage_smallship01, 0.000000, 400
    fuse = intermed_damage_smallship02, 0.000000, 200
    fuse = intermed_damage_smallship03, 0.000000, 133
    ;.... (about 250kb of this)
     
    Brad Baxter, Feb 11, 2009
    #20
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. lallous
    Replies:
    6
    Views:
    6,263
    P.Hill
    Dec 24, 2003
  2. mirco

    [OT] ini files

    mirco, Nov 11, 2003, in forum: C++
    Replies:
    3
    Views:
    636
    Karl Heinz Buchegger
    Nov 12, 2003
  3. Jazzhouse

    INI files

    Jazzhouse, Oct 2, 2004, in forum: C++
    Replies:
    4
    Views:
    2,542
    Stewart Gordon
    Oct 6, 2004
  4. Sebastian Bassi
    Replies:
    5
    Views:
    603
    Sebastian Bassi
    Jul 20, 2005
  5. Sebastian Bassi
    Replies:
    0
    Views:
    410
    Sebastian Bassi
    Jul 20, 2005
Loading...

Share This Page