reading in a nested formatted data structure with quirks

E

el.dodgero

I'm having a brainfart day, so I'll just throw this up to see if
there's some easy solution already out there that I don't know about...

I have data in a format that looks like it should be easy to parse, but
it's not acting that way...

For the most part, most of the data is in a simple sort of structure,
like so:

something whatever:2
{
property blah
otherproperty blahblah
}

However some of the time it's more like this:

something foo:3
{
property blah
otherthings
{
otherthingType1 identifier
{
property1 1
property2 2
moreInfoThanYouNeed
{
data 1 yadda
data 2 yaddayadda
data 3 yaddayaddayadda
}
}
otherthingType2 identifier
{
property1 0
property2 foo
}
}

The trick is that the whole {} block of anything is one biug property
of the stuff above it. The nesting could go on and on forever in theory
though normally sits at about 0-5 deep

I'm wanting to parse this stuff into a data structure. The thing is, if
I go line by line, then by the time I'm at a { line to know I'm in a
subnested block, I've already passed by the thing I have to set it as a
property of. Thus I'm thinking two things:

1) The reading-in of any given {} block should be a subroutine that
knows to call itself. This makes it convenient that all these files
start and end with a { and } respectively, putting the whole thing
inside a big block like this.

2) I should keep a reference to the last thingy I set, the previous
token, so to speak.

3) I should make a simple little subroutine called 'assign' that
simplifies the process of assigning a property to a token that may
already have it -- since I don't want to redefine such property of a
token, I should assign or append to an array. Since I don't know what
will need an array ahead of time for sure, a subroutine that handles
the assigning would be good, something like this:

sub assign ($$) {
my $thing = shift;
my $newval = shift;
if (defined $thing) {
unless (ref $thing eq 'ARRAY') {
$thing = [$thing];
}
push @{$thing}, $newval;
}
else {
$thing = $newval;
}
}

But anything i'm not thinking of... well, I can't exactly ask 'Am I not
thinking of this?' if I have no clue that 'this' may be. So this is
sort of that whole 'pointing and question marks' thing....

BTW if no one answers this, it's okay.
This is sort of therapeutic to write it all out like this too.
Who knows, maybe someone else has a similar scenario where my thoughts
will help them. *shrug*

BTW -- the vasy majority will either pass this by without a second
thought, suggest things to me that will help, or say that something I
said helped them with their problem. The first don't bother me, the
second I thank (and will, not just in advance), and the third, no
worries.

However, there is someone out there who's wanting to answer this in
some asshole way just to get some smartass comment in for no good
reason than their own narcissistic joy at seeing the oh-so-clever thing
they typed. I know this deep in my soul because I know this is usenet.
So, if you are that person, just get a life instead, ok?
 
E

el.dodgero

Jim said:
I'm having a brainfart day, so I'll just throw this up to see if
there's some easy solution already out there that I don't know about...

I have data in a format that looks like it should be easy to parse, but
it's not acting that way...

For the most part, most of the data is in a simple sort of structure,
like so:

[nested data structure snipped]

BTW -- the vasy majority will either pass this by without a second
thought, suggest things to me that will help, or say that something I
said helped them with their problem. The first don't bother me, the
second I thank (and will, not just in advance), and the third, no
worries.

However, there is someone out there who's wanting to answer this in
some asshole way just to get some smartass comment in for no good
reason than their own narcissistic joy at seeing the oh-so-clever thing
they typed. I know this deep in my soul because I know this is usenet.
So, if you are that person, just get a life instead, ok?

Hey! You are trying to take all the fun out of Usenet for some people!

Well, you know *L*
This sounds like a job for Parse::RecDescent. You will have to define a
grammar for your file format, and it might be overkill, but it would be
a good opportunity to learn the technique.

I will look at this, thanks.
I would also consider putting the whole file into one scalar and
picking it apart with Text::Balanced.

I actually considered it, but...

The actual files are Curious Labs Poser markup files (CR2, HR2, PP2,
PZ2, etc). While some are relatively smallish, others contain morph
targets and custom geometry --

As a for-instance, Victoria 2 from daz3d.com (a bit of an older model,
but still the best example) contains some 100-odd morphs in her CR2
file.Each of these morphs is composed of up to some 10,000 odd vertex
deltas (and if a newer figure was all-in-one like this it would be
worse -- the geometry for Victoria 3 is 72,712 vertices for the full
figure's mesh, and half that in the head:1 actor -- if that had the
same number of morphs in the CR2 itself like V2 did, augh!). As it is,
V2's CR2 file is already some 20MB. Somehow I'm thinking that a 20MB
scalar... not such a good idea B^)

And yeah that's my bad -- I didn't indicate the potential for hugeness
these files might get to.

Thanks!
 
E

el.dodgero

Jim said:
I would also consider putting the whole file into one scalar and
picking it apart with Text::Balanced.

Oooh, thanks again actually...
'cause ... well... now that I think about it (and had a better think
about why I didn't go that route)...

I could theoretically take out all the morphs and custom geeomtry
*first*... store them out seperately and then put a marker of some sort
that indicates that I mean to reference those there...

Like, a morph deltas' subblock looks like this:

deltas
{
d 1 .00004 .00032 -.00045
d 2 .00003 .00031 -.00041
...
d 1754 .00024 -.00415 .00012
}

Where the ... means all or most of the numbers I skipped (ones where
the results would be like d i 0 0 0 are left out thankfully) -- but if
I pulled that out, stored it in an array in a hash, or even in a DBM or
something --- I could then refer to that by replacing it like, for
instance:

deltas
{
deltasref 1
}

And the same for embedded meshes...

ANd then *that* would all fit in a scalar because it would be hard
pressed to be more than a few k, and usually far less. It's jus tthe
deltas and geometry that kill things.
 
M

Mumia W.

I'm having a brainfart day, so I'll just throw this up to see if
there's some easy solution already out there that I don't know about...

I have data in a format that looks like it should be easy to parse, but
it's not acting that way...
[...]

It's complicated enough that you might want to use the Text::Balanced
module and a custom-made recursive subroutine.
 
D

Dr.Ruud

(e-mail address removed) schreef:
However, there is someone out there who's wanting to answer this in
some asshole way just to get some smartass comment in for no good
reason than their own narcissistic joy at seeing the oh-so-clever
thing they typed. I know this deep in my soul because I know this is
usenet. So, if you are that person, just get a life instead, ok?

Pot. Kettle. Black.
 
D

Dodger

David said:

After scanning through for other posts by David Squire, though he won't
read this, I want to thank him. By killfiling me, he'll never reply to
one of my posts. Therefore, I wont ever have to get my hopes up that
someone has posted an answer when it was just him.
 
T

Tad McClellan

At this point it was merely speculation that you are an ass.

Yeah, you. **** off.


It is no longer just speculation.

Thanks for providing the confirmation.
 
D

Dodger

Tad said:
It is no longer just speculation.
Thanks for providing the confirmation.

Absolutely. I'm totally not someone who needs to waste time on little
trolls who think delivering 'clever' little smartarse responses are the
best thing to do with it. I didn't think I'd given that impression in
the first place.

I'm perfectly happy to help anyone else and share. However, pointless
little snarks simply crowd my inbox (and those of anyone else watching
a given thread) with the mistaken impression that someone had something
worthwhile to say.

Of course, it was disappointing to have someone almost immediately
respond with one to an expression that I'd rather not deal with them.
I'm using Google groups, due to a lack of a proper usenet server
(thanks to Comcast). I don't have a killfile, and honestly the little
twit armies who think it's oh-so-important that they post their snarky
nonsense for everyone to see are simply making places like usenet less
useful for people who really want to share knowledge.

Or did you mean something else?
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

No members online now.

Forum statistics

Threads
473,769
Messages
2,569,580
Members
45,054
Latest member
TrimKetoBoost

Latest Threads

Top