Using References to Formats?

[Veli-Pekka Tätilä]

Hi,
Browsing perldiag, I noticed messages related to format references. So being
curious and wishing to continue my exploration of Perl's dark and archaic
corners, I decided to write a sample program to see how format references
could be used in Perl. first is an account of what I've attempted, the
relevant code in small chunks and the output received. The mail ends with
the full program source and output from a sample run.

Curiously, references to formats are not documented in perlref, perlform
etc... I'm running ActiveState Perl v5.8.7 (build 815, XP Pro SP2 English).

And now to the program:

To motivate taking references to formats I started out with a rather useless
toy function that generates formats using eval. The format name and the
number of chars it extracts from a global named $text can be parameterized.

sub genForm
{ # A simple format named $name outputting $n chars of $text.
my($name, $length) = @_;
eval
(
"format $name =\nFirst $length chars: @" .
'<' x $length . "\n\$text\n."
); # eval
die $@ unless $@ eq ''; # Eval failed.
} # sub

There's a related output function, which given a format name, writes it out
to the default file handle:

sub writeForm
{ # Write out the specified format.
local $~ = shift;
write;
} # sub

At this point I started wondering whether I could use a real reference to a
format in stead of an "indirect format" (in analogy to indirect file
handles). First I had to use the *foo{THING} syntax to get at a format. The
following statement, using main's symbol table and *foo{FORMAT} did the
trick for me:

my $formRef = *{ $::{$name} }{FORMAT}; # *foo{FORMAT} syntax in main
package.

To test what info could be gleaned from a format reference I made a function
for that, too. Here it is:

sub dumpForm
{ # Dump info on a format reference.
my $formRef = shift;
print "The formref $name is:";
print "Stringified: $formRef";
print "of type: " . ref($formRef);
print "Dumped: ";
eval { print Dumper($formRef) };
} # sub

Oddly, neither the docs for the ref built-in nor Data:umper mentioned
references to formats. Despite this the ref function and stringification
worked all right but Dumper didn't. Here's some output:

The formref eight is:
Stringified: FORMAT(0x18c4ecc)
of type: FORMAT
Dumped:
cannot handle ref type 14 at C:/Perl/lib/Data/Dumper.pm line 167.
$VAR1 = ;

I wonder if the debugger does any better. I have not tested it yet.

To make format references useful at all, I suppose one would have to be able
to dereference them somehow. IS that possible, and if so how? Formats have
no sigil so I myself have absolutely no idea how they could be dereferenced.
Would being able to work with format references bring any benefits compared
to refering to formats by name? I suppose not though using format references
does seem to sort of work.

The first thing that occurred to me was to try assigning a format reference
to $~, as opposed to a format name. The same writeForm function could be
used, just passing it a reference:

eval { writeForm($formRef) };
print "Using formref for $~: $@";

This strategy didn't work all that well. The statement printing the eval
error outputs:

Using formref for STDOUT: Undefined format "FORMAT(0x18c4eb4)" called at
C:\programming\plx\test.plx line 29.

Apparently no magical dereferencing is going on here. Starting to run out of
ideas, I thought of testing what would happen if I tried to dereference the
format as a scalar. I have no real rationale for that apart from scalar
derefs working for elements in arrays and hashes. I did realize right from
the start this wouldn't work for formats but typed in the following
nevertheless:

eval { writeForm(${$formRef}) };
print "Using desperate scalar deref for $~: $@";

And the output is:

Using desperate scalar deref for STDOUT: Not a format reference at
C:\programming\plx\test.plx line 29.

Quite right, not a format reference. But the thing that puzzles me here is
that the error is phrased as though Perl expected a format reference. Yet
when I give it one, as in the previous attempt, it doesn't seem to like it
any better, either. It just takes the stringified form of the reference to
be a format name which is no good.

Finally, here's the full code followed by some sample output:

Full code:

use strict; use warnings;
use Data:umper;
our $text = 'this is a test';
(my $name, local $\) = ('eight', "\n");
genForm($name , 8);
writeForm($name);
my $formRef = *{ $::{$name} }{FORMAT}; # *foo{FORMAT} syntax in main
package.
dumpForm($formRef);
# Try using formatref in stead of format name for writing the data.
eval { writeForm($formRef) };
print "Using formref for $~: $@";
eval { writeForm(${$formRef}) };
print "Using desperate scalar deref for $~: $@";

sub genForm
{ # A simple format named $name outputting $n chars of $text.
my($name, $length) = @_;
eval
(
"format $name =\nFirst $length chars: @" .
'<' x $length . "\n\$text\n."
); # eval
die $@ unless $@ eq ''; # Eval failed.
} # sub

sub writeForm
{ # Write out the specified format.
local $~ = shift;
write;
} # sub

sub dumpForm
{ # Dump info on a format reference.
my $formRef = shift;
print "The formref $name is:";
print "Stringified: $formRef";
print "of type: " . ref($formRef);
print "Dumped: ";
eval { print Dumper($formRef) };
} # sub

Sample output:

First 8 chars: this is a
The formref eight is:
Stringified: FORMAT(0x18c4eb4)
of type: FORMAT
Dumped:
cannot handle ref type 14 at C:/Perl/lib/Data/Dumper.pm line 167.
$VAR1 = ;

Using formref for STDOUT: Undefined format "FORMAT(0x18c4eb4)" called at
C:\programming\plx\test.plx line 29.

Use of uninitialized value in scalar assignment at
C:\programming\plx\test.plx line 28.
Using desperate scalar deref for STDOUT: Not a format reference at
C:\programming\plx\test.plx line 29.

 

[attn.steven.kuo]

Veli-Pekka Tätilä wrote:

(snipped)

At this point I started wondering whether I could use a real reference toa
format in stead of an "indirect format" (in analogy to indirect file
handles). First I had to use the *foo{THING} syntax to get at a format. The
following statement, using main's symbol table and *foo{FORMAT} did the
trick for me:

my $formRef = *{ $::{$name} }{FORMAT}; # *foo{FORMAT} syntax in main
package.

To test what info could be gleaned from a format reference I made a function
for that, too. Here it is:

sub dumpForm
{ # Dump info on a format reference.
my $formRef = shift;
print "The formref $name is:";
print "Stringified: $formRef";
print "of type: " . ref($formRef);
print "Dumped: ";
eval { print Dumper($formRef) };
} # sub

Oddly, neither the docs for the ref built-in nor Data:umper mentioned
references to formats. Despite this the ref function and stringification
worked all right but Dumper didn't. Here's some output:

The formref eight is:
Stringified: FORMAT(0x18c4ecc)
of type: FORMAT
Dumped:
cannot handle ref type 14 at C:/Perl/lib/Data/Dumper.pm line 167.
$VAR1 = ;

You can use Devel:eek instead of Data:umper
if you want to look at the guts of a format reference.

For me Devel:eek:ump prints:

SV = RV(0x1821258) at 0x182ad40
REFCNT = 1
FLAGS = (PADBUSY,PADMY,ROK)
RV = 0x182ae60
SV = PVFM(0x4489e0) at 0x182ae60
REFCNT = 3
FLAGS = (CLONE)
IV = 0
NV = 0
COMP_STASH = 0x0
START = 0x448b90 ===> 5120
ROOT = 0x448c90
XSUB = 0x0
XSUBANY = 0
GVGV::GV = 0x182ae9c "main" :: "eight"
etc.,

I wonder if the debugger does any better. I have not tested it yet.

To make format references useful at all, I suppose one would have to be able
to dereference them somehow. IS that possible, and if so how? Formats have
no sigil so I myself have absolutely no idea how they could be dereferenced.
Would being able to work with format references bring any benefits compared
to refering to formats by name? I suppose not though using format references
does seem to sort of work.

Well, just use another typeglob for dereferencing.
Throw in the 'reftype' function from the
Scalar::Util module and then you can
pass either a format NAME or format
reference to writeForm:

sub writeForm
{ # Write out the specified format.
local $~ = shift;
write;
} # sub

becomes:

use Scalar::Utiil ('reftype');

sub writeForm {
if (reftype $_[0] and reftype $_[0] eq 'FORMAT') {
local *FOO;
*FOO = $_[0];
local $~ = 'FOO';
write;
} else {
local $~ = shift;
write;
}
}

 

[Veli-Pekka Tätilä]

Hi,
Replying a bit late and slightly out-of-order, too.

To make format references useful at all, I suppose one would have to be
able to dereference them somehow. IS that possible, and if so how?

Well, just use another typeglob for dereferencing.

sub writeForm
{ # Write out the specified format.
local $~ = shift;
write;
} # sub

becomes:

use Scalar::Utiil ('reftype');

sub writeForm {
if (reftype $_[0] and reftype $_[0] eq 'FORMAT') {
local *FOO;
*FOO = $_[0];
local $~ = 'FOO';
write;
} else {
local $~ = shift;
write;
}
}

Hey, thanks for the code. Works fine for me. Somehow aliasing the name of
the format to be able to refer to it by another name didn't occur to me. But
as the built-in functions seem to support using a stringified scalar as a
format name, I doubt if using format refs have any benefits at all. It just
serves to make the syntax more gory, eh. No wonder they haven't been
documented propperly.

You can use Devel:eek instead of Data:umper
if you want to look at the guts of a format reference.

Ah, nice. Seems most of the fields are the same as for the rest of the Perl
data types. The Peek module can be quite handy, I think. I initially
overlooked it, when going through all the built-in modules that semed
interesting, because the title talks about XS and I'm not ready to tackle it
yet.

Your note about the function got me playing around with other Perl variables
and browsing PErlguts to understand at a high level what's going on behind
the scenes.

In particular, I've been tracking when exactly scalars change their datatype
from integer to double and how frequently the string portions are updated.
Again I don't think even Perlguts covers the rules but I might be wrong, as
I didn't read it all through.

I've only experimented a little but it would seem to me that:

- Perl is pretty smart in keeping integer values integers. Even if you add
and subtract doubles you might still end up with an integer value, if the
result happens to fit exactly to an int.

- Using an operator or a built-in with a double and an int usually converts
the int to a double if you assign the result to a variable. THis is natural,
of course.

- There don't seme to be many instances in which variables which are now
doubles would be converted to ints. Even the int function doesn't do that,
if I'm interpreting these funny type abbreviations right.

- Perl appears to update all but the most recent types lazily. Sometimes the
string or double values can lag behind considerably. They do get updated
when current values for the types are needed such as in stringification or
calling functions taking doubles.