Binary I/O in Javascript

[Patient Guy]

Has anyone written code that successfully manipulates binary file data
using Javascript?

It might---and in the case of doing I/O, will---make use of browser-
specific functions (ActiveX/COM with Internet Explorer, XPCOM/XPConnect
with Mozilla/Firefox).

I am writing client-side code that will generate binary data for producing
a GIF file through an OBJECT element. (The GIF is an image of a line and
points on a two-axis plot.)

In order to do some tests to see how the OBJECT element performs, I am
trying to read in binary file data (using XPConnect functions in Firefox).

The problem is that plain Javascript and these browser-specific functions
will not marry with one another (for instance, they cannot be manipulated
as strings, and the read() functions of these things return strings as
well). This is from what little I know. Dated posted in various
newsgroups on the subject get comments saying that if those creating file
I/O libraries in client-side code for text file I/O might as well make it
possible for manipulating binary data in I/O, since the point of no return
is authorizing any access to the file system, and not whether it is text or
binary access.

If I don't get an answer here, I will take this to the JScript and XPCOM
groups, where I can hope for more from the latter than the former. If it
can't be done in MSIE though, I suppose I should forget it (and try to
write a plugin of some kind instead). Unless I hear that MSIE browsers are
being used by less than 100,000 people around the world.

 

[VK]

Has anyone written code that successfully manipulates binary file data
using Javascript?

It might---and in the case of doing I/O, will---make use of browser-
specific functions (ActiveX/COM with Internet Explorer, XPCOM/XPConnect
with Mozilla/Firefox).

I am writing client-side code that will generate binary data for producing
a GIF file through an OBJECT element. (The GIF is an image of a line and
points on a two-axis plot.)

In order to do some tests to see how the OBJECT element performs, I am
trying to read in binary file data (using XPConnect functions in Firefox).

The problem is that plain Javascript and these browser-specific functions
will not marry with one another (for instance, they cannot be manipulated
as strings, and the read() functions of these things return strings as
well). This is from what little I know. Dated posted in various
newsgroups on the subject get comments saying that if those creating file
I/O libraries in client-side code for text file I/O might as well make it
possible for manipulating binary data in I/O, since the point of no return
is authorizing any access to the file system, and not whether it is text or
binary access.

If I don't get an answer here, I will take this to the JScript and XPCOM
groups, where I can hope for more from the latter than the former. If it
can't be done in MSIE though, I suppose I should forget it (and try to
write a plugin of some kind instead). Unless I hear that MSIE browsers are
being used by less than 100,000 people around the world.

I'm doing my jsFileManager as universal FSO/XPConnect wrapper.
I'd really like to finish up the Gesko part by this week-end if I'll be
able to keep staying late after my main job. IE part is already done.

Briefly - yes, it is perfectly possible in the universal way (despite
the internal mechanics is as different as black and white, plus buggy
as a street dog from the Microsoft side at least).

jsFileManager will be free open library any way so if you have some
burning questions - I'm glad to try to answer.

 

[VK]

As an addon:
Actually my first realease planned to be text files reading/writing
plus lounch applications. Binary data reading/writing can be done
additionally as it really not a challenge.

Any file in the file system is binary in meaning that it consists of
bytes allocated in some sequences - either it's a text file or a
picture or an application - it doesn't matter. So you can read any file
as text or as binary.
TextStream differs from ByteStream only in usage of internal tokenizer.
This tokenizer makes TextStream tread differently line breaks and
special char sequences (if not ASCII encoding). Also reading goes from
one system line break to another.
So to handle ByteStream you simply need to read/write data by blocks of
bytes (say 80 bytes per block) not by lines. And you have to ensure
that the encoding set to base ASCII to avoid bytes transformations by
tokenizer. This is all real lore of byte I/O.

 

[Julian Turner]

Has anyone written code that successfully manipulates binary file data
using Javascript?

It might---and in the case of doing I/O, will---make use of browser-
specific functions (ActiveX/COM with Internet Explorer, XPCOM/XPConnect
with Mozilla/Firefox).

[snip]

You can certainly manipulate binary data in a rough fashion, using a
Javascript String. A Javascript String format is in Unicode pairs I
believe. I.e. each character can hold a value from 0000 to FFFF, so
the binary stream could be represented either as a sequence where the
higher pair is 00, and the byte is the lower, i.e. 0000, 0001, 0002
.... 00FF, or each pair of bytes is encoded in to the Unicode pairs,
e.g. 0001, 0203, 0304, ... FEFF. The latter case is problematic for
odd numbered streams.

The trick however is getting the binary codes from files into and out
of the String object in the first place, whether at all, or
efficiently.

I have been experimenting myself, and so far Javascript alone does not
seem to be enough.

INTERNET EXPLORER

There are two ActiveX objects to consider:-

(a) Scripting.FileSystemObject

Through Javascript, it seems to be impossible to use this to save
strings containing a binary stream. If encoding is set to ASCII, an
error will be thrown for char codes in the range 80 - A0 which you try
to write to a file. If encoding is Unicode, then the resulting file is
prefixed with FF FE, and it only works for even numbered streams.

You can use it to load binary into Strings, but you need again to
watch for characters 80-A0, as they are converted during the load
process to some other unicode characters. This is consistently done,
so once you know the conversion, i.e. 80->2030 etc, you can easily
reverse it.

However, oddly, in VBScript, with encoding set to ASCII you can
actually write binary using Scripting.FileSystemObject. I think this
is because VBScript has other data types which are more acceptable to
the ActiveX object.

So you can adopt a mixture of VBScript and Javascript to get a kind of
binary i/o.

(a) ADODB.Stream

This gives full binary file system access. I think however it is
disabled by default on later versions of windows. In any event, it has
one limitation again in Javascript if you want to write binary data
from a String to the stream: the WriteText method again fails with
characters in the range 80-A0.

(c) XML dataType

A third area to explore is setting an XML node data type to bin.hex or
bin.base64. You could convert a String to base64 or hex, and set the
nodeTypedValue with that encoded string. This is more useful for
HTTPRequests, but conversion to base64 and hex is very slow. When you
subsequently bet the nodeTypedValue it will return a Variant byte
buffer containing the binary. Useless in Javascript, but might be
useable with ADODB.Stream or VBScript.

MOZILLA

I am not as familiar with the XPCOM components - see www.xulplanet.com,
but they may have some scriptable input and output streams. Again
don't know how they would interact with the Javascript String type.

Julian

 

[Jim Ley]

I am writing client-side code that will generate binary data for producing
a GIF file through an OBJECT element. (The GIF is an image of a line and
points on a two-axis plot.)

If your goal is generating a 2axis plot, why are you bothering to
generate a GIF? Just use the vector graphic methods available to
both IE and mozilla, and forget mucking around with GIF's.

Jim.

 

[Dr John Stockton]

JRS: In article <[email protected]>, dated Mon, 14
Nov 2005 10:25:47, seen in Patient Guy

Has anyone written code that successfully manipulates binary file data
using Javascript?

Not I; but you may care to know that <URL:http://www.merlyn.demon.co.uk/
js-misc0.htm#IEEE> shows some conversion between javascript Numbers and
equivalent more-ostensibly-binary forms.

 

[Aaron Gray]

I am writing client-side code that will generate binary data for producing

If your goal is generating a 2axis plot, why are you bothering to
generate a GIF? Just use the vector graphic methods available to
both IE and mozilla, and forget mucking around with GIF's.

Please elaborate, do you mean SVG ? As that requires a plug in.

There is Walter Zorn's div based JavaScript graphics :-

http://www.walterzorn.com/jsgraphics/jsgraphics_e.htm

Otherwise please do tell us the magic vector graphics methods

Aaron

 

[Jim Ley]

Please elaborate, do you mean SVG ? As that requires a plug in.

VML or DirectAnimation in IE
Native SVG in Opera, FireFox 1.5 or Safari beta's
Canvas in Safari, Opera or ..
Flash or SVG plugins...

Jim.

 

[Kevin]

Has anyone written code that successfully manipulates binary file data
using Javascript?

What do you mean by manipulate? Concatenate segments? Add binary
numbers? Save or load to/from disk? Or ???

I am writing client-side code that will generate binary data for producing
a GIF file through an OBJECT element. (The GIF is an image of a line and
points on a two-axis plot.)

I'd like to help, but why do you need to pass binary data to a plot
object?

Just trying to get a grip on what you're after.

Kev

 

[VK]

I'd like to help, but why do you need to pass binary data to a plot
object?

Just to use JavaScript (run in a browser) as an application platform I
guess. What's wrong in that?

And yes in many languages (not JavaScript only) a byte-in-byte
in's/out's transform into an idiotically complicated problem floating
around the same question: "How the hell to kill or at least to make it
shut up the build-in string tokenizer?!?"

A perfect sample is Java where you can *manually* form IP packets if
you need to, but you cannot send byte-in-byte sequence to say printer
w/o loosing time and the rest of your good mind.

 

[Patient Guy]

What do you mean by manipulate? Concatenate segments? Add binary
numbers? Save or load to/from disk? Or ???


I'd like to help, but why do you need to pass binary data to a plot
object?

Just trying to get a grip on what you're after.

Suppose you want to render an x-y plot showing, for instance, a
calibration curve. This is very common in any natural science discipline.

Now how do you do it with the tools you have for doing it on a web page?

First I create an user interface using the HTML form elements: the user
enters their abscissa-ordinate data in textboxes and then clicks a button
to do any stastistical work (least-squares for a line fit, maybe
polynomial fit, or other kind). You then want a VISUAL display (not just
tabular) of the plots and any fitted lines. How would you do that?

1. There are text (ASCII char) representations using dashes/hyphens, plus
symbols, 'x' characters, and other sorts of stuff, usually loaded into a
PRE element created. But that often looks rather bad. Easily done, but
poor appearance.

2. The alternative is to use script to manipulate data to create a
properly formatted image file to render the plot. The appearance is
stunningly much better. The GIF is the most suitable form for images that
are not photographs (as a plot is). An OBJECT element is created
dynamically, its 'type' attribute set (='image/gif') and its 'data'
attribute then set with encoded binary (the encoding is likely to be
base64). For perhaps a 700 x 400 pixel gif image of a plot, we're talking
about maybe 2K bytes of data, right? Not too much for a script engine to
work through in a decent amount of time.

The problem is that core Javascript----from apparently what little I know
of it---- does not seem to work with binary data too well.

Actually, I should not say that. I believe that you can work with Array
objects with each element specified as a Unicode char. One poster in this
thread seems to be saying that very thing.

But I want to do some testing with file I/O, and the file I/O library
functions for the most-used browsers I know of (the ActiveX
FileSystemObject, Mozilla/Firefox XPCOM/XPConnect) have read() and write()
functions that only seem to work with String objects, and the characters
allowed in a string object are limited; if some outside the ASCII charset
are allowed, the char '\u0000' (or '\x00') is one used by the script to
know when to terminate strings.

There are clearly other individuals who have experimented more with what
core Javascript can do than I have done, and I am trying to learn from
them what they have learned. One poster in this thread has developed a
library, and from reading through his code, I will see what he has
learned. But I want to see if this poster has dealt with the file I/O
problem, and how he has gotten round it. I will put this question to the
groups who know about ActiveX FSO and XPCOM. I also want to include any
other browser-specific code for other browsers (Opera, whatever) with
possibly their own interfaces for dealing with file I/O in binary, unless
they add as interfaces those of the major browsers.

The suggestion to use vector graphics (SVG, a W3C "recommendation") is
probably the way I should go, although clearly (not all) the major
browsers do not have built-in component to render this type of object
(i.e., requires a plug-in). I will be delayed for a time browsing through
a 700-page SVG specification to learn what it is I need to do make that
rendering.

 

[Jim Ley]

The suggestion to use vector graphics (SVG, a W3C "recommendation") is
probably the way I should go, although clearly (not all) the major
browsers do not have built-in component to render this type of object

No, I didn't recommend SVG, I recommended vector graphics, there are
non-plugin vector graphics available in all the latest versions of the
4 largest browsers, add in plug-ins and you'll support all the desktop
browsers released in the last 4 years.

Certainly the coverage is much larger than file access (opera has none
for example) and it will be much, much faster than a GIF generator in
javascript.

Jim.

 

[VK]

The suggestion to use vector graphics (SVG, a W3C "recommendation") is
probably the way I should go, although clearly (not all) the major
browsers do not have built-in component to render this type of object
(i.e., requires a plug-in). I will be delayed for a time browsing through
a 700-page SVG specification to learn what it is I need to do make that
rendering.

My advise was to use the combination of
VML (Vector Markup Language) build into all supported versions of
Internet Explorer and
SVG (Scalable Vector Graphics) build into incoming version of Firefox
1.5 and having a bunch of plugins to hold on now. You may look what
Internet Explorer can to by default for a longest time here:
<http://www.geocities.com/schools_ring/archives/VML_SVG.html>
<http://www.geocities.com/schools_ring/archives/VML_SVG.zip>

Both VML and SVG are rather easy to understand (especially VML)
Unfortunately Microsoft did a big mistake: they right away announced
VML as open standard and filed standard proposal papers to W3C under
their name. This alone ensured that W3C did not accept VML and started
to develop another standard (SVG)
And Mozilla Foundation accepts only W3C signatures and Microsoft
doesn't give a damn about W3C relevations and.. any way, there are two
standards now we have to deal with.
Next time Microsoft should use some 3rd party bogus company to file
standards to W3C - they will have much more chance to pass through

VML and SVG are still pretty close: really, vector is vector and line
is line, whoever's describing them. VML is more evident and user
friendly.
SVG is based on VRML project and inherits its academical look and
confusing coordinates notation. But the ability to go away from ASCII
pseudo-graphics and from micro DIV's chaos pays for troubles to learn
both.
IMHO

 

[Aaron Gray]

My advise was to use the combination of

VML (Vector Markup Language) build into all supported versions of
Internet Explorer and
SVG (Scalable Vector Graphics) build into incoming version of Firefox
1.5 and having a bunch of plugins to hold on now. You may look what
Internet Explorer can to by default for a longest time here:
<http://www.geocities.com/schools_ring/archives/VML_SVG.html>
<http://www.geocities.com/schools_ring/archives/VML_SVG.zip>

Great stuff.

Thanks I was not aware of VML.

Aaron

 

[VK]

I have been experimenting myself,

Julian, did you keep your testcases? If so, could you give me some?

 

[VK]

.... not for criticism of any kind, I'd just like so save some of my
time/

 

[Morgan]

There is Walter Zorn's div based JavaScript graphics :-

http://www.walterzorn.com/jsgraphics/jsgraphics_e.htm

Wow! Neat link, I've been looking for something along these lines for
ages.

 

[Aaron Gray]

There is Walter Zorn's div based JavaScript graphics :-

Wow! Neat link, I've been looking for something along these lines for
ages.

Beware every dot is a <div>

VML and SVG are better solutions but not as portable as Walters solution.

Google for VML and SVG they are specs on W3C website.

VML runs in IE 5.0+. There are FireFox plugins but for SVG the latest beta
has SVG built in.

VML is easier to program than SVG, but SVG is more comprehensive.

Aaron

 

[Morgan]

Beware every dot is a <div>

VML and SVG are better solutions but not as portable as Walters solution.

Google for VML and SVG they are specs on W3C website.

VML runs in IE 5.0+. There are FireFox plugins but for SVG the latest beta
has SVG built in.

VML is easier to program than SVG, but SVG is more comprehensive.

Aaron

Thanks,

 

[Kevin]

I'd like to help, but why do you need to pass binary data to a plot
object?

[...]
2. The alternative is to use script to manipulate data to create a
properly formatted image file to render the plot. The appearance is
stunningly much better. The GIF is the most suitable form for images that
are not photographs (as a plot is). An OBJECT element is created
dynamically, its 'type' attribute set (='image/gif') and its 'data'
attribute then set with encoded binary (the encoding is likely to be

Ah. I had misread your first posting, thinking you already had written
an ActiveX display or plotting object. So I couldn't figure out why
you wouldn't simply pass the plot parameters as text or numbers to that
object for display generation.

For example, I did an ActiveX signature control that is used for both
capture and display. You get and set compressed vector data (as a
string) to draw lines or whatever.

But you can't just create an HTML element of type OBJECT and expect it
to know how to display (or do) anything. So if your idea was to
create GIF data and pass it to an inanimate element of type OBJECT,
nothing would happen. Unless there's some magic OBJECT I don't know
about.

[...]
But I want to do some testing with file I/O, and the file I/O library
functions for the most-used browsers I know of (the ActiveX
FileSystemObject, Mozilla/Firefox XPCOM/XPConnect) have read() and write()
functions that only seem to work with String objects, [...]

This is what I thought your original Q was about... reading in binary
test data to pass to an ActiveX object. For that, you could use
XMLHTTP to read in a file://, and pass its responseBody as binary data
into a control.

The suggestion to use vector graphics (SVG, a W3C "recommendation") is
probably the way I should go, although clearly (not all) the major

I like SVG myself, and have done some complicated, rotating and moving
gizmos under it. Google around and you should be able to find sample
SVG code for plotting graphs... a very popular use for SVG.

If it's just simple graphing, though, you could get away with using
DIVs / tables and CSS to create some nice images. Bar graphs easiest,
of course, but the finer grain you go, the more it'd look like a line,
too.

Luck!
Kevin