javascript preprocessor instructions?

[nick]

Has anyone here used preprocessing instructions in their js source,
for example to import other files, conditionally include bits of code
based on compile time options, etc?

If so, how did you do it?

I've discovered that using gcc with the -E option seems to work pretty
well. It feels hackish though, I don't totally trust it. Lines
beginning with # and : have to be removed from the output. The benefit
is that pretty much any C(++) preprocessor stuff can be used with
javascript... Is there a cleaner way to do this though?

# preprocess as chromium extension
cp src.js temp.c
echo '/** my chromium extension */' > out.js
gcc -DMY_VERSION=1.23 -DEXT_CHROMIUM=1 -E temp.c \
| sed s/^[#:].*// | jsmin >> out.js
rm temp.c

 

[nick]

I've used preprocessor instructions to build JS applications before, but
 that turned out to be too inflexible after a while. The next step was a
custom Perl script, which basically replicated some of the CPP
functionality, but accepted directives in a slightly different format,
and allowed leading whitespace:

  //#ifdef WITH_REGEX
    //#include "regexp.js"
    ...
  //#endif

Hmm, it would be pretty easy to add that functionality with another
sed before preprocessing.

What you might find interesting is that you don't need to run gcc if you
only need the preprocessor (cpp)

Good to know, it looks like I can pipe the cpp file right through cpp.
Thanks

 

[nick]

pipe the cpp file

s/cpp/js/

 

[nick]

Hmm, it would be pretty easy to add that functionality with another
sed before preprocessing.

This works:

sed s/^\\s*\\/\\/#/#/

 

[nick]

On 27/04/10 03:38, nick wrote:

That's right. The Perl script did more than that: it also handled
options and preferences, paths, dependency resolution, cleanup, lint
checking, minification, and deployment/packaging.

That sounds very useful. Is the script still around somewhere?

 

[nick]

Well... it's still around, but it was never intended for release in this
form, which means that there's no documentation at all, and most of the
options were added to fit one specific internal project. Since this was
my first attempt at writing a build system, I made some pretty bad
design decisions; it soon became clear that this wasn't the way to go,
and I abandoned it. If you want to see it anyway, I'd have to clean it
up a little first, or at least remove any references to the internal
project it was written for (and I'll also need a valid email address to
send it to, because I don't want to post half-baked code on Usenet).

Well, that sounds like a lot of trouble... I wasn't sure whether it
was something you had written or part of a larger build system.

The next version was written in PHP, and didn't use any preprocessing
instructions in the JS files anymore (I was excited about them at first,
but as it turned out, I hardly ever used them). A few months later, I
took the lessons learned in the first two attempts and rewrote the whole
thing from scratch in Java - as you can see, code reuse is not one of my
strong points . This version is also still undocumented and mostly
tailored to my current personal requirements, but it's faster than the
other two, more flexible, and could potentially be useful for others. I
was planning to document it and make it available for download later
this year (probably rewriting it again in the process, but I could
send you a preview tarball, if you want.

That sounds pretty interesting, I wouldn't mind taking a look at it.
I'd like to see what options are out there before I commit to one way
of doing it. The shell script seems to do the trick for now though.

In the last 3 years, I've read about a number of other build systems for
JS/CSS files. Most of them are more polished and better documented than
what I'm using, and everybody except me is probably better off using one
of those.

I can relate. It's helpful if the build system uses the same language
as your web app, you can compile all the js on the page into a
resource file to cut down on requests.

I've seen a few other build systems posted on people's blogs, but
nothing really caught my interest... I'll have to take another look at
what's out there.

It seems like the //#instruction syntax is mutually agreed upon across
most of the implementations I've looked at so far.

 

[Garrett Smith]

That's right. The Perl script did more than that: it also handled
options and preferences, paths, dependency resolution, cleanup, lint
checking, minification, and deployment/packaging.

Some of the current JS frameworks (YUI, for example) have specialized
build systems, which can be quite complex. I don't remember any specific
details, but I know that some of them can be used stand-alone to build
other projects. Could be interesting to look at what they're doing.

I have been using Ant to make custom builds. I use concat task and run
the result through YUI Compressor. Typically I have an application js
file and a page js file. The application file is something that most
pages will share. That file can benefit from caching. The page js file
is javascript something that is specific to the page.

The build process I use for applications (not Ape Library) is in a
thread somewhere here, but I can't find it now. Basically, I copy from
src -> build -> deploy. In build task, I combine and minify files. The
js deploy task copies to the webserver and usually a dependency of
deploy jsps (<target name="jsp.deploy" depends="js.deploy" ...).

Ah, found it.
http://groups.google.co.kr/group/comp.lang.javascript/browse_thread/thread/c5ec3a0ee6ebc28b

For some reason teh search result included Korean site for google groups.

The shortcoming of that is it does not allow for runtime builds of code.
For example, a page with a number of features, where the features used
varies depending on http request info or stored user data.

 

[Ry Nohryb]

(...)
It seems like the //#instruction syntax is mutually agreed upon across
most of the implementations I've looked at so far.

It makes sense.

$ cat build.js
//#include </Users/jorge/Desktop/assert.js>
//#include </Users/jorge/Desktop/buffer.js>
//#include </Users/jorge/Desktop/child_process.js>
//#include </Users/jorge/Desktop/crypto.js>

$ cat build.js | sed s/^\\s*\\/\\/#/#/ | cpp -E | sed s/^[#:]/\\/\\/#/
| tee raw.o.js | jsmin | tee o.js | gzip -n -9 -c > o.js.gz

 

[Ry Nohryb]

$ cat build.js
//#include </Users/jorge/Desktop/assert.js>
//#include </Users/jorge/Desktop/buffer.js>
//#include </Users/jorge/Desktop/child_process.js>
//#include </Users/jorge/Desktop/crypto.js>

$ cat build.js | sed s/^\\s*\\/\\/#/#/ | cpp -E | sed s/^[#:]/\\/\\/#/
| tee raw.o.js | jsmin | tee o.js | gzip -n -9 -c > o.js.gz

$ cat /Users/jorge/Desktop/{assert,buffer,child_process,crypto}.js \
| tee raw.o.js | jsmin | tee o.js | gzip -9 > o.js.gz

That's what I meant when I said "I hardly ever used" preprocessor
instructions - there's usually a simpler way to do the same thing ;-)

Isn't she (the bash shell) lovely ?

(...) {assert,buffer,child_process,crypto}.js (...)

Hardcoding the files to build into the .sh sounds to you like a good
idea ? Agreed that this particular build.js was a dumb one, but
usually there would be much more meat into it, ifdefs, etc.

 

[Ry Nohryb]

$ cat /Users/jorge/Desktop/{assert,buffer,child_process,crypto}.js \
| tee raw.o.js | jsmin | tee o.js | gzip -9 > o.js.gz

And maybe you should not bypass the preprocessor unless you know
beforehand that no src.js has any meat for it inside...