Develop Update Server in Java

[Simon]

Dear all,

I am looking for ideas, patterns, practices to develop a client/server
architecture that can be used to update clients when a new release is
uploaded to the server. Does anyone know any web resources considering
this topic? (WebStart is not what I want.)

(Yes, I know Google, but "update sever java" is not a good query string )

Cheers,
Simon

 

[John B. Matthews]

I am looking for ideas, patterns, practices to develop a client/server
architecture that can be used to update clients when a new release is
uploaded to the server. Does anyone know any web resources considering
this topic? (WebStart is not what I want.)

I recall this project having a very simple scheme: java.net.URL for a
quick version check and java.awt.Desktop to navigate to the project's
download page for updates:

<http://sourceforge.net/projects/jmcad/>

Why not WebStart? It conveniently integrates digital code signing. The
approach can't tell me whether I should trust your code; but, once I
decide to accept it, I'll know it's the code you intended:

<http://www.ibm.com/developerworks/java/library/j-webstart/>

 

[Simon]

You haven't provided much information. What kind of "client" are

you talking about, and what do you mean by "update?" Or, turn the
question around: What is it about your situation that makes Web Start
inappropriate? What do you want to do that Web Start doesn't (or
what does Web Start do that you'd rather it didn't)?

I want to do what almost every consumer software product does these
days, e.g. Firefox, Adobe Reader, etc: When it starts, check whether
updates are available, if they are, annoy the user with an "Update Me
Now!" dialog, and download and install the update.

Regarding WebStart, there are four issues I see:

- Our application comes bundled with a JRE which also has to be updated.
I cannot bundle a download version with a JRE with WebStart, because
WebStart requires the JRE to be installed.

- Part of the application requires a license key to authenticate the
download.

- The software must also be able to display a list of "Available
Extensions" among which the user can choose.

It might be that all of the above can be addressed by WebStart, but I'm
afraid the most important reason is this one:

- People are not used to WebStart. They easily become irritated by
things they have never seen. They want to download an EXE file and run
it. (This is not my opinion, it's just whart I observe, and it *is* a
reason, although we may not like it.)

Best,
Simon

 

[Lew]

You can, however, specify what Java version(s) the application
requires, and tell the user to install it if it's not already present.
(Note that he may have obtained Java from some site other than yours;
why push him a redundant copy?)

There's probably a way to give some assistance, but I don't think
you can automate it altogether. (Note that you can't completely
automate the other upgrades you mention, either.)

Why must you bundle the JRE?

Why can't the user use their own?

 

[Arne Vajhøj]

I want to do what almost every consumer software product does these
days, e.g. Firefox, Adobe Reader, etc: When it starts, check whether
updates are available, if they are, annoy the user with an "Update Me
Now!" dialog, and download and install the update.

Regarding WebStart, there are four issues I see:

- Our application comes bundled with a JRE which also has to be updated.
I cannot bundle a download version with a JRE with WebStart, because
WebStart requires the JRE to be installed.

- Part of the application requires a license key to authenticate the
download.

- The software must also be able to display a list of "Available
Extensions" among which the user can choose.

It might be that all of the above can be addressed by WebStart, but I'm
afraid the most important reason is this one:

- People are not used to WebStart. They easily become irritated by
things they have never seen. They want to download an EXE file and run
it. (This is not my opinion, it's just whart I observe, and it *is* a
reason, although we may not like it.)

JWS has the core functionality you need.

I find it very likely that it will be faster to develop the
bells and whistles you need on top of it than to start developing
the core functionality from scratch.

Arne

 

[Andrew Thompson]

     You can, however, specify what Java version(s) the application
requires, and tell the user to install it if it's not already present.

Sun supplies the deployJava.js - which will work even
if no Java is currently installed. It will check for
a suitable JRE and if not found, either guide the user
through installing it (Windows), or drop them onto the
Sun download page (Linux).

See
<http://java.sun.com/javase/6/docs/technotes/guides/jweb/
deployment_advice.html>
for further details.

     I don't know how to do this, but I'd bet its do-able.

See the ExtensionInstallerService. Demo at

     Maybe I'm missing something, but that doesn't look like a problem.

See the DownloadService. Crude demo at
<http://pscode.org/jws/api.html#ds>.

 

[Philipp]

I am looking for ideas, patterns, practices to develop a client/server
architecture that can be used to update clients when a new release is
uploaded to the server. Does anyone know any web resources considering
this topic? (WebStart is not what I want.)

The OSGi framework offers a flexible update mechanism. This is how
Eclipse does it. It also supports updating extensions separately from
the core elements of your app (actually it supports updating "bundles"
separately).

Several OSGi implementations exist: Equinox, Apache Felix and others,
have a look. It's very powerful but will require some adaptation to
your existing app.

HTH Phil

 

[alexandre_paterson]

One very good reason not to use JWS is that on OS X:

- users expects an app to be packaged the OS X way

- a *lot* of OS X users think that "Java sucks" (I don't
care about the reason for such thinking, I'm pragmatic and
want to make sales, not enter in some flamewar)

JWS screems "Java" everywhere... While a Java app bundled
using "Jar Bundler" and then put in a .dmg can "fly under
the radar".

These are two *very* good reasons to consider JWS unacceptable
and to 100% rule that option out.

cat flameswar > /dev/null

Another *very* good reason not too suggest JWS is that the OP
stated "WebStart is not what I want"

I am looking for ideas, patterns, practices to develop a client/server
architecture that can be used to update clients when a new release is
uploaded to the server.

Patterns and best practice, I don't know, but ideas sure

I've got a client-side Java app deployed on a lot of systems
and at every app startup they contact a server (which is a Java
webapp server) to verify that their licence is OK and that
they're running the latest version.

I'm not automatically pushing updates that said but in case
it helps...

The client-side app simply does an HTTP GET into which we put
the infos in a Base64-like (URL friendly) fashion:

http://example.com/Z/43QYfqvPYT7wGOAaIlftM8w_D43u7XbGC2j51W7UGh6yousIw8M0LiPa...

The info I send to the server are:

* version of our app
* JVM version
* OS
* proguard(ed) stack trace from last run if any
* etc.

Formatted in a way we have control over.

Then we zip/make-URL friendly and sent it using an HTTP GET.

The Java webapp server decodes the Base64-like then unzips
and we get all the infos we want back.

There's not limit on the size of an HTTP GET defined in RFC2616,
Tomcat by default accepts 8K GET but you can bump this up to 64K
if you wish (of course a POST would be fine too and would be more
'RESTful' but a GET is just sooo easy).

It's really amazing how much infos you can send using
zipped "strings" in a single HTTP GET request.

The client app reads the answer and there's a field that
contains the message to display to the user, if any.

If there's a message, the client-side app pops-up a message
to the user.

So it can be an automated "Version 1.07 is available" that
is displayed if user is running 1.06 or "Version 1.07 build 418
is available" if user is running "Version 1.07 build 316" and
was found affected by one of the bug that got fixed meanwhile
(we know this for stacktraces, if any, are send in that same
HTTP GET).

But it can also be message targetted to certain users. For
example we have a list of people who used to be beta-testers,
they tend to like to try "cutting edge" revisions, so we offer
them first the ability to download latest builds, etc.

You can simply drop newer builds on the server in a specific
directory and then the server chooses which message, if any,
to display to user, depending on the major/minor/build version.

Proguarded stack traces that comes back are automatically
'retraced' by our webapp server using the correct proguard-mapping
file and emailed to us as "crash logs".

New builds are simply 'dropped' on the webapp server and the
download page updates automatically as well as the response
to the HTTP GETs clients are making.

Once again we're not pushing updates automatically (but such
a scheme can be trivially modified to download latest .jar and
to use, at launch time, the latest .jar available)...

On the client side it's something like 30 lines of code, zipping,
Base64URlFriendly encoding and popping of the
JOptionPane.showMessageDialog
included

It may give you ideas

Alex

 

[alexandre_paterson]

Re,

I didn't read your second message before posting my first one...

- Part of the application requires a license key to authenticate the
download.

Same for our app... Amongst the info sent we sent the user's
product key and this influence messages send back by the server
(say "7 days left in your trial-version").

It might be that all of the above can be addressed by WebStart, but I'm
afraid the most important reason is this one:

- People are not used to WebStart. They easily become irritated by
things they have never seen. They want to download an EXE file and run
it. (This is not my opinion, it's just whart I observe, and it *is* a
reason, although we may not like it.)

Oh I hear you...

I gave the example about OS X in the other post.

There's no way, unless you want to loose 90% (guestimate) of your
OS X customers, that you'll be offering them a JWS app instead
of a .dmg file.

Pragmatism rules the day

As you said, it's a reason: it's not about liking that
fact on technical merits.

It's about that *fact*.

Alex,

 

[Andrew Thompson]

One very good reason not to use JWS is that on OS X:

I think you will find there are even more good reasons
not to 'use' an EXE on OS X.

 

[Roedy Green]

Dear all,

I am looking for ideas, patterns, practices to develop a client/server
architecture that can be used to update clients when a new release is
uploaded to the server. Does anyone know any web resources considering
this topic? (WebStart is not what I want.)

(Yes, I know Google, but "update sever java" is not a good query string )

Cheers,
Simon

See my student project outlines:
http://mindprod.com/project/deltacreator.html
http://mindprod.com/project/prebrandedsoftwarerental.html
http://mindprod.com/project/autoupdate.html
--
Roedy Green Canadian Mind Products
http://mindprod.com

When your language is nowhere near Turing-complete, syntactic sugar can be your friend.
~ Eric S. Raymond (born: 1957-12-04 age: 51) The Cathedral and the Bazaar

 

[Simon]

Why must you bundle the JRE?

Why can't the user use their own?

Because people don't know what a JRE actually is? Telling the user to go
to Sun's web page and try to find the suitable download for their system
will loose half the people on the way.

(Of course those who do know can download an unbundled version.)

I wasn't aware of the deployJava.js script Andrew mentions. That may
solve the problem.

Cheers,
Simon

 

[Simon]

Andrew Thompson wrote:

[some good examples and links]

I wasn't aware of these things, thanks Andrew. I may change my mind
about, can't guarantee the rest of us will, however

 

[Simon]

Thanks all for your answers. Mabe I'll change my opinion on JWS.

Some more things:

- I know that you can specify heap size with JWS. But is it possible to
set the heap size to some percentage of the free memory (at the time it
is launched)?

- Is it possible to let the user define a heap size from inside the
application? (Not inside some JWS configuration tool?)

- The application consists of two parts, one with a GUI and one command
line version. The latter will often be scheduled by cron or some other
tool on machines that may not even have a display. Will the user have to
enter "javaws -something" as the command to be executed by its scheduler
or can I provide some less confusable executable? At best this
executable would not be placed somewhere deeply hidden in a JWS directory.

- In the task manager on Windows, what do you see when you launch a JWS
application? "javaws" or the application name?

Best,
Simon

 

[markspace]

It isn't necessary or useful. On many OSes, such as some Linuces, there
isn't any "free" memory. The OS fills it all with program pages and
caches, swapping out as demanded. You might see 50 KiB of "free" memory
yet safely can allocate 1 GiB of heap.

This.

A swap file is used on any modern OS. I can (and have) run multiple JVM
instances set for 1.2 Gigs on my 2 Gig machine. Even with 10 of them
running, they all work just fine. So set your heap limit to what you
know works for your app, and let the OS figure it out.

There may be some corner-cases where this doesn't work (netbooks? Micro
edition?) but if you're running desktop apps then I'm sure that this
will work for more than 99.999% of your users.

 

[Simon]

It isn't necessary or useful.

As is any fixed number defined at deploy time.

On Windows, building a native launcher, this works well.

A swap file is used on any modern OS. I can (and have) run multiple JVM
instances set for 1.2 Gigs on my 2 Gig machine. Even with 10 of them
running, they all work just fine. So set your heap limit to what you
know works for your app, and let the OS figure it out.

There is no such number. It is a data analysis tool. If the user decides
to process an 8GB gene data file, then you need 8GB. Still, setting the
heap size to 8GB in general is not a good idea since Java will simply
not start if the machine does not have 8GB. More importantly, if the
heap size is too large and the OS starts to swap, this is worse than
getting an OutOfMemoryError. Swapping will almost freeze the app with no
chance of telling the user what the actual problem is.

Best,
Simon

 

[markspace]

There is no such number. It is a data analysis tool. If the user decides
to process an 8GB gene data file, then you need 8GB. Still, setting the

Well, you can't set an 8Gb heap size on a 32 bit JVM period. It's
limited to about 1.2G to 1.4G, depending on other parameters.

heap size to 8GB in general is not a good idea since Java will simply
not start if the machine does not have 8GB. More importantly, if the

Have you actually tried this? Because I don't think it works the way
you say.

On a 64 bit JVM, you can set 8Gb regardless of physical memory. The OS
just uses the swap file for any "extra" memory that it needs. That's
what virtual memory does. It effectively converts disc memory into RAM.
So you can most definitely set an 8Gb heap size on a machine with less
physical memory.

heap size is too large and the OS starts to swap, this is worse than
getting an OutOfMemoryError. Swapping will almost freeze the app with no
chance of telling the user what the actual problem is.

Have you actually seen this kind of swapping (what most folks call
"thrashing") on your app? I doubt it, because I don't think you're
actually testing with a 64 bit JVM.

However, if you have seen this behavior, there's garbage collector
tweaks you can use to alleviate it. I'd have to dig through the docs to
find them, but they're there. It might be worth checking in to if you
really think you need high amounts of JVM memory.

 

[markspace]

For what it's worth, the previous poster didn't actually write "physical
memory". It might have been what he meant, but he also might have
simply meant "does not have 8GB of memory".

That may be true. However, in my test, I made 10 JVM's with 1.2 Gig
each as the heap limit. My swap file has about 2x my physical memory,
for about 6 Gigs total. Yet I had 1.2G x 10 = 12 Gigs "allocated".

The heap limit is just that -- a limit. The JVM does not go out and
Bogart 8 Gig of memory as soon as you start it up, it just does not ever
allocate more than 8 Gigs.

So I believe that a JVM with a 8 Gig heap limit could be started on a
machine with less than 8 Gigs total available. You'd just never get to
the 8 Gig limit, the OS would stop allocating memory for you first.
Although I admit I have not tried this.

The confusion comes in, I think, when one tries to allocate 1.8 Gigs or
more on a 32 bit JVM, like most desktop JREs. The 32 bit JVM cannot
physically handle that much memory, period, so it issues an error
message immediately.

And when running in a VM environment, "memory" really means "disk
space". The physical memory is really just a cache of sorts; the disk

The disc cache on a VM system is usually referred to as the "backing
store." That's an old name, but I still see it used. The "swap file"
is an implementation of the backing store. "A cache of sorts" is... not
often used in the technical literature. Just sayin'.

 

[markspace]

What OS?

Windows Vista.

I admit, I don't know the exact implementation of the JVM. But I doubt
it's as you say. If it were, there would be no need to specify a limit
at all, and the default would just be the theoretical maximum size for

Consider the other case: plenty of RAM, (let's say 16 Gigs RAM and 32
Gigs in the backing store) and one wants to limit JVM usage to some
fraction of that. It's pretty common to allocate memory usage on large
servers manually. Say, 2G for the OS, 6G for the JVM, 8Gs for the
database, etc. Memory limits enforce that, without having the OS
terminate a process for exceeding its limit.

There was a big commotion in the MySQL users group meeting I attended
when someone brought up the fact that MySQL memory limit parameters
weren't accurate. You'd often get a little more or a little less memory
used than you asked for. This is a big deal for the bleeding-edge
database admin types, I gather. It kinda shied me alway from using
MySQL too, if they can't control the memory they allocate.

That directly contradicts your claim that Java doesn't pre-allocate the
heap. In particular, the only hard limit on 32-bit Windows is the 2GB

No, it just means that the JVM has compiled-in limits, and checks its
parameters first to make sure they're sane. If it finds an illegal
combination, it issue an error message and halts.

 

[Lew]

In other words, if you get a failure starting the 32-bit JVM under
Windows at some heap size smaller than 2GB, that's a very strong
indication that Java is in fact pre-allocating the heap.

No, it has to do with address space in a 32-bit pointer. The JVM "knows" that
it needs some memory for itself, so it only allows what's left over for the
"-Xmx" parameter.

GIYF:
<http://java.sun.com/docs/hotspot/HotSpotFAQ.html#gc_heap_32bit>
<http://publib.boulder.ibm.com/infoc...a/_1vg00014884d287-11c3fb28dae-7ff6_1001.html>