Aspect questions?

[Arne Vajhøj]

I'm surprised that your common class is so small in terms of number of
classes, especially after 10 years! I would have thought you'd have
written many more by now....

I've got 40 classes in mine already and those are just the ones I
consider more-or-less finished. I've got many more in my CommonCandidates
project where I store the ones that are still just half-baked (or
less!).I'm regularly thinking of new ones to add....

You will know that you need to split when you start
spending time looking after the right class scrolling up
and down, because the list is long.

I think I would have split before 40.

Arne

 

[Arne Vajhøj]

I disagree, with respect for your position.

Not that the number will be greater than one, but that it's a motivator
for the distinctions.

The difference between 'com.novice.common' and
'com.novice.common.lookups' and 'com.novice.common.utilities' should
take no note of how many types (NOT just classes!) would go in each
package, but whether those in the first are merely "common", those in
the second are common to "lookups", and those in the third are common
"utilities".

If you have a hard time semantically distinguishing one package from
another, then you shouldn't have more than one package. Regardless of
the number of types in it.

A package is a namespace, not some finite-sized bucket that holds some
particular number range of items. Its purpose is to semantically group
related types together. What determines co-residence in a package is
semantic similarity, not quantity.

Hmmm.

I can not disagree that logical grouping is a driver for package
structure.

But I don't believe that there is a single possible logical
grouping.

Let us compare:

1)

com.foobar

2)

com.foobar.a
com.foobar.b

3)

com.foobar.a.x
com.foobar.a.y
com.foobar.b.z
com.foobar.b.w

4)

com.foobar.a.x.bla
com.foobar.a.x.blabla
com.foobar.a.y.zzz
com.foobar.a.y.zzzzz
com.foobar.b.z.one
com.foobar.b.z.two
com.foobar.b.w.lion
com.foobar.b.w.tiger

In all of these 4 options the classes can be logical grouped.

The question is at what granularity do you want to split.

Everything in one package is not good.

One package per class is not good.

But somewhere in between one need to pick a level of
granularity.

And packages was not invented for fun but to help the
developer.

So one should pick the level of granularity that makes it
easiest to find things.

According to this source:

http://binstock.blogspot.com/2008/04/perfecting-oos-small-classes-and-short.html
then Thougthworks has recommended "no more than 10 classes per package".

That seems a bit on the low side for me.

YMMV

Arne

 

[Lew]

Hmmm.

I can not disagree that logical grouping is a driver for package
structure.

But I don't believe that there is a single possible logical
grouping.

I agree with you on this point.

Let us compare:

1)

com.foobar

2)

com.foobar.a
com.foobar.b

3)

com.foobar.a.x
com.foobar.a.y
com.foobar.b.z
com.foobar.b.w

4)

com.foobar.a.x.bla
com.foobar.a.x.blabla
com.foobar.a.y.zzz
com.foobar.a.y.zzzzz
com.foobar.b.z.one
com.foobar.b.z.two
com.foobar.b.w.lion
com.foobar.b.w.tiger

In all of these 4 options the classes can be logical grouped.

The question is at what granularity do you want to split.

Everything in one package is not good.

One package per class is not good.

But somewhere in between one need to pick a level of
granularity.

And packages was not invented for fun but to help the
developer.

So one should pick the level of granularity that makes it
easiest to find things.

According to this source:

http://binstock.blogspot.com/2008/04/perfecting-oos-small-classes-and-short.html
then Thougthworks has recommended "no more than 10 classes per package".

That seems a bit on the low side for me.

YMMV

You illustrate one way in which programming is a matter of art.

Ultimately you do what makes the most sense to you at the time. This is where
Patricia's advice to refactor willingly (but not willy-nilly, of course) takes
the heat off having to have one right answer right away.

“A good plan violently executed now is better than a perfect plan next week.â€
— General George S. Patton, Jr., U.S. Army

(Gods! I hear this in George C. Scott's voice. I can't help it.)

 

[Arved Sandstrom]

Typical:
- one logger instance per class
- one log file per app

Note that configuration of log files and other destinations for
logging should not be in the code but in the log config file.

Arne

Also typical: _several_ log files per app. The subsystems that comprise
an application may have such different audiences, log levels, log
volume, and log file treatment (rollover policies, compression etc),
among other things, that it simply makes sense to configure logging for
multiple output files. sed/grep/Splunk filtering and so forth only goes
so far; it's sensible to use the logging framework itself to do some of
the heavy lifting too.

Some cautions with respect to that: it's easier to filter than it is to
combine. And logs are most effective when all of the entries, at all
levels, that pertain to something are in the same place (file, console,
whatever) in chronological order. So if log output does get split the
different sets of entries should be quite independent.

AHS

 

[Jeff Higgins]

You do mean these questions and your proposed exercise rhetorically,
right?
No.

I'm willing to actually do them if it would help me do a better
job

Look at other projects, lots of them.
I guess I should have added:
Apply critical thinking to each.

but I'm inclined to think that you don't mean that you literally want
me to do this right in the newsgroup.

This is usenet, not a study hall.
Do your study in a comfortable quiet place.

Correct me if I'm wrong. (I'm
speaking about the suggestions in your follow-on posts too.)

Speakers outline their speeches before writing them.
Authors outline their papers before writing them.
Programmers outline their programs before writing them.
What better way to outline a Java program than javadoc?

 

[Lew]

To be fair, both Unix and Windows have master logs that are shared by
several different processes (syslog and windows event log) without
that appearing to cause much in the way of headaches.

A syslog is conceptually for a single app, the OS. Also each app may have its
own log besides the syslog. It's true, furthermore, that not every project or
app on Windows or *nix writes to the syslog or event log. It's also true that
Windows apps don't write to *nix syslogs as a rule, nor vice versa.

As before, every "rule" I present may have exceptions. In the context of the
OP's question, it was important to distinguish that in general different apps
have different logs. At no point did I suggest the ratio had to be one to one.

I also did not say that it did not have to be one to one. Your post is very
useful for making that point.

 

[Arne Vajhøj]

I agree with you on this point.


You illustrate one way in which programming is a matter of art.

Or craftsmanship.

Arne

 

[Arne Vajhøj]

Also typical: _several_ log files per app. The subsystems that comprise
an application may have such different audiences, log levels, log
volume, and log file treatment (rollover policies, compression etc),
among other things, that it simply makes sense to configure logging for
multiple output files. sed/grep/Splunk filtering and so forth only goes
so far; it's sensible to use the logging framework itself to do some of
the heavy lifting too.

I have not seen different log files for different audiences much.

I have seen log file for everything and some other destination for
the higher levels rather frequently.

Arne

 

[Martin Gregorie]

I'm surprised that your common class is so small in terms of number of
classes, especially after 10 years! I would have thought you'd have
written many more by now....

I don't add anything to my environ package until (a) its been written
because I needed it in some program I was working on and (b) I needed it
in another unrelated program. Unless/until both conditions are met it
remains a part of the original project-specific package.

I'd previously done the same in C: this library would be similarly small
except that programs that use it are written to run on OS/9, DOS/Windows
and Linux. The functions making up the standard C libraries are not the
same. As a result the library exists in three versions, one for each OS,
and each is augmented with functions that I need that are standard in one
of the other OSen. At most this requirement has doubled the size of the
environ library and, as you might expect, this part of its content is
similarly static. Not having to write these compatibility functions is a
big advantage of using a WORA language like Java.

 

[Novice]

On 2/28/2012 12:04 AM, Lew wrote:
...
...

When it does come down to an art, after applying whatever science I
have available, I often resort to the "What is easiest to document?"
test. For example, I like a method for which it is really easy to
write a Javadoc comment, especially the first sentence.

To apply this to a packaging issue, try to write the Javadoc
description(s).

If you find yourself, within one package, having to say a lot about
some particular subset of the classes, consider making that subset a
separate package. The subset has a lot of commonality or interaction.

If you find yourself doing a lot of cross-referencing between package
descriptions, or repeating a lot of material, you may have divided
things up too much.

I like this approach. As someone who often starts with comments and then
writes code, this appeals to me.

 

[Novice]

Oops, I got the standard API and the Apache Commons Collections API
mixed up. Sorry.


No, I made a mistake. There is such a thing but I didn't really mean
to recommend something outside the standard API.

No worries!

But if you ever do want 'TreeList', here's what I was misremembering:
<http://commons.apache.org/collections/api-release/org/apache/commons/c
ollections/list/TreeList.html>

Commons Collections is a very popular package that aims to expand and
augment the standard Collections API.

I'm already a fan of the Collections classes in regular Java and use them
eagerly. I must have a look at Commons Collections now....

And I need to look at List and ArrayList in more detail too. I've already
glanced at them and expect to start a thread on Interfaces soon so that I
can hash out some of my conceptions and misconceptions about interfaces
vs. classes. It might take a couple of days because I've got some other
things that I need to do first....

 

[Novice]

Look at other projects, lots of them.
I guess I should have added:
Apply critical thinking to each.

Excellent advice!

This is usenet, not a study hall.
Do your study in a comfortable quiet place.


Speakers outline their speeches before writing them.
Authors outline their papers before writing them.
Programmers outline their programs before writing them.
What better way to outline a Java program than javadoc?

I have no problem with that. In fact, I often write comments before I
write code. Even a single sentence laying out what I'm trying to
accomplish helps me clarify and organize what the code needs to do.

 

[Novice]

You will know that you need to split when you start
spending time looking after the right class scrolling up
and down, because the list is long.

I think I would have split before 40.

I'm torn on that. I could certainly resplit them but I really don't see
much relationship between most of them. I do have a small handful that
are really just String manipulation that could go together. But most of
the rest are standalone.

I also have a few classes that are basically used to create panels in
GUIs. For instance, I have several class that will create a small panel
containing a single field (text field, text area, spinner, checkbox or
whatnot) with an optional label, but they're all independent of one
another. It's entirely possible just to use one on a given GUI without
ever having one of the others. Those don't feel related to me beyond the
fact that they are JPanels so I'm not inclined to lump them together.

 

[Novice]

I don't add anything to my environ package until (a) its been written
because I needed it in some program I was working on and (b) I needed
it in another unrelated program. Unless/until both conditions are met
it remains a part of the original project-specific package.

I'm basically doing the same thing, except that in some cases when I
write something new for Project X, I feel sure it will get used again in
upcoming projects (or be retrofitted to existing incomplete projects) so
I put it in Common straightaway.

I'd previously done the same in C: this library would be similarly
small except that programs that use it are written to run on OS/9,
DOS/Windows and Linux. The functions making up the standard C
libraries are not the same. As a result the library exists in three
versions, one for each OS, and each is augmented with functions that I
need that are standard in one of the other OSen. At most this
requirement has doubled the size of the environ library and, as you
might expect, this part of its content is similarly static. Not having
to write these compatibility functions is a big advantage of using a
WORA language like Java.

I was a neophyte C programmer in the early/mid 90s and had written one
real program in that language to run in DOS. (A couple of friends who
were very skilled in C helped me over the rough spots.) I was just
starting to be keen on the language but then discovered that it was a big
can of worms to try to port C to other platforms. I got very uneasy about
that and looked around for other alternatives. I discovered that Java,
which was very much in its infancy, was designed to run on lots of other
platforms without all the work involved in porting C and that it started
with the same syntax as C _and_ added classes like C++ (which I was also
in the very early stage of learning). Well, I was sold and moved over to
Java right away and have few regrets. (There are times I wish I had also
improved my C and learned more about C++ but I suspect that would have
started muddling them altogether a bit if I had done that.)

I envy you for being fluent in both C and Java (and probably others!). I
have other languages but they're pretty much all very rusty from lack of
use. Mind you, I have found that I can relearn things pretty quickly even
after a long gap. I had occasion to look at a COBOL program a few years
back and found it very familiar. Mind you, I doubt I'd say the same about
C if I were to try that again ;-)

 

[Novice]

You do ask very good questions.

Each program should have its own log. I don't know what you mean when
you say "project", but conventionally a project is tied to one
specific program or application.


And as there is a different target for each archer, there should be a
different log for each program.

Really, I've already given you the advice you need to answer your own
question. I told you to design logs to benefit people who need them to
troubleshoot.

How will it help an operations engineer or reliability engineer to
have to sift through dozens of applications' output to find what they
need? It's bad enough to sift through one application's log,
especially the way most programmers fail to think through their
logging output.

I'm 100% in agreement with you. I would certainly lean toward fewer logs
than more logs. I would embrace One Big Log long before I would choose to
have a separate log for each class!

I must have remembered this wrong. I coulda sworn someone suggested to
use the "-D" option per
<http://docs.oracle.com/javase/7/docs/technotes/guides/logging/overview
.html> which in the section entitled "Configuration File"
<http://docs.oracle.com/javase/7/docs/technotes/guides/logging/overview
.html#a1.8> points you to "See the LogManager API Specification for
details."
<http://docs.oracle.com/javase/7/docs/api/java/util/logging/LogManager.
html> where it tells you

As I said, I remember seeing something about a -D option too. But I tried
REALLY hard to find it again and couldn't. I still don't know where that
post went. So thanks for repeating the information.

«In addition, the LogManager uses two optional system properties that
allow more control over reading the initial configuration:

"java.util.logging.config.class"
"java.util.logging.config.file"
These two properties may be set via the Preferences API, or as command
line property definitions to the "java" command, or as system property
definitions passed to JNI_CreateJavaVM.»

I don't have these link chains memorized, dude. I'm literally reading
the documentation on your behalf and reporting what I discover.

There's only so far you can get as a programmer asking others to read
the documentation for you.

Sorry, I do get a bit lazy on occasion. Just say no if you sense that
happening. But I really couldn't find those links and I spent at least a
half hour looking.

study the Javadocs for
'getResource[AsStream]()'.

Do I actually need to open the logging.properties file and read it,
as if it were a routine text file, to find the desired log path? That
seems improbable....

I'm only suggesting you study those Javadocs to understand how the
system searches for resources.

For what it's worth, someone else told me to add it to the classpath
for the project so I went into Run Configurations, chose the
Classpath tab and added the folder containing logging.properties
under "User Entries". I also

Umm - cargo cult.

In retrospect, yes, you're right. If I had found those links again or
tried a different google search, I might have found that pesky -
Djava.util.logging.config.file parameter and realized it was the
linchpin. But I was going crazy trying to find that darned log file so
that I could actually TRY some of the things people have been suggesting.

Didn't someone already ask you to share your logging.properties file
with us? Have you done so?

Yes, I did. But this thread is so unwieldy now that it's hard to find
things again. I'm having to click on a great many of the posts in the
thread just to find things. I'm in mid-conversation with several people
at the same time but I can't find any way to bookmark individual posts in
this newsreader....

Regardless, you are asking questions now that the documentation
answers. I have read the documentation for you and reported what I
found, to a degree.

So will your next post be another request to read the documentation
for you?

No, that will not be necessary. You didn't hear from me yesterday and
that's because I had a great day yesterday. I finally put that
-Djava.util.logging.config.file parameter in my VM arguments and pointed
it to my logging.properties file. I can finally see the bleeping log that
I'm writing!!!

And that let me get a great deal of useful work done. I reworked my Foo
and Common projects so that no loggers or locales are passed between
classes. In each class that needs to do logging - and no others, I do a
Logger logger = Logger.getLogger(getClass().getName());

In classes where only one or two methods do logging, logger is a local
variable. In classes where lots of methods do logging, I make it an
instance variable and instantiate it in the constructor.

I've become very fond of logging.properties because I figured out how to
assign my desired custom formatter there rather than having to do it
programmatically. I was very surprised to find that I was getting log
records from various java.awt.*, java.swing.*, and sun.* classes but I
keep all but the WARNING and SEVERE ones out of the log via the
appropriate lines in the logging.properties file.

In short, I am a lot closer to doing logging the way it is supposed to be
done rather than the messed-up way I was doing it before. I still need to
work with it a bit to fine-tune my approach to what should be logged and
what the exact level and content of each log record should be but that is
going to be an ongoing thing.

I even understand that advice in the article which Arved recommended
where the author said not to put the class name in the log record. Now
that I've got the actual class name as the Logger value in the record, I
can see that I don't need it again elsewhere in the same log record.

I know that some people do really well by just hearing principles and
concepts; with that under their belts, they can do exactly what needs to
be done with minimal hesitation. But I tend to need to see things
actually working with my own two eyes for concepts and principles to gel
properly. Before that, I'm not sure if I'm really envisioning things
accurately. That's why I was getting so antsy about seeing that log. Now
that I can see and try things and see real log records, things are
starting to solidify for me.

So thank you (everyone who contributed to this thread but especially Lew)
VERY MUCH for your patience.

I'll go try finishing up any dangling conversations within this thread
now and then get on to my next issue ;-)

 

[Novice]

Typical:
- one logger instance per class
- one log file per app

That's exactly the kind of generalization I was looking for elsewhere in
the thread.

Note that configuration of log files and other destinations for
logging should not be in the code but in the log config file.

Now that I've found my own log files and worked with them a bit, this all
makes sense. The log config file is becoming my new best friend. Your
comment is very helpful here because I was still inclined to assume that
I should have a single logging.properties file for my entire system. But
it follows from what you've said that I should have a different
logging.properties for each app. Therefore, my Foo project, which is
essentially a single app - it's a tiny menu with the choice of two
programs, a main one and a supporting one - is going to have its own
logging.properties. Projects Bar and Baz are each going to have their own
distinct logging.properties as well. Each logging.properties file will
independently manage the level of logging of the classes/packages used by
the code invoked by that project.

This logging stuff is FINALLY coming together for me. Thanks a lot to
everyone for their help with this!

 

[Daniel Pitts]

No worries!


I'm already a fan of the Collections classes in regular Java and use them
eagerly. I must have a look at Commons Collections now....

Apache commons has a lot of useful libraries that are missing from core
Java. The StringUtils are very useful as well.

 

[Lew]

And I need to look at List

and ArrayList

in more detail too. I've already
glanced at them and expect to start a thread on Interfaces [sic] soon so that I
can hash out some of my conceptions and misconceptions about interfaces
vs. classes. It might take a couple of days because I've got some other
things that I need to do first....

I favor a style of programming I call "type-based programming" that relies on
interfaces and generics.

Since you say you write comments first, this should be natural for you.

An interface is like a compilable comment that specifies what classes are
supposed to accomplish. Classes are specific implementations of what their
interfaces promise.

In general, per the advice of /Effective Java/ (2nd ed.) by Joshua Bloch* and
other wise people, you should declare variables, return types, etc., as the
most general (closest to interface) type that promises what you need.

If all your list needs is what's promised by 'List<E>', you declare the type
as the interface type. If it needs additional methods or attributes declared
by 'ArrayList<E>' but not in the interface, you use the more specific type. If
you don't even need the "listiness" but just that it's a collection, declare
the type 'Collection<E>'.

When you design a program, design it in terms of interfaces. Defer
implementations as far as makes sense. This lets you focus on _what_ the
program should do rather than _how_.

I frequently write interfaces even when I'm only writing one implementing class.

So taking lists as the example:

List<String> options = new ArrayList<>();

I declared the 'options' variable as a 'List<String>', not as an
'ArrayList<String>'. If I discover that the array-backed list wasn't giving me
the performance or other implementation-specific behavior I need, but a linked
list does, I can refactor more easily:

List<String> options = new LinkedList<>();

The rest of the program, depending only on the "list"ness and not the
"array"ness, is unharmed.

(What performance differences are there? Excellent question! Pop quiz - please
answer this post with your answer - where would you find out about performance
characteristics of 'ArrayList<E>' and 'LinkedList<E>'?)

Summary:
Interface: what to do
Class: how to do it

 

[Novice]

And I need to look at List

and ArrayList

in more detail too. I've already
glanced at them and expect to start a thread on Interfaces [sic] soon
so that I can hash out some of my conceptions and misconceptions
about interfaces vs. classes. It might take a couple of days because
I've got some other things that I need to do first....

I favor a style of programming I call "type-based programming" that
relies on interfaces and generics.

Since you say you write comments first, this should be natural for
you.

An interface is like a compilable comment that specifies what classes
are supposed to accomplish. Classes are specific implementations of
what their interfaces promise.

In general, per the advice of /Effective Java/ (2nd ed.) by Joshua
Bloch* and other wise people, you should declare variables, return
types, etc., as the most general (closest to interface) type that
promises what you need.

If all your list needs is what's promised by 'List<E>', you declare
the type as the interface type. If it needs additional methods or
attributes declared by 'ArrayList<E>' but not in the interface, you
use the more specific type. If you don't even need the "listiness" but
just that it's a collection, declare the type 'Collection<E>'.

When you design a program, design it in terms of interfaces. Defer
implementations as far as makes sense. This lets you focus on _what_
the program should do rather than _how_.

I frequently write interfaces even when I'm only writing one
implementing class.

So taking lists as the example:

List<String> options = new ArrayList<>();

I declared the 'options' variable as a 'List<String>', not as an
'ArrayList<String>'. If I discover that the array-backed list wasn't
giving me the performance or other implementation-specific behavior I
need, but a linked list does, I can refactor more easily:

List<String> options = new LinkedList<>();

The rest of the program, depending only on the "list"ness and not the
"array"ness, is unharmed.

(What performance differences are there? Excellent question! Pop quiz
- please answer this post with your answer - where would you find out
about performance characteristics of 'ArrayList<E>' and
'LinkedList<E>'?)

Summary:
Interface: what to do
Class: how to do it

I need to understand what you've said a bit better before I try the pop
quiz ;-)

It makes perfect sense to me to figure out what a program needs to do
before you worry about how it gets done. You've got to design a house - how
many bedrooms, how many bathrooms, what style of house (ranch or bungalow
or apartment building), etc. etc. - before you choose the exact fixtures.

But when you talked about designing interfaces, I assumed you meant NEW
interfaces. Your examples seem to be about selecting which of the existing
interfaces - which kind of lists - you wanted. But those kinds already
exist so you're really just choosing them, not inventing new ones to do new
things. It would be helpful to me if you also talked about new interfaces
and how those come about at design time.

By the way, I completely see your point in the examples. I have already
benefited from the same by using Collections. I use more Sets than Lists
and I will often start with a HashSet but if I start using it and find it
doesn't have things in the order I wanted, it's a trivial thing to change
it to another ordering by choosing a TreeSet or LinkedHashSet. That is
REALLY convenient.

Now that I think about it, I suppose I know enough to tackle the pop quiz
after all.

I have the feeling that I'm supposed to answer with something like "the
API" or maybe based on something you said in one of your posts. Well, I'm
going to "over-answer" the question and say this:

I know that the API or the various Java Tutorials (like the one on
Collections) often gives clues about performance and tells you explicitly,
for example, that a HashSet normally performs best and a TreeSet worst with
a LinkedHashSet typically performing almost as well as a HashSet. So that's
obviously a good place to start.

But you can obviously go a lot further. Various options come to mind:
- Simply try executing a few different implementations. This is worthwhile
especially if it is a matter of changing just one line of code to get an
alternate implementation. It might be completely impractical if we're
talking about developing hundreds or thousands of lines of code to get
another implementation.
- Ask other people. If you have doubts about a given approach, ask on
newsgroups like this one what their experience has been. The veterans here
can probably tell you from personal experience that Approach A is awful and
Approach B is much better and Approach C is even better.
- Fire up your debugger and try tracing through your implementation to see
how efficient it seems. You can sometimes find bottlenecks that way that
convince you to try a different approach.
- Performance tools. I have very little experience with these but I've
heard whispers that there are tools you can apply to get real numbers as
your program runs so that you can see if it is a pig. I expect the better
tools in this category will even suggest changes to your code (or
environment) to improve performance and maybe even make those changes for
you.
- Benchmarking. Set up your own benchmark to compare different
implementations that solve a problem. Potentially lots of work involved but
might be necessary if all else fails.

Okay, back to you.... ;-)

 

[Martin Gregorie]

I envy you for being fluent in both C and Java (and probably others!). I
have other languages but they're pretty much all very rusty from lack of
use. Mind you, I have found that I can relearn things pretty quickly
even after a long gap. I had occasion to look at a COBOL program a few
years back and found it very familiar. Mind you, I doubt I'd say the
same about C if I were to try that again ;-)

Just about all I use these days are C and Java plus a few scripting
languages (awk, PHP, bash shell scripting and Perl if you insist). In
another life I wrote much more COBOL than was good for me, so could
probably get up to speed fast with that too. There are a raft of others I
used for single projects (PL/1) or that were specific to particular
hardware (TAL, PL/9, filetab, RPG III and various assemblers).

I'm not sure its useful to know a lot of languages: idioms often don't
transfer don't at all well and if you're not careful you can end up
writing the nasty sort of code best summarized as "a Real Programmer can
write FORTRAN in any language".