Concurrent, persistent background process for a J2EE container

A

Arne Vajhøj

Ignatius said:
Writing logs to a database... why? The logs should be simple, so that
they are present even when things don't work. How are you going to
debug database problems when the logs didn't get written because the
database was supposed to be hold the logs in the first place? This
seems circular, and unnecessarily complex. Even baroque.

If I got an app that wrote logs to a database, I think I'd curse your
name. Just saying....

It is a goofiest use of logs if you need an audit trail.

I fatten with your comments for unacceptable troubleshooting log.

Arne



- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Ibrahim Nafie Al-Ahram, Egypt, November 5

"Is it anti-semitism ? Or is it a question of recognising
expansionist and aggressive policies?

Israel's oft-stated weapon of anti-semitism has become truly
exposed ...

Tel Aviv has been called upon to explore the reasons behind
the Middle East conflagration. It is these reasons that make
Israel a rogue state in the real sense of the word.
Enough of crying 'anti-semitism' to intimidate others."
 
D

Donkey Hottie

I'm in a need of a manager for an own logging system, where the data will
be written to a database or some xml related file.

I would like to dedicate a process or a thread for writing the log, and in
addition to that for some cleanup and other management.

I'm not allowed to use JMS.

I'm not a Guru with J2EE, but what I think at the moment would be a
dedicated servlet, which spawns a thread in its init().

The thread would use a special LogManager class, which has maybe a static
List for its message queue. That would be "written" and "read" in
synchronised methods.

Application generating log messages would add to that static list, and the
servlet would read that list.

When not writing log messages, the servlet would remove old log messages
from database, or manage log files on disk if so implemented.

I'm I on a decent track?

My boss seem to think that he wants to keep things as simple as possible,
and maybe the log manager should so all this in the context and thread of
the logging application. That might be well possible, because out app run
on intranets, and there will not be much traffic.

But I just fancy that the messages should be put on a queue, and managed by
a singleton.
 
M

Mark Space

Donkey said:
My boss seem to think that he wants to keep things as simple as possible,
and maybe the log manager should so all this in the context and thread of
the logging application. That might be well possible, because out app run
on intranets, and there will not be much traffic.

I think your boss is correct. The Java LogManager is pretty robust.
I'd definitely try that first, and only abandon it when it was proven it
couldn't work. The LogManager is pretty sophisticated, it'll even
rotate your logs for you.

Writing logs to a database... why? The logs should be simple, so that
they are present even when things don't work. How are you going to
debug database problems when the logs didn't get written because the
database was supposed to be hold the logs in the first place? This
seems circular, and unnecessarily complex. Even baroque.

If I got an app that wrote logs to a database, I think I'd curse your
name. Just saying....
 
D

Donkey Hottie

I think your boss is correct. The Java LogManager is pretty robust.
I'd definitely try that first, and only abandon it when it was proven
it couldn't work. The LogManager is pretty sophisticated, it'll even
rotate your logs for you.

Writing logs to a database... why? The logs should be simple, so that
they are present even when things don't work. How are you going to
debug database problems when the logs didn't get written because the
database was supposed to be hold the logs in the first place? This
seems circular, and unnecessarily complex. Even baroque.

If I got an app that wrote logs to a database, I think I'd curse your
name. Just saying....

We can't use java.util.logging for this. Boss wants "structured" data, not
flat text files.

Actually our logging needs are 3 fold.

We have a web type server managing identities and access, kind of an
Identity and Access management solution. It's requests and actions must be
logged, and plain text web server type log file is not enough.

Then we have and Administration GUI Web Service for it, and every action
there must be logged, identifying actions, entity and it's attributes, when
updates, old and new values logged. Very "structural" data.

Finally we have "normal" debug log, which can be handled with
java.util.logging (currently we use Log4j, but boss wants to get rid of
open source).

Our admin/access audit logs will be written to (implementation may vary) to
a database or disk file in some structured format, and those logs will be
available in the Management Console GUI. Debug logs must not be visible in
the GUI, but they must be turned on and off from said GUI. I'm prepared to
let log4j go and use java.util.logging for that, it works ok.
 
D

Donkey Hottie

If I got an app that wrote logs to a database, I think I'd curse your
name. Just saying....


Yes. Different kinds of logs. Error and debug to normal logger file, but
Audit data to database, and accessible to the end users via browser.
 
M

Michael Justin

Donkey said:
Finally we have "normal" debug log, which can be handled with
java.util.logging (currently we use Log4j, but boss wants to get rid of
open source).


From Wikipedia:

"Sun's goal is to replace the parts that remain proprietary and
closed-source with alternative implementations and make the class
library completely free and open source."

http://en.wikipedia.org/wiki/Java_(software_platform)#Free_software



So boss wants to get rid of Java altogether soon?

SCNR :)
 
A

Arne Vajhøj

Donkey said:
We can't use java.util.logging for this. Boss wants "structured" data, not
flat text files.

Even java.util.logging support custom formatters and handlers.

(Log4j is better though)
Our admin/access audit logs will be written to (implementation may vary) to
a database or disk file in some structured format, and those logs will be
available in the Management Console GUI.

java.util.logging and Log4j can be customized for a task like that.

Arne
 
A

Arne Vajhøj

Donkey said:
but boss wants to get rid of
open source).

Does he also want to avoid files with multiple of 13 number
of lines because 13 is an unlucky number ?

Because that is about as serious.

If an open source product solves the business and problem
and the license is compatible with your usage, then there
are really no reason to avoid it.

Arne
 
A

Arne Vajhøj

Mark said:
Writing logs to a database... why? The logs should be simple, so that
they are present even when things don't work. How are you going to
debug database problems when the logs didn't get written because the
database was supposed to be hold the logs in the first place? This
seems circular, and unnecessarily complex. Even baroque.

If I got an app that wrote logs to a database, I think I'd curse your
name. Just saying....

It is a common use of logs if you need an audit trail.

I agree with your comments for traditional troubleshooting log.

Arne
 
D

Donkey Hottie

Even java.util.logging support custom formatters and handlers.

(Log4j is better though)


java.util.logging and Log4j can be customized for a task like that.

Arne

Can java.util.logging take a parameter like a own rolled Java Object, not a
String as a thing to log? or a String of xml formatted object?

Ok, maybe it can. XML string is a string. Have to think about this.. maybe
no need for a custom formatter, put the xml in, and read it out. As long as
it's on one LINE in the log it might work.
 
M

Mark Space

Donkey said:
We can't use java.util.logging for this. Boss wants "structured" data, not
flat text files.

So you know about XMLFormatter for logging, right? I'm curious: why
doesn't that work?

It'd consider writing your formatted data to a socket, just on the
general principle that it might be more robust and more portable than
permanently wiring your logs to a database. However, you probably know
your requirements best. I do think writing to a socket would obviate
the need for any kind of multi-threading however.
 
D

Donkey Hottie

So you know about XMLFormatter for logging, right? I'm curious: why
doesn't that work?

Nope. But you got me interested! Like I said, I'm no guru in Java EE.
It'd consider writing your formatted data to a socket, just on the
general principle that it might be more robust and more portable than
permanently wiring your logs to a database. However, you probably
know your requirements best. I do think writing to a socket would
obviate the need for any kind of multi-threading however.

Socket? But there always has to be an another thread or process in the
other end of the socket, no?

Socket programming is nothing new to me, 20+ years with C++ but J2EE
practices are new to me.
 
A

Arne Vajhøj

Can java.util.logging take a parameter like a own rolled Java Object, not a
String as a thing to log? or a String of xml formatted object?

Ok, maybe it can. XML string is a string. Have to think about this.. maybe
no need for a custom formatter, put the xml in, and read it out. As long as
it's on one LINE in the log it might work.

See the code below for an example of what a custom formatter can do.

If you need it written in a custom way we will also need a custom
handler.

Arne

PS: The code below is demo code - you may want to make it a bit more
robust for production.

===============================================================

import java.util.logging.Formatter;
import java.util.logging.LogRecord;
import java.util.logging.Logger;

public class CustomFormatter extends Formatter {
@Override
public String format(LogRecord record) {
return "####" + record.getMessage() + "####" +
System.getProperty("line.separator");
}
public static void setup() {
Logger log = Logger.getLogger("foobar");
while(log.getParent() != null) log = log.getParent();
log.getHandlers()[0].setFormatter(new CustomFormatter());
}
public static void main(String[] args) {
CustomFormatter.setup();
Logger logger1 = Logger.getLogger("test");
logger1.info("This is a test");
Logger logger2 = Logger.getLogger("test.test");
logger2.info("This is also a test");
}
}
 
D

Donkey Hottie

Can java.util.logging take a parameter like a own rolled Java Object,
not a String as a thing to log? or a String of xml formatted
object?

Ok, maybe it can. XML string is a string. Have to think about this..
maybe no need for a custom formatter, put the xml in, and read it
out. As long as it's on one LINE in the log it might work.

See the code below for an example of what a custom formatter can do.

If you need it written in a custom way we will also need a custom
handler.

Arne

PS: The code below is demo code - you may want to make it a bit more
robust for production.

===============================================================

import java.util.logging.Formatter;
import java.util.logging.LogRecord;
import java.util.logging.Logger;

public class CustomFormatter extends Formatter {
@Override
public String format(LogRecord record) {
return "####" + record.getMessage() + "####" +
System.getProperty("line.separator");
}
public static void setup() {
Logger log = Logger.getLogger("foobar");
while(log.getParent() != null) log = log.getParent();
log.getHandlers()[0].setFormatter(new CustomFormatter());
}
public static void main(String[] args) {
CustomFormatter.setup();
Logger logger1 = Logger.getLogger("test");
logger1.info("This is a test");
Logger logger2 = Logger.getLogger("test.test");
logger2.info("This is also a test");
}
}

It outputs

####This is a test####
####This is also a test####



The base of if is the record.getMessage()

I need a structured message, not a String.

Ok, I could extend LogRecord, and get its getters for info. But how about
Logger.


I want a log record like this

<LogEntry>
<Login>admin</login>
<from>192.168.1.57</from>
<action result="success">MODIFY</action>
<entity type="Customer">
<id>35252</id>
<attributes>
<attribute name="address">
<oldvalue>Ankkalinna 11</oldvalue>
<newvalue>Hanhilinna 12</newvalue>
</attribute>
</attributes>
</entity>
</LogEntry>

How In earth do I that with logger.info()
 
M

Mark Space

Donkey said:
Nope. But you got me interested! Like I said, I'm no guru in Java EE.

java.util.logging.XMLFormatter is a regular old Java class, nothing to
do directly with Java EE really. Other than Java EE uses the regular
old Java API.
Socket? But there always has to be an another thread or process in the
other end of the socket, no?

Socket programming is nothing new to me, 20+ years with C++ but J2EE
practices are new to me.

Well I'm speaking mostly from general network programming experience
myself. I don't admin systems, but I know from monkeying around with my
own boxes that real OSs (Unix) have supported logging through sockets
for a long time. It's a standard configuration parameter on all Unix
loggers, and I think now for Windows too.

But the standard process to read the logs wouldn't do anything special
with them, just write them to disk. If the text is already XML
formatted that might be enough. But there's likely to be delays and
such, so this might not update instantly for you web interface.

It seems reasonable that someone may have written a utility to read
standard logs from a socket and stuff them in a database, but I don't
happen to know any such utilities.

What I think I'm trying to say is that Java EE fundamentally isn't
different from other types of programming, and you should eschew
re-inventing the wheel, just like regular old programming. Writing text
to a buffer (even XML formatted) is simple, fast, obviates the need for
complex multithreading schemes, and all around seems simpler to me. But
yes of course someone has to read the socket on the other end.

But you know your requirements best, and there seems to already an
expectation that systems actually write some of their logs to a database
to gain other features.
 
A

Arne Vajhøj

It outputs

####This is a test####
####This is also a test####

The base of if is the record.getMessage()

I need a structured message, not a String.

Ok, I could extend LogRecord, and get its getters for info. But how about
Logger.

I want a log record like this

<LogEntry>
<Login>admin</login>
<from>192.168.1.57</from>
<action result="success">MODIFY</action>
<entity type="Customer">
<id>35252</id>
<attributes>
<attribute name="address">
<oldvalue>Ankkalinna 11</oldvalue>
<newvalue>Hanhilinna 12</newvalue>
</attribute>
</attributes>
</entity>
</LogEntry>

How In earth do I that with logger.info()

I think the best approach will be to do the Object->String
before calling the logging.

logger.info(myobj.toLogString());

and the use a formatter that does not "add unwanted stuff".

Arne
 
D

Donkey Hottie

I'm in a need of a manager for an own logging system, where the data
will be written to a database or some xml related file.

I would like to dedicate a process or a thread for writing the log,
and in addition to that for some cleanup and other management.

I'm not allowed to use JMS.

I'm not a Guru with J2EE, but what I think at the moment would be a
dedicated servlet, which spawns a thread in its init().

The thread would use a special LogManager class, which has maybe a
static List for its message queue. That would be "written" and "read"
in synchronised methods.

Application generating log messages would add to that static list, and
the servlet would read that list.

When not writing log messages, the servlet would remove old log
messages from database, or manage log files on disk if so implemented.

I'm I on a decent track?

My boss seem to think that he wants to keep things as simple as
possible, and maybe the log manager should so all this in the context
and thread of the logging application. That might be well possible,
because out app run on intranets, and there will not be much traffic.

But I just fancy that the messages should be put on a queue, and
managed by a singleton.

I still need opinions on the thing in the topic: a concurrent, persistent
background process in a J2EE container.

I have used Quartz, but it is Open Source, which my boss does not want.

My idea was a logmanager-servlet, which creates a Thread in its init(). Is
this idea good or bad, and why?
 
D

Donkey Hottie

I think your boss is correct. The Java LogManager is pretty robust.
I'd definitely try that first, and only abandon it when it was proven
it couldn't work. The LogManager is pretty sophisticated, it'll even
rotate your logs for you.

Writing logs to a database... why? The logs should be simple, so that
they are present even when things don't work. How are you going to
debug database problems when the logs didn't get written because the
database was supposed to be hold the logs in the first place? This
seems circular, and unnecessarily complex. Even baroque.

If I got an app that wrote logs to a database, I think I'd curse your
name. Just saying....

Toying with java.util.logger

Logger formats and logs a LogRecord object.

I'm trying to extend it for my purposes, to get that structure in it.

Seems Logger does not call format() for my extended object.

The code to toy with is here. The missing error is in the testUrl method,
which is part of my another "toy". It is quaranteed to throw an error,
and get the message logged. But no log entry there. format() will not be
called.

Bummer.


----------------------------<snip>---------------------------

package xmltoys;

import java.util.logging.Formatter;
import java.util.logging.LogRecord;
import java.util.logging.Logger;
import java.util.logging.Level;
import java.net.* ;

public class Main extends Formatter
{
@Override
public String format(LogRecord record)
{
if (record instanceof Something)
{
Something some = (Something) record ;
return some.toString() +
System.getProperty("line.separator");
}
return "####" + record.getMessage() + "####" +
System.getProperty("line.separator");
}


public class Something extends LogRecord
{
private String value ;

public Something(String value)
{
super(Level.ALL, value) ;
// super.setMessage(value);
this.value = value ;
}

public String toString()
{
return value ;
}
}

public static void setup()
{
Logger log = Logger.getLogger("foobar");
while (log.getParent() != null)
{
log = log.getParent();
}
log.getHandlers()[0].setFormatter(new Main());
}

void testUrl()
{
try
{
new URL("ldap://localhost:389") ;
}
catch (Exception ex)
{
Something some = new Something(ex.getMessage()) ;
Logger.getLogger("testUrl").log(some);
}
}

public static void main(String[] args)
{
Main.setup();

Main blah = new Main() ;
blah.testUrl() ;
Logger logger1 = Logger.getLogger("test");
logger1.info("This is a test");
Logger logger2 = Logger.getLogger("test.test");
logger2.info("This is also a test");
}
}
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

Forum statistics

Threads
473,769
Messages
2,569,580
Members
45,054
Latest member
TrimKetoBoost

Latest Threads

Top