ultralog: new concept of logging API

[Mikhail Vladimirov]

Hello, Java developers community.

Let me present "ultralog" project (http://code.google.com/p/ultralog/) which main purpose is to demonstrate new concept of how logging APIs could be structured.

The idea is that each logger in the application has its own interface specific to the purpose of the logger. This interface defines all messages that could be logged via this logger. Here is an example of such interface:

protected interface HelloLogger extends BasicLogger
{
@LogMessage (level = INFO, template = "Hello, ${1}!")
void hello (String user);
}

The logging framework generates implementations of such interfaces on the fly or at compile time by compiling message templates into Java bytecode. Here is how this logger could be used:

HelloLogger logger = LoggerManager.createLogger (HelloLogger.class);

if (logger.isInfoEnabled ())
logger.hello (System.getProperty ("user.name"));

Advantages of this approach:
- Clear syntax.
- All log messages for the logger are defined in one place which makes it easier to review and restyle them in consistent manner.
- Log messages are defined as methods, so IDEs "find references" feature could be used to fine all places where particular message is logged.
- Values of arbitrary types including primitive types could be substituted into log message without allocating any temporary objects.

See project WiKi (http://code.google.com/p/ultralog/wiki/TableOfContents) for more information and feel free to provide feedback.

 

[markspace]

Advantages of this approach:

It's an interesting idea. However I don't think it really does enough
to make it better than existing logging solutions.

I don't think, for example, the syntax of your approach is any more
clear than calling a method. (Your approach is, frankly, a little less
clear.)

I don't see how defining messages in one place is different than
existing solutions, which have the same capability. Nor do I see how it
interacts with IDEs any better than the current approaches.

So to make this review a little more positive, here's a few things I
think I'd like in a logger.

The main idea I'd like is the ability to just plain write logging
statements for me. For example, given the following code:

package my.package;

public class MyStuff {
public static void main( String... args ) {
System.out.println( "Hello World." );
}
}

I'd like to be able to have the logger, on the fly, instrument this with:

- a static or instance logger declaration
- method invocation
- parameter values on invocation
- return value, if any
- variable assignment within the method

All at varying levels of debug priority. For example, the middle three
are typically at a level of FINER, while the last one should be FINEST.
(The first one obviously has to happen all the time, in order for
anything to work.)

There's probably more logging patterns, I'm just making one for discussion.

The idea here, and this is very important for the requirements, is
similar to the idea of Aspect Oriented programming and Cross-Cuts.
Certain things are necessary in programming (logging is a classic
example), but are also "noise" when a developer looks at the source code
(logging is again the classic example of this problem).

Could you imagine the source code above with all of the logging
statements I asked for? It would be a mess! And all for one little
line of code.

Yet if we instrument the code on the fly, we gain all the benefits of
fully instrumented code, without having to pay nearly as much price in
maintenance.

That's what I'm really going for here: logging, but without the cost in
lines of code. That would be a real win, imo.

 

[vladimirow]

It's an interesting idea. However I don't think it really does enough

to make it better than existing logging solutions.

Ultralog is not supposed to be better than existing solutions. It is supposed to be better choice for high-performance applications. Especially for those applications that want to be garbage- and lock-free.

I don't think, for example, the syntax of your approach is any more
clear than calling a method. (Your approach is, frankly, a little less
clear.)

For me the following:

logger.userLoggedIn (userName, address);

is clearer than

logger.info ("User `" + userName + "' logged in from " + address);

or

logger.info ("User `{}' logged in from {}", userName, address);

or

logger.infoT ("User `{}' logged in from {}").p (userName).p (address).endT ();

Anyway this is matter of personal preference.

 

[Joerg Meier]

For me the following:

logger.userLoggedIn (userName, address);

is clearer than

logger.info ("User `" + userName + "' logged in from " + address);

But the latter still has to be SOMEWHERE. You just hide it behind a method.
But your logger isn't magic, it can't guess what I want the log entry to
look like, so I still have to specify it somewhere. Your way just forces me
to write half a dozen extra classes/methods for every class.

It also seems like there would be next to no code reuse. When do you ever
want to log the exact same thing in two or more different places ?

Liebe Gruesse,
Joerg

 

[Mikhail Vladimirov]

Actually, I disagree with both claims, but regardless, even if you

prefer the first version, I fail to see the advantage in your way of
doing it compared to just doing this:

When I read the code which logs something, what I usually need to know, is what information is logged here and which level at. I may need to know this because I'm doing code review and I want to make sure that proper information is logged at proper levels. Also I may need this during issue investigation because I don't see something in the log that logically should be there and I want to make sure that I didn't forget to put logging in the code.. I usually don't care about exact words and commas in the log message. Only what information is logged and which level at.

The real cases when I need to know log message literally, are:
1. I want to review all log message to make sure they obey our policy regarding style, formatting and levels.
2. I found something in the log and I need to find place in code where it was logged.

The later two things are hard to do when log messages are spread among the code and are written in the form of concatenations. Especially hard is to find in the code log messages that was taken from log file if you don't know exactly which parts of the message are hardcoded and which parts were substituted from variables.

Anyway, the idea of ultralog was to offer garbage-free yet clear syntax, not syntax that is clearer than even non garbage-free ones.

 

[Mikhail Vladimirov]

But the latter still has to be SOMEWHERE. You just hide it behind a method.

But your logger isn't magic, it can't guess what I want the log entry to
look like, so I still have to specify it somewhere. Your way just forces me
to write half a dozen extra classes/methods for every class.

I'm not sure you checked examples in ultralog's WiKi before guessing.

Simple syntax like

logger.info ("Hello, " + user + "!");

or

logger.error ("Oh, My!", exception);

is still available in ultralog and is supposed to be used in places where we just need to quickly add some logging and we don't care about garbage, orin cases where we want to log plain messages without any substitutions.

When you use new syntax like:

logger.userLoggedIn (userName, address);

in Eclipse you just need to put mouse pointer over method name and wait fora moment to popup appear to know literal message template and level. And you need to just Ctrl+Click on it to start editing message template.

Ultralog is not magic, but it tries to lock like magic. You need to definemethod for parametrized log messages, but you never need to implement them, so there is no executable code in them.

The idea is that each class that needs high-performance logging, will define it's own logger interface, usually as inner interface with protected access level, and then define all log messages in it. Once each logger interface belongs to separate class, they are not supposed to be reused.

 

[Arved Sandstrom]

Hello, Java developers community.

Let me present "ultralog" project (http://code.google.com/p/ultralog/) which main purpose is to demonstrate new concept of how logging APIs could be structured.

The idea is that each logger in the application has its own interface specific to the purpose of the logger. This interface defines all messages that could be logged via this logger. Here is an example of such interface:

protected interface HelloLogger extends BasicLogger
{
@LogMessage (level = INFO, template = "Hello, ${1}!")
void hello (String user);
}

The logging framework generates implementations of such interfaces on the fly or at compile time by compiling message templates into Java bytecode. Here is how this logger could be used:

HelloLogger logger = LoggerManager.createLogger (HelloLogger.class);

if (logger.isInfoEnabled ())
logger.hello (System.getProperty ("user.name"));

Advantages of this approach:
- Clear syntax.
- All log messages for the logger are defined in one place which makes it easier to review and restyle them in consistent manner.
- Log messages are defined as methods, so IDEs "find references" feature could be used to fine all places where particular message is logged.
- Values of arbitrary types including primitive types could be substituted into log message without allocating any temporary objects.

See project WiKi (http://code.google.com/p/ultralog/wiki/TableOfContents) for more information and feel free to provide feedback.

I think you're thinking the right things, I'm just not sure you've got
much of a better solution. Yet.

Let me describe what I typically do, let's say with log4j:

1. Write a little helper or utility class (these days I just copy it
from an existing project) that wraps core log4j calls, basically to
format log messages according to project/app guidelines. MessageFormat
usually figures prominently.

2. As I work through and actually start to log it, messages start to
amass in one or more logging .properties files. The property key syntax
I use for logging is usually a dotted notation, to help search for
messages when there are already hundreds or more entries.

So just by doing what I already do I get absolutely everything you tout
as an advantage of your new logging system.

AHS

 

[Mikhail Vladimirov]

So just by doing what I already do I get absolutely everything you tout

as an advantage of your new logging system.

Main advantage of ultralog is an ability to substitute variables into log messages without allocating new objects, yet having readable code. This advantage cannot be achieved on top of log4j because log4j allocates temporary objects internally.

 

[Arved Sandstrom]

Main advantage of ultralog is an ability to substitute variables into log messages without allocating new objects, yet having readable code. This advantage cannot be achieved on top of log4j because log4j allocates temporary objects internally.

Well, I'll do this, I'll give your product a trial run in a real project
I am working on. I can't really critique it effectively without doing that.

AHS

 

[Mikhail Vladimirov]

Well, I'll do this, I'll give your product a trial run in a real project

I am working on. I can't really critique it effectively without doing that.

Release currently available for download is marked as 'preview' because it misses essential feature: an ability to configure logging levels and destinations via configuration file. Now the only way to configure logging is touse API calls. This should be fine of experiments and performance testingbut not for real integration.

 

[Lew]

Main advantage of ultralog is an ability to substitute variables into log messages without allocating new objects, yet having readable code. This advantage cannot be achieved on top of log4j because log4j allocates temporary objects internally.

You keep touting "advantages" that your potential users state as not very helpful,
or not superior to existing tools.

Your sense of "advantages" is irrelevant to marketing. The market's sense is what matters
and all that matters.

Arguing with prospects does not make them customers.

Good luck with that. The answers you've given others in this thread have convinced me
unequivocally to avoid your project.

Good job.

 

[Mikhail Vladimirov]

You keep touting "advantages" that your potential users state as not very helpful,

or not superior to existing tools.
Your sense of "advantages" is irrelevant to marketing. The market's sense is what matters
and all that matters.

Ultralog focuses on demands of high performance applications, which is quite a narrow market. Advantages of ultralog may be helpless outside high performance domain.

 

[Lew]

You keep touting "advantages" that your potential users state as not very helpful,
or not superior to existing tools.

Your sense of "advantages" is irrelevant to marketing. The market's sense is what matters
and all that matters.

Ultralog focuses on demands of high performance applications, which is quite a narrow market. Advantages of ultralog may be helpless outside high performance domain.

Made me wrong again, huh, Dale Carnegie?

I think your market might be considerably narrower than you imagine.

 

[Mikhail Vladimirov]

In what way? Which benefits do Ultralog bring to the table for high

performance applications re logging?

The most important things are:
1. Garbage-free logging. You can format and send out log message without allocating any new objects.
2. Lock-free logging. you can format and send out log message without acquiring any locks.
3. Log message templates are compiled into Java bytecode in run time or even in compile time, which makes message formatting really fast.

Check ultralog WiKi for more details: http://code.google.com/p/ultralog/wiki/TableOfContents

 

[Arne Vajhøj]

The most important things are:
1. Garbage-free logging. You can format and send out log message without allocating any new objects.
2. Lock-free logging. you can format and send out log message without acquiring any locks.
3. Log message templates are compiled into Java bytecode in run time or even in compile time, which makes message formatting really fast.

Check ultralog WiKi for more details: http://code.google.com/p/ultralog/wiki/TableOfContents

Any measurements of the impact for some realistic code?

Arne

 

[Mikhail Vladimirov]

1. Garbage-free logging. You can format and send out log message without allocating any new objects.

Any measurements of the impact for some realistic code?

The idea that high-performance application may want to process data withoutproducing garbage is not something new. Check, for example, this link: http://oreilly.com/catalog/javapt/chapter/ch04.html

The same could be said about lock-free design. It is well known approach to increase performance of multi-threaded applications.

Ultralog is not the first Java library designed to be garbage- and lock-free and focused on demands of high performance applications. See, for example, LMAX Disruptor: http://lmax-exchange.github.com/disruptor/ It is all about hot to avoid lock and garbage creation and it states itself as "High Performance Inter-Thread Messaging Library".

Ultralog is even not the first attempt to create garbage-free logging framework for Java. See, for example, the following link: https://bitbucket.org/vladimir.dolzhenko/gflogger/wiki/Home It states its goal as "to create ad-hoc logger for low latency (latency critical) applications (to be precise for latency critical execution path) which will affect application explicitand implicit (though gc pauses) as less as it possible". It does not claim itself to be lock-free though.

What is new in ultralog in comparison with gflogger, is that it demonstrates that garbage-free logging could be done without sacrificing code readability.

 

[Lew]

Mikhail Vladimirov wrote:

DUDE! Attribute quotes!

Please.

The idea that high-performance application may want to process data without producing garbage is not something new. Check, for example, this link: http://oreilly.com/catalog/javapt/chapter/ch04.html

The same could be said about lock-free design. It is well known approachto increase performance of multi-threaded applications.

Ultralog is not the first Java library designed to be garbage- and lock-free and focused on demands of high performance applications. See, for example, LMAX Disruptor: http://lmax-exchange.github.com/disruptor/ It is all about hot to avoid lock and garbage creation and it states itself as "High Performance Inter-Thread Messaging Library".

Ultralog is even not the first attempt to create garbage-free logging framework for Java. See, for example, the following link: https://bitbucket.org/vladimir.dolzhenko/gflogger/wiki/Home It states its goal as "to create ad-hoc logger for low latency (latency critical) applications (to be precise for latency critical execution path) which will affect application explicit and implicit (though gc pauses) as less as it possible". It does not claim itself to be lock-free though.

What is new in ultralog in comparison with gflogger, is that it demonstrates that garbage-free logging could be done without sacrificing code readability.

In other words, no, no measurements.

 

[Mikhail Vladimirov]

In other words, no, no measurements.
I think that discussion about why one may want to make applications garbage-free and why garbage-free applications may perform faster and how much faster they can perform is off-topic here.

Once application is decided, for whatever reason, to be garbage-free, whichmeans that normal data processing flow in the application does not allocate any temporary objects, and once normal data processing flow involves logging, the application has to use some garbage-free logging solution, either home-grown or third party. In this case mainstream logging frameworks simply does not fit, because they are not garbage-free. Ultralog demonstrates how API for garbage-free logging framework can be structured without sacrificing code readability. It does not need to be faster than mainstream frameworks and switching to ultralog in application that is not garbage-free itself should not necessarily lead to performance benefit.

Performance tests I have shows that ultralog is usually not slower than System.out.println() and is not slower than log4j.

 

[Lew]

Mikhail Vladimirov wrote:

Still not attributing quotes, I see.

[Lew wrote:]

In other words, no, no measurements.

I think that discussion about why one may want to make applications garbage-free and why garbage-free applications may perform faster and how much faster they can perform is off-topic here.

Of course it is, because you have no answer for this so you want to ban thequestion.

You're not convincing me you're anything but a con artist with such tactics..

Once application is decided, for whatever reason, to be garbage-free, which means that normal data processing flow in the application does not allocate any temporary objects, and once normal data processing flow involves logging, the application has to use some garbage-free logging solution, either home-grown or third party. In this case mainstream logging frameworks simply does not fit, because they are not garbage-free. Ultralog demonstrates how API for garbage-free logging framework can be structured without sacrificing code readability. It does not need to be faster than mainstream frameworks and switching to ultralog in application that is not garbage-free itself should not necessarily lead to performance benefit.

Why not? What other benefit does avoiding GC provide?

Performance tests I have shows that ultralog is usually not slower than System.out.println() and is not slower than log4j.

That tells us nothing. What sort of "performance" tests? Under what protocols? And "not slower" than
'println()' is useless; we already know log4j is generally less impactful on performance than the
'System.out' technique. "Not slower" than log4j, even if your tests are notjust crocks of porcine
excrement, is not impressive.

You have yet to tout a real benefit. You get diversionary when people ask about your benefits.
You were asked about "any impact" (beneficial or otherwise) and you chose to interpret it as
a question about performance, then tried to claim that performance, the topic you introduced,
was off topic.

And again, when potential users ask about things that matter to them, you try to tell them that
their concerns are not valid. That's shitty marketing.

I think that your product, far from being garbage free, is nothing but garbage, based on what
you've told us and how you disrespect us.

 

[Mikhail Vladimirov]

Still, never mind, at least you're trying to do something constructive.
Lew is doing good work bumping this discussion up. This is why I'm feeding the troll