Draft PEP on RSON configuration file format

[Patrick Maupin]

All:

Finding .ini configuration files too limiting, JSON and XML to hard to
manually edit, and YAML too complex to parse quickly, I have started
work on a new configuration file parser.

I call the new format RSON (for "Readable Serial Object Notation"),
and it is designed to be a superset of JSON.

I would love for it to be considered valuable enough to be a part of
the standard library, but even if that does not come to pass, I would
be very interested in feedback to help me polish the specification,
and then possibly help for implementation and testing.

The documentation is in rst PEP form, at:

http://rson.googlecode.com/svn/trunk/doc/draftpep.txt

Thanks and best regards,
Pat

 

[Paul Rubin]

I have started work on a new configuration file parser....
The documentation is in rst PEP form, at:...

Noooooooo.... not another... there are too many already. :-(

-1

 

[Steven D'Aprano]

All:

Finding .ini configuration files too limiting, JSON and XML to hard to
manually edit, and YAML too complex to parse quickly, I have started
work on a new configuration file parser.

I call the new format RSON (for "Readable Serial Object Notation"), and
it is designed to be a superset of JSON.

Wait a minute... if JSON is too hard to edit, and RSON is a *superset* of
JSON, that means by definition every JSON file is also a valid RSON file.
Since JSON is too hard to manually edit, so is RSON.

I would love for it to be considered valuable enough to be a part of the
standard library,

Come back when you actually have MANY users other than yourself using
this is real-world projects. Until then, it is too early to even consider
adding it the std library. Python comes with batteries included, but not
experimental batteries that aren't even built yet, let alone proven that
they work.

 

[Chris Rebert]

  You're not supposed to edit those formats manually.

Huh? True for XML /perhaps/, but for the others I quite disagree.
Perhaps you could explain your reasoning?

Cheers,
Chris

 

[John Nagle]

All:

Finding .ini configuration files too limiting, JSON and XML to hard to
manually edit, and YAML too complex to parse quickly, I have started
work on a new configuration file parser.

You're not supposed to edit those formats manually.

John Nagle

 

[Patrick Maupin]

On Feb 28, 9:18 pm, Steven D'Aprano > Wait a minute... if JSON is too
hard to edit, and RSON is a *superset* of

JSON, that means by definition every JSON file is also a valid RSON file.
Since JSON is too hard to manually edit, so is RSON.

Well, Python is essentially a superset of JSON, with string escape
handling being ever so slightly different, and using True instead of
true, False instead of false, and None instead of null. YMMV, but I
find it possible, even probable, to write Python that is far easier to
edit than JSON, and in fact, I have used Python for configuration
files that are only to be edited by programmers or other technical
types.

Come back when you actually have MANY users other than yourself using
this is real-world projects. Until then, it is too early to even consider
adding it the std library. Python comes with batteries included, but not
experimental batteries that aren't even built yet, let alone proven that
they work.

I know that it is too early to be included in the library. But there
is a chicken-and-egg problem. PEP 1 says you should circulate a
design for discussion before writing code...

I fully expect this sort of negative feedback, fully expect that it
will be years (if ever) before the code makes it into the standard
library, and yet I offer the design for your perusal, just on the slim
chance that I might get a few constructive criticisms on the actual
design to go along with all the silly strawmen reductio ad absurdum
arguments by people who aren't even going to bother to read the thing.

Best regards,
Pat

 

[Patrick Maupin]

    You're not supposed to edit those formats manually.

                                        John Nagle

Opinions on that will vary, and as the saying goes, "everybody is
entitled to his own opinions, but not his own facts."

I submit as fact that there are many initialization files in each of
these formats, and I would be willing to place a wager that the
majority of the people on this list have used a text editor to modify
at least one file in this category at least once.

IOW, in my opinion (and it is only an opinion, not a fact), you are
actually helping to make the case that a better format is needed.

Regards,
Pat

 

[Robert Kern]

On Feb 28, 9:18 pm, Steven D'Aprano

I know that it is too early to be included in the library. But there
is a chicken-and-egg problem. PEP 1 says you should circulate a
design for discussion before writing code...

Ignore it. That comment really doesn't apply to this case. That's for things
that only make sense in the language or standard library, like context managers.
For libraries like this, Steven's summary is correct. It needs to have a useful
life as a third party package for a few years before you should propose it for
inclusion into the standard library. By all means, distribute a design document
for comment before you implement things; it's a reasonable practice. But don't
bother with a PEP yet.

--
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth."
-- Umberto Eco

 

[Patrick Maupin]

Ignore it. That comment really doesn't apply to this case. That's for things
that only make sense in the language or standard library, like context managers.
For libraries like this, Steven's summary is correct. It needs to have a useful
life as a third party package for a few years before you should propose it for
inclusion into the standard library. By all means, distribute a design document
for comment before you implement things; it's a reasonable practice. But don't
bother with a PEP yet.

So, I went and re-read PEP 1, and in the fine print it directs me to
PEP 2. Mea culpa -- I though I understood the PEP.

Nonetheless, the fact that I created what I call a "PEP draft" seems
to have had more read into it than I meant. I will bear this in mind
when I start future projects. I have a couple of projects, and
contribute to another, which in no way should wind up in the standard
library. But I find the lack of a good, up-to-date, configuration
reader in the library to be an issue, which is why I had the temerity
to try to start a discussion on what a good standard configuration
file reader should be.

Yes, I would like to create something compelling enough to go into the
standard library. No, I don't expect it to wind up there for a very
long time, if ever. BUT, at the same time, following the PEP format
is very valuable for my project. It forces me to think more deeply
about the problem I am solving and it forces me to write down some
documentation. Really, it's no more burdensome than any other similar
useful template.

Or, to put it another way, it is my goal (which I may or may not be
smart enough to reach) to write a module that anybody would want to
use; that is good enough to put into the standard library. Whether
this happens or not, one useful tool to help the module down that path
(and produce a better module for me and other users even if it never
becomes part of the standard library) is to think about what it takes
to get into the standard library. Part of this involves writing the
PEP. Writing a PEP is a good exercise; the format itself helps me to
focus on important issues.

So I was writing the contents of the PEP anyway, just for me. The
real question, then, is when to release the PEP (vs. the code). I've
already got some coding done, but it is not yet good enough to share,
and if there is anybody else out there grappling with the same issues
at this time, I would prefer not to go into production with it until
they have had a chance to look over the design and offer insights and
criticisms.

Best regards,
Pat

 

[Paul Rubin]

it is my goal (which I may or may not be smart enough to reach) to
write a module that anybody would want to use;

But you are working on a solution in search of a problem. The really
smart thing to do would be pick something more useful to work on. We
don't need another configuration language. I can't even say "yet
another" because there's already a "yet another" called yaml.

 

[Robert Kern]

So, I went and re-read PEP 1, and in the fine print it directs me to
PEP 2. Mea culpa -- I though I understood the PEP.

Nonetheless, the fact that I created what I call a "PEP draft" seems
to have had more read into it than I meant. I will bear this in mind
when I start future projects. I have a couple of projects, and
contribute to another, which in no way should wind up in the standard
library. But I find the lack of a good, up-to-date, configuration
reader in the library to be an issue, which is why I had the temerity
to try to start a discussion on what a good standard configuration
file reader should be.

Yes, I would like to create something compelling enough to go into the
standard library. No, I don't expect it to wind up there for a very
long time, if ever. BUT, at the same time, following the PEP format
is very valuable for my project. It forces me to think more deeply
about the problem I am solving and it forces me to write down some
documentation. Really, it's no more burdensome than any other similar
useful template.

Certainly. The PEP format is a useful one. I've used it myself for some numpy
design documents. But can you see why people might get confused about your
intentions when you call it a draft PEP and post it to python-dev? If you stop
calling it a PEP and stop talking about putting it in the standard library,
people will stop being distracted by those issues.

--
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth."
-- Umberto Eco

 

[Daniel Fetchinson]

it is my goal (which I may or may not be smart enough to reach) to

But you are working on a solution in search of a problem. The really
smart thing to do would be pick something more useful to work on. We
don't need another configuration language. I can't even say "yet
another" because there's already a "yet another" called yaml.

And in case you are new here let me assure you that Paul is saying
this with his full intention of being helpful to you. I also would
think that working on such a project might be fun and educational for
you but completely useless if you have users other than yourself in
mind. Again, I'm trying to be helpful here, so you can focus on a
project that is both fun/educational for you and also potentially
useful for others. This RSON business is not one of them.

Cheers,
Daniel

 

[Patrick Maupin]

But you are working on a solution in search of a problem.  The really
smart thing to do would be pick something more useful to work on.  We
don't need another configuration language.  I can't even say "yet
another" because there's already a "yet another" called yaml.

The only "in search of" here is that, instead of working on a point
solution for my particular problem, I am "in search of" a solution
that is a bit more elegant and general, and that might help solve
other people's problems too. If you are not one of those that has
this sort of problem at this point in time, then feel free to kill-
file this thread.

Thanks,
Pat

 

[Patrick Maupin]

Certainly. The PEP format is a useful one. I've used it myself for some numpy
design documents. But can you see why people might get confused about your
intentions when you call it a draft PEP and post it to python-dev? If you stop
calling it a PEP and stop talking about putting it in the standard library,
people will stop being distracted by those issues.

As I mentioned, I didn't see the fine print in PEP 1 about PEP 2 being
the document for library modules. As I mentioned, mea culpa. It is
painfully obvious that some don't like the way I have gone about
describing the project. They obviously view me announcing this as
premature, or presumptuous, or something, and they have some sort of
visceral reaction to that.

However, I do not believe that any people (other than me) were really
confused in the process. I made my intentions clear, and some people
reacted badly to that because I didn't follow the process (for which I
apologize again). But calling it a draft PEP is a distraction
(because of the visceral reaction), but is not really what I would
call confusing. My intention actually is to try to build something
that is worthy of the standard library, and to eventually try to get
it accepted, because I perceive a hole there, with a lot of point
solutions being done to solve a common problem, and I believe the
pushback is coming from people who fully understood that intention
from my posting.

I will try to say "hey -- here's a hole in the library and a proposal
for how to fix it" more diplomatically and in the correct forum in the
future, but it would be disingenuous for me to disown my goal of
getting a better configparser into the standard library.

Regards,
Pat

 

[Patrick Maupin]

And in case you are new here let me assure you that Paul is saying
this with his full intention of being helpful to you. I also would
think that working on such a project might be fun and educational for
you but completely useless if you have users other than yourself in
mind. Again, I'm trying to be helpful here, so you can focus on a
project that is both fun/educational for you and also potentially
useful for others. This RSON business is not one of them.

OK, but I am a bit unclear on what you and/or Paul are claiming. It
could be one of a number of things. For example:

- There is a preexisting file format suitable for my needs, so I
should not invent another one.

- If I invent a file format suitable for my needs, it couldn't
possibly be general enough for anybody else.

- Even if it was general enough for somebody else, there would only be
two of them.

I've been known to waste time (or be accused of wasting time) on
various endeavors, but I like to know exactly *why* it is perceived to
be a waste.

Regards,
Pat

 

[Paul Rubin]

- There is a preexisting file format suitable for my needs, so I
should not invent another one.

There are in fact quite a few--json, yaml, .ini, xml, Python literals
(http://code.activestate.com/recipes/364469-safe-eval/), s-expressions,
actual Python code that the application can import, and so forth.

The problem isn't that you're trying to invent a useless file format per
se, but rather that in trying to get other people to learn it and use
it, you're also trying to appropriate a chunk of the user community's
scarce and precious brain cells without good justification. Newbie
designers are often lured into that and they're unfortunately (i.e. to
the community's detrimtent) often more successful than they really
should be.

Your one complaint about yaml is that it's slow to parse. Why do you
care about the parsing speed of a config file? If the existing
implementation is too slow, why not write a faster one instead of
designing yayaml? Even yaml is excessive in my view. "Yet another" was
an ok plan when Steve Johnson started the trope by writing Yacc 30 years
ago. These days, don't do yet another without very convincing reasons
for rejecting what is already there.

 

[Patrick Maupin]

There are in fact quite a few--json, yaml, .ini, xml, Python literals
(http://code.activestate.com/recipes/364469-safe-eval/), s-expressions,
actual Python code that the application can import, and so forth.

Yes, I know about those.

The problem isn't that you're trying to invent a useless file format per
se, but rather that in trying to get other people to learn it and use
it, you're also trying to appropriate a chunk of the user community's
scarce and precious brain cells without good justification.  Newbie
designers are often lured into that and they're unfortunately (i.e. to
the community's detrimtent) often more successful than they really
should be.

Can't disagree with the statement about newbie designers. On the flip
side, sometimes things wind up being defacto standards, simply because
nobody could be bothered to write a better one, and they have lots of
little niggling warts that also collectively take up a lot of time and
attention. There is obviously a good balance somewhere.

Your one complaint about yaml is that it's slow to parse.  Why do you
care about the parsing speed of a config file?  If the existing
implementation is too slow, why not write a faster one instead of
designing yayaml?  Even yaml is excessive in my view.  "Yet another" was
an ok plan when Steve Johnson started the trope by writing Yacc 30 years
ago.  These days, don't do yet another without very convincing reasons
for rejecting what is already there.

One of my complaints. If you had read the document you would have
seen others. I actually have several complaints about YAML, but I
tried to write a cogent summary.

Regards,
Pat

 

[Paul Rubin]

One of my complaints. If you had read the document you would have
seen others. I actually have several complaints about YAML, but I
tried to write a cogent summary.

Yaml sucks, but seems to have gotten some traction regardless.
Therefore the Python principle of "there should be one and only one
obvious way to do it" says: don't try to replace the existing thing if
your new thing is only slightly better. Just deal with the existing
thing's imperfections or make improvements to it. If you can make a
really powerful case that your new thing is 1000x better than the old
thing, that's different, but I don't think we're seeing that here.

Also, XML is used for pretty much everything in the Java world. It
sucks too, but it is highly standardized, it observably gets the job
done, there are tons of structure editors for it, etc. Frankly
I'd rather have stayed with it than deal with Yaml.

There are too many of these damn formats. We should ban all but one of
them (I don't much care which one). And making even more of them is not
the answer.

 

[Patrick Maupin]

Yaml sucks, but seems to have gotten some traction regardless.

Yes, that's actually one of the reasons I want to do this. I've heard
that some of the YAML people want that in the standard library, and
IMHO that would be a huge mistake.

Therefore the Python principle of "there should be one and only one
obvious way to do it" says: don't try to replace the existing thing if
your new thing is only slightly better.

But for my use-case, YAML is irretrievably broken. Sure, it looks
reasonably nice, but it increases regression runtime unacceptably.
Also, despite their current protestations that Yaml AIN'T markup
language, the syntax still belies its markup roots. I'm looking for a
configuration language, not a markup language.

 Just deal with the existing thing's imperfections or make improvements to it.

Well, I've looked at the YAML parser and I can assure you that I will
not be contributing to that project.

If you can make a really powerful case that your new thing is 1000x better than the old thing, that's different, but I don't think we're seeing that here.

Well, there's no way I'm hitting three orders of magnitude, but I
don't think the bar is really that high. We will see.

Also, XML is used for pretty much everything in the Java world.  It
sucks too, but it is highly standardized, it observably gets the job
done, there are tons of structure editors for it, etc.  Frankly
I'd rather have stayed with it than deal with Yaml.

XML can certainly be made readable by humans. It's hard to make it
writeable by (average) humans just using a regular text editor, and
even though there are tons of structure editors, requiring non-
technical people to find one of those and start using it is a non-
starter in many cases. But I can't strongly disagree with the opinion
that YAML doesn't offer all that much over it.

There are too many of these damn formats.  We should ban all but one of
them (I don't much care which one).  And making even more of them is not
the answer.

Well, I think that XML and YAML both overreach by trying to be all
things to all applications. So I'm aiming a lot lower than that. I
just want to make really good configuration files that are easy to
modify in a text editor.

None of the existing formats are suitable. I may be the only one who
feels that way. OTOH, if we never invented new things, then I
wouldn't be enjoying restructured text, and I'd be missing out on a
lot there...

Regards,
Pat

 

[Paul Rubin]

But for my use-case, YAML is irretrievably broken. Sure, it looks
reasonably nice, but it increases regression runtime unacceptably.

How big are the files that you want to parse with it? Sheesh.

Well, I've looked at the YAML parser and I can assure you that I will
not be contributing to that project.

So write a new one that parses the same syntax, but cleaner and faster.

XML can certainly be made readable by humans. It's hard to make it
writeable by (average) humans just using a regular text editor,

I do it all the time; it's a bit dreary but not difficult. And there is
absolutely no way to get anything done in this field anymore without
dealing with XML from time to time. So given that we all have some
experience using it, it's sensible to stick with it.

None of the existing formats are suitable. I may be the only one who
feels that way. OTOH, if we never invented new things, then I
wouldn't be enjoying restructured text, and I'd be missing out on a
lot there...

ReST is another abomination that should never have gotten off the
ground. It is one of the reasons I react so negatively to your
config format proposal. It just sounds like more of the same.