Negative array indicies and slice()

[Andrew Robinson]

By the way Andrew, the timestamps on your emails appear to be off, or
possibly the time zone. Your posts are allegedly arriving before the
posts you reply to, at least according to my news client.

-- yes, I know about that problem. Every time I reboot it shows up
again...
It's a distribution issue, my hardware clock is in local time -- but
when the clock is read by different scripts in my distribution, some
refuse to accept that the system clock is not UTC.
I'll be upgrading in a few weeks -- so I'm just limping along until
then. My apology.

Then I look forward to seeing your profiling results that show that the
overhead of subclassing list is the bottleneck in your application.

Until then, you are making the classic blunder of the premature optimizer:

"More computing sins are committed in the name of efficiency (without
necessarily achieving it) than for any other single reason — including
blind stupidity." — W.A. Wulf

I'm sure that's true. Optimization, though, is a very general word.

On a highway in my neighborhood -- the government keeps trying to put
more safety restrictions on it, because it statistically registers as
the "highest accident rate road" in the *entire* region.

Naturally, the government assumes that people in my neighborhood are
worse drivers than usual and need to be policed more -- but the truth
is, that highway is the *ONLY* access road in the region for dozens of
miles in any direction for a densely populated area, so if there is
going to be an accident it will happen there; the extra safety
precautions are not necessary when the accident rate is looked at from a
per-capita perspective of those driving the highway.

I haven't made *the* blunder of the premature optimizer because I
haven't implemented anything yet. Premature optimizers don't bother to
hold public conversation and take correction.
OTOH: people who don't ever optimize out of fear, pay an increasing
bloat price with time.

I am not impressed by performance arguments when you have (apparently)
neither identified the bottlenecks in your code, nor even measured the
performance.

Someone else already did a benchmark between a discrete loop and a slice
operation.
The difference in speed was an order of magnitude different.
I bench-marked a map operation, which was *much* better -- but also
still very slow in comparison.

Let's not confound an issue here -- I am going to implement the python
interpreter; and am not bound by optimization considerations of the
present python interpreter -- There are things I can do which as a
python programmer -- you can't. I have no choice but to re-implement
and optimize the interpreter -- the question is merely how to go about it.

You are essentially *guessing* where the bottlenecks are,
and *hoping* that some suggested change will be an optimization rather
than a pessimization.

Of course I may be wrong, and you have profiled your code and determined
that the overhead of inheritance is a problem. If so, that's a different
ball game. But your posts so far suggest to me that you're trying to
predict performance optimizations rather than measure them.

Not really; Inheritance itself and it's timing aren't my main concern.
Even if the time was *0* that wouldn't change my mind.

There are man hours in debugging time caused by not being able to wrap
around in a slice. (I am not ignoring the contrary man hours of an API
change's bugs).

Human psychology is important; and it's a double edged sword.

I would refer you to a book written by Steve Maguire, Writing Solid
Code; Chapter 5; Candy machine interfaces.

He uses the "C" function "realloc()" as an excellent example of a bad
API; but still comments on one need that it *does* fulfill -- "I've
found it better to have one function that both shrinks and expands
blocks so that I don't have to write *ifs* constructs every time I need
to resize memory. True, I give up some extra argument checking, but
this is offset by the *ifs* that I no longer need to write (*and
possibly mess up*).

* Extra steps that a programmer must take to achieve a task are places
where bugs get introduced.

* API's which must be debugged to see what particular operation it is
performing rather than knowing what that operation is from looking at
the un-compiled code are places where bugs get introduced.

These two points are not friendly with each other -- they are in fact,
generally in conflict.

Not necessarily. Not only can monkey-patches conflict, but they can
combine in bad ways. It isn't just that Fred assumes X and Barney assumes
not-X, but also that Fred assumes X and Barney assumes Y and *nobody*
imagined that there was some interaction between X and Y.

They *STILL* made contradictory assumptions; each of them assumed the
interaction mechanism would not be applied in a certain way -- and then
used it based on that assumption.

Not carefully enough. He notes that he was using a provocative title and
that he doesn't actually think that Ruby is being destroyed. But the
actual harm he describes is real, e.g. bugs that take months to track
down.

Yes. I read that *carefully*.
BTW: I'm not planning on monkey patching -- I did say your comments
were well taken.

I haven't mentioned namespaces. Nothing I have said has anything to do
with namespaces.

keywords are a namespace. xrange is a keyword, etc.
You don't want pre-defined API method name to have its operation
altered; you want the original names preserved. Overloading pollutes
namespaces, etc.

If you want to write a language that is not Python, go right ahead.

That's not the only possibility. It will still be called Python and for
good reason.

Python has certain public interfaces. If your language does not support
those public interfaces, then it might be an awesome language, but it is
not Python.

I didn't say I wouldn't support it. How do you mean this?

Iterators cannot replace slices, because once you have looked up an
iterator value, that value is gone, never to be seen again.

Yes they can replace slices, but not always.
For example; Lets say an iterator were allowed to replace a slice;

then a[ 1:3 ] would still be a slice, but a[ xrange(1,3) ] would not.
The second piece of code does not deny that slices exist, it just allows
an iterator to be passed to __getitems__.

Tuples cannot replace slices, because tuples do not have start, step and
stop attributes;

I refer you to my comments to IAN.

And none of them have a public indices method.

Hmm.. a new point. Do you have a link?

Hell, the actual *need* for slicing hasn't been demonstrated! Or Python!
Since C doesn't have slicing, nobody needs it! Am I right?

I asked about the need for slices() not the others, in the OP.
Thanks for your comments on the names, etc. That's helpful -- and I
hope I haven't driven your blood pressure up too much unintentionally.

Needed or not, that is what Python has, so if your language doesn't have
it, it isn't Python.

Fine. But if mine still has these, but only as an *optional* wrapper
function -- it still *RUNS* all python progams.

Sure. And a tuple can also be () or (1, "x", [], None, None, None, None).
Now your list.__getitem__ code has to deal with those instead.

And the present __getitem__ can be passed a perverted slice() as IAN
demonstrated and the bug-list examined as "goofy". These strange things
simply cause an exception.

So you want to get rid of slices because you want to save every byte
of memory you can, and your way to do that is to introduce a new,
*heavier* type that does more than slices? Oh man, that's funny.

I'm not Introducing a new type !
Besides; tuples will likely be lighter on memory in my target and I
don't recall saying a tuple does more than slices. I did say that
*iterators* were more *flexible* -- is that what you are thinking of?

 

[Ian Kelly]

File a bug report?

Looks like it's already been wontfixed back in 2006:

http://bugs.python.org/issue1501180

 

[Mark Lawrence]

Looks like it's already been wontfixed back in 2006:

http://bugs.python.org/issue1501180

Absolutely bloody typical, turned down because of an idiot. Who the
hell is Tim Peters anyway?

 

[Ian Kelly]

Absolutely bloody typical, turned down because of an idiot. Who the hell is
Tim Peters anyway?

I don't really disagree with him, anyway. It is a rather obscure bug
-- is it worth increasing the memory footprint of slice objects by 80%
in order to fix it?

 

[Ian Kelly]

D'Apriano mentioned the named values, start, stop, step in a slice() which
are an API and legacy issue; These three names must also be stored in the
interpreter someplace. Since slice is defined at the "C" level as a struct,
have you already found these names in the source code (hard-coded), or are
they part of a .py file associated with the interface to the "C" code?

You mean the mapping of Python attribute names to C struct members?
That's in sliceobject.c:

static PyMemberDef slice_members[] = {
{"start", T_OBJECT, offsetof(PySliceObject, start), READONLY},
{"stop", T_OBJECT, offsetof(PySliceObject, stop), READONLY},
{"step", T_OBJECT, offsetof(PySliceObject, step), READONLY},
{0}
};

 

[Chris Angelico]

I don't really disagree with him, anyway. It is a rather obscure bug
-- is it worth increasing the memory footprint of slice objects by 80%
in order to fix it?

Bug report: If I take this gun, aim it at my foot, and pull the
trigger, sometimes a hole appears in my foot.

This is hardly normal use of slice objects. And the penalty isn't a
serious one unless you're creating cycles repeatedly.

ChrisA

 

[Ian Kelly]

D'Apriano mentioned the named values, start, stop, step in a slice() which
are an API and legacy issue; These three names must also be stored in the
interpreter someplace. Since slice is defined at the "C" level as a struct,
have you already found these names in the source code (hard-coded), or are
they part of a .py file associated with the interface to the "C" code?

You mean the mapping of Python attribute names to C struct members?
That's in sliceobject.c:

static PyMemberDef slice_members[] = {
{"start", T_OBJECT, offsetof(PySliceObject, start), READONLY},
{"stop", T_OBJECT, offsetof(PySliceObject, stop), READONLY},
{"step", T_OBJECT, offsetof(PySliceObject, step), READONLY},
{0}
};

Note that the slice API also includes the slice.indices method.

They also implement rich comparisons, but this appears to be done by
copying the data to tuples and comparing the tuples, which is actually
a bit ironic considering this discussion.

 

[Mark Lawrence]

I don't really disagree with him, anyway. It is a rather obscure bug
-- is it worth increasing the memory footprint of slice objects by 80%
in order to fix it?

Thinking about it I entirely agree. An 80% increase in memory foorprint
where the slice objects are being used with Python 3.3.0 Unicode would
have disastrous consequences given the dire state of said Unicode, which
is why some regular contributors here are giving up with Python and
using Go.

Oh gosh look at the time, I'm just going for a walk so I can talk with
the Pixies at the bottom of my garden before they go night nights.

 

[Mark Lawrence]

I would refer you to a book written by Steve Maguire, Writing Solid
Code; Chapter 5; Candy machine interfaces.

The book that took a right hammering here
http://accu.org/index.php?module=bookreviews&func=search&rid=467 ?

 

[Michael Torrie]

Let's not confound an issue here -- I am going to implement the python
interpreter; and am not bound by optimization considerations of the
present python interpreter -- There are things I can do which as a
python programmer -- you can't. I have no choice but to re-implement
and optimize the interpreter -- the question is merely how to go about it.

As this is the case, why this long discussion? If you are arguing for a
change in Python to make it compatible with what this fork you are going
to create will do, this has already been fairly thoroughly addressed
earl on, and reasons why the semantics will not change anytime soon have
been given.

 

[Steven D'Aprano]

Absolutely bloody typical, turned down because of an idiot. Who the
hell is Tim Peters anyway?

I see your smiley, but for the benefit of those who actually don't know
who Tim Peters, a.k.a. the Timbot, is, he is one of the gurus of Python
history. He invented Python's astonishingly excellent sort routine,
Timsort, and popularised the famous adverbial phrase signoffs you will
see in a lot of older posts.

Basically, he is in the pantheon of early Python demigods.


stop-me-before-i-start-gushing-over-the-timbot-ly y'rs,

 

[Mark Lawrence]

I see your smiley, but for the benefit of those who actually don't know
who Tim Peters, a.k.a. the Timbot, is, he is one of the gurus of Python
history. He invented Python's astonishingly excellent sort routine,
Timsort, and popularised the famous adverbial phrase signoffs you will
see in a lot of older posts.

Basically, he is in the pantheon of early Python demigods.

stop-me-before-i-start-gushing-over-the-timbot-ly y'rs,

4 / 10, must try harder, the omission of the Zen of Python is considered
a very serious matter

 

[88888 Dihedral]

(e-mail address removed)æ–¼ 2012å¹´10月29日星期一UTC+8上åˆ11時12分11秒寫é“：

The slice operator does not give any way (I can find!) to take slices from negative to positive indexes, although the range is not empty, nor the expected indexes out of range that I am supplying.



Many programs that I write would require introducing variables and logical statements to correct the problem which is very lengthy and error prone unless there is a simple work around.



I *hate* replicating code every time I need to do this!



I also don't understand why slice() is not equivalent to an iterator, butcan replace an integer in __getitem__() whereas xrange() can't.





Here's an example for Linux shell, otherwise remove /bin/env...

{{{#!/bin/env python

a=[1,2,3,4,5,6,7,8,9,10]

print a[-4:3] # I am interested in getting [7,8,9,10,1,2] but I get [].

}}}

I'll suggest to use the reverse method
to get what you want.

Of course, the reverse method is not efficient for
a list of a huge number of objects in python.