Equivalents of Ruby's "!" methods?

[Simon Mullis]

Hi All,

Quick question, I can't seem to find the answer online (well, at the
moment I think the answer is a simple "no" but I would like to
confirm).

Consider the following hash:

h = { "1" : "a\r", "2" : "b\n" }

In order to strip the dict values in Python I (think) I can only do
something like:

for k,v in h.items:
h[k] = v.strip()

While in Ruby - for the equivale dict/hash - I have the option of an
in-place method:

h.each_value { |v| val.strip! }

Are there Python equivalents to the "!" methods in Ruby?

The reason I ask is that I have some fairly complex data-structures
and this would make my code alot cleaner... If this is not an accepted
and pythonic way of doing things then please let me know... and I'll
stop!

Thanks in advance

SM

 

[Paul McGuire]

Consider the following hash:

h = { "1" : "a\r", "2" : "b\n" }

In Python, this called a "dict". "hash" sounds so... perlish.

In order to strip the dict values in Python I (think) I can only do
something like:

for k,v in h.items:
    h[k] = v.strip()

While in Ruby - for the equivale dict/hash - I have the option of an
in-place method:

h.each_value { |v| val.strip! }

Are there Python equivalents to the "!" methods in Ruby?

You have pretty much answered your own question. "!" methods are not
specially named or annotated in Python, str.strip is the method that
corresponds to strip!. It looks a little different since you left off
the ()'s in the Ruby example (as is your privilege and right in Ruby).

Pythonistically speaking, even though a dict is a mutable thing, I'm
learning that the accepted practice for methods like this is not so
much to update in place as it is to use generator expressions to
construct a new object. For example, given a list of integers to 100,
instead of removing all of the even numbers (with the attendant
hassles of updating a list while iterating over it), just create a new
list of the numbers that are not even.

In your dict case, this would just be:

h = dict( k,v.strip() for k,v in h.iteritems() )

Perhaps I am overgeneralizing from comments I have read here on c.l.py
about creating new lists instead updating old ones. But in your loop,
you *will* end up doing a re-assignment of every value in the dict,
even if the original value had no whitespace to strip.

-- Paul

 

[Simon Mullis]

Thanks to all for the quick responses.

This is a 'dict' instance in Python. A 'hash' is a different concept.

In order to strip the dict values in Python I (think) I can only do
something like:

for k,v in h.items:
h[k] = v.strip()

The above won't do what you describe, since 'h.items' evaluates to
that function object, which is not iterable. If you want the return
value of the function, you must call the function:

for (k, v) in h.items():
h[k] = v.strip()

Yes - absolutely, a typo on my part.

This will create a new value from each existing value, and re-bind
each key to the new value for that key. Clear, straightforward, and
Pythonic.

You can also create a new dict from a generator, and re-bind the name
to that new dict:

h = dict(
(k, v.strip())
for (k, v) in h.items())

Also quite Pythonic, but rather less clear if one hasn't yet
understood generator expressions. Very useful to have when needed,
though.

Thanks for this - I'll have a look!

I'm not overly familiar with Ruby, but the feature you describe above
seems to rely on mutating the string value in-place. Is that right?

There are a number of methods that can be used to change things
"in-place" such as:
=> ["upcase!", "gsub!", "downcase!", "chop!", "capitalize!", "tr!",
"chomp!", "swapcase!", "tr_s!", "succ!", "strip!", "delete!",
"lstrip!", "squeeze!", "next!", "rstrip!", "slice!", "reverse!",
"sub!"]

Or,
=> ["map!", "shuffle!", "uniq!", "reject!", "compact!", "slice!",
"sort!", "flatten!", "collect!", "reverse!"]

They normally have a non-"!" partner which is used only for a return
value and does not affect the original object.

But! This isn't a Ruby group so I'll stop now... ;-)

Strings in Python are immutable (among other reasons, this allows them
to meet the requirement of dict keys to be immutable, which in turn
allows dict implementations to be very fast), so you can only get a
new value for a string by creating a new string instance and re-bind
the reference to that new value.

Normally I would use a Ruby symbol as a hash key:

h = { :key1 => "val1", :key2 => "val2" }

From the stdlib docs:

"The same Symbol object will be created for a given name or string for
the duration of a program's execution, regardless of the context or
meaning of that name. Thus if Fred is a constant in one context, a
method in another, and a class in a third, the Symbol :Fred will be
the same object in all three contexts. "

There is no equivalent in Python (as far as I know, and I'm only in my
second week of Python so I'm more than likely incorrect!).

If you're interested:
http://www.randomhacks.net/articles/2007/01/20/13-ways-of-looking-at-a-ruby-symbol

Thanks again for the pointers.

 

[bearophileHUGS]

Simon Mullis:

h = { "1" : "a\r", "2" : "b\n" }
I have the option of an in-place method:
h.each_value { |v| val.strip! }

This in-place version may be the closer code for Python 2.x:

d = {"1": "a\r", "2": "b\n"}
d.update((k, v.strip()) for k, v in d.iteritems())

You may also want to use v.rstrip() there instead.

Bye,
bearophile

 

[Hrvoje Niksic]

There is no equivalent in Python (as far as I know, and I'm only in
my second week of Python so I'm more than likely incorrect!).

Python uses strings for that purpose, which works well due to their
immutability. Python automatically interns strings used in source
code that look like identifiers, so memory is saved.

An even closer equivalent of symbols are interned strings (see the
"intern" built-in), but they are not as nice to work with as "real"
symbols.

 

[Steven D'Aprano]

In Python, this called a "dict". "hash" sounds so... perlish.

A hash is also a mixture, a jumble or mess, as in hash browns, or as in
"to make a hash of something".

Also, a hash table is the conventional Computer Science term for what
Python calls dicts and Perl calls associative arrays.

 

[Terry Reedy]

Pythonistically speaking, even though a dict is a mutable thing, I'm
learning that the accepted practice for methods like this is not so
much to update in place as it is to use generator expressions to
construct a new object.

I disagree. The convention is that mutation methods should return None.

> For example, given a list of integers to 100,
instead of removing all of the even numbers (with the attendant
hassles of updating a list while iterating over it), just create a new
list of the numbers that are not even.

This is because each removal is O(n), making the entire process O(n*2),
whereas a new list is O(n) -- and the hassle of remembering to iterate
in reverse when doing removals.

Perhaps I am overgeneralizing from comments I have read here on c.l.py
about creating new lists instead updating old ones.

I think so. Removals and insertions are importantly different from
change in place. If I wanted to square each number in a list, and *did
not need the original list*, I would not hesitate to do it in place.

The newish sorted() and reversed() built-ins were meant to complement
list.sort and list.reverse, not replace them.

tjr

 

[Grzegorz Staniak]

The newish sorted() and reversed() built-ins were meant to complement
list.sort and list.reverse, not replace them.

BTW, is there a reason why sorted() on a list returns a list, while
reversed() on the same list returns an iterator?

GS

 

[Fredrik Lundh]

BTW, is there a reason why sorted() on a list returns a list, while
reversed() on the same list returns an iterator?

the algorithm required to sort a sequence is something entirely
different from the algorithm required to loop over a sequence in
reverse. we went through this a few days ago.

</F>

 

[Steven D'Aprano]

BTW, is there a reason why sorted() on a list returns a list, while
reversed() on the same list returns an iterator?

Until the day that somebody discovers how to sort a list without seeing
all the items first, there's no point in sorted() returning an iterator.
It has to generate a copy of the entire list to sort it, and so might as
well just return the list -- there's no advantage to turning it into an
iterator after you've already built the list.

On the other hand, reversed() can supply it's items lazily. Although it
does need access to the entire source, it doesn't need to return an
entire list. It can just return the items one at a time, starting from
the last one.

That however does mean there's one gotcha: if you mutate a list after
calling sorted() on it, the result of the sorted() doesn't change. But
the same doesn't hold for reversed():

L = range(5)
it = reversed(L) # expecting [4, 3, 2, 1, 0] as an iterator
it.next() 4
L[3] = 'mutate'
it.next()

'mutate'

The solution to that is simple: call list(reversed(L)). Or don't mutate
the original.

 

[mumbler]

Until the day that somebody discovers how to sort a list without seeing
all the items first, there's no point in sorted() returning an iterator.

To nitpick, this is not strictly true: sure, you're at best O(nlogn)
on sorting the entire list, but you could return the first element of
the 'sorted' list in O(n) (if you don't mind using a O(n^2) algorithm
for the whole sort). i.e. if you have a use case where you're only
likely to look at the first few elements of a sorted list, it would
make some sense to have an iterator.