f python?

Discussion in 'Python' started by Xah Lee, Apr 8, 2012.

1. Xah LeeGuest

hi guys,

sorry am feeling a bit prolifit lately.

today's show, is: ã€ˆFuck Pythonã€‰
http://xahlee.org/comp/fuck_python.html

------------------------------------
**** Python
By Xah Lee, 2012-04-08

**** Python.

just fucking spend 2 hours and still going.

here's the short story.

so recently i switched to a Windows version of python. Now, Windows
version takes path using win backslash, instead of cygwin slash. This
fucking broke my find/replace scripts that takes a dir level as input.
Because i was counting slashes.

Ok no problem. My sloppiness. After all, my implementation wasn't
portable. So, let's fix it. After a while, discovered there's the
ã€Œos.sepã€. Ok, replace ã€Œ"/"ã€ to ã€Œos.sepã€, done. Then, bang, all hell
went lose. Because, the backslash is used as escape in string, so any
regex that manipulate path got fucked majorly. So, now you need to
find a quoting mechanism. Then, **** python doc incomprehensible
scattered comp-sci-r-us BNF shit. Then, **** python for â€œos.pathâ€ and
â€œosâ€ modules then string object and string functions inconsistent
ball. And **** Guido who wants to **** change python for his idiotic
OOP concept of â€œeleganceâ€ so that some of these are deprecated.

So after several exploration of â€œrepr()â€, â€œformat()â€, â€œâ€¹strâ€º.count()â€,
â€œos.path.normpath()â€, â€œre.split()â€, â€œlen(re.search().group())â€ etc,
after a long time, let's use â€œre.escape()â€. 2 hours has passed. Also,
discovered that â€œos.path.walkâ€ is now deprecated, and one is supposed
to use the sparkling â€œos.walkâ€. In the process of refreshing my
python, the â€œos.path.walkâ€ semantics is really one fucked up ****.
Meanwhile, the â€œos.walkâ€ went into incomprehensible OOP object and
iterators ****.

now, it's close to 3 hours. This fix is supposed to be done in 10 min.
I'd have done it in elisp in just 10 minutes if not for my
waywardness.

This is Before

def process_file(dummy, current_dir, file_list):
current_dir_level = len(re.split("/", current_dir)) -
len(re.split("/", input_dir))
cur_file_level = current_dir_level+1
if min_level <= cur_file_level <= max_level:
for a_file in file_list:
if re.search(r"\.html$", a_file, re.U) and os.path.isfile(current_dir + "/" + a_file): replace_string_in_file(current_dir + "/" + a_file) This is After def process_file(dummy, current_dir, file_list): current_dir = os.path.normpath(current_dir) cur_dir_level = re.sub( "^" + re.escape(input_dir), "", current_dir).count( os.sep) cur_file_level = cur_dir_level + 1 if min_level <= cur_file_level <= max_level: for a_file in file_list: if re.search(r"\.html$", a_file, re.U) and
os.path.isfile(current_dir + re.escape(os.sep) + a_file):
replace_string_in_file(current_dir + os.sep + a_file)
# print "%d %s" % (cur_file_level, (current_dir + os.sep +
a_file))

Complete File

# -*- coding: utf-8 -*-
# Python

# find & replace strings in a dir

import os, sys, shutil, re

# if this this is not empty, then only these files will be processed
my_files = []

input_dir = "c:/Users/h3/web/xahlee_org/lojban/hrefgram2/"
input_dir = "/cygdrive/c/Users/h3/web/zz"
input_dir = "c:/Users/h3/web/xahlee_org/"

min_level = 2; # files and dirs inside input_dir are level 1.
max_level = 2; # inclusive

print_no_change = False

find_replace_list = [

(
u"""<iframe style="width:100%;border:none" src="http://xahlee.org/
footer.html"></iframe>""",
u"""<iframe style="width:100%;border:none" src="../footer.html"></
iframe>""",
),

]

def replace_string_in_file(file_path):
"Replaces all findStr by repStr in file file_path"
temp_fname = file_path + "~lc~"
backup_fname = file_path + "~bk~"

input_file = open(file_path, "rb")
input_file.close()

num_replaced = 0
for a_pair in find_replace_list:
num_replaced += file_content.count(a_pair[0])
output_text = file_content.replace(a_pair[0], a_pair[1])
file_content = output_text

if num_replaced > 0:
print "â—† ", num_replaced, " ", file_path.replace("\\", "/")
shutil.copy2(file_path, backup_fname)
output_file = open(file_path, "r+b")
preserve file creation date
output_file.seek(0)
output_file.write(output_text.encode("utf-8"))
output_file.truncate()
output_file.close()
else:
if print_no_change == True:
print "no change:", file_path

# os.remove(file_path)
# os.rename(temp_fname, file_path)

def process_file(dummy, current_dir, file_list):
current_dir = os.path.normpath(current_dir)
cur_dir_level = re.sub( "^" + re.escape(input_dir), "",
current_dir).count( os.sep)
cur_file_level = cur_dir_level + 1
if min_level <= cur_file_level <= max_level:
for a_file in file_list:
if re.search(r"\.html$", a_file, re.U) and os.path.isfile(current_dir + re.escape(os.sep) + a_file): replace_string_in_file(current_dir + os.sep + a_file) # print "%d %s" % (cur_file_level, (current_dir + os.sep + a_file)) input_dir = os.path.normpath(input_dir) if (len(my_files) != 0): for my_file in my_files: replace_string_in_file(os.path.normpath(my_file) ) else: os.path.walk(input_dir, process_file, "dummy") print "Done." Xah Lee, Apr 8, 2012 1. Advertising 2. Steven D'ApranoGuest On Sun, 08 Apr 2012 04:11:20 -0700, Xah Lee wrote: [...] I have read Xah Lee's post so that you don't have to. Shorter Xah Lee: "I don't know Python very well, and rather than admit I made some pretty lousy design choices in my code, I blame Python. And then I cross-post about it, because I'm the most important person in the Universe." When the only tool you know how to use is a hammer, everything looks like a nail. Instead of using regexes ("now you have two problems"), use the right tool: to count path components, split the path, then count the number of path components directly. import os components = os.path.split(some_path) print len(components) No matter what separator the OS users, os.path.split will do the right thing. There's no need to mess about escaping separators so you can hammer it with the regex module, when Python comes with the perfectly functional socket-wrench you actually need. -- Steven Steven D'Aprano, Apr 8, 2012 1. Advertising 3. Xah LeeGuest On Apr 8, 4:34 am, Steven D'Aprano <steve > wrote: > On Sun, 08 Apr 2012 04:11:20 -0700, Xah Lee wrote: > > [...] > > I have read Xah Lee's post so that you don't have to. > > Shorter Xah Lee: > > "I don't know Python very well, and rather than admit I made > some pretty lousy design choices in my code, I blame Python. > And then I cross-post about it, because I'm the most important > person in the Universe." > > When the only tool you know how to use is a hammer, everything looks like > a nail. Instead of using regexes ("now you have two problems"), use the > right tool: to count path components, split the path, then count the > number of path components directly. > > import os > components = os.path.split(some_path) > print len(components) > > No matter what separator the OS users, os.path.split will do the right > thing. There's no need to mess about escaping separators so you can > hammer it with the regex module, when Python comes with the perfectly > functional socket-wrench you actually need. Lol. i think you tried to make fun of me too fast. check your code. O, was it you who made fun of my python tutorial before? i was busy, i'll have to get back on that down the road. Xah Xah Lee, Apr 8, 2012 4. Martin P. HellwigGuest On 08/04/2012 12:11, Xah Lee wrote: <cut all> Hi Xah, You clearly didn't want help on this subject, as you really now how to do it anyway. But having read your posts over the years, I'd like to give you an observation on your persona, free of charge! You are actually a talented writer, some may find your occasional profanity offensive but at least it highlights your frustration. You are undoubtedly and proven a good mathematian and more important than that self taught. You have a natural feel for design (otherwise you would not clash with others view of programming). You know a mixture of programming languages. Whether you like it or not, you are in the perfect position to create a new programming language and design a new programming paradigm. Unhindered from all the legacy crap, that keep people like me behind (I actually like BNF for example). It is likely I am wrong, but if that is your destiny there is no point fighting it. Cheers and good luck, Martin Martin P. Hellwig, Apr 8, 2012 5. Steven D'ApranoGuest On Sun, 08 Apr 2012 11:34:56 +0000, Steven D'Aprano wrote: > When the only tool you know how to use is a hammer, everything looks > like a nail. Instead of using regexes ("now you have two problems"), use > the right tool: to count path components, split the path, then count the > number of path components directly. > > import os > components = os.path.split(some_path) > print len(components) Which is completely wrong. How embarrassing. Serves me right for not testing my code before sending. Nevertheless it is easy enough to write a split path function, or even to use it in place: def splitpath(path): return os.path.normpath(path).split(os.path.sep) -- Steven Steven D'Aprano, Apr 8, 2012 6. David CanziGuest Xah Lee <> wrote: >hi guys, > >sorry am feeling a bit prolifit lately. > >today's show, is: '**** Python' >http://xahlee.org/comp/fuck_python.html > >------------------------------------ >**** Python > By Xah Lee, 2012-04-08 > >**** Python. > >just fucking spend 2 hours and still going. > >here's the short story. > >so recently i switched to a Windows version of python. Now, Windows >version takes path using win backslash, instead of cygwin slash. This >fucking broke my find/replace scripts that takes a dir level as input. >Because i was counting slashes. > >Ok no problem. My sloppiness. After all, my implementation wasn't >portable. So, let's fix it. After a while, discovered there's the >'os.sep'. Ok, replace "/" to 'os.sep', done. Then, bang, all hell >went lose. Because, the backslash is used as escape in string, so any >regex that manipulate path got fucked majorly. When Microsoft created MS-DOS, they decided to use '\' as the separator in file names. This was at a time when several previously existing interactive operating systems were using '/' as the file name separator and at least one was using '\' as an escape character. As a result of Microsoft's decision to use '\' as the separator, people have had to do extra work to adapt programs written for Windows to run in non-Windows environments, and vice versa. People have had to do extra work to write software that is portable between these environments. People have done extra work while creating tools to make writing portable software easier. And people have to do extra work when they use these tools, because using them is still harder than writing portable code for operating systems that all used '/' as their separator would have been. If you added up the cost of all the extra work that people have done as a result of Microsoft's decision to use '\' as the file name separator, it would probably be enough money to launch the Burj Khalifa into geosynchronous orbit. So, when you say **** Python, are you sure you're shooting at the right target? -- David Canzi | TIMTOWWTDI (tim-toe-woe-dee): There Is More Than One | Wrong Way To Do It David Canzi, Apr 8, 2012 7. Kaz KylhekuGuest ["Followup-To:" header set to comp.lang.lisp.] On 2012-04-08, David Canzi <> wrote: > Xah Lee <> wrote: >>hi guys, >> >>sorry am feeling a bit prolifit lately. >> >>today's show, is: '**** Python' >>http://xahlee.org/comp/fuck_python.html >> >>------------------------------------ >>**** Python >> By Xah Lee, 2012-04-08 >> >>**** Python. >> >>just fucking spend 2 hours and still going. >> >>here's the short story. >> >>so recently i switched to a Windows version of python. Now, Windows >>version takes path using win backslash, instead of cygwin slash. This >>fucking broke my find/replace scripts that takes a dir level as input. >>Because i was counting slashes. >> >>Ok no problem. My sloppiness. After all, my implementation wasn't >>portable. So, let's fix it. After a while, discovered there's the >>'os.sep'. Ok, replace "/" to 'os.sep', done. Then, bang, all hell >>went lose. Because, the backslash is used as escape in string, so any >>regex that manipulate path got fucked majorly. > > When Microsoft created MS-DOS, they decided to use '\' as > the separator in file names. This is false. The MS-DOS (dare I say it) "kernel" accepts both forward and backslashes as separators. The application-level choice was once configurable through a variable in COMMAND.COM. Then they hard-coded it to backslash. However, Microsoft operating systems continued to (and until this day) recognize slash as a path separator. Only, there are broken userland programs on Windows which don't know this. > So, when you say **** Python, are you sure you're shooting at the > right target? I would have to say, probably yes. Kaz Kylheku, Apr 8, 2012 8. Peter J. HolzerGuest On 2012-04-08 17:03, David Canzi <> wrote: > If you added up the cost of all the extra work that people have > done as a result of Microsoft's decision to use '\' as the file > name separator, it would probably be enough money to launch the > Burj Khalifa into geosynchronous orbit. So we have another contender for the Most Expensive One-byte Mistake? Poul-Henning Kamp nominated the C/Unix guys: http://queue.acm.org/detail.cfm?id=2010365 hp -- _ | Peter J. Holzer | Deprecating human carelessness and |_|_) | Sysadmin WSR | ignorance has no successful track record. | | | | __/ | http://www.hjp.at/ | -- Bill Code on Peter J. Holzer, Apr 8, 2012 9. Jürgen ExnerGuest "David Canzi" <> wrote: >Xah Lee <> wrote: Please check whom you are replying to. Do not feed the trolls, please. jue Jürgen Exner, Apr 8, 2012 10. Kaz KylhekuGuest On 2012-04-08, Peter J. Holzer <> wrote: > On 2012-04-08 17:03, David Canzi <> wrote: >> If you added up the cost of all the extra work that people have >> done as a result of Microsoft's decision to use '\' as the file >> name separator, it would probably be enough money to launch the >> Burj Khalifa into geosynchronous orbit. > > So we have another contender for the Most Expensive One-byte Mistake? The one byte mistake in DOS and Windows is recognizing two characters as path separators. All code that correctly handles paths is complicated by having to look for a set of characters instead of just scanning for a byte. DOS backslashes are already mentioned in that page, but alas it perpetuates the clueless myth that DOS and windows do not recognize any other path separator. Worse, the one byte Unix mistake being covered is, disappointingly, just a clueless rant against null-terminated strings. Null-terminated strings are infinitely better than the ridiculous encapsulation of length + data. For one thing, if s is a non-empty null terminated string then, cdr(s) is also a string representing the rest of that string without the first character, where cdr(s) is conveniently defined as s + 1. Not only can compilers compress storage by recognizing that string literals are the suffixes of other string literals, but a lot of string manipulation code is simplified, because you can treat a pointer to interior of any string as a string. Because they are recursively defined, you can do elegant tail recursion on null terminated strings: const char *rec_strchr(const char *in, int ch) { if (*in == 0) return 0; else if (*in == ch) return in; else return rec_strchr(in + 1, ch); } length + data also raises the question: what type is the length field? One byte? Two bytes? Four? And then you have issues of byte order. Null terminated C strings can be written straight to a binary file or network socket and be instantly understood on the other end. Null terminated strings have simplified all kids of text manipulation, lexical scanning, and data storage/communication code resulting in immeasurable savings over the years. Kaz Kylheku, Apr 8, 2012 11. BartCGuest "Kaz Kylheku" <> wrote in message news:... > Worse, the one byte Unix mistake being covered is, disappointingly, just a > clueless rant against null-terminated strings. > > Null-terminated strings are infinitely better than the ridiculous > encapsulation of length + data. > > For one thing, if s is a non-empty null terminated string then, cdr(s) is > also > a string representing the rest of that string without the first character, > where cdr(s) is conveniently defined as s + 1. If strings are represented as (ptr,length), then a cdr(s) would have to return (ptr+1,length-1), or (nil,0) if s was one character. No big deal. (Note I saw your post in comp.lang.python; I don't about any implications of that for Lisp.) And if, instead, you want to represent all but the last character of the string, then it's just (ptr,length-1). (Some checking is needed around empty strings, but similar checks are needed around s+1.) In addition, if you want to represent the middle of a string, then it's also very easy: (ptr+a,b). > Not only can compilers compress storage by recognizing that string > literals are > the suffixes of other string literals, but a lot of string manipulation > code is > simplified, because you can treat a pointer to interior of any string as a > string. Yes, the string "bart" also contains "art", "rt" and "t". But with counted strintgs, it can also contain "bar", "ba", "b", etc.... There are a few advantages to counted strings too... > length + data also raises the question: what type is the length field? One > byte? Two bytes? Four? Depends on the architecture. But 4+4 for 32-bits, and 8+8 bytes for 64-bits, I would guess, for general flex strings of any length. There are other ways of encoding a length. (For example I use one short string type of maximum M characters, but the current length N is encoded into the string, without needing any extra count byte (by fiddling about with the last couple of bytes). If you're trying to store a short string in an 8-byte field in a struct, then this will let you use all 8 bytes; a zero-terminated one, only 7.) > And then you have issues of byte order. Which also affects every single value of more than one byte. > Null terminated > C strings can be written straight to a binary file or network socket and > be > instantly understood on the other end. But they can't contains nulls! > Null terminated strings have simplified all kids of text manipulation, > lexical > scanning, and data storage/communication code resulting in immeasurable > savings over the years. They both have their uses. -- Bartc BartC, Apr 8, 2012 12. NobodyGuest On Sun, 08 Apr 2012 04:11:20 -0700, Xah Lee wrote: > Ok no problem. My sloppiness. After all, my implementation wasn't > portable. So, let's fix it. After a while, discovered there's the > os.sep. Ok, replace "/" to os.sep, done. Then, bang, all hell > went lose. Because, the backslash is used as escape in string, so any > regex that manipulate path got fucked majorly. So, now you need to > find a quoting mechanism. if os.altsep is not None: sep_re = '[%s%s]' % (os.sep, os.altsep) else: sep_re = '[%s]' % os.sep But really, you should be ranting about regexps rather than Python. They're convenient if you know exactly what you want to match, but a nuisance if you need to generate the expression based upon data which is only available at run-time (and re.escape() only solves one very specific problem). Nobody, Apr 9, 2012 13. Xah LeeGuest Xah Lee wrote: Â« http://xahlee.org/comp/fuck_python.html Â» David Canzi wrote Â«When Microsoft created MS-DOS, they decided to use '\' as the separator in file names. Â This was at a time when several previously existing interactive operating systems were using '/' as the file name separator and at least one was using '\' as an escape character. Â As a result of Microsoft's decision to use '\' as the separator, people have had to do extra work to adapt programs written for Windows to run in non-Windows environments, and vice versa. Â People have had to do extra work to write software that is portable between these environments. People have done extra work while creating tools to make writing portable software easier. Â And people have to do extra work when they use these tools, because using them is still harder than writing portable code for operating systems that all used '/' as their separator would have been.Â» namekuseijin wrote: > yes, absolutely. Â But you got 2 inaccuracies there: Â 1) Microsoft didn't create DOS; 2) fucking DOS was written in C, and guess what, it uses \ as escape character. Â Fucking microsoft. > > > So, when you say **** Python, are you sure you're shooting at the > > right target? > > I agree. Â **** winDOS and fucking microsoft. No. The choice to use backslash than slash is actually a good one. because, slash is one of the useful char, far more so than backslash. Users should be able to use that for file names. i don't know the detailed history of path separator, but if i were to blame, it's **** unix. The entirety of unix, unix geek, unixers, unix fuckheads. **** unix. ã€ˆOn Unix Filename Characters Problemã€‰ http://xahlee.org/UnixResource_dir/writ/unix_filename_chars.html ã€ˆOn Unix File System's Case Sensitivityã€‰ http://xahlee.org/UnixResource_dir/_/fileCaseSens.html ã€ˆUNIX Tar Problem: File Length Truncation, Unicode Name Supportã€‰ http://xahlee.org/comp/unix_tar_problem.html ã€ˆWhat Characters Are Not Allowed in File Names?ã€‰ http://xahlee.org/mswin/allowed_chars_in_file_names.html ã€ˆUnicode Support in File Names: Windows, Mac, Emacs, Unison, Rsync, USB, Zipã€‰ http://xahlee.org/mswin/unicode_support_file_names.html ã€ˆThe Nature of the Unix Philosophyã€‰ http://xahlee.org/UnixResource_dir/writ/unix_phil.html Xah Xah Lee, Apr 9, 2012 14. Alex MizrahiGuest >> Ok no problem. My sloppiness. After all, my implementation wasn't >> portable. So, let's fix it. After a while, discovered there's the >> os.sep. Ok, replace "/" to os.sep, done. Then, bang, all hell >> went lose. Because, the backslash is used as escape in string, so any >> regex that manipulate path got fucked majorly. So, now you need to >> find a quoting mechanism. > > if os.altsep is not None: > sep_re = '[%s%s]' % (os.sep, os.altsep) > else: > sep_re = '[%s]' % os.sep > > But really, you should be ranting about regexps rather than Python. > They're convenient if you know exactly what you want to match, but a > nuisance if you need to generate the expression based upon data which is > only available at run-time (and re.escape() only solves one very specific > problem). It isn't a problem of regular expressions, but a problem of syntax for specification of regular expressions (i.e. them being specified as a string). Common Lisp regex library cl-ppcre allows to specify regex via a parse tree. E.g. "(foo[/\\]bar)" becomes REGISTER SEQUENCE "foo" CHAR-CLASS #\/ #\\) "bar")) This is more verbose, but totally unambiguous and requires no escaping. So this definitely is a problem of Python's regex library, and a problem of lack of support for nice parse tree representation in code. cl-ppcre supports both textual perl-compatible regex specification and parse tree. I would start with a simple string specification, then when shit hits fan I can call cl-ppcre:arse-string to get those parse trees and replaces forward slash with back slash. Moreover, I can automatically convert regexes: (defun scan-auto/ (regex target-string) (let ((fixed-parse-tree (subst 'char-class #\/ #\\) 'char-class #\/) (cl-ppcre:arse-string regex) :test 'equal))) (cl-ppcre:scan-to-strings fixed-parse-tree target-string))) CL-USER> (scan-auto/ "foo[/]bar" "foo\\bar") "foo\\bar" #() Alex Mizrahi, Apr 9, 2012 15. Roy SmithGuest In article <4f82d3e2$1$fuzhry+tra$>,
Shmuel (Seymour J.) Metz <> wrote:

> >Null terminated strings have simplified all kids of text
> >manipulation, lexical scanning, and data storage/communication
> >code resulting in immeasurable savings over the years.

>
> Yeah, especially code that needs to deal with lengths and nulls. It's
> great for buffer overruns too.

I once worked on a C++ project that used a string class which kept a
length count, but also allocated one extra byte and stuck a null at the
end of every string.
Roy Smith, Apr 9, 2012
16. Kaz KylhekuGuest

On 2012-04-09, Shmuel Metz <> wrote:
> In <>, on 04/08/2012
> at 07:14 PM, Kaz Kylheku <> said:
>
>>Null-terminated strings are infinitely better than the ridiculous
>>encapsulation of length + data.

>
> ROTF,LMAO!
>
>>For one thing, if s is a non-empty null terminated string then,
>>cdr(s) is also a string representing the rest of that string
>>without the first character,

>
> Are you really too clueless to differentiate between C and LISP?

In Lisp we can burn a list literal like '(a b c) into ROM, and compute (b c)
without allocating any memory.

Null-terminated C strings do the same thing.

In some Lisp systems, in fact, "CDR coding" was used to save space when
allocating a list all at once. This created something very similar to
a C string: a vector-like object of all the CARs, with a terminating
convention marking the end.

It's logically very similar.

I need not repeat the elegant recursion example for walking a C string.

That example is not possible with the length + data representation.
(Not without breaking the encapsulation and passing the length as a separate
recursion parameter to a recursive routine that works with the raw data part of
the string.)

>>Null terminated strings have simplified all kids of text
>>manipulation, lexical scanning, and data storage/communication
>>code resulting in immeasurable savings over the years.

>
> Yeah, especially code that needs to deal with lengths and nulls.

To get the length of a string, you call a function, in either representation,
so it is not any more complicated from a coding point of view. The function is,
of course, more expensive if the string is null terminated, but you can code
with awareness of this and not call length wastefully.

If all else was equal (so that the expense of the length operation were
the /only/ issue) then of course the length + data would be better.

However, all else is not equal.

One thing that is darn useful, for instance, is that
p + strlen(p) still points to a string which is length zero, and this
sort of thing is widely exploited in text processing code. e.g.

size_t digit_prefix_len = strspn(input_string, "0123456789");
const char *after_digits = input-string + digit_prefix_len;

if (*after_digits == 0) {
/* string consists only of digits: nothing after digits */
} else {
/* process part after digits */
}

It's nice that after_digits is a bona-fide string just like input_string,
without any memory allocation being required.

We can lexically analyze a string without ever asking it what its length is,
and as we march down the string, the remaining suffix of that string is always
a string so we can treat it as one, recurse on it, whatever.

Code that needs to deal with null "characters" is manipulating binary data, not
text, and should use a suitable data structure for that.

> It's great for buffer overruns too.

If we scan for a null terminator which is not there, we have a buffer overrun.

If a length field in front of string data is incorrect, we also have a buffer
overrrun.

A pattern quickly emerges here: invalid, corrupt data produced by buggy code
leads to incorrect results, and behavior that is not well-defined!
Kaz Kylheku, Apr 9, 2012
17. Kaz KylhekuGuest

On 2012-04-09, Roy Smith <> wrote:
> In article <4f82d3e2$1$fuzhry+tra$>, > Shmuel (Seymour J.) Metz <> wrote: > >> >Null terminated strings have simplified all kids of text >> >manipulation, lexical scanning, and data storage/communication >> >code resulting in immeasurable savings over the years. >> >> Yeah, especially code that needs to deal with lengths and nulls. It's >> great for buffer overruns too. > > I once worked on a C++ project that used a string class which kept a > length count, but also allocated one extra byte and stuck a null at the > end of every string. Me too! I worked on numerous C++ projects with such a string template class. It was usually called std::basic_string and came from this header called: #include <string> which also instantiated it into two flavors under two nicknames: std::basic_string<char> being introduced as std::string, and std::basic_string<wchar_t> as std::wstring. This class had a c_str() function which retrieved a null-terminated string and so most implementations just stored the data that way, but some of the versions of that class cached the length of the string to avoid doing a strlen or wcslen operation on the data. Kaz Kylheku, Apr 9, 2012 18. Rainer WeikusatGuest Shmuel (Seymour J.) Metz <> writes: [...] >>For one thing, if s is a non-empty null terminated string then, >>cdr(s) is also a string representing the rest of that string >>without the first character, > > Are you really too clueless to differentiate between C and LISP? In LISP, a list is a set of conses (pairs) whose car (first element of the pair) contains a value and whose cdr (second element of the pair) links to the next cons that's part of the list. The end of a list is marked by a cdr whose value is nil. A so-called 'C string' is a sequentially allocated sequence of memory locations which contain the characters making up the string and the end of it is marked by a memory location holding the value 0. This is logically very similar to the LISP list and it shouldn't be to difficult to understand that 'cdr(s) is also a string representing the rest of the string' means 'given that s points to a non-empty C string, s + 1 points to a possibly empty C string which is identical with s with the first character removed'. >>Null terminated strings have simplified all kids of text >>manipulation, lexical scanning, and data storage/communication >>code resulting in immeasurable savings over the years. > > Yeah, especially code that needs to deal with lengths and nulls. It's > great for buffer overruns too. This is, I think, a case where the opinions of people who have used C strings and the opinions of people who haven't differ greatly. A nice German proverb applicable to situations like that would be 'Was der Bauer nicht kennt das frisst er nicht' ... Rainer Weikusat, Apr 9, 2012 19. Rainer WeikusatGuest Rainer Weikusat <> writes: > Shmuel (Seymour J.) Metz <> writes: > > [...] > >>>For one thing, if s is a non-empty null terminated string then, >>>cdr(s) is also a string representing the rest of that string >>>without the first character, >> >> Are you really too clueless to differentiate between C and LISP? > > In LISP, a list is a set of conses (pairs) whose car (first element of > the pair) contains a value and whose cdr (second element of the pair) > links to the next cons that's part of the list. The end of a list is > marked by a cdr whose value is nil. Addition: This can also be implemented very neatly in Perl by using two element array references as 'cons cells', toy example ----------- sub car { return$_[0][0];
}

sub cdr
{
return $_[0][1]; } sub list { @_ && [shift, &list]; }$l = list(0 .. 100);
while ($l) { print(car($l), ' ');
$l = cdr($l);
}
print("\n");
-----------

and for algorithms which are well-suited for linked lists, this can
even outperform (when suitably implemented) an equivalent algorithm
using arrays.
Rainer Weikusat, Apr 9, 2012
20. BartCGuest

"Shmuel (Seymour J.)Metz" <> wrote in
message news:4f8410ff$2$fuzhry+tra\$...
> In <>, on 04/09/2012
> at 06:55 PM, Kaz Kylheku <> said:

>>If we scan for a null terminator which is not there, we have a
>>buffer overrun.

>
> You're only thinking of scanning an existing string; think of
> constructing a string. The null only indicates the current length, not
> the amount allocated.
>
>>If a length field in front of string data is incorrect, we also have
>>a buffer overrrun.

>
> The languages that I'm aware of that use a string length field also
> use a length field for the allocated storage. More precisely, they
> require that attempts to store beyond the allocated length be
> detected.

I would have thought trying to *read* beyond the current length would be an
error.

Writing beyond the current length, and perhaps beyond the current allocation
might be OK if the string is allowed grow, otherwise that's also an error.

In any case, there is no real need for an allocated length to be passed
around with the string, if you are only going to be reading it, or only
modifying the existing characters. And depending on the memory management
arrangements, such a length need not be stored at all.

--
Bartc
BartC, Apr 10, 2012