Forums
New posts
Search forums
Members
Current visitors
Log in
Register
What's new
Search
Search
Search titles only
By:
New posts
Search forums
Menu
Log in
Register
Install the app
Install
Forums
Archive
Archive
Python
hashing strings to integers for sqlite3 keys
JavaScript is disabled. For a better experience, please enable JavaScript in your browser before proceeding.
You are using an out of date browser. It may not display this or other websites correctly.
You should upgrade or use an
alternative browser
.
Reply to thread
Message
[QUOTE="Chris Angelico, post: 5161901"] And they trade off that time and memory on the basis of X years of development expertise. A while ago (about ten years or so, now... wow, that's quite a while) I wrote a C++ program that needed an ever-growing array; for simplicity, I went with a very basic system of doubling the size every time, from a base of something like 1024 or 8192. (Note that it was storing and moving around only pointers, so it's comparable to Python's list.) That means it has an average 25% slack space at all times, more until its first allocation, and every now and then it has a huge job of copying a pile of pointers into a new array. (Imagine it's now at 16777216 and it needs to add the 16,777,217th string to the array. Bye-bye CPU caches.) These boundaries became *user-visible pauses*, fortunately at predictable points, but on a not-terrible computer it could cause a >1s pause just copying heaps of pointers. How do you think a Python list will perform, under the same workload (periodic appending of single strings or small groups of strings, very frequent retrieval based on index - probably about a 20:1 read:write ratio)? Not only would it be far more convenient, it's probably going to outperform my hand-rolled code - unless I'm so brilliant (or lucky) that I can stumble to something as good as can be achieved with years of dedicated development. Yeah, I don't think so. Same goes for hashing algorithms. I can at least boast that I've never written one of those... although I have several times written a binary tree of one form or another, in order to provide comparable features to a dict. And once again, now that I know how convenient and performant high level languages can be (which may or may not have been true 15 years ago, but I certainly didn't know it), I don't rewrite what can be done better by just tossing in a couple of curly braces and saying "here, map this to that". ChrisA [/QUOTE]
Verification
Post reply
Forums
Archive
Archive
Python
hashing strings to integers for sqlite3 keys
Top