Why 'files.py' does not print the filenames into a table format?


N

Nick the Gr33k

Hello,

Trying to browse http://superhost.gr/?page=files.py with tailing -F of
the error_log i noticed that error log outputs no error!

So that means that the script is correct.

here are the directory app's files.

(e-mail address removed) [~/www/data/apps]# ls -l
total 412788
drwxr-xr-x 2 nikos nikos 4096 Jun 12 12:03 ./
drwxr-xr-x 6 nikos nikos 4096 May 26 21:13 ../
-rwxr-xr-x 1 nikos nikos 13157283 Mar 17 12:57 100\ Mythoi\ tou\
Aiswpou.pdf*
-rwxr-xr-x 1 nikos nikos 29524686 Mar 11 18:17 Anekdotologio.exe*
-rw-r--r-- 1 nikos nikos 42413964 Jun 2 20:29 Battleship.exe
-rw-r--r-- 1 nikos nikos 51819750 Jun 2 20:04 Luxor\ Evolved.exe
-rw-r--r-- 1 nikos nikos 60571648 Jun 2 14:59 Monopoly.exe
-rwxr-xr-x 1 nikos nikos 1788164 Mar 14 11:31 Online\ Movie\ Player.zip*
-rw-r--r-- 1 nikos nikos 5277287 Jun 1 18:35 O\ Nomos\ tou\ Merfy\
v1-2-3.zip
-rwxr-xr-x 1 nikos nikos 16383001 Jun 22 2010 Orthodoxo\ Imerologio.exe*
-rw-r--r-- 1 nikos nikos 6084806 Jun 1 18:22 Pac-Man.exe
-rw-r--r-- 1 nikos nikos 45297713 Jun 10 12:38 Raptor\ Chess.exe
-rw-r--r-- 1 nikos nikos 25476584 Jun 2 19:50 Scrabble.exe
-rwxr-xr-x 1 nikos nikos 49141166 Mar 17 12:48 To\ 1o\ mou\ vivlio\ gia\
to\ skaki.pdf*
-rwxr-xr-x 1 nikos nikos 3298310 Mar 17 12:45 Vivlos\ gia\ Atheofovous.pdf*
-rw-r--r-- 1 nikos nikos 1764864 May 29 21:50 V-Radio\ v2.4.msi
-rw-r--r-- 1 nikos nikos 3511233 Jun 4 14:11 Ευχή\ του\ ΙησοÏ.mp3
-rwxr-xr-x 1 nikos nikos 66896732 Mar 17 13:13 Κοσμάς\ Αιτωλός\ -\
ΠÏοφητείες.pdf*
-rw-r--r-- 1 nikos nikos 236032 Jun 4 14:10 Σκέψου\ έναν\ αÏιθμό.exe


The code is as follows:

#
=================================================================================================================
# Convert wrongly encoded filenames to utf-8
#
=================================================================================================================
path = b'/home/nikos/public_html/data/apps/'
filenames = os.listdir( path )

utf8_filenames = []

for filename in filenames:
# Compute 'path/to/filename'
filename_bytes = path + filename
encoding = guess_encoding( filename_bytes )

if encoding == 'utf-8':
# File name is valid UTF-8, so we can skip to the next file.
utf8_filenames.append( filename_bytes )
continue
elif encoding is None:
# No idea what the encoding is. Hit it with a hammer until it stops
moving.
filename = filename_bytes.decode( 'utf-8', 'xmlcharrefreplace' )
else:
filename = filename_bytes.decode( encoding )

# Rename the file to something which ought to be UTF-8 clean.
newname_bytes = filename.encode('utf-8')
os.rename( filename_bytes, newname_bytes )
utf8_filenames.append( newname_bytes )

# Once we get here, the file ought to be UTF-8 clean and the Unicode
name ought to exist:
assert os.path.exists( newname_bytes.decode('utf-8') )


# Switch filenames from utf8 bytestrings => unicode strings
filenames = []

for utf8_filename in utf8_filenames:
filenames.append( utf8_filename.decode('utf-8') )

# Check the presence of a database file against the dir files and delete
record if it doesn't exist
cur.execute('''SELECT url FROM files''')
data = cur.fetchall()

for url in data:
if url not in filenames:
# Delete spurious
cur.execute('''DELETE FROM files WHERE url = %s''', url )


#
=================================================================================================================
# Display ALL files, each with its own download button
#
=================================================================================================================
print('''<body background='/data/images/star.jpg'>
<center><img src='/data/images/download.gif'><br><br>
<table border=5 cellpadding=5 bgcolor=green>
''')

try:
cur.execute( '''SELECT * FROM files ORDER BY lastvisit DESC''' )
data = cur.fetchall()

for row in data:
(filename, hits, host, lastvisit) = row
lastvisit = lastvisit.strftime('%A %e %b, %H:%M')

print('''
<form method="get" action="/cgi-bin/files.py">
<tr>
<td> <center> <input type="submit" name="filename" value="%s"> </td>
<td> <center> <font color=yellow size=5> %s </td>
<td> <center> <font color=orange size=4> %s </td>
<td> <center> <font color=silver size=4> %s </td>
</tr>
</form>
''' % (filename, hits, host, lastvisit) )
print( '''</table><br><br>''' )
except pymysql.ProgrammingError as e:
print( repr(e) )

===========================================
PLEASE take a look, its not a huge code, the encoding was of Steven
idea's, so from another thread is a bit more or less already known to
the most of you.

I just want to know why it doesn't print anything.

Thank you and please whoever does not feel like helping, please at least
not spam the thread.
 
Ad

Advertisements

N

Nick the Gr33k

Nick, at this point, you need to hire someone to do your work for you.

The code is completely ready.
Some detail is missing and its not printing the files as expected.

Irrelevant to my question i just noticed weird behavior about my
pelatologio.py script which can be seen here:

http://superhost.gr/?show=stats

The first 3 files are of my doing.
All the rest are of someone else's that managed to append entries into
my counters database utilizing this code:

================

try:
#find the needed counter for the page URL
cur.execute('''SELECT ID FROM counters WHERE url = %s''', page )
data = cur.fetchone() #URL is unique, so should only be one

if not data:
#first time for page; primary key is automatic, hit is defaulted
cur.execute('''INSERT INTO counters (url) VALUES (%s)''', page )
cID = cur.lastrowid #get the primary key value of the new record
======================

Does someone want to state something?
 
J

Joshua Landau

The code is completely ready.
Some detail is missing and its not printing the files as expected.

Look, Nick,

A lot of people are frustrated by you. You should understand that. If
you cannot, you need to step back and consider, or you really are a
troll.

Now, obviously it's not going to get you any help to have half of the
forum angry at you. People have stopped helping, at least in large.
This is fine; people here are volunteers. But you want help.

So, Nick, listen. You need to learn how to ask *smart* questions. If
you do, I *guarantee* that people will respect you a lot more. I'll be
willing to give a bit of time to explain what I mean.

1) What is your problem. Not "I want to know why it doesn't print
anything." Here's an example, for some random idea:
I've written some code to find the first file in a directory which
is not UTF-8. Lines 40-42 are meant to print out the file found
to a log ("/home/joshua/.logs/log"). Unfortunately, although
there is no error, no file is printed to the log.

2) What have you tried? What debugging have you done? For someone of
your skill level, it's also important to tell us what you think your
code is doing. Example:
I've tried checking for a failure - when there is no non-UTF-8 file
in the directory the appropriate error is raised. I think this should
mean that the "else" after the "for" loop would be run, and this
should run the lines 40-42 above when there *is* a non-UTF-8
file.

3) If possible, give us an example we can run.
To make helping easier, I've removed the code that searches the
directory as I know that works, and instead there's a list of BytesIO
and StringIO objects that pretend to be them. The bug is still
there.

Do you see the difference?
Irrelevant to my question i just noticed weird behavior about my
pelatologio.py script which can be seen here:

http://superhost.gr/?show=stats

The first 3 files are of my doing.
All the rest are of someone else's that managed to append entries into my
counters database utilizing this code:

================

try:
#find the needed counter for the page URL
cur.execute('''SELECT ID FROM counters WHERE url = %s''',
page )
data = cur.fetchone() #URL is unique, so should only
be one

if not data:
#first time for page; primary key is automatic, hit
is defaulted
cur.execute('''INSERT INTO counters (url) VALUES
(%s)''', page )
cID = cur.lastrowid #get the primary key
value of the new record
======================

Does someone want to state something?

Sure. Here I go:

What's the question?
 
M

Mark Lawrence

Thank you and please whoever does not feel like helping, please at least
not spam the thread.

Your arrogance clearly has no bounds. This is a public forum and people
can say what they like. You've wasted enough time as it is, so why
don't you simply bugger off.

--
"Steve is going for the pink ball - and for those of you who are
watching in black and white, the pink is next to the green." Snooker
commentator 'Whispering' Ted Lowe.

Mark Lawrence
 
C

Chris Angelico

cur.execute('''SELECT ID FROM counters WHERE url = %s''', page )
cur.execute('''INSERT INTO counters (url) VALUES (%s)''', page )

Sure, whoever wrote that code is a fool.

http://xkcd.com/327/

They didn't sanitize your database inputs.

I assume you're talking about the above two lines of code? They're not
SQL injection targets. The clue is that the %s isn't in quotes. This
is an out-of-band argument passing method (actually, since he's using
MySQL (IIRC), it's probably just going to escape it and pass it on
through, but it comes to the same thing), so it's safe.

ChrisA
 
S

Steven D'Aprano

Your arrogance clearly has no bounds. This is a public forum and people
can say what they like. You've wasted enough time as it is, so why
don't you simply bugger off.

Congratulation. You have just entered an extremely exclusive club. See
you in a month.


*plonk*
 
Ad

Advertisements

N

Nick the Gr33k

Your arrogance clearly has no bounds.

Your spamming to my threads in an unproductive and yet bitching way has
no bounds either.
This is a public forum

Yes it is.
and people can say what they like.

Only if its relative to the OP's question, otherwise its trolling to an
other's person thread.
You've wasted enough time as it is, so why don't you simply bugger off.

The only time i'm wasting is that of folk's trying to respond to my
questions.

You are the one that wants to waste his time if you take the time and
read my posts and also take more time to bitch-respond.

I said to you and others before. Kill-file me, or mute my threads or
ignore me if you do not like me and my questions.
 
N

Nick the Gr33k

I assume you're talking about the above two lines of code? They're not
SQL injection targets.

Then how those page entries found in the database Chris?
The clue is that the %s isn't in quotes.

What happens if i write it like this?

cur.execute('''SELECT ID FROM counters WHERE url = "%s"''', page )

How quoting of %s helps here?
This is an out-of-band argument passing method (actually, since he's using
MySQL (IIRC), it's probably just going to escape it and pass it on
through, but it comes to the same thing), so it's safe.

Yes iam using a comma and not a substitute operator, so input is mysql
validates.

Please explain what is an "out-of-band argument passing method"

What your idea of those entries made it to the counters database table?
 
N

Nick the Gr33k

I assume you're talking about the above two lines of code? They're not
SQL injection targets. The clue is that the %s isn't in quotes. This
is an out-of-band argument passing method (actually, since he's using
MySQL (IIRC), it's probably just going to escape it and pass it on
through, but it comes to the same thing), so it's safe.

ChrisA

Chris or someone else please explain a bit whats happening here because
that list is getting bigger and bigger as we speak.

look: http://superhost.gr/?show=stats

At least i have secured 'pelatologio.py' form prying eyes.
 
M

Mark Lawrence

Your spamming to my threads in an unproductive and yet bitching way has
no bounds either.


Yes it is.


Only if its relative to the OP's question, otherwise its trolling to an
other's person thread.


The only time i'm wasting is that of folk's trying to respond to my
questions.

You are the one that wants to waste his time if you take the time and
read my posts and also take more time to bitch-respond.

I said to you and others before. Kill-file me, or mute my threads or
ignore me if you do not like me and my questions.

I have no intention of kill-filing you, muting your threads or ignoring
you. I do intend hounding you until with any luck you crawl off into a
hole somewhere and leave this group in peace.

--
"Steve is going for the pink ball - and for those of you who are
watching in black and white, the pink is next to the green." Snooker
commentator 'Whispering' Ted Lowe.

Mark Lawrence
 
N

Nick the Gr33k

I assume you're talking about the above two lines of code? They're not
SQL injection targets. The clue is that the %s isn't in quotes. This
is an out-of-band argument passing method (actually, since he's using
MySQL (IIRC), it's probably just going to escape it and pass it on
through, but it comes to the same thing), so it's safe.

ChrisA

Here is how i think i have dealt with it:

=================
path = '/home/nikos/public_html/'
cgi_path = '/home/nikos/public_html/cgi-bin/'

file = form.getvalue('file') # this comes from .htaccess
page = form.getvalue('page') # this comes form index.html or metrites.py

if not page and os.path.exists( file ):
# it is an html template
page = file.replace( path, '' )

......
......

#find the needed counter for the page URL
if os.path.exists( path + page ) or os.path.exists( cgi_path + page ):
cur.execute('''SELECT ID FROM counters WHERE url = %s''', page )
data = cur.fetchone() #URL is unique

==================

Do you think i'am sfae now from those kind of attacks?
Do you see some other way, better, to write the above?
 
Ad

Advertisements

N

Nick the Gr33k

I have no intention of kill-filing you, muting your threads or ignoring
you. I do intend hounding you until with any luck you crawl off into a
hole somewhere and leave this group in peace.

No such luck i'm afraid for you.
And it seems to me that you are the one that doesn't leave this group in
piece, not me.


ps to other members: Is there any way in ThunderBird that i kill file
Mark? Never have to used kill-filing before but i'll start now.
 
D

Denis McMahon

PLEASE take a look, its not a huge code

First, you need to start writing your code to less than 80 columns if
you're going to keep posting it to usenet. I'm sure I'm not the only
person who can't be bothered to unwrap it.

Secondly, the code you posted only tells part of the story - it's
obviously missing either relevant imports or defined functions or
possibly both.

Third, it would help to see examples of (a) what you expect it to
generate, and (b) what it actually generates. You obviously have a web
server available to you - you could put both code (just append .txt to
the filename) and screenshots from your browser there with no difficulty
at all and just include links.
 
N

Nick the Gr33k

Nikos,

Have you considered subscribing to this?

http://mail.python.org/mailman/listinfo/python-greece


Possibly some of these concepts will be easier for you to understand if
explained to you in your native language. Or you might be able to join a
local Users Group who can help you.

Thank you Steven i don't want to enter there as mail but wish to find it
as a newsgroups, which i tried to subscribe but TB couldn't find it.

Also i have no trouble understand you guys in English or express myself
here. I like English.

And i'm under the impression that foreigners are more helpful from
Greeks. At least that's what experience have tought me in a local linux
group for many years.

I prefer staying here but i can also subscribe there as well if you teel
me what the groups name.
 
N

Nick the Gr33k

First, you need to start writing your code to less than 80 columns if
you're going to keep posting it to usenet. I'm sure I'm not the only
person who can't be bothered to unwrap it.

TB behaves for me the same way. Any line > 80 chars gets a newline.
Why this is happening? Why not post up to 256 chars in a single line?
Secondly, the code you posted only tells part of the story - it's
obviously missing either relevant imports or defined functions or
possibly both.

Third, it would help to see examples of (a) what you expect it to
generate, and (b) what it actually generates. You obviously have a web
server available to you - you could put both code (just append .txt to
the filename) and screenshots from your browser there with no difficulty
at all and just include links.
Actually i twas a short story since i have asked this already in 2
previous threads of mine, but here it is the whole thing pasted in
pastebin. Its not so biug and with your talent you could understand it
in aprox. 5 mins.

http://pastebin.com/XgWKuXUC
 
Ad

Advertisements

D

Denis McMahon

TB behaves for me the same way. Any line > 80 chars gets a newline. Why
this is happening? Why not post up to 256 chars in a single line?

Because this is usenet. Read the RFCs if you must know!
 
S

Steven D'Aprano

Nikos,

Have you considered subscribing to this?

http://mail.python.org/mailman/listinfo/python-greece
[...]
I prefer staying here but i can also subscribe there as well if you teel
me what the groups name.

Nikos, this is exactly the sort of thing that makes it painful to try to
help you. I've given you the URL. The name of the list is in the URL, and
even if it isn't, you can just click on it and see for yourself.

Let me repeat the URL in case you cannot see it above:

http://mail.python.org/mailman/listinfo/python-greece

I will not answer any more questions about the python-greece list,
because I do not know any more about it than what you can see by
following that list.
 
F

Ferrous Cranus

Nikos,

Have you considered subscribing to this?

http://mail.python.org/mailman/listinfo/python-greece
[...]
I prefer staying here but i can also subscribe there as well if you teel
me what the groups name.

Nikos, this is exactly the sort of thing that makes it painful to try to
help you. I've given you the URL. The name of the list is in the URL, and
even if it isn't, you can just click on it and see for yourself.

Let me repeat the URL in case you cannot see it above:

http://mail.python.org/mailman/listinfo/python-greece

I will not answer any more questions about the python-greece list,
because I do not know any more about it than what you can see by
following that list.
i did Steven that why i asked in the 1st place

To post a message to all the list members, send email to
(e-mail address removed).

this is not a valid nrewgroup name/
 
Ad

Advertisements

M

Mark Lawrence

i did Steven that why i asked in the 1st place

To post a message to all the list members, send email to
(e-mail address removed).

this is not a valid nrewgroup name/

Not valid in the same way that (e-mail address removed) is not valid?

--
"Steve is going for the pink ball - and for those of you who are
watching in black and white, the pink is next to the green." Snooker
commentator 'Whispering' Ted Lowe.

Mark Lawrence
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Top