Another copy from the link:
http://www.absolutefreebies.com/phpBB2/viewtopic.php?t=7959851&highlight=cache
Support this web site by purchasing from one of our Amazon affiliate stores:
Canada | France | Germany | Japan | United Kingdom | United States
Absolute Freebies Forums
Cursing is forbidden.
Posting of affiliate codes by Newbies is forbidden.
Nudity is forbidden (not by me, by hosting company, i rather like nudity).
Body modification of housepets is forbidden (except for Bonsai Kittens).
FAQ Search Memberlist Chat (0) Usergroups
Profile You have no new messages Log out [ qllonceagain ]
MrRat's APF - Weaknesses - your thoughts please.
Absolute Freebies Forums Forum Index -> MrRat's Script Support
View previous topic :: View next topic
Author Message
Dean
Patron of all things Cheesy
Joined: 26 Jul 2004
Posts: 1588
Location: Lancaster, Lancashire, United Kingdom
Posted: Tue Apr 05, 2005 8:43 pm Post subject: MrRat's APF -
Weaknesses - your thoughts please.
----------------------------------------------------------------------------
----
THE PREAMBLE
As promised a few days ago here are my thoughts about weaknesses in MrRat's
Amazon Products Feed (APF). As before - this is NOT an attack on MrR, or on
his work - merely my thoughts on what is not perfect. Some of these 'issues'
can be improved, some are limitations imposed from elsewhere - but which I
believe should still be borne in mind, for example so that feature requests
can be made to Amazon - or alternative workarounds sought.
See this earlier thread, for a run down of APF's strengths.
http://www.absolutefreebies.com/phpBB2/viewtopic.php?t=7959821
In a day or two I will publish my ideas for specific tweaks and
improvements, based upon my own ideas and building upon your thoughts and
feedback. I hope that everyone who uses the script will join in this effort
to highlight the strengths, weaknesses and perhaps future direction of
MrRat's APF script.
WEAK POINTS
One Man Show - this is MrRat's baby.
I listed this (him) as the script's greatest strength - well the corollary
of this is that the future of the script and any future development is
dependent upon one man. It is inherently in our own interests to keep MrRat
interested in this project - You will all shortly be receiving your
commission payments - I will say no more on the matter.
Documentation is perhaps a weakpoint.
In particular installation instructions. Newcommers to the script do seem to
struggle - and the same few issues do seem to come up time after time:
users NOT uploading all of the required files.
File and folder permissions.
How to debug server 500 errors.
How to use the templates to produce VALID HTML (avoiding the multiple
<html><head><body> tags)
Creating and using a second (basic) template set for integrating into
existing pages (which already have <html> and <body> tags) using Server Side
Includes or php Includes.
Some of the difficulties experienced can be put down to users wanting to run
websites but who have no HTML, CSS, FTP or basic scripting knowledge and who
either miss the docs or decide to just ask questions first and do their
reading later - or never. I know that most who experience difficulties do
NOT fit this description.
The documentation DOES jump straight from an install guide (probably
misleadingly named the beginners guide), straight to the 'Advanced guide'
with template variables, apf_ loop logic and Server Side Includes / php
includes.
BTW: to improve trouble-shooting, the script could be distributed with two
copies of the script, one named normally the other with the _test.cgi string
in the name (enabling additional debugging output). Additionally MrRat,
those of us who use .pl file extensions still have to edit the script to
allow debugging with .pl file suffix.
AMAZON
Most of the major issues with the script are in my humble opinion down to
Amazon. These may not be directly related to this script but bear with me
for a moment anyway. If enough people raise these issues with Amazon in a
persuasive enough manner we may make some headway.
Co-branded Shopping Cart
The lack of co-branding on the landing pages / shopping cart pages at Amazon
shows a certain lack of value in affiliates. I wonder whether AWS / ECS has
all been about driving up Amazon's own rankings in Google. I wonder about
the long term future of AWS / ECS.
Returning 'OUR' visitors to us AFTER the purchase process is completed.
This would be a sign that associates were valued - rather than just a means
of inflating Amazon's Page Rank, and link count. After all once the order
process is complete why would shoppers want to be pointed back to Amazon's
front page? This is even more relevent now that opening new windows is
becoming less polite / more difficult for the avarage surfer. If we can't
open Amazon in new windows then we lose our open page in the browser.
The 'Price too low to display'
(the price isn't too low to display - typically these are high value items -
it is just a cynical marketing move they don't want to tell the shopper the
price before they try and buy it.) This is disgraceful practise and it is on
OUR sites. Imagine a supermarket not putting the price on the shelf but
instead telling you that you can find out the price at the checkout:
Availability: This item is currently not available by this merchant
I am not sure where the responsibility for this one lies - I suspect
Amazon - but I don't have the in-depth understanding about AWS/ECS needed to
'prove the point'. It could be that MrRat's Why are product that are not
available for purchase returned in the listings.
Purple elephant eaters from the planet Tharg aren't available at the
moment - that is why I don't have any listed on my site. Imagine if we all
went around shouting about what is not available.
Searching and Sorting
Searching and Sorting are just plain awful - no two ways about it. I am sure
the problems here lie firmly at Amazon's door - They sort (badly) by one
variable whilst displaying another. For example - I hit upon the idea of
integrating a product search with the encyclopaedia on the Professional
Researcher web site, but after a month or so I removed the feature, the
products were unrelated wherever the article title (Which we used as teh
search string) in question was more than one word. This is currently
recognised by Amazon as an issue - but this is the second or third release
of AWS where it is listed as an issue - so it is not a high priority then.
THE APF SCRIPT ITSELF
Error Message Handling
Error messages come either from Amazon, or in the case of the Unable to
process in a timely manner - are hardcoded into the script. I would like to
see:
1. Better Error Messages - possibly from the language files, or from a
template file. Perhaps not showing the error message from Amazon, but a user
configurable message based upon the Amazon Error Code.
2. Link to Amazon - Easy way to offer the old 'Please try amazon directly'
type messages.
3. A hook - for logging / recording / handling error.
The above refers to errors returned by Amazon in the XML. For example when a
search returns no matches. I also have concerns about Internal Server Errors
and catching those by placing the main 'script control block' inside an eval
block (You know the bit I mean MrR - the seven or eight calls to
sub-routines with the START_PROCESSING_LABEL: in amongst them). A friendly
error message could then be shown to the browser, and a hook could enable
logging / e-mailing to the webmaster in the event of code that would have
produced Server 500s.
The Cache File
There have been multiple reports of issues with the cache file -
particularly from sites that are hit hard with high traffic requests - such
as a concerted burst from Google (although traffic throttling server modules
may help here).
Anyone who gets into difficulty with the script - particularly problems with
the script working through the menus but not displaying product listings, or
high CPU warnings from their host should try deleting the cachefile and the
related lock file.
I know that when the cache file becomes corrupt it can cause problems - but
I disagree with the stance that some take that this means it is useless. In
fact for high traffic sites I would have thought that the cache can be very
useful. I know my (medium traffic) sites benefit greatly from cacheing -
there is a recognisable repetition in products viewed when items become
'hot' or when a page gets a good ranking in the Search Engine Results Pages.
I think that this is an issue though and that at some point it would benefit
from some attention. I think MrRat has indicated an intent to review the
cache system (although that was some time ago - I don't know if he changed
his mind, or just got sidetracked).
I wonder what versions of Perl those who experience frequent cache problems
are running, whether they have the most up-to-date versions of Fcntl or
DM_File (if these are external modules???)
Code:
eval 'use Fcntl'; $dbm_error = $@;
eval 'use DB_File'; $dbm_error .= $@;
I don't know if these particular modules have known performance / concurrent
user limitations, but there may be better performing modules that carry out
similar functions.
Navigation
I think that Navigation is still an issue - if I find an APF site whilst
searching the web, I often have a little poke around - I often find myself
on pages with NOWHERE to go - almost literally. Especially after errors.
MrRat's 'showcase' store - with no nav menu is a prime example, and whilst I
understand that he wants to show the basic script in action I don't think it
is a good 'shop front' for his masterpiece.
Two solutions spring to mind:
1. Improve navigation in the core script / default template.
2. Alternatively, show an 'out of the box' installation and a 'look at what
you could have' installation, which includes a few select mod files to
improve the navigation.
Template Difficulties
Template design can be a little tricky sometimes - trying to position
template variables when you are not certain whether the variable is going to
produce any output. For example, placing a <hr />above and below a product
description only to find that there is no product description but two <hr/>
for no apparent reason. For those of you who struggle with this sort of
thing there is a hack for this - available in the mods section - it is a
little technical, but it works.
I would like to see this hack, or something like it built into the core
program - but then I am biased. (MrR - it is yours if you want it) I know it
adds an extra layer of complexity - but only for those who want it.
Another issue is the exclamation mark / exclamation point - might be known
by different names in other parts of the world ( bang? ). Whatever you call
it - it looks like this --> !
Because of the parsing of apf!!! commands used in templates using
exclamation marks can cause things to behave unpredictably. This includes
<!-- html --> comments or exclamation marks that you might enter into the
language files. Other than changing the parse routines - and invalidating
every template in use - I see no solution to this, other than awareness -
and a mention in any FAQ / help documentation.
The 'Header' or breadcrumb trail or path
Bestsellers > Books > Topic here. When you go to the sub topic the
breadcrumb becomes:
Bestsellers > Books > SubTopic rather than Bestsellers > Books > Topic >
SubTopic
This inability of the script to maintain a more informative trail is a
weakness - it is not MrRat's fault, but it is inefficient, ineffective and
at times confusing.
I can only think of one solution to this at present - and that is to build a
static set of browsenode xml files of our own that incorporate the extra
data required, similar to my workaround for the on/off UK browsenode lookup
issue. On balance this would probably be a lot of work to maintain, would be
error prone and only available for the UK and US locales. Then again just
because we haven't got a solution doesn't mean it is not an issue - at lease
if we recognise it as such we can look for solutions.
Too many query string variables
I know that these variables are how the script knows what to do - but there
a lot of them at times. I think some search engines will still balk at 5 or
more key=value pairs after the question mark. I have identified the
following situations where these could be dropped.
Following the use of: link_templates=default
all urls show this (forever more) templates=default - no need to propogate
these - just ugly and don't achieve anything. Similarly the number of
results to retrieve max_results / link_max_results
Basenodes - The starting point within each SearchIndex
Choices have been made as to the 'basenode' for each SearchIndex (product
category such as books DVD software). This is necessary because there is no
true ZERO NODE or start node within SearchIndexes. By which I mean there is
no one books node from which ALL book nodes spring. So potentially books are
available listed by Topic, by publisher, bargain books, used books. By
choosing 'by topic' above the others, many books and book categories are not
easily available - they are available - but perhaps not as readily, you
would have to build some other link to these alternate schema.
I quite like the idea of constructing, perhaps as an option - some form of
zero node - perhaps distributing a zero node xml file for each SearchIndex.
For example, when books is selected in the browse menus - the script could
load our home grown xml file which when processed as normal would offer the
user the options organised by topic, organised by publisher, etc. But there
I go proffering solutions rather than detailing issues.
Title and Meta Tags / Search Engine Optimisation
NO script should be designed from the ground up just to appeal to Search
Engines, and I believe MrRat has expressed his intention NOT to incorporate
SEO 'techniques' into APF (could be wrong on that one - I have a vague
memory of this but can't find it with a quick search) I think that there are
some rough edges. The subject tag after a search is 'sub optimal' and
because of the way that Store and Subject tags are defined (and possibly
redefined) throughout the script these are difficult issues to fix.
Look ahead processing - Processor / memory usage, and bandwidth usage.
This one is purely personal opinion (what do you mean most of it has been?)
and is built on deduction rather than 'absolute' knowledge of the script's
workings. I wonder whether the script isn't a little inefficient in its
logic. Data is requested and processed and variables are calculated /
concatenated even though they may not be needed or used in the final 'page
build'. As an example, I know of one fool (alright it was me) who when
switching from the script's own shopping cart ( %%shopping_cart_button%% )
to using Amazon's cart (%%buy_button%%) left the variations_html variable
lying around. I suspect the script requested variation data, created the
variations_html template variable (as well as the buy_button variable), and
for what. Without the %%shopping_cart_button%% present in the template what
was the point? In fact look at the harm - variations requested, variables
built, etc.
I wonder how many situations would benefit from a little look ahead logic -
to predetermine what is needed and only request data / build variables that
are needed.
Showing standard text / html pages within APF templates.
(I know this one has cropped up a few times recently MrR, but for others who
didn't catch it)
I think many people would benefit from the ability to show standard text /
html 'pages' within the apf script - embedded with the template structure.
Trying to link to a static privacy policy external to the script is easy,
but assume the shopper is not in your default locale. They click to your
privacy policy, when they come to click to re-enter your store, they will
end up in the default locale. What are you going to do - educate them to use
their brower back button? Make them select from a list of countries to click
back to where they were? Both of these could work, but both are sub-optimal.
Postal / shipping rates, privacy policy, index pages, special offers, even
the apf_config.cgi setup script could ALL be incorporated directly into the
main script.
These are the issues that I feel are worthy of attention - what do you
think? Am I being harsh, are there other issues you think are a higher
priority. If you think I am plain wrong then please step up and say so.
Once again, just so everyone is clear of my intentions I still recommend
APF, and I commend MrRat for all of his hard work.
So tell me - and MrRat, what do you think are the script's weaknesses.
Dean Marshall
PS - as I said earlier, please hold fire for a few days and I will summarise
everyone else's thoughts and suggestions and together with my own ideas we
will see if we can't add a few items to MrRat's TODO list.
PPS - I have a minor 'analysis' of APF's mod file system - I will probably
post that next, before I wrap up the process with the aforementioned
summary.
_________________
DeanMarshall.co.uk - Professional Researcher - UK Computer Store
Back to top
Nintendo
Patron of all things Cheesy
Joined: 30 Nov 2002
Posts: 1583
Location: Winston, OR, USA
Posted: Wed Apr 06, 2005 9:37 am Post subject:
----------------------------------------------------------------------------
----
Ouch!!!!!!
1. You can't block main categories using the config file. ie, keep the full
video game section out, unless I think if you edit one of the files.
2. Special characters in URLs!!!!! ie those music on CD links. (Not sure how
they are in the newest version.)
3. Cache files. I think by default it's on. Blarg!!!! (I know, I know, I'm
the one who always says it's worthless!!)
4. Too EASY to set up!!!! Upload files, change permission on two files, go
to config file, and bang, you are done!!!! There for you get a ton of
competition!!!
5. It's free!!!! Start charging every one!!!!!!!!
6. Amazon.com has a crummy AWS system!!!!! Rumor has it that there allready
working on AWS 4, even with AWS 3 covered with bugs!!!
_________________
Amazon/Webmaster Services (AWS)
Public Domain Content
News Rant
Back to top
Dean
Patron of all things Cheesy
Joined: 26 Jul 2004
Posts: 1588
Location: Lancaster, Lancashire, United Kingdom
Posted: Thu Apr 07, 2005 7:51 pm Post subject:
----------------------------------------------------------------------------
----
Nintendo wrote:
Ouch!!!!!!
Do you think I was harsh - I hope I haven't offended MrRat. I expect if
someone delivered such an analysis of something I had put so much time and
effort into I would probably not go back to the project. Usually when he
goes this quiet he comes back with some amazing new features. I hope the
latter is the case rather than the former.
Nintendo wrote:
1. You can't block main categories using the config file. ie, keep the full
video game section out, unless I think if you edit one of the files.
Not quite with you here. SearchIndexes can be excluded individually in
config.cgi on a per country basis.
I expect that you mean something else and that I am being dim.
Nintendo wrote:
2. Special characters in URLs!!!!! ie those music on CD links. (Not sure how
they are in the newest version.)
Not sure what the current status is on this one - I am no longer on the
B?le[ea]ding edge - MrRat's work rate is too high for me.
Nintendo wrote:
3. Cache files. I think by default it's on. Blarg!!!! (I know, I know, I'm
the one who always says it's worthless!!)
I know you are the one who calls it worthless - I wrote a rant telling you
not to be so rude the last time (but held off from posting while I counted
to 10) in the meantime MrR posted with an emoticon sticking his tongue out
or similar. So I figured he had had his say. I know you find the cache file
error prone in your 'unique' situation. By the way - have you tried any of
the bandwidth throttling server modules to calm Google down when it hits you
hard? I don't think G penalises based on server load, and the lower
performance demands might make the cache more stable.
Nintendo wrote:
4. Too EASY to set up!!!! Upload files, change permission on two files, go
to config file, and bang, you are done!!!! There for you get a ton of
competition!!!
5. It's free!!!! Start charging every one!!!!!!!!
I have some sympathy here. I think it should be distributed in a crippled
format that only those with a modicum of programming knowledge can resolve.
Still there are some who still struggle with setup - usually total scripting
newbies or people with unusual server configuarations.
Nintendo wrote:
6. Amazon.com has a crummy AWS system!!!!! Rumor has it that there allready
working on AWS 4, even with AWS 3 covered with bugs!!!
I am not sure if there is some subtle humour at work here (too subtle for me
if so) or whether your numbering is deliberately off by one for some other
reason. As I understand it APF4 is based on AWS/ECS4. Then again I am
probably missing something.
Your point about Amazon is valid though - I posted elsewhere today that
having read "expendable's" posts on the Amazon boards (although he does seem
argumentative in the extreme at times) I am warming to his overall
conclusions. I think the developement of AWS/ECS is both a means of gaining
favourable press coverage and a potential future revenue stream that we are
just the guinea pigs for. In addition our sites gain Amazon backlinks in
Google which doesn't do Amazon any harm.
Most computer developments approach some sort of respectable stability by
version 3 and maturity by version 4. I am not convinced that Amazon's ECS is
close to either of these definitions.
Having said all that, fair play to MrRat for all of his efforts. I have a
fair understanding of what is involved in providing this project (although I
admit I don't have the discipline or commitment - or frankly the ability -
to do anything similar) I respect and thank him for all of his efforts, even
moreso because of my own failings.
Dean.
_________________
DeanMarshall.co.uk - Professional Researcher - UK Computer Store
Back to top
Nintendo
Patron of all things Cheesy
Joined: 30 Nov 2002
Posts: 1583
Location: Winston, OR, USA
Posted: Fri Apr 08, 2005 1:20 am Post subject:
----------------------------------------------------------------------------
----
:::: Do you think I was harsh - I hope I haven't offended MrRat.
Ouch as in your post here was much bigger than the other one!
:::::::::Not quite with you here. SearchIndexes can be excluded individually
in config.cgi on a per country basis. I expect that you mean something else
and that I am being dim.
Now I think I can see.....
Limit US SearchIndexes to browse (comma separated and case sensitive)
(The 'COUNTRY SearchIndexes' links there are dead links now, so I'm not
sure.)
Ie, don't want a books section, or don't want the DVD section, or video game
section. I'm guessing that is where to take it out.
:::I know you are the one who calls it worthless - I wrote a rant telling
you not to be so rude the last time (but held off from posting while I
counted to 10)
Ouch!!! You should of posted it!!!! I probably deserverd it since I flame
the cache all the time!
:::I know you find the cache file error prone in your 'unique' situation.
At some where around 20 stores, at the rate of each cache crashing an
average of about every six months, that would be a crash every 9 days, and
the server wasn't at all happy when that happened! I had to go to technical
support to figure it out when it happened the first time! So yah, I
understand why the web hosting companies ban the script when this
occures!!!! If I was a web hosting company I would of done the same thing,
unless I knew how simple it was to fix it!!
::::By the way - have you tried any of the bandwidth throttling server
modules to calm Google down when it hits you hard?
Nope. I've never had to worry about bandwidth, I got a terabyte of that a
month and don't even use close to that, not even 1/3rd of that a month!
:::I am not sure if there is some subtle humour at work here (too subtle for
me if so) or whether your numbering is deliberately off by one for some
other reason
er my brain is off by one year!!! Rumor has it that there allready working
on AWS 5, even with AWS 4 being covered with bugs!!!
:::having read "expendable's" posts on the Amazon boards (although he does
seem argumentative in the extreme at times)
He HATES me, BIG time!!!!!
::In addition our sites gain Amazon backlinks in Google which doesn't do
Amazon any harm.
Unless you use MrRats script, which uses a redirect link.
_________________
Amazon/Webmaster Services (AWS)
Public Domain Content
News Rant
Back to top
MrRat
The Big Cheese
Joined: 28 Nov 2000
Posts: 3764
Location: Alabama
Posted: Mon Apr 11, 2005 10:33 am Post subject:
----------------------------------------------------------------------------
----
Nintendo wrote:
:::: Do you think I was harsh - I hope I haven't offended MrRat.
Ouch as in your post here was much bigger than the other one!
rotflmao
Quote:
:::I know you are the one who calls it worthless - I wrote a rant telling
you not to be so rude the last time (but held off from posting while I
counted to 10)
Ouch!!! You should of posted it!!!! I probably deserverd it since I flame
the cache all the time!
i think the idea of leaving the cache off by default is worth considering.
i don't think Amazon would like that thought. they really want my script to
cache.
Back to top
Nintendo
Patron of all things Cheesy
Joined: 30 Nov 2002
Posts: 1583
Location: Winston, OR, USA
Posted: Mon Apr 11, 2005 7:24 pm Post subject:
----------------------------------------------------------------------------
----
Or just place a <H1> warning next to the cache option.
<H1>Turn this off if you don't want your site crashing every three or four
months generating a mad E-Mail from your web host!!! Don't use with
mod_rewrite. Don't use if the Googlebot likes your store. Dont use if your
store doesn't get over 10,000 pageviews a day. Don't use if....... </H1>
_________________
Amazon/Webmaster Services (AWS)
Public Domain Content
News Rant
Back to top
Display posts from previous: All Posts1 Day7 Days2 Weeks1 Month3 Months6
Months1 Year Oldest FirstNewest First
Absolute Freebies Forums Forum Index -> MrRat's Script Support All
times are GMT - 6 Hours
Page 1 of 1
Watch this topic for replies
Jump to: Select a forum AbsoluteFreebies.com----------------General
DiscussionFreebies MrRat.com----------------MrRat's Script SupportModsBeta
TestingGeneralGraphics
You can post new topics in this forum
You can reply to topics in this forum
You can edit your posts in this forum
You can delete your posts in this forum
You can vote in polls in this forum
Powered by phpBB ? 2001, 2005 phpBB Group
----------------------------------------------------------------------------
