Help with research

[elena]

I'm a Java/C++ developer who is also studying psychology.

I would really appreciate it if you would complete a survey that I'm
using for a research project on programmers.

It's easy [Yes/No answers] and takes about 5 minutes.

I will be presenting the results at the American Psychological
Association convention in August.

The study link is:

http://www.elena.com

The survey measures "cognitive style" (analytical/intuitive) which
describes how you process information and learn. The people I've
pre-tested it with found it to be pretty interesting.

I can go to my friends, however it occurred to me that it might be
better to post in a newsgroup and get a larger, more diverse, and
random sample.

Thanks again for your time,

Elena

 

[infobahn]

I'm a Java/C++ developer who is also studying psychology.

Try comp.lang.java.programmer or comp.lang.c++ (if you're not merely
a spambot, which is the explanation that most immediately comes to
mind).

 

[Steven]

Even if she is a Java or C++ programmer, shouldn't the post be in a
phychology newsgroup?

Interesting, she has her name (an awefully common one too) as an URL...

 

[Steven]

Even if she is a Java or C++ programmer, shouldn't the post be in a
phychology newsgroup?

Interesting, she has her name (an awefully common one too) as an URL...

 

[Walter Roberson]

:I'm a Java/C++ developer who is also studying psychology.

:I would really appreciate it if you would complete a survey that I'm
:using for a research project on programmers.

That would be "self-selection", and such a proposal would not pass
the ethics department at any reasonable university that I have
heard of. Or don't research studies require approval anymore?

 

[Default User]

:I'm a Java/C++ developer who is also studying psychology.

:I would really appreciate it if you would complete a survey that I'm
:using for a research project on programmers.

That would be "self-selection", and such a proposal would not pass
the ethics department at any reasonable university that I have
heard of. Or don't research studies require approval anymore?

Walter is absolutely right. Without a properly selected and controlled
survey group, the results are meaningless.

According to the web site, she's with Northcentral University, which
seems to be one of those "distance learning" universities.

http://www.universities.com/Distance_Learning/Northcentral_University.html



Brian

 

[Mark McIntyre]

Walter is absolutely right. Without a properly selected and controlled
survey group, the results are meaningless.

Not necessarily - a randomish sample also has meaning, provided you
understand and document the nature of its randomness. Whats at least as
important is the questions asked and how they're phrased.

 

[Mark McIntyre]

Walter is absolutely right. Without a properly selected and controlled
survey group, the results are meaningless.

Not necessarily - a randomish sample also has meaning, provided you
understand and document the nature of its randomness. Whats at least as
important is the questions asked and how they're phrased.

 

[osmium]

Not necessarily - a randomish sample also has meaning, provided you
understand and document the nature of its randomness. Whats at least as
important is the questions asked and how they're phrased.

How about providing an example of a self selected sample where the results
are useful? As opposed to, interesting, say. Of course the questions are
important. Why bring that up??

I think this kind of sampling is done all the time and the results are
certainly readable, but I don't know what anyone can actually *do* with such
results. Perhaps one could refine the questions for a real survey to be
made later. Other than that, AFAIAC, self selected samples are interesting
but useless.

There was a show on PBS earlier this week on Kinsey, he used nonsensical
sampling techniques. And I suppose he made a lot of money out of it. But I
can't think of a single reliable inference one could make based on his
books.

 

[Mark McIntyre]

Walter is absolutely right. Without a properly selected and controlled
survey group, the results are meaningless.

Not necessarily - a randomish sample also has meaning, provided you
understand and document the nature of its randomness. Whats at least as
important is the questions asked and how they're phrased.

 

[Dave Vandervies]

:

How about providing an example of a self selected sample where the results
are useful?

Alumni income surveys?

Unless you mean statistically useful, and not just useful for what you're
trying to accomplish.


dave

 

[elena]

To answer as many of these questions as possible:

Yes, I am a software engineer with more than 25 years experience. Not a
spambot. I used my name as a sign-in because my name was very uncommon
in the US up to a few years ago (just a habit). Northcentral is an
accredited university that allows me to earn a PhD (I have Masters in
CS and Psychology) while still making a living (alghough given
off-shoring, this may change). Sampling of this sort is valid when the
population has certain characteristics. Self-selection is a limitation,
but one that I am willing to accept given that I'm only looking at 2
factors. This study concerns only Software Developers. This group
should consist mainly of software developers. For an interesting paper
on web research and the social sciences, see:

http://www.psychologie.unizh.ch/sowi/reips/papers/exppsy/ExPsyReipsReprint.pdf

 

[Mark McIntyre]

Walter is absolutely right. Without a properly selected and controlled
survey group, the results are meaningless.

Not necessarily - a randomish sample also has meaning, provided you
understand and document the nature of its randomness. Whats at least as
important is the questions asked and how they're phrased.

 

[Alan Balmer]

How about providing an example of a self selected sample where the results
are useful?

There may not be any ;-) However, I think *all* survey samples are
self-selected, in that people usually make the choice as to whether to
participate or not. Forced participation, I suspect, would result in
even more skewed results. The professional survey people claim that
they compensate for this effect, though I'm not convinced.

As opposed to, interesting, say. Of course the questions are
important. Why bring that up??

Probably because the questions are the weakest part of any survey, and
can easily swamp any bias caused by poor sampling.

I think this kind of sampling is done all the time and the results are
certainly readable, but I don't know what anyone can actually *do* with such
results. Perhaps one could refine the questions for a real survey to be
made later.

Define "real survey." Not being sarcastic, I just don't know what
constitutes a useful survey, or how you prove it.

Other than that, AFAIAC, self selected samples are interesting
but useless.

There was a show on PBS earlier this week on Kinsey, he used nonsensical
sampling techniques. And I suppose he made a lot of money out of it. But I
can't think of a single reliable inference one could make based on his
books.

You can infer that he made a bunch of money ;-)

 

[Alan Balmer]

To answer as many of these questions as possible:

Yes, I am a software engineer with more than 25 years experience. Not a
spambot. I used my name as a sign-in because my name was very uncommon
in the US up to a few years ago (just a habit). Northcentral is an
accredited university that allows me to earn a PhD (I have Masters in
CS and Psychology) while still making a living (alghough given
off-shoring, this may change). Sampling of this sort is valid when the
population has certain characteristics. Self-selection is a limitation,
but one that I am willing to accept given that I'm only looking at 2
factors. This study concerns only Software Developers. This group
should consist mainly of software developers. For an interesting paper
on web research and the social sciences, see:

http://www.psychologie.unizh.ch/sowi/reips/papers/exppsy/ExPsyReipsReprint.pdf

A comment on the questions - The choices given are true, false, and
uncertain. Uncertain is not a good choice, and I answered as if it
meant "sometimes."

Incidentally, you contribute to the self-selection problem by
selecting those who are (1) willing to give you an email address or
(2) willing to give you a false email address.

 

[elena]

I received similar feedback from others on the "uncertain" option.
Actually, the survey is a well-known instrument (not my own) introduced
in the early nineties. I'll be giving feedback to its developers.

On self-selection, this problem varies depending on the theory being
studied. A cognitive attribute, such as learning style, indicates a
smaller self-selection affect. Also, self-selection will make results
less generalizable. But that is not a concern here because I'm
comparing software developers from 2 cultures.

BTW, I did ask permission on the comp.lang.java.programmers group and
got one response that basically said it was fine to ask for
participants, as long as discussion did not occur there. I had expected
more feedback on the level of intrusiveness, but didn't get any. So I
figured I would just give it go and see what happened. I apologize if
this was too far off-topic.

Elena

 

[osmium]

Define "real survey." Not being sarcastic, I just don't know what
constitutes a useful survey, or how you prove it.

I think a real survey is possible, but one can't prove it as a mathematical
proof. But you can prove to the standards of a civil trial in the USA.

Consider this hypothetical:

Q:"Have you or members of your immediate family ever rented a video tape?"

A:
yes 85%
no 11%
no response 4%.

If this is done right you can conclude that somewhere around 80% or so of
the non-institutionalized adults under 80 years old have indeed rented a
video tape.

It is hard to imagine a reason to lie to such a survey, although it will
still happen. Questions on politics, sexuality, honesty and such-like would
be much harder to judge.

Note also that the respondent can almost surely understand the question and
come up with a non-fudged answer if he wishes. It's not some fuzzy thing
like "Are you a member of the middle class?"

 

[Chris Croughton]

A comment on the questions - The choices given are true, false, and
uncertain. Uncertain is not a good choice, and I answered as if it
meant "sometimes."

I assumed that as well.

Incidentally, you contribute to the self-selection problem by
selecting those who are (1) willing to give you an email address or
(2) willing to give you a false email address.

I just left it blank, it didn't seem to complain...

Chris C

 

[Walter Roberson]

:Q:"Have you or members of your immediate family ever rented a video tape?"
:A:
:yes 85%
:no 11%
:no response 4%.

:If this is done right you can conclude that somewhere around 80% or so of
:the non-institutionalized adults under 80 years old have indeed rented a
:video tape.

But it isn't done right -- instead, the question gets asked at
video rental stores and in movie-related newsgroups. With self-
selection, you seldom reach the non-affluent or minorities.
People who spend a lot of time in comp.lang.c are those with
leisure time, which implies at least moderately well off; their
opinions and cognative styles could be quite different from the
person who is a business programmer by day and a movie usher by night
to make ends meet.

 

[Mark McIntyre]

How about providing an example of a self selected sample where the results
are useful?

Heck, even the percentage that self-select is itself useful info.

As opposed to, interesting, say. Of course the questions are
important. Why bring that up??

Because as any fule no, the 'right' question can totally change the
results.