Sat in a philosophy class today.

[Daldolma]

Originally Posted by Alawen [You must be logged in to view images. Log in or Register.]

I'm going to start with the very basics of psychometrics, so my apologies if this is review. I mentioned the elements of standardization, validity, and reliability in a previous post.

In brief, standardization is a given test being delivered in the same way given different physical locations, test administrators, and so on. If you've taken the SAT or the ACT, you've experienced their attempts to create a very controlled environment for taking the test.

Validity is the quality of testing what you are trying to test. This is what HBB is questioning--whether IQ is a valid test of intelligence. Validity is viewed from three perspectives: content, criterion, and construct. This is why the underlying theory of intelligence is important. I think Loraen is mostly objecting to the Cattell model of generalized intelligence; he would probably be more comfortable with Gardner's theory of multiple intelligences.

Reliability is the consistency between assessments and this is what you're questioning. It also has three subcategories, which are stability (also called test-retest), alternate form, and internal consistency. Obviously, stability is the characteristic you're calling into question.

You use a coefficient of variation to measure test-retest reliability (the quotient of standard of deviation and mean). Coefficients for both Standford-Binet and the Wechsler tests are around .90, and coefficients between the two tests, which have very different histories, are approximately .85. These are not results from a single study--this is a heavily researched area.

Most people have never taken a real IQ test. They may have received an estimated IQ based on a standardized achievement test, or taken some hokey short form internet test, but these approaches fail at all of the above. All of the standardized IQ tests require a skilled tester and quite a bit of time. As a result, they're expensive. As an aside, the popular Jung-based Myers-Briggs assessment fails horribly at reliability. Of the four elements, the only consistent measurement comes from the introvert/extrovert axis.

Returning to your question for explanations of the variance that does exist, it probably makes the most sense to consider Cattell's model. Up to this point, I've done very little speculation. This part is mostly my own thoughts. It seems unlikely that fluid reasoning (Gf) is subject to much variation. It doesn't seem to me that people learn things like curiosity and creativity. They seem to have them from a young age or not. However, it is the expressed purpose of a liberal arts education to teach logic, rhetoric, and critical analysis. Perhaps there is evidence of fluctuation in Gf between high school seniors and those same students graduating from a four-year liberal arts curriculum.

Within the limits of the model, then, variation in intelligence would most likely come from crystallized intelligence, Gc. It makes perfect sense that skills, knowledge, and experience could expand or contract with continued learning or disuse. Some of that will almost completely atrophy given a decade or two of neglect. On the other hand, some people are always learning. I like to think of myself in that group, though my skills in higher math are abominable now. I have to look up almost everything beyond simple trigonometry.

In brief, variance in test-retest reliability for standardized IQ tests is pretty acceptable, depending on what you're looking for. I think there's a lot of confusion from the simple fact that most people have never taken a real IQ test.

okay, so it seems like the main issue is that i'm just not that familiar with real IQ tests. my limited experience has been with estimates based on other, more popular standardized tests. and with those tests, your score can fluctuate drastically over the course of a few years. to me, that would signal such an egregious lack of stability that it casts doubt on the value of the test as iq predictive

i imagine you would view those tests as unacceptable for iq translation, but i know that mensa (just as an example) will accept scores on tests similar to the SAT as grounds for entry

[Alawen]

Originally Posted by Daldolma [You must be logged in to view images. Log in or Register.]

okay, so it seems like the main issue is that i'm just not that familiar with real IQ tests. my limited experience has been with estimates based on other, more popular standardized tests. and with those tests, your score can fluctuate drastically over the course of a few years. to me, that would signal such an egregious lack of stability that it casts doubt on the value of the test as iq predictive

i imagine you would view those tests as unacceptable for iq translation, but i know that mensa (just as an example) will accept scores on tests similar to the SAT as grounds for entry

This is exactly the confusion I was referencing. As it happens, Mensa is the lowest of the high-IQ societies and their standards are correspondingly lax. Mensa's intention is to include people over the 98th percentile, which is 1 of every 50 individuals. That's not very discriminating.

In contrast, the Triple Nine Society no longer accepts scores from tests like SAT, ACT, or GRE. They will, however, accept a GMAT score over 750. Their standard is the 99.9th percentile--1 in 1000.

The Prometheus Society seems to be the most exclusive of the stable high IQ societies with a threshold at the 99.997th percentile--one in 30,000 or a full four standard deviations above norm. The only score they currently accept is an MAT (Miller Analogies Test) of 500 or above.

Approximate scores are probably good enough for a low standard like Mensa, but any estimate made from an achievement test introduces a lot of variation and becomes more about acquired knowledge than generalized intelligence. I personally find the MAT and GMAT somewhat troubling in this respect. I consider Stanford-Binet and WAIS (WAIS is the adult test from Wechsler, WISC is for children) the current standards in IQ testing.

[Alawen]

I should mention that four standards of deviation is the limit of the Stanford-Binet test. It is very difficult to measure IQs over 160.

[Hasbinbad]

Hey alawen, what happens if you eat a large meal of fish 12 hours before you take an IQ test?

[Samoht]

Originally Posted by Alawen [You must be logged in to view images. Log in or Register.]

The only score they currently accept is an MAT (Miller Analogies Test) of 500 or above.

holy, crap.

just looked up some MAT sample questions.

i only got six of them right.

[Samoht]

i don't know if i could sit through 2 hours of those, but these are fucking easy in comparison

[Barkingturtle]

All easy u dum.

[Alawen]

Originally Posted by Samoht [You must be logged in to view images. Log in or Register.]

holy, crap.

just looked up some MAT sample questions.

i only got six of them right.

They're tough, but they don't do a very good job of measuring fluid reasoning at all. The MAT is almost all accumulated knowledge. It's almost a trivia test. Liberal arts majors should crush STEM majors, and that's just silly.

[Samoht]

MAT is clearly a combination of reasoning and working knowledge of the world. would do.

[Splorf22]

What I am trying to say is that I feel the human brain is extremely specialized. Not just something like multiple intelligences but literally case by case, problem by problem.

Making this claim requires some explanation for the correlations we see (some people seem to be able to solve more problems on average than others). I am saying that some of these correlations are probably genetic (person x has more neurons or faster or more connections or whatever) but a lot of it probably has to do with motivation.

Also all of those theories of intelligence are basically BS. For a great (if longwinded and somewhat difficult to read) explanation, see http://vserver1.cscs.lsa.umich.edu/~...eblog/523.html. The basic idea is that just because you run PCA on your data and get some dominant eigenvalues doesn't mean that your new basis vectors actually have physical meaning. Sometimes they do, sometimes they dont.