Whenever I ordered Hideaway Pizza for our family while we still had teenage boys living in the house, I learned quickly to grab what I wanted before announcing “Boys, the pizza is here”; else I would be left looking at crumbs, empty boxes, and maybe a pizza crust or two.

That’s kind of how I’m feeling now.

Yesterday morning, the highly anticipated report on the 2012 Program for International Student Assessment (PISA) was released to the public. In addition to Diane Ravitch, Valerie Strauss, and several other national bloggers, my colleague at okeducationtruths was quick out of the gate and quickly devoured two large pieces of the PISA pie: “Leaning Trendline of PISA” and “A Lunch Slice of PISA.” I do love these titles and both blogs were spot on! You can read both of these posts HERE and HERE.

So yes, bloggers can be somewhat competitive. Fortunately, unlike Hideaway pizza within reach of perennially hungry teenage boys, there is lots of PISA to go around in this case.

There are 92 pages of data in the initial PISA report left to glean. Even more fun over the next few days and weeks will be observing the “shocked” reactions and reading the “doom and gloom” public statements of the many politicians or policy makers who will seek to leverage these results to promote their respective narrative or reform idea.

This report is a gold mine for anyone with an agenda. Depending on how one chooses to parse the data, you can create a credible and convincing argument for almost any point of view and be right….or be completely wrong. Very few people will call you on it either way.

If you look at the comparative data, the performance of American students on the PISA appears to be relatively stagnant for the past 9 years (the test is administered every three years). Therefore, one could reasonably argue that our country needs more reforms or, conversely…fewer reforms. The data also shows that our students do very well against international comparisons when disaggregated by socioeconomic level, yet not too impressively when compared as a whole to other nations in the sample. There is that pesky correlation between poverty and student achievement again!

You will also notice that Shanghai China had the highest scores for mathematics in the world. Shanghai was also the top PISA “nation” in the 2009 results. Good for them. However, something you won’t read in most reports is that China actually had twelve provinces participate in the 2012 PISA tests, yet only chose to release the results from the three highest scoring provinces. Here is a rhetorical question: Do you think the American students’ performance would look different if we excluded the bottom 75% of our sample and reported only our top 25%?

Three states (scores in parenthesis)—Massachusetts (514), Connecticut (506), and Florida (467) each participated in the PISA separate from the larger American sample. The first two states outperformed both the national (481) and international averages (494) in mathematics. On the other hand, Florida scored significantly below the national and international averages.

I guess you could say the Florida miracle does not seem to be quite as miraculous as was previously reported. I wonder how Jeb Bush is going to shine this turd. And, are we not excited that nearly every major reform our state has enacted in the past few years has been adopted from Jeb and his “wonderful” Florida model?

Let’s face it though, how many people in America do you think are really going to take the time to read this full report, analyze the research methodology and sampling techniques, and dissect and disaggregate the data to try to derive any statistically valid conclusions? Let’s just say it will be a small number.

Speaking of small numbers, I found the American sample size to be very interesting. As oktruths pointed out, only 6,111 American students took the PISA test in 2012 out of nearly four million students in the overall national sample. This equates to 15-hundredths of one percent of all eligible students (0.15%). Even after I read the PISA Sampling and Data Collection Protocol on their website, this number just seems too small.

For comparative context, if we used this sampling protocol to randomly select students from Jenks Middle School to take the state reading test, we would only need to test THREE students out of 1,640. This sounds good to me, but I don’t suspect the state would let us get away with this, do you?

I also find it ironic that just this Monday, we were told by the OSDE’s new Harvard researcher, Megan Clifton, that the OU/OSU Report on the A-F grading system was faulty because the researchers used only 15,000 data points, or 3% of the available scores in the state. However, compared to the PISA consortium’s sampling ratio, the OU researchers used a sample size that is 20 times (or 2000%) larger than PISA! I wonder what, if anything, they will say anything about this.

Anyhow, I feel better that I at least got one slice of PISA down before heading to bed, even if it was a little cold.

Tomorrow, I will be back to discuss something with a few more toppings on it; specifically, how national scores on the PISA exam compare to measures of student creativity and entrepreneurial attributes. You will be surprised to see the relationship. I will also talk more about why Shanghai China doesn’t celebrate having the top scores in the world.

If it snows tonight and school is cancelled, just stay inside, be safe, and order out for a little more PISA.