PPP vs. Rasmussen

I disagree with your presupposition

"The most accurate American pollsters are the left-leaning Public Policy Polling (PPP) and right-leaning Rasmussen Reports (RR). Their poll number results are typically not biased"

Rasmussen hasn't been on the most accurate pollster list since 2004, and their polling has systematically been off in a rightward direction in recent years. I truly wish they get their formulas back in order, as they were invaluable during the 2002 and 2004 cycles.

I am a Tom Rukavina Democrat.

---

by: OGGoldy @ Thu May 03, 2012 at 22:47:47 PM UTC

Rasmussen not the most accurate.

1. Fordham professor had Rasmussen has the most accurate in 2008 presidential election.

2. The non-partisan TownPagesNet.com named RR the most accurate in 2008.

3. RR not perfect, but many Democratic analysts have credited RR as one of the best.

4. Most of their 2008 U.S. Senate election polls and 2008 presidential polls were accurate.

Male, PA-15, Libertarian leaning-Republican

---

by: AppleCon @ Thu May 03, 2012 at 23:04:06 PM UTC

[ Parent ]

RR is not #1 most accurate...**but one of the most accurate**

Male, PA-15, Libertarian leaning-Republican

---

by: AppleCon @ Thu May 03, 2012 at 23:05:14 PM UTC

[ Parent ]

See below.

(-9.38, -7.49), Blood type "O", social anarchist, KY 01, "When smashing monuments, save the pedestals. They always come in handy."


-- Stanisław Lem

---

by: Setsuna @ Thu May 03, 2012 at 23:15:28 PM UTC

[ Parent ]

if I recall

The best pollster is Quinnipiac.

26, R, PA-01.

---

by: krazen1211 @ Thu May 03, 2012 at 22:52:00 PM UTC

Pollsters ranked by accuracy

based on polls from the final 21 days prior to the 2010 election:

1). Quinnipiac:
Polls: 21
Average error: 3.3
Bias: R+0.7

2). SurveyUSA
Polls: 30
Average error: 3.5
Bias: R+0.8

3). YouGov
Polls: 35
Average error: 3.5
Bias: R+1.1

4). Public Policy Polling (PPP):
Polls: 45
Average error: 3.8
Bias: R+0.3

5). Mason-Dixon
Polls: 20
Average error: 4.6
Bias: D+0.4

6). Marist:
Polls: 14
Average error: 4.9
Bias: R+4.0(!!!)

7). CNN/Opinion Research:
Polls: 17
Average error: 4.9
Bias: R+2.1

8). (last place) Rasmussen Reports:
Polls: 105
Average error: 5.8(!)
Bias: R+3.9

.

Above data taken from FiveThirtyEight

You state:

In 2008 and 2010, their poll numbers were similar, and ended up being among the most accurate.

Rasmussen had the worst track record in 2010, being off be an average of almost 6 points and having a Republican bias of a little under 4 points. 2010 was not a good year for most pollsters, but Rasmussen was worst, almost every pollster has a slight Republican bias, but few so egregious as Rasmussen. PPP showed no such bias toward Democrats, indeed they had a 3/10 of a percentage point bias toward Republican candidates. Their survey's were also closer to the results by 2 full percentage points, performing just 1/5 of a percentage point worse than the best performing pollster.

Neither made the top-two of best performing pollsters in 2010.

(-9.38, -7.49), Blood type "O", social anarchist, KY 01, "When smashing monuments, save the pedestals. They always come in handy."


-- Stanisław Lem

---

by: Setsuna @ Thu May 03, 2012 at 23:14:57 PM UTC

This study is Statistical Abuse

and can only be called a joke.

1st RAS is credited for multiple polls in the same race in that three week time period.  So exactly how can you justify that in any sort stat analysis? It could have been 2-7-9 points off in a race but pollster A is only in there once.  Martist could have polled that race 3 points off at the same time RAS was two points but it comes off as statistical more accurate.  Now how that work out stat wise?  Oh I see if you pile a lot of numbers up people get fooled.

2nd RAS is rated in races that no one else did.  So if they  missed HI senate by 20 points and no one else on this list polled it how do rate them a better pollster then RAS? I guess only in a 538 survey.

We get think joke survey posted here in some form of another every couple of months.  Posting it over and over again does not make it any more accurate.

Lets just say that even the CBO would score the analysis in the survey a "zero".

---

by: rdelbov @ Thu May 03, 2012 at 23:46:14 PM UTC

[ Parent ]

My analysis below

I excluded any race where there was only one pollster and only included the most recent poll. Thus, my number of polls each pollster did was lower than 538.

R, CA-37; hometown: PA-2

---

by: Left Coast Libertarian @ Fri May 04, 2012 at 00:00:15 AM UTC

[ Parent ]

But you also objected

when I did a comparison of PPP and Rasmussen's final polls in each state where they both polled. What evidence would you accept?

(-9.38, -7.49), Blood type "O", social anarchist, KY 01, "When smashing monuments, save the pedestals. They always come in handy."


-- Stanisław Lem

---

by: Setsuna @ Fri May 04, 2012 at 00:06:29 AM UTC

[ Parent ]

somewhat of an artificial metric

Really just means that Ras should do fewer polls.  

26, R, PA-01.

---

by: krazen1211 @ Thu May 03, 2012 at 23:47:20 PM UTC

[ Parent ]

My analysis

I'm going to show my analysis, which, while Rasmussen didn't do that well, doesn't show them as the worst. I included how often they were within 4 points and how often they predicted the winner. Average error can have a few outliers have a big impact.

Quinnipiac
Within 4 points: 70%
Predict winner:  80%
Bias: R+0.9

PPP
Within 4 points: 61%
Predict winner:  81%
Bias: R+1.3

Suffolk
Within 4 points: 60%
Predict winner:  90%
Bias: D+3.0

Reuters/Ipsos
Within 4 points: 57%
Predict winner:  86%
Bias: R+3.4

CNN/Time
Within 4 points: 55%
Predict winner:  80%
Bias: R+0.2

Fox News
Within 4 points: 50%
Predict winner:  71%
Bias: R+3.0

Rasmussen
Within 4 points: 47%
Predict winner:  87%
Bias: R+3.3

Survey USA
Within 4 points: 47%
Predict winner:  88%
Bias: R+0.7

McClatchey/Marist
Within 4 points: 44%
Predict winner:  89%
Bias: R+4.8

Mason-Dixon
Within 4 points: 36%
Predict winner:  64%
Bias: D+0.3

My analysis, like the 538 analysis, only accounts for polls within the three weeks before the election. That means that any polls before that aren't included. PPP moved several points to the right once they switched to a likely voter model. Until they did that, Rasmussen was a far better pollster.

R, CA-37; hometown: PA-2

---

by: Left Coast Libertarian @ Thu May 03, 2012 at 23:58:27 PM UTC

[ Parent ]

I tend to think you need to look at other pollsters

to determine who's right and who's wrong. PPP is probably right in Virginia and Rasmussen is closer to right in Florida.

---

by: GradyDem @ Fri May 04, 2012 at 00:13:05 AM UTC

blerg!

Nobody's 'right' now.  It's all a different mix of voters.  Between now an election day:
1.  The races can and will move because of the campaigns.
2.  The races can and will move because of outside influences.
3.  The races can move because of the enthusiasm of various partisans.  
4.  Momentum exists.  No one says the polls were 'wrong' when they had Coakley up 20 in November 2009.  The race moved, and it moved fast. "If the election were held today" is meaningless this far out.

You can't be 'right' in May.  You just can't.  If, in November, your poll from May was correct and everyone's else was wrong, sweet!  But it really doesn't mean you did something great.  You got lucky.  Your 'snapshot' of the race 6 months early hit it.  Now, your final poll the weekend before, that should be how you are judged.    

---

by: pstchrisp @ Fri May 04, 2012 at 01:51:06 AM UTC

[ Parent ]

one-day samples

One of the biggest reasons for Rasmussen's supposed inaccuracy is that they do quick one-day samples without any callbacks, thus usually getting only the most informed voters.  That is probably why their name ID numbers are usually so much higher than other pollsters.

These one-day samples can lead to wild swings in the results.  For example, RR's Wisconsin Senate poll in February had Tommy Thompson up 50-36 on Tammy Baldwin, with Baldwin even losing 41-40 to Jeff Fitzgerald.  However, the March poll had it return to a more believable 48-44 lead for Tommy Thompson, with Baldwin back up 48-40 on Fitzgerald.  Did the results really move that much? No.  Did Rasmussen "fix" one or both of the polls? No.  They likely just got a big fluctuation due to a warped one-day sample, which also shows up occasionally in the daily tracking poll.

This is probably why Quinnipiac usually, but not always, shows the most stable results, as they have a huge sample size with a margin of error almost always under 3%.

The one thing that annoys me about Rasmussen is the weird variations between their Presidential and Senate polls in the same state with the exact same sample.  For example, if Obama only wins by 11% in MA, Scott Brown would be easily re-elected, not losing by 1%.  Likewise, in Florida, their March poll had Obama leading Romney 46-43, while Connie Mack led Bill Nelson 43-36.  In reality, Romney probably has a narrow advantage in FL, while Bill Nelson is favored to win the Senate race.  Then the April poll went back to reality with Romney taking a 46-45 lead, and Nelson winning 47-36.  In the unlikely event Obama wins Florida again, Bill Nelson would prevail by about 10%, not losing by 7.

CO-6, socially liberal, fiscally conservative, devout atheist

---

by: skibum59 @ Fri May 04, 2012 at 02:34:55 AM UTC

Agree

Rasmussen seems to do better when they do multi-day samples.  

27, Republican, PA-6

---

by: Ryan_in_SEPA @ Fri May 04, 2012 at 11:52:57 AM UTC

[ Parent ]