Surveys And Slovin’s Formula
I was reading some threads in one of the famous Filipino Forums (Pinoyexchange.com) otherwise known as PEX for short, and I found this article posted (see complete article below). I don’t know exactly who originally wrote this but it made sense to me.. at first that is (let me explain)
I’m not a mathematician or a statistics major but certainly I can do some calculations using my spreadsheet.
It seems like the author is right in saying that if you backtrack the formula, using the 2100 respondents value, you can come up with the population assumed which is 13,125 (for the SWS survey) and for the pulse asia survey, at 1800 respondents, working on the formula would give 6,429 as the population.
So let me try out the formula and plugin an assumed population of 50,000 voters. I get a value of 2380.95 which is not at all far from the 2100 value used by SWS.
n = N / (1 + Ne^2)
where n = sample size
N = population size
e = margin of error
Then I tried out 50,000,000 and plug it into my spreadsheet and see how much ’sample size’ will a population of 50,000,000 would need. I was hoping to see something like a few hundred thousand but when I saw the result, using the formula, it says that I only need a sample size of 2499.88 HUH???
50,000 population needs 2381 sample size and 50 million needs 2500 sample size? That’s a difference of only (2500 – 2381 = 119).
Meaning to say, if you have 50,000 voters, with the Slovin’s formula you can accurately predict by talking to just 2381 people and if you ask a couple more people (119 more) you can say that you can actually predict for a population of 50,000,000 voters? That doesn’t sound right.
Does this mean that theorically if SWS or Pulse asia conducted a survey with 2,500 respondents then statisticians can say that it is a good and valid sample size?
Check it out. Here’s a screen cap of the spreadsheet
See the screen cap above. You can see that the difference between the population size results in very little changes in the sample size. I don’t know about you but for me, I really don’t think that such a small sample size can accurately predict the outcome of the 2010 elections.
But then again, I totally agree with the writer’s point at the end which is
Before you believe that SWS and/or Pulse Asia survey results are what can actually be expected if elections were held then and there, think more than twice; it is also highly likely that the results may not really be reflective of what the entire Filipino electorate may actually and ultimately reflect, from a statistical standpoint.
Here’s the complete acticle I was talking about
http://www.pinoyexchange.com/forums/showthread.php?p=42735928
A Take on SWS and Pulse Asia Surveys
Not only have I been studying and teaching statistics as a college instructor, but I have also been using it as a scientist and researcher for quite some time now. However, it was only during the early morning of March 26, 2010 that I thought of trying to scrutinize SWS and Pulse Asia surveys from a statistical standpoint. Although SWS and Pulse Asia never reveal how they actually conduct the surveys (aside from indicating the questions asked, the number of respondents, and the margin of error they set) and that they usually argue that their methods are “tried and tested” ones, I think it would not hurt if we try to take a look at how representative their survey results are of the entire population of registered voters, using another generally accepted and “tried and tested” method we use in statistics.
I’m talking about Slovin’s formula.
It is only from members of a sample (as respondents) that data would be obtained through a survey, since a census (or gathering data from the entire population) is not feasible for data gathering given a short span of time and limited resources. It is important, however, that the sample used be as representative of the population as possible, so that inferences derived from analysis of sample data may be more or less applicable to the whole population. This may be ensured by using appropriate sampling methods and using appropriate sample sizes.
In statistics, Slovin’s formula is a generally accepted way of how to determine the size appropriate for a sample to ensure better representation of the population of a known size. The formula may be expressed as follows:
n = N / (1 + Ne^2)
where n = sample size
N = population size
e = margin of error
Again, in the context of SWS and Pulse Asia surveys, these groups could argue that they use formulas other than Slovin’s formula in coming up with sample sizes of 2,100 and 1,800 respondents respectively (I did a review of news clips from the ABS-CBN News web site, and noted that these two figures are the most commonly used sample sizes of the two survey groups). However, if we use Slovin’s formula (which is, again, a generally accepted and commonly used method in statistics), rather alarming ideas may be derived (alarming, considering how some Filipinos base and defend their decisions on who to vote for on survey results).
Considering that both survey groups usually set the margin of error at plus or minus 2 percent (or 0.02), here’re what Slovin’s formula says about the SWS and Pulse Asia sample sizes (anybody with a considerable aptitude in algebra may verify these):
* SWS’s survey over 2,100 respondents, with margin of error set at 0.02, assumes a population composed of only 13,125 individuals. In the context of election-related surveys, that would point to the survey results being possibly representative of a population of 13,125 registered voters nationwide.
* Pulse Asia’s survey over 1,800 respondents, with margin of error set at 0.02, assumes a population composed of only 6,429 individuals.In the context of election-related surveys, that would indicate that the survey results may be representative of a population of 6,429 registered voters nationwide.
Now let’s see…. Based on Slovin’s formula, the SWS sample size seems to assume that there are only 13,125 registered voters, while the Pulse Asia sample size seems to assume that there are only 6,429 registered voters nationwide. How many registered voters are there in the country? Can anyone provide the actual population size of registered voters? Is there anybody reading this who knows anyone from COMELEC? I’m sure there’s a definite figure.
Well, with or without the actual figures from COMELEC, I believe 13,125 and 6,429 are gross underestimations of the actual number of registered voters in the country.
I’m not trying to disprove SWS or Pulse Asia here. Again, it is highly likely that they are using methods that do not include Slovin’s formula. However, here’s my case in point: Before you believe that SWS and/or Pulse Asia survey results are what can actually be expected if elections were held then and there, think more than twice; it is also highly likely that the results may not really be reflective of what the entire Filipino electorate may actually and ultimately reflect, from a statistical standpoint.
And that is not yet considering the sampling method employed by these survey groups.
Hence, to the SWS and Pulse Asia survey frontrunners and their supporters, I suggest for you not to keep your hopes too high, or you may end up disappointing yourselves if the actual results of the elections will not reflect the trends reflected by those survey results. And to survey tailenders and their supporters, there may actually be valid bases for you not to give much credence to these survey results. Quoting from Sen. Gordon: “The real ’survey’ is on May 10, 2010.”
6 Responses to “Surveys And Slovin’s Formula”
Leave a Reply
Great article, I especially like what you had to say in the last paragraph.
Yes, that’s how Slovin’s Formula really worked. You noticed that part of the formula for the quotient (the one dividing the population) becomes larger as the population also increases. This means that a 50 million population has a sample size of approx 2,000 that’s proportionally small. Whereas if you try the formula to say 100 population you get a sample size of around 96. that’s proportionally large right. My point is the result of the formula is not to be interpreted in terms of proportion between the population and the sample size but rather on the representativeness of these sample in order to generalize the whole population. If a sample size of 96 can represent a population of 100 then so can 2,500 respondents represent a 50M participants because it’s the same formula right.
The value of the determination of the sample size is for practical reasons. If you are going to survey the sample size, the results you you have is approximately the same when you survey the whole population. so why not just use the sample for your survey right. because it’s more practical.
The best analogy i can give regarding sampling is when you taste your coffee in the morning to test if it has enough sugar. You don’t need to drink the whole coffee to try if its too sweet. you just take a sip and you know how the whole cup tastes.
I don’t have any knowledge on how SWS or Pulse Asia determines their sample size but the issue doesn’t end there. The Slovin’s Formula assumes that the selection of the respondents will be randomized and that’s where we should focus on the issue. How were the respondents selected randomly? That should be our main questions in trying to focus on the credibility of the surveys. Because if its not properly randomized (unbiased selection) then it cannot be used to generalize the population.
very nice article it is all true, these surveys will only disappoint people who has high hopes for their candidate and it only delivers wrong information to the public…tsk..tsk..tsk…
dapat hindi tlga natin gmitin yn mga survey n yan mali-mali naman ang mga binibigay nilang impormasyon nakakalungkot lng bka pg yung kulelat sa survey ang manalo sbhin nung nangunguna s survey dinaya cla dahil sbi s survey cla ang nanagunguna. pang gulo lng tlga yn survey n yan
Slovin’s formula would give us a valid sample of the whole population. Of course there are some sampling errors along the way but it is being patched up by margin or error. Hence, it is highly valid.
with all due respect to the author, I think its about time we should be aware that surveys are scientific and a reflection of the people’s views on issues, which is especially true for election opinion polls. the May 2010 election results speak for the surveys’ validity. the SWS exit polls declares a 0.4 difference from the official tally. with that, Sen.Gordon is now facing a Compulsory Counterclaim for moral damages amounting to P1 million for his maliciously debasing SWS’ reputation. rather, let us examine how Filipino’s vote and decide come election day, without putting credible and honest opinion polling organizations dedicated to public service be put under scrutiny. kudos to the hardworking men and women of SWS and Pulse Asia.