8/1: Methodology Should be Methodology, whether a Public Poll* or an Academic Survey

August 1, 2013 by  
Filed under Blog, Featured, Guest Bloggers

It’s no secret that academic survey researchers and public pollsters have very different goals for their work. Public pollsters are gathering data to quickly measure the current environment and find out who thinks what.  In contrast, academic researchers ponder why the environment is what it is, and what the implications of their findings are for theoretical approaches in their discipline.   I have worked on both sides of this equation, and have been questioned for my career choice quite a bit given my graduate school and postdoctoral training in academia. Even though I’m working for an academic institute, the perceived focus on public polling and not having “professor” in my job title seems to lead to the idea that somehow I “sold out.” But “we’re not so different, you and I.” All credible surveys rely on complex, rigorous methodology, regardless of purpose.

Natalie Jackson

The difference in purpose is best illustrated by comparing public pre-election polls to the American National Election Study (ANES), conducted for every presidential election since 1948. Pre-election polls are ubiquitous prior to elections. If the data were collected more than a few days ago, the numbers are considered “old” and no longer relevant. On the other hand, the ANES gathers two waves of data which are processed, cleaned, and carefully combed through prior to public release several months after the election. ANES data are primarily used in academic publications that may emerge years or even decades after the initial data collection. Public polling data chronicle public reactions to the give and take of campaigns and events for a 24/7 news cycle, magazines, and popular books.

Public polls often stoke more controversy than academic work, especially election polling because there is something to compare the election polls to in order to verify accuracy—the actual election returns. Reputations are gained and lost very quickly over one errant poll (which, by the way, is statistically guaranteed to happen to everyone sooner or later simply by the laws of probability).  There are clear expectations of transparency in methods of public polls, and they are dissected by anyone with an opinion and a keyboard.

On the other hand, there are usually few guidelines to provide checks on conclusions about how or why people think the way they do or for the vast majority of survey sample findings in academic work. And, rarely do they have quantifiable verification.  Academic reputations are gained by publication, which is generally earned through the perceived importance of the findings in a peer review process and are not usually the subject of public scorn, argument or debate about “bias,” as the public polls were in the 2012 election cycle.

Despite these differences, the intersection of these two diverse approaches is found in the underlying methodology.  How do we best measure what we’re trying to measure? The question is the same regardless of the goal or use of the data. We are united by the desire to get the best quality data, from the best and most representative samples, and to do this within budget constraints that are often outside our control. There is also a need to innovate and stay on the cutting edge of technology while being careful to not step beyond the scientific bounds to achieve reliability and validity of our samples and polls.

We each think statistically, creatively, and scientifically about survey sampling and data collection. But it doesn’t end there. We think psychologically and ethically about the people who so graciously (or through some artful persuasion) provide us with the information we seek. How do people hear questions? What does each word mean to 1,000 or more different people? How does the order of our words and our questions affect what people hear and how they answer? In some modes, an additional question is how the interviewer administering the survey affects the respondent’s experience and how they hear and answer questions.

These difficulties are shared by everyone who engages in any type of survey research—whether academic, public polling, market research, private polling, or evaluation work—if we care about producing high-quality reliable and valid data. We are trying to achieve statistical precision about human behavior by crafting methodologies that weave together sampling theory, statistics, psychology, sociology, political science, evaluation theories, and technology.

Creating this tapestry is the heart of survey research, and what better place to do methods work than a place where journalists, partisan devotees with agendas, and the general public are constantly looking over your shoulder questioning how your polls were done and whether you were “right” about a given election? I deal with the same methodological puzzles, struggles, and need for innovation that I dealt with in my academic positions, only in a fast-paced and very public environment. So please, can we stop pretending that academic survey researchers and public pollsters are so different, or that one is better than the other? We all put our pants on one leg at a time—and we all learned our sample statistics one t-score at a time.  Besides, last time I checked there were a lot of smart and talented PhDs around here too.


*”Public polling” and “public pollsters” refer only to those who do nonpartisan, non-candidate funded work. Funding structures may vary, but public polls are done for the benefit of the general public, not a party, candidate, PAC, Super PAC, or any politically organized entity.



11/1 Undecided Voters in the 2012 Presidential Election

November 1, 2012 by  
Filed under Blog, Featured, Guest Bloggers

In Presidential elections some people will always vote for the Democrat and some will always vote for the Republican.  But, there’s that group of people in the middle who make up their mind as the campaign moves toward Election Day that often decides who wins.  The exact proportion of undecided voters varies from campaign to campaign—sometimes voters have a more difficult time deciding who to vote for and as much as 10 to 15% of likely voters are undecided.  But in some years, like 2012, voters pick sides early and there are few undecided voters left by Labor Day, the start of the fall campaign.  This election season, polls have consistently shown few undecided likely voters.  So, who are these people? How much do they matter? If they do vote, in whose direction will they break?

Natalie Jackson


In order to look at these questions, the data from the NBC News/Wall Street Journal/Marist battleground state polls of Florida, Virginia, and Ohio were combined.  This dataset comprises three polls from each state in September and October, two before the debates started and one after, for a total of 9 polls. The combined samples include 11,510 registered voters, of which 1,168–approximately 10%–indicated that they were unsure of their vote preference.  About 40% of these respondents chose a side in the follow-up question that asked if they were leaning toward a candidate, but the majority said they were still not sure.

Will the undecideds vote? Perhaps, the single biggest question is whether these people will vote.  The answer could hold the key to who wins the election. Undoubtedly, some will stay home. However, of the 1,168 undecided voters in the polls, 61% of them are classified as likely voters.  The Marist Poll likely voter model takes into account chance of vote, interest in the election, past voting behavior, and turnout expectations.  These undecided voters likely to cast a ballot will probably make a decision at some point, or even at the last minute, and vote.  They account for 8% of all likely voters in nine battleground states and could make a difference in the race.[1]  In this close race, that is a substantial voting bloc.

Beyond the likely voter classification, there are other factors that point to a substantial number of undecided voters actually casting a ballot.  Perhaps, most strikingly, is that 92% of undecided likely voters report having participated in past presidential elections, and 90% say their chance of voting this year is either excellent or good.  Sixty-two percent of undecided likely voters also say they are very or somewhat enthusiastic about voting.  Over two-thirds—70%– indicate they are very interested in the campaign.  These numbers suggest undecided likely voters are not apathetic, uninformed or uninterested.  They are genuinely interested in voting—they simply have not decided who to vote for yet.

Who are these undecided voters?  This group of voters is not demographically exceptional.   Fifty-four percent are women, 52% are married, and 39% are college graduates. Thirteen percent are between the ages of 18 and 29, 26% are between ages 30 and 44, 29% are in the 45-49 age range, and just about a third are over 60.  Seventy-six percent identify as white, 10% are African American, and 9% are Latino.

Politically, most of these people are often not committed to a party or ideology.  Twenty-two percent identify as Democrats, compared to 15% that identify as Republicans.  The largest group, 61%, identify as independents. The same pattern is evident with ideology: 33% are either conservative or very conservative, compared with only 15% who are liberal or very liberal, and the remaining 52% are moderates.  Thirteen percent say they support the Tea Party.

Why are they undecided?  If these people are engaged, enthusiastic, or, most importantly, likely to vote, why have they not made a decision yet? It is often assumed people who have not decided who to vote for are not paying attention or do not care.  The data offer some insights.  Of the 92% of current undecided likely voters who voted in the previous presidential election, 59% of them recall voting for President Obama, while only 32% recall voting for Senator McCain.  The remainder either thinks they voted for another candidate or cannot recall.  But, they divide about Obama’s job approval.  Thirty-six percent approve of the job he is doing as President and 36% disapprove.  The remaining 28% are unsure.  Additionally, 53% think the nation is headed in the wrong direction, and only 33% say it is headed in the right one.

Altogether, these results indicate that most of these undecided likely voters are conservatives or moderates, who are dissatisfied with Obama’s presidency and the state of the nation.  But, these individuals are not fond of Romney either.  Romney’s favorability rating is upside-down among this group. Only 31% give him a favorable rating and 42% rate him unfavorably.  Even among Republicans in this undecided likely voter group, Romney has only a 39% favorability rating and a 32% unfavorable one.

Who will they vote for?  As noted, about 40% of undecided voters provided an answer when asked if they lean toward one candidate or another.  Although the majority of undecided likely voters still say they do not know, 25% state they lean toward Obama, and 15% lean toward Romney.  A handful of undecided partisans lean toward the candidate across the aisle: 7% of undecided Democrats lean toward the Republican ticket, and 12% of undecided Republicans lean toward the Democratic ticket.    Interestingly, 16% of those who identify as Tea Party supporters lean toward Obama, and 26% lean toward Romney.  The favorability ratings advantage Obama.  Among undecided likely voters, Obama nets a plus 15 points compared with Romney’s net 11 points in a negative direction.  Still, 58% remain undecided even when asked if they are leaning toward a candidate.  For them, it is anyone’s guess.

As we approach the final few days of the campaigns, the proportion of undecided likely voters has continued to shrink.  Some might have already voted and made their decision, but many of these individuals will not choose until they get into the voting booth on Election Day—if they choose to vote.  The numbers indicate that a good proportion of undecided registered voters are likely to vote.  The campaigns have many competing goals and issues to address, but one of the most crucial remains attracting undecided voters, even in a year in which there are few undecideds left. The close race means these few undecided voters could make the difference.


[1] The Marist Poll releases tables which show a smaller percentage of undecided registered and likely voters.  These tables do not categorize undecided voters who are leaning toward a candidate as undecided but instead include them within the candidate’s numbers.  For this analysis, undecided voters include voters who may be leaning toward a candidate.

4/4: My Quest for 30

April 4, 2012 by  
Filed under Blog, Featured, Guest Bloggers

Everyone has their bucket list.  And mine has typical things like skydiving and visiting where my dad was born in Italy.  However, numero uno on my list is to attend a game at every Major League Baseball stadium.  Anyone who knows me, knows how much I love baseball.  I, like 16% of national adults, am a Yankee fan.  I have been since I was a kid.  But, it wasn’t until a once-in-a-lifetime undergraduate internship at the Baseball Hall of Fame in 2003 that I became a true fan of the game.  Sure, I liked baseball, but it was truly a pinstripe-centric affair.  I liked going to games (Yankee games), and I liked watching games on TV (Yankee games).

baseball field under sky

Photo Courtesy of Stephanie L. Calvano


However, in a short 3 months surrounded by interns from across the country and a great Hall staff, I learned to appreciate what the game of baseball has to offer, not just the Yankees.  Through my own research for projects, information that was taught by Hall of Fame staff, and an opportunity to interact with baseball greats, I started to realize that baseball was about so much more than cheering for one team.  Baseball is one of the greatest history books one can “read.”  So much of history is mirrored on the diamond and is a testimony to American life.  From Jackie Robinson joining the Brooklyn Dodgers which broke Major League Baseball’s color barrier to the emergence of social media and its impact, baseball reflects the times in which we live.

After my internship at the Hall, I was not only a Yankee fan but a true baseball fan!  In fact, that summer I attended my first Major League game in which the Yankees were not one of the competing teams! And, so my quest for 30 began!

I have been to the following ball parks:

  • Yankee Stadium (both old and new) (Yankees)
  • Shea Stadium/Citi Field (Mets)
  • Veterans Stadium (old Phillies stadium)
  • Jacob’s Field (Indians and now called Progressive Field)
  • Ballpark at Arlington (Rangers; At one called AmeriQuest Park and now called Rangers Ballpark in Arlington)
  • Tropicana Field (Rays)
  • Pro-Player (old Marlins stadium; Still home of the Miami Dolphins)
  • Camden Yards (Orioles)
  • Wrigley (Cubs)
  • Nationals Park (Nationals)
  • Roger’s Centre (Blue Jays)
  • Safeco Field (Mariners)

I try to plan vacations to places that have baseball stadiums I have yet to see.  And when a business trip comes up, if there’s a baseball team nearby, my first stop is to their website in hopes they will be home when I am visiting.  Sometimes it works out and sometimes it doesn’t, but I always try!

So, now here I am in 2012.   I’m still plugging away trying to take in a game at all the parks, but this is my dilemma……

I plead to all MLB teams….please, stop building new ballparks!!!  Two of the stadiums on my list are no longer the home of a baseball team.  In 2004, the Phillies opened Citizen’s Bank Ballpark and this season the Marlins will call Marlins Park home.  I’m sure the Trop won’t make it much longer either.  So, with each stadium I cross off, I fear that before I get to all 30 parks, I will have no choice but to start again!

Sure, one could argue, “just go by the team and not the stadium.”  But, that’s not gonna cut it!  If I had unlimited time and resources, I would take in all 30 in one season.   But, until I hit the lotto, my quest will continue.  And as new baseball stadiums are built, I will just have to find a way to visit again!  As they say in A Field of Dreams, “if you build it, they will come.”

3/30: What Is a Yankees Fan to Do?

March 30, 2011 by  
Filed under Blog, Featured, Guest Bloggers

Play ball!!  I’ve waited through a very long, cold winter to hear those words.  And now it’s here, it’s here … it’s finally here!  It’s baseball season!  Although the 2011 baseball season is still in its infancy, in New York, it’s never too soon to talk about the postseason.  In a recent New York City Marist Poll, we asked Yankees and Mets fans who they would root for in the most unfortunate World Series matchups.  Mets fans were asked who they would root for if the Yankees were playing the Philadelphia Phillies in the World Series.  61% of New York City Mets fans said they would keep it in the New York family and cheer on the Yankees, and 34% would support the Phillies.  I wonder if this was actually the case in 2009?!?  When asked who they would root for if the Mets were playing the Boston Red Sox in the World Series, Yankees fans were a little kinder to their cross-town rivals.  83% of New York City Yankees fans said they would be in the Mets corner, while 14% said they would rally behind the BoSox.

Stephanie Calvano

As a Yankees fan, the thought of this hypothetical matchup put my stomach in knots and left me wondering what I would do if this horror actually came to pass.  After much thought and rationalization, I still can’t definitely say what I would do.  Initially, I thought I would support the Mets over the Red Sox.  If my New York team didn’t make it to the World Series, why not root for another New York team?  But, upon further reflection I started digging a little deeper.  In order for the Red Sox to advance to the postseason and ultimately the World Series, they would have had to have a better season than the Yankees.  So, if the Sox won the World Series, at least I would know that my Bombers were beaten by the best.  It falls under the same logic that had me rooting for the Packers in the Super Bowl.  They beat the Giants and it made me feel better about Big Blue’s demise to know it was the Super Bowl Champs that put the brakes on their postseason chances.  This kind of thinking also made me a fan of the Duke women’s basketball team after they eliminated the Marist Red Foxes from the 2011 NCAA tournament in a close contest.

So, what’s my conclusion?  The answer is “I don’t know.”  All I can hope for is that it’s a decision I never have to make.  And, if that World Series matchup does ever happen, check back with me … I might be rooting for the New York Red Sox or the Boston Mets!

Until then … happy baseball season!

7/7: World Cup (Vuvuzela) Buzz

July 7, 2010 by  
Filed under Blog, Featured, Guest Bloggers

GOOOOOOOOOOAAAAAAAAAALLLLLLLLLL!!!!!!!!!!  Get your vuvuzelas ready because the World Cup semi-finals are underway … or you can do what I did and download the vuvuzela app!

Stephanie Calvano

Stephanie Calvano

After nearly a month of competition, a new country will be crowned World Cup King on July 11th.  A quick look at the history books will tell you the first FIFA World Cup was held in 1930 where host Uruguay won it all, and believe it or not, the United States placed third overall that year.  With the game clock still running in 2010 … the 19th World Cup is being played in South Africa, and there have been plenty of upsets (reigning World Cup winner Italy and runner-up France were knocked out after the first round), controversies (bad calls, disallowed goals), and excitement, not to mention the constant buzz of vuvuzelas.

But, have people in the U.S. been watching the World Cup?  According to the latest national Marist Poll, 37% of Americans, me included, say they are watching most or some of the month-long event.  As someone who has been watching since it started in early June, and continues to watch even though U.S.A. has been eliminated, that finding was bittersweet. At first glance, I thought, “only 37%.”  Then, as I thought about it further, I realized that nearly 4 out of 10 watchers isn’t so bad considering supposedly “nobody watches soccer.”

I personally equate the World Cup to the Olympics. Regardless of one’s interest in the game of soccer, the bottom line is that, as a country, we rally behind our guys.  You may not know all the players’ names, that the field is called the “pitch,” or understand all the rules, but you know that there’s a group of guys representing the red, white, and blue and want them to succeed.  I was watching the USA’s final World Cup game against Ghana at a restaurant.  There were “ohs” and “ahs” when a great opportunity to score was missed, there was cheering, there was an entire restaurant on the edge of their seats when Landon Donovan took his penalty kick and then sent it into the back of the net to tie the game.  That day, everyone there was a soccer fan, because everyone was a USA fan.

In 1998, I was on a school trip to France and Spain the year the World Cup was in France.  I remember we were getting on the metro to go to dinner, and it was packed with people coming from a game.  Their faces were painted, and they were singing.  It was incredible to see and a moment I will never forget.  My classmates and I, as well as other hotel patrons, hovered around the small TV in the lobby to watch the games (when our educational tour schedule permitted, of course).  At 15, it didn’t sink in then, but it’s incredible that one event can have such a unifying effect.  We didn’t all have the same native language, but we all spoke one language … The World Cup.

Soccer is a language I’ve been speaking for a while.  It has been an important part of my life since I first stepped onto a field at 6 years old.  For the next 12 years, it was something I could not do without.  I played on local town teams until I aged out, joined the JV team in middle school, and played Varsity soccer in high school.  And, because playing in the fall wasn’t enough, I played indoor soccer in the winter.  I even sustained an injury that left me with six screws and a plate in my ankle as mementos.  Granted, it has been a decade since the last time I played soccer competitively, but that doesn’t mean I enjoy the sport any less.  As it always does, watching the World Cup makes me want to get back out there and play again.

I believe playing soccer as a child makes me more inclined to follow the World Cup.  In fact, when you look at the results from Marist’s latest national survey, of USA residents who played soccer when they were young, nearly 6 in 10 — 58% — say they are watching most or some of the World Cup.

It’s no secret that professional soccer in the United States doesn’t hold a candle to “futbol” in Europe and South America. But, this World Cup has been making headlines with atrocious officiating, talks of using instant replay, upsets, and even “off-the-pitch”drama.  Is that enough to make Americans want to take in a soccer game?   Or does Major League Soccer need to revamp and find ways to “Americanize” the game to make it more attractive here?  I don’t know the answer, but I do hope that Major League Soccer can ride the World Cup wave and maybe, just maybe, at the next World Cup in 2014, it won’t take a bad call, a milestone goal, or a tabloid story for everyone to know the group of guys that are Team USA!

4/28: The Tribeca Film Festival’s Identity Crisis

April 28, 2010 by  
Filed under Blog, Featured, Guest Bloggers, Michael Avila

We’re already at the halfway point for the ninth session of the Tribeca Film Festival, yet unless you live downtown, it’s hard to notice. The festival isn’t exactly a hot topic around the water cooler.

Michael Avila, TV producer and pop culture writer.

Michael Avila, TV producer and pop culture writer.

That’s not an indictment against the quality of films.

Documentaries, a traditionally strong category at Tribeca, have made a strong impression, thanks to the eco-warning “Climate of Change,” the Shea Stadium love letter “Last Play at Shea” and the festival’s most talked-about entry, the work-in-progress about New York’s disgraced ex-governor “Untitled Eliot Spitzer Project.” Several features, like “Legacy” and “Spork,” have also earned strong reviews.

Still, there doesn’t seem to be much electricity around town for TFF. A new poll by Marist indicates 79% of New Yorkers have no intention of attending a single TFF event. That’s a staggering number.

The star-studded premiere parties and the countless sponsor events are still making it into Page Six. How is that different than any other night in Manhattan? There is nothing approaching the buzz that turns Park City, Utah, into mini-Hollywood every January for Sundance, or the excitement in Toronto as likely Oscar contenders are screened there in the fall.

It’s not exactly fair to compare Tribeca to Sundance or Toronto or Cannes, for that matter. Those festivals are established events with longer histories. They also benefit from better placement on the calendar (Sundance in January, Cannes in May, Toronto in September). And you can’t ignore the fact that the options for entertainment in Manhattan are slightly more plentiful than Park City or Toronto.

For all these reasons and more, Tribeca has been forced to dig a little deeper than other festivals to find quality pictures.

Fortunately, there are plenty of great independent movies out there. And TFF has shown a knack for finding gems in the foreign markets, with “Buried Land,” “Dog Pound” and “My Brothers” being good examples this year.

It’s not like the festival isn’t drawing crowds. Last year, nearly 350,000 people attended. They can’t all be out-of-towners, can they? But attendance figures can’t explain Tribeca’s lack of cultural cachet.

You can go ahead and blame the festival’s one-size-fits-all approach to programming for that. Co-founders Jane Rosenthal and Robert De Niro have embraced mainstream popcorn pictures such as “Star Wars: Episode II – Attack of the Clones” and “Spider-Man 3″ since the festival began in 2002, as a way to drum up publicity. But kicking off a film festival with “Shrek Forever After” – this year’s Opening Night movie – isn’t the way to gain street cred on the film festival circuit.

By striving for national attention with splashy red-carpet premieres of upcoming summer blockbusters, Tribeca sabotaged its own efforts to position itself as a central player in independent film.

But it may not be too late to fix that.

Festival organizers have drastically scaled back the number of feature films they screen, from 176 in 2005 to a more manageable 85 this year. Yes, they did premiere with “Shrek 4,” but at least it was a 3D movie. The recent explosion in 3D moviemaking goes hand-in-hand with TFF’s mission to embrace new film technology.

For the first time, Tribeca has made festival movies such as Ed Burns’ “Nice Guy Johnny” available for viewing online. In addition, TFF struck deals with several cable companies to offer movies through Video On Demand. Those films will become available at the same time they premiere at the festival.

The old independent film business model is undergoing drastic change. People are watching movies online and on their iPhone, when they want to watch. TFF is embracing these new platforms as a means to finding new ways to expose independent films to wider audiences. By doing so, the Tribeca Film Festival may have finally figured out a solution to its longtime identity crisis.

Who knows? Maybe New Yorkers will finally get jazzed about Tribeca if they know they can see the films without having to take the A train downtown.

This article is written by Michael Avila, a television producer and pop culture writer.

4/1: For the Love of the Game

April 1, 2010 by  
Filed under Blog, Featured, Guest Bloggers

Rogers Hornsby said, “People ask me what I do in winter when there’s no baseball.  I’ll tell you what I do.  I stare out the window and wait for spring.”  I share this sentiment with the legendary second baseman.

Stephanie Calvano

Stephanie Calvano

The Yankees take top honors among baseball fans nationally with 11% of those fans saying they support the Pinstripes.  In New York, 58% of the state’s registered voters who are baseball fans do the same.  As a person who shares this allegiance, I couldn’t have imagined a more picture-perfect ending to the 2009 season than a 27th World Series Championship, but with my elation came a sense of sadness.  Baseball season was over … now what?  I think A. Bartlett Giamatti put it perfectly… “It breaks your heart.  It is designed to break your heart.  The game begins in spring, when everything else begins again, and it blossoms in the summer, filling the afternoons and evenings, and then as soon as the chill rains come, it stops and leaves you to face the fall alone.”

Yeah, sure, I enjoy football season, and it gives me my sports fix through the winter, but it’s not the same.  I don’t watch football with the same knowledge and passion that I do with baseball.  I wait and wait for the day pitchers and catchers report to spring training.  It’s a signal that a new season (as well as a visit to Florida for spring training) is just around the corner!

To say the least, I can’t wait for Sunday!  No, not because it’s Easter, and I have an excuse to eat chocolate all day, but because its baseball’s opening night!  And, as if the start of the 2010 season wasn’t exciting enough for this baseball fan, the game is a marquee matchup between my Bronx Bombers and rival Boston Red Sox.  The baseball gods must really love me this year!!

As the 2010 season begins, and my quest to attend a game at every Major League Baseball stadium continues, I have one thing to say, “Play ball!!!”