Poll Dancing

Image from Cartoonstock.com

Ok, it is time to talk polls. For the second Presidential election in a row, the lack of accuracy from major polling services has been an issue. Before the election, there was skepticism expressed by many, since the predictions of a blue wave as detected by the polls did not match the gut feelings of people on the ground, especially in those states declared to be battleground states. It is always difficult to determine the slope of a line with less than three data points, but in this case, since presidential polling only gets tested every four years, it is appropriate to declare a trend and try to understand why it is occurring. In this regard, I have no knowledge about the internals that polling firms have seen. I am only looking at trends in society in general, and extrapolating them to the polling results.

First, polls are very valuable in estimating the characteristics of a large population, if three criteria are met. Those criteria are:

  • Those polled are a representative sample of the population
  • Those who respond to polls are honest in their answers
  • The technology used to reach those who are sampled matches the technology used by those who are sampled.

The first and third criteria are closely interrelated. Since most polling still depends upon land line responses, the audience for polling is becoming further and further divorced from the population as a whole. That is because fewer and fewer people use a land line, but instead are totally dependent upon their cell phones. If you look over the past decade, the growth of cell phone penetration has been explosive. And another factor that comes into play is that many people automatically disregard phone calls from an unknown number. So if you attempt to contact people on cell phones, you are likely to be ignored by an increasing percentage of the population. Finally, once you have answered the phone, you have the opportunity to opt in to being polled. I normally will opt in unless I am in the midst of doing something else and can’t split my attention. But I would be interested to see if there is a difference in behavior between those who lean left and those who lean right in terms of voluntary opt in percentages. Since so many of those on the right politically now distrust the government and the established elites, my sense is that more people on the right will decline to participate in a survey.

The second criteria, being honest in their answers, is the most subtle factor in determining whether a poll is accurate. Sometimes folks just want to throw a monkey wrench into the works, and so they will deliberately answer inaccurately in order to influence the results. The number who choose this option may be small, but when you are trying to assess a smaller population (like a state), the smaller sample size means each response is proportionally more important. So it can appeal to those who feel powerless in society to try to exert more influence on polls than normal by screwing with the results. For this to affect polling accuracy, it would mean that more people on one side of the electoral continuum would use this than those on the other side. Sounds like a good project for a social scientist to take on over the upcoming years.

Why has polling been so heavily used over the past few decades? Because it worked. When the US was a more homogenous nation, and we all shared a common communications technology (the telephone), it was possible to ensure that you could select a random slice of the population. Call someone up, have them answer a few questions regarding age, sex, and race, and you could slot them into one of the acceptable demographic categories for a poll. In case you haven’t noticed, we no longer fit neatly into categories as we used to. And the longer we go with alternative communications technologies, the further we stray from the easy-to-sample population we had from the 50’s through the 90’s.

Now, as to how the polls are used, you have to stray into the world of mathematics. One of the most common terms you hear is “Margin of Error”. That phrase is bandied about by the Steve Kornacki’s of the cable world along with many others of the pundit class. The formula for margin of error is this:

The margin of error in a sample = 1 divided by the square root of the number of people in the sample

This is what is amazing to understand. It doesn’t matter what is the size of the population being sampled, it only matters what is the size of the sample. That is why having a representative, but random sample of the population is so important. Incidentally, for a +3% margin of error, the sample size would need to be 1090. For a +5% margin of error, the sample size would need to be 400. Usually national samples are larger in order to ascertain valid statistics for subgroups (male, female, white, black, age groupings). But if just the top result is desired with a +3% margin of error, it is possible to sample the entire population of the US with a sample size of slightly over 1000 individuals. This is the magic of polling.

When someone speaks about the margin of error being +3%, what that means is that you would expect the true value for the population to be equal to the sampled value, +3% for 95% of the time. The 95% is a standard confidence limit in statistics, used often to determine if an effect is real or may be just a chance result. So if someone shows a poll support of 45% with a margin of error of +3%, then we would expect the real value to be within 42% to 48% for 95% of the time. If two candidates are being sampled, you look to see if there is any overlap between the 95% confidence intervals for the two. In this case, if candidate A had 45%, and candidate B had 49%, there would be some overlap between the 95% confidence intervals for the two. The range from 46% to 48% would fit both of these candidates. Now, if there is only slight overlap between the two, it is more likely that the one who samples higher is truly ahead, but it is not outside of the standard of 95%.

The 95% confidence interval is used many times in science. It is used in testing of drugs and medical treatments. I used it in production trials in a chemical plant, when we were attempting to determine whether one set of conditions was better than another. Once you are familiar with the math behind sampling, you can use that math in many different ways.

But once again, it all depends upon whether the population that responds to a survey is truly a representative sample of the population as a whole. It seems obvious that at least in the US, there is something wrong with the methodology used to select a random, representative sample. It remains to be seen whether these problems can be diagnosed and fixed before the next huge use of polling coming up in 2024.

Of Subtle Snares and Nanny States

payday loan

Why is it whenever I try to explain a log-normal distribution to folks, eyes start to glaze over? You’d think that such a useful concept would be intuitively understood by everyone, right? Well, the correct response is that very few people have the background in mathematics, and particularly statistics, to understand how a log-normal distribution works, and why it is important.

Log-normal distributions are one thing. Understanding compound interest is something else altogether. That is a consumer survival skill in this day and age where it can work both for you and against you. It works for you if you use it early in your lifetime to start saving for retirement. It works against you if you depend upon borrowing in order to make it through your life. And what is amazing is the “legitimate” financial growth industry that has developed out of what the mafia used to call loan sharking. Payroll advance services, legal in many states, charge an annualized rate of interest of up to 780% (don’t use one of these services in Louisiana). But the people who use these lenders of last resort are the poor who are just trying to stave off eviction or keep the lights on, or fix a failing vehicle. They are indeed the ones who have the least math literacy, and our free enterprise business system takes advantage of their illiteracy by trapping them in a cycle of loans and renewals of loans. Then there’s the car title businesses – I don’t even want to know the details of their business model.

Now, the Consumer Financial Protection Bureau (CFPB) issued proposed regulations in 2016 that would cut down the maximum annualized interest rate, including fees for lending, all the way down to 390%. It also is requiring that lenders be prohibited from issuing new loans (with additional fees) for borrowers to pay off their old loans, and also that they verify a payer’s ability to repay a loan. Those regulations were put out for comment back in June 2016, and comments were supposed to close in October. Then November 2016 happened, and now the CFPB is engaged in an existential struggle with the Trump administration. The CFPB has been flagged as a flagrant excessive regulation generator. So as of today, no proposed final regulations have been issued.

Very few issues highlight the ideological divide between Republicans and Democrats like the issue of the CFPB. Republicans invoke the ideology of the free market, and view any interference with its exercise as a violation of their unwritten social contract to allow any predation upon society as a whole, as long as it is deemed legal. Look at sites like Forbes as exemplars of the capitalist ideal in order to receive indoctrination that the CFPB only desires to reduce the choices that poor consumers with no credit options have to meet daily needs. The CFPB is epitomizing the nanny regulatory state that is holding back economic growth.

Democrats on the other hand view the efforts of the CFPB as biblically based social justice, where the poor are protected from being preyed upon by the powerful and moneyed interests of the nation. Yes, there is an element of “We know what’s good for you” in this, in the perspective of the Democrats. Does this version of a nanny state mentality outweigh the monetary crack offered by the payday loan businesses that has resulted in the virtual slavery to the customers of the payday loan complex?

It is obvious in a capitalistic society, keeping score matters. The ultimate score keeping in this instance is the share price of the publically traded companies involved with payday loan operations. Since the CFPB announced potential regulation in June 2016, one would assume that the share price of these companies would have gone down since then had they taken the possibility of these regulations coming into effect seriously. But for two of the largest publically traded companies, their share price does not reflect much of a fear premium. In the past year, two of the largest companies have seen their share price go up by 51% and 14% apiece. In the realm of Trump, it is obvious that for the free market, anything goes.

It is doubtful that the administration of President Donald Trump will allow an agency to issue final regulations that are so opposed to the core interests of the moneyed aristocracy occupying the cabinet. So we will remain with a patchwork of state regulations – in some states, strict prohibitions against this parasitic industry are in effect, and maximum loan interest is capped at 30%. In many others, it is a wild west of freedom, and libertarians can celebrate their freedom to choose to pay annualized interest rates of 600-800% for the honor of accepting a payday loan.

What neither party recognizes is that the demand that built the payday loan industry into a $50 billion annual business is real, and we need to acknowledge that and work to provide real world business solutions. Republicans insist that the exorbitant prices charged by the payday loan providers is necessary to serve the market, since there is a high risk of default. Democrats insist that the providers of the service not be abusers of their customers, profiting exorbitantly on the backs of those who can least afford it.

I ask why do people find it necessary to resort to these lenders of next to last resort? What are the societal issues that keeps people needing to rely upon exorbitant interest and fee rate lenders, and how can we mitigate this need? It galls me to see a huge industry develop over the past few decades where it is touted as an investment opportunity, but it builds its profitability upon the backs of our poor. In our Episcopal hymnal there is a hymn which resonates with me whenever I hear it. It says,

For sins of heedless word and deed, for pride ambitions to succeed, for crafty trade and subtle snare to catch the simple unaware, for lives bereft of purpose high, forgive, forgive, O Lord, we cry

Far too often the rules in our society are stacked against the poor, and those with less education. And then we in our arrogance, blame these victims while we reap the riches they provide to the companies that profit from the subtle snares.