As a kid, I loved math. The answer was right or wrong, black or white. You could easily see where you went wrong if your answer wasn’t correct and could easily remedy that to arrive at the expected answer.
Once in college, I decided to abandon my calculus career path and delve into statistics. I quickly determined that statistics is not math (despite the fact it fulfilled required math credits) but data—namely the analysis and interpretation of data. While I loved the black and white nature of math, the gray nature of statistics intrigued me, because of the narrative it allows. A percentage itself is black and white—40%, for example—but how it is presented, the stories it can be used to convey, are gray—only two out of five, or nearly half or less than half. The positioning begins the narrative.
Take the way in which the decline of defined benefit (DB) pension plans has been discussed. I believe valuable data has led to a narrative that is a bit disingenuous. Although the number of defined benefit plans has decreased, this hasn’t translated to a loss of income for Americans as is frequently implied.
Dallas Salisbury, former president of the Employee Benefit Research Institute (EBRI), would frequently point out that, although pensions are a good source of retirement income for workers employed by one company for many years, they won’t necessarily provide for those who change employers often. And, though it is often said that American workers switch jobs more now than in the past, data show that most American workers have always changed jobs multiple times over the course of a career. This means that, for most retirees, the corporate move away from defined benefit plans has not had a significant—or potentially any—impact on their long-term retirement savings or financial well-being. So, although the data narrative is correct in that there are fewer plans, Americans’ less stable retirement income is not necessarily, a result.
In addition to questioning the narrative that comes out of interpretation, it’s important to be mindful of year-over-year reporting when questionnaires change. Statistics are completely dependent on inputs. Take EBRI’s recent criticism of the U.S. Census Bureau’s Current Population Survey (CPS). EBRI alleged that the recently released CPS estimation of major declines in employment-based plan participation miscounted how many Americans participate in employer-sponsored retirement plans. Workers with the sharpest drops in participation include older employees, higher earners and workers with larger employers, all of whom are actually those most likely to engage in retirement plans. EBRI reported that these miscalculations are due to a major redesign of CPS’ questions pertaining to income two years ago. The Census Bureau initiated the 2014 redesign following previous research that indicated the survey misclassified and generally underreported income—particularly pension income. EBRI points out that, while the changes appear to have improved the accuracy of data on pension income, they also created the appearance of historically sharp reductions in the levels of worker participation in employment-based retirement plans.
What is the lesson in this for the retirement plan industry? We’ve written in these pages about the importance of sponsors evaluating and benchmarking their retirement plans by leveraging industry and company data. But the concept of a data narrative reminds all of us that, in order to properly leverage that data, sponsors and advisers have to look at the inputs and be mindful of how the numbers are then interpreted.
Think about evaluating the use of catch-up contributions. If only 10% of participants are taking advantage of them, that might appear to be low usage. But what if that 10% actually represents 40% of all eligible participants? Then that would show strong usage.
At PLANSPONSOR, we believe it is incumbent to help our readers make sense of the deluge of research and commentary available in the industry every day. We report on research findings, and conduct and produce our own proprietary research, featured in this issue. When we communicate what we’ve learned, we consider not just the numbers themselves but how they are presented, and by whom they are being reported. Is a financial wellness company the source behind the research showing such programs work? Is an asset manager showing the value of active management?
The more that the industry looks to data to help address plan-design or demographics issues, the more important it is for all involved to ensure that we access the right research and that the data be questioned where appropriate.