Should I Pay Off My Mortgage Balance Early? - Rodgers & Associates

Should I Pay Off My Mortgage Balance Early?

One decision that many people nearing retirement or recently retired face is the question: Should I pay off the balance on my mortgage? While few actually like having a mortgage, retirees generally fall into one of two camps: people who are okay with having a mortgage and people who aren’t. While there are many economic reasons for either keeping or paying off your mortgage, there are plenty of emotional reasons as well which may be equally as important as the monetary reasons. Just in case you’re still on the fence, I’ll try to lay out the case from both sides. As you’ll see, there are very logical arguments on both sides of this debate.

Tax DeductionI can take a deduction each year for the interest portion of my payment.I don’t want to pay 100% of the interest only to get roughly a quarter of it back on my taxes. I’d rather not pay interest and just keep what I would have paid in my pocket.

While the mortgage interest deduction is a valuable deduction for many taxpayers, it’s important to analyze exactly how much of a benefit you’re actually getting from it. For retirees, you’ll want to consider how much of your payments are actually interest as opposed to principal repay­ments. If you’re 25 years into a 30 year mortgage, there’s a decent chance you’re not getting that much of a benefit from it. However, if you upgraded to a new home during your peak earning years and you’re in year three or four of a 30 year mortgage, a big part of your payments are still going to be interest. Additionally, the standard deduction for 2015 is $12,600 and if your total itemized deduc­tions (which include real estate tax, state income or sales tax, chari­table donations, medical expenses, miscel­la­neous deduc­tions, and of course, mortgage interest) are less than $12,600, you’re not getting any benefit from the mortgage interest.

Investment ReturnsIf my mortgage is 4% and my portfo­lio’s growing at more than 4%, I’m better off investing the money rather than paying off my mortgage.I don’t want to risk a guaranteed oblig­ation (i.e. a mortgage payment) for the chance that my portfolio might be able to earn more than the interest rate on my mortgage.

With rates as low as they are, there is some sense to the pro-mortgage argument, with a few large caveats. There have been long periods in history where paying your mortgage off has been a better return on your money than the average return for a balanced portfolio of stocks and bonds (think the 1970s and 80s)[1]. There have also been long periods in history when investing in a mix of stocks and bonds would have given you a better return (think 1950 when average rates averaged 4.08%). While there’s no way to say with any certainty that stock prices will outperform mortgage rates over the next 30 year period, you’ve certainly got history on your side that they will[2].

The other large caveat that I’d add here is that if having the mortgage adds a level of discomfort that would make you more inclined to sell at the bottom of a bear market, it’s not worth it. In other words, this is a strategy that has the potential to work but only if you stick with it and only if stock returns continue to be above mortgage rates.

Flexi­bilityI don’t want to exchange investment assets that can be easily sold for cash, for an asset that I can only access by either selling my house or taking on a new debt.Having a monthly payment and a balance hanging over my head is an uncom­fortable feeling, partic­u­larly now that I don’t have a paycheck anymore.

When I was growing up I would play Monopoly against my Dad and my brother. My Dad would always win. My brother and I begged him to tell us his secret and he reluc­tantly explained that his secret to winning was making sure he had the right mix of cash he could spend and real estate. The same principle holds true for retirees. You need to carefully consider how much of your money is going to be in fixed assets and how much of your money can be quickly converted to cash you can spend. Once you pay down your mortgage, you can’t get that money back without selling your house or taking on a new loan.

Taking on a new loan as a retiree isn’t impos­sible, but it’s a bit more challenging than it was when you had a paycheck. I asked Elaine Leech of Caliber Home Loans in Lancaster, PA about the compli­ca­tions of getting a mortgage during retirement. She said, “Just like when you’re working, quali­fying for a mortgage during retirement is still based on your income and your collateral. Unlike when you’re working, we’re now looking at social security, pensions, and the ability to take an income stream from retirement accounts.” She encourages her clients approaching retirement to try to qualify for a home equity line of credit while they still have a paycheck.

The question of whether it makes sense to pay off a mortgage is also dependent on what the mix of your account types looks like (IRAs, 401(k)s, Roth IRAs, or non-retirement accounts). For example, if all of your money is in IRA or 401(k) accounts, you’ll have to pay ordinary tax rates to take money out of those accounts which is a pretty costly way to access money. For someone in a 25% tax bracket to pay off a $100,000 mortgage, you’d have to take out about $134,000 to get $100,000 after taxes. Additionally, this large withdrawal could end up putting you into a higher tax bracket.

Another key consid­er­ation is how much you’d be left with in savings after paying off the mortgage. If you have a portfolio of $1,000,000 and you’re consid­ering paying off a $200,000 mortgage you’ll need to consider if $800,000 is still going to be enough for you to live the life you want. If you can still fund your lifestyle and it makes you feel better not having a mortgage, pay it off. While that might not be the way to get the absolute highest rate of return on your money, if you can still accom­plish your goal and it helps you sleep better at night, it’s probably worth it.

As you can see, there are definite merits to both schools of thought. The ultimate decision is often just as much an emotional one as it is a financial one. While the emotional aspect of this decision is in your court, it’s important to consider the financial side as well. Working with a financial planner who can consider the emotional, tax, investment, and estate planning aspects of the decision will make you a more informed consumer.

[1] https://​research​.stlou​isfed​.org/​f​r​e​d​2​/​g​r​a​p​h​/​?g=WYb