[imageframe lightbox=”no” style_type=”none” bordercolor=”” bordersize=”0px” borderradius=”0″ stylecolor=”” align=”none” link=”” linktarget=”_self” animation_type=”0″ animation_direction=”down” animation_speed=”0.1″ class=”” id=””] [/imageframe][fusion_text]
Have you ever seen a bad backtest?
[/fusion_text][fusion_text]Statistics lie. Well they can, at least. They don’t always. There is a common phrase, “There are lies, damn lies, and statistics.” The implication is clear. Statistics are not to be trusted. A few of us here at Adam Smith Advisors like to go one step further. We say, “There are lies, damn lies, statistics…and then there are backtests.”
Are you familiar with backtests? When someone comes up with a new investment idea, they go back and see how it would have performed historically. If it did poorly, you and I will never see it. They know it will never sell. But if it looks good, then it might end up in my inbox.
The problem is most backtests go back 5 or 10 years at the most. There’s a lot of noise in numbers from that short of a time period. Heck, it’s impossible to tell if someone got lazy and did a bunch of backtests to see what worked and then published the winner as fact. You don’t want to go down that rabbit hole.
There are some things that can be done to help you get me more comfortable with a new investment strategy. Investment theories can be tested back almost 100 years in the U.S. and decades internationally. Then once you create a fund, I like to see live results for 5-10 years before I draw any meaningful conclusions. Ideally I’d like to see 5+ years of live results and 30+ years of backtested data. That helps build confidence that there’s really something to this new idea.
The biggest area where I’m seeing a lot of backtests today is in factor/smart beta funds. I’m glad there’s so much research being done here. We’ve been invested in factors like small company stocks and low priced stocks for years. I’m sure there are improvements to be made, but it takes a long time to sort out the brilliant ideas from the sales garbage. New entrants need to show me at least 5 years before I take their backtest seriously. And then, they had better show dramatic improvement over what we’re using today.
The moral of the story is that it takes a lot longer than you’d think to prove a good investment strategy. Good strategies can fail for 5-10 years at a time and bad strategies can succeed for longer than that. I hope you like numbers, or have someone you trust to sort through them for you. I’m happy to help. [/fusion_text][separator style_type=”single” top_margin=”” bottom_margin=”” sep_color=”” icon=”” width=”” class=”” id=””][fusion_text]
Legal Disclaimer: These posts do not constitute an offer or recommendation to buy or sell any securities or instruments or to participate in any particular investment or trading strategy. They are for informational purposes only. ASA gathers its data from sources it considers reliable. However, ASA makes no express or implied warranties regarding the accuracy of this information or any opinions expressed by the author and may update or change them without prior notification.