Posted on

Quantopian: Fooled by Back Testing.

I have realized one thing. I can show all the proof and the data, but naysayers like you who simply don’t like the 50% drawdown will always come up with some excuse as to why all backtests are wrong. Regardless, since 2008 was a sustained VIX spike the algo did fabulously well. you may not want to believe it, but that’s your choice.

Trading Volatility

I understand the fellow’s frustration but I think there is another side to the story. I don’t happen to like this particular algo he is talking about but have spent several months devising my own to trade XIV and VXX. My back testing was done by backfilling the missing ETN prices to 2004 using futures prices.

In back testing the algo produces a CAGR of 98% with a max DD of around 70%. It has a single parameter.

I have been trading the algo now for a couple of months with small capital.

Unfortunately my experience has been over many years that whatever back testing says things will turn out very differently going forward.

I fully expect XIV to go bust at some stage or at least reach the 80% drawdown at which the sponsors are entitled or obliged to shut it down. I hope that my algo will have me out before then but I am not counting on it.

I will rebalance between the algo and cash at regular intervals but of course even this will lead to ruin if the algo goes bust several times.

Our greatest physicists are unable to predict the future. Will the universe end with a bang or a whimper? We do not know. Does time even exist? We do not know.

So “as to why all backtests are wrong”: they are not “wrong” as regards what they are competent to show (the past) but it is a brave man (or a foolish one) who believes they say much, if anything, about the future.

Trading forums are populated by an odd mix of characters. There are no-hopers looking for an easy way out of their dull jobs, competent (and incompetent) statisticians and software designers, people with maths skills, people with no apparent skills, successful “traders”, unsuccessful traders. At different times I have fitted some of these categories and certainly my success has gone and come and come and gone.

There is however no certainty. And that is what almost everyone here is missing.

The biggest fools of all are those who believe otherwise.

4 thoughts on “Quantopian: Fooled by Back Testing.

  1. There is literally nothing wrong with looking for the Holy Grail. What if you find it? Maximum risk => maximum reward.

    1. Indeed not; I have been engaged in that very process for years! However my favourite book is Candide by Voltaire. Like Candide I have decided that “cultivating my garden” is probably the only true version of the Holy Grail, or Eldorado.

      For me, cultivating my garden takes the form of extreme quiet and privacy combining daily physical exercise with mental stimulation. A hermetic, monastic existence.

      Currently I am using the market (since it is all I know) to explore machine learning. Not with any particular hope of profiting financially but more because I am fascinated by the nature of consciousness and convinced that in the end we will “know it all”.

      Regarding profitability I am still of the view that in order to profit from financial markets it is best to profit from some built in structural aspect rather than to try to predict. Medallion and the like appear to profit from the bid offer spread and front running among other techniques. VIX traders profit from contango (a version of the option writers premium but for the futures markets) which is a form of “earning” rather than “prediction”. Its about charging for a service if you like. Its offering insurance for a premium. Yes, there is of course prediction involved (critically) but the dice are loaded in your favour.

      None of this is putting me off seeking the “answer”. Perhaps the Philosopher’s Stone really does exist.

      After all we now know the alchemists of old were right all along: ultimately everything is made from the same material so we can, in theory, turn base material into gold.

      1. Quite related, at least the first part.

  2. Hedonic Treadmill -very good , thanks

Leave a Reply