John, the big question is does ML have any place in making predictions in the financial markets. You mention “automated trade management” but that assumes you have some sort of system to choose the stocks in the first place.
As to “optimisation” all of ML seems to have that object at its heart. But is the prediction of financial instrument pricing a suitable topic for ML? If not, Keras et al is a waste of time.
As to “time series data” it presumably boils down to the independence or otherwise of the price movements of financial instruments. Determinism or randomness. Some claim they are a random walk others that patterns such as trends or mean reversion are to be found.
As a laymen in the world of ML (but an interested one) it seems to me so far that ML’s great successes have come in deterministic systems. The rules of chess, Go, cards, poker. Where the probabilities can be accurately assesses and odds forecasted.
Although some improvement seems to have been made in predicting chaotic systems such as the weather over the last 50 years would it be accurate to say that ML is better at predicting what is than what may be? Recommender systems on Netflix and Amazon rely on the fact that while people’s taste can change in general it doesn’t and the same people will carry on liking the same books and films.
Reinforcement learning to fly a toy helicopter seems to have been a great success but again I assume that there may be at least a partly deterministic environment involved with the exception of the wind; Newtonian laws presumably dominate.
So, out of the vast plethora of machine learning algos out there from Baysian classification to neural networks to genetic algorithms most of them actually, broadly, aim to do the same thing. And so the question remains that while ML may be of use in image recognition and astronomical classification is it actually of much use in predicting the behaviour of an apparently chaotic system such as the stock market?
Or is the stock market perhaps NOT random and chaotic? Certainly not in the long term – it is merely a reflection of increased wealth from 300 years of technological revolution.
But in particular have you personally had any success in generating alpha from using ML? And if so, what approach and class of ML algo do you prefer?