General Trading Chat

Sorry didn't read your response....
Anyways as I said, I am using KNN on 7-8 parameters like oil prices, other stock exchanges, dollar values and learning over the pattern for last 45 days....
I am building an inverse covarince matrix - based on the distances that I get for the stock with global parameters...
That is basic AI knowledge- but trick was to do it realtime....there is not much rocket science in this...
And then I use Sharpe Ratios and some indices as weights to the stocks - and look for its tendency to go up....
So for example - my prediction for NIFTY says that it has the tendency to go to 11691.
Not it might start from there after 9:15 or reach there after 12.
But overall it has a tendency to fall as well - but overall seems to be stabalizing in 11691
 
Sorry didn't read your response....
Anyways as I said, I am using KNN on 7-8 parameters like oil prices, other stock exchanges, dollar values and learning over the pattern for last 45 days....
I am building an inverse covarince matrix - based on the distances that I get for the stock with global parameters...
That is basic AI knowledge- but trick was to do it realtime....there is not much rocket science in this...
And then I use Sharpe Ratios and some indices as weights to the stocks - and look for its tendency to go up....
Oh wow !!! This is something new. So far we have not had anyone use AI for predicting prices. At least I don't remember anything in this regard. But we have experts from various fields looking in here. I hope someone takes up a discussion on this topic with you. I certainly I am zero where KNN and inverse covarince matrix are concerned :)
 

Riskyman

Well-Known Member
Oh wow !!! This is something new. So far we have not had anyone use AI for predicting prices. At least I don't remember anything in this regard. But we have experts from various fields looking in here. I hope someone takes up a discussion on this topic with you. I certainly I am zero where KNN and inverse covarince matrix are concerned :)
I have used KNN in the past. The biggest problem was not the predictive outcome but the amount of past data to be sampled. I used various data sets. Eventually, got fed up and decided to use god's gift of grey matter.
Happy with the decision.

Btw, KNN is great if you are a geek and have patience to back test multiple data sets/parameters etc etc.
Should give it a try.

Edit: You can find/buy some KNN algos written for MT 4/5 if you are keen. Some of them are pretty good.
 
By the way just a clarification - in my monsoon predictions I use Outgoing LongRange Predictions and this time - there was weather modification happened during April and May months - so my predictors are catching the bias...So predictions for monsoon made in June is the fourth column which is marked MostProbable - which averages to remove bias in the system....

Just clarifying as I don't want to confuse the readers as to why June predictions show good monsoon. Actual prediction is the most probable case. This scenario never happened before - but thanks to these modifications not many people died during the two months due to heat-stroke...When it stopped heat stroke happened....So please I don't want to give wrong science - please follow the last column against each state...

If okay, I will the top ten stocks tomorrow before market opens just to help test the models. Then I could go into the details...
 

DanPSup

Hedge Strategy Trader in Options and Futures
Oh wow !!! This is something new. So far we have not had anyone use AI for predicting prices. At least I don't remember anything in this regard. But we have experts from various fields looking in here. I hope someone takes up a discussion on this topic with you. I certainly I am zero where KNN and inverse covarince matrix are concerned :)
Yea, me too. :)) But Stdv or any calculating AFl which is officially available on any what ever math calculation could help about it to give a better shot about it and clear all that stuff and secrectly help about it on any TF. Any way: It is just an other way of many how trading can be done.

So what: Our strategy have to fix and our believes and works proved have to work with it and this is it. :))
 

ncube

Well-Known Member
By the way just a clarification - in my monsoon predictions I use Outgoing LongRange Predictions and this time - there was weather modification happened during April and May months - so my predictors are catching the bias...So predictions for monsoon made in June is the fourth column which is marked MostProbable - which averages to remove bias in the system....

Just clarifying as I don't want to confuse the readers as to why June predictions show good monsoon. Actual prediction is the most probable case. This scenario never happened before - but thanks to these modifications not many people died during the two months due to heat-stroke...When it stopped heat stroke happened....So please I don't want to give wrong science - please follow the last column against each state...

If okay, I will the top ten stocks tomorrow before market opens just to help test the models. Then I could go into the details...
@Arindam Bhattacharjee Good to see you are using Data analytics to make predictive analysis. This is an upcoming field and we are already part of AI based ecosystem and will see lot more in near future.

Few comments based on the information you have shared and for understanding of others:
1. Any particular reason you have used KNN (K-Nearest Neighbor) Algorithm? As this algorithm is more suitable to solve classification problems. For example, in predictive recommendation systems such as those used in online portals (Amazon, Flipkart etc) where you see "you may also like this" kind of recommendations.
2. I presume you are using this algorithm to 1st predict only the probable direction the stock will move based on the 6-7 parameters and then use some regression algorithm to measure the probable extent of that move?
3. Just a suggestion, you may explore using simple ensemble models of Random Forrest along with linear regression algorithm. I have observed they give better results.
4. These ML models just provide static analysis, you can try making it more interesting & efficient by using Reinforcement learning (Deep learning) techniques on top to make it a true AI application which will learn dynamically from its mistakes and successes and take intelligent calls (Similar to how computers play Chess/Games these days and there are lot of research papers available on this topic).

However word of caution based on my experience:
1. Strategies based on these type of predictive models will work best if they are fully automated else if human intervention is involved, he will most likely second guess the trade signals when the strategy enters draw-down period. Hence the models should be clear to understand and not completely black box so that one has confidence in using it.
2. Each ML model is a strategy based on machine learned rules, it does not mean it will not fail in future. It is just like we running one of our successful strategies mechanically, but there will be periods/conditions after which the strategies stop working and we many need to make changes to it. Similarly ML based models/strategies also require regular monitoring/refining/optimization.
3. The edge in these types of models are very small, i.e it cannot scale well and if it is posted on the open forum the edge will be lost much faster. I remember testing a strategy mentioned by @UberMachine in his automation thread and it works but the edge is very small and it cannot scale up well if many trade it and overhead costs are included.
4. Hence best way to apply machine learning based models is to diversify it, i.e select multiple strategies (5-10) which are not co-related and run them all in parallel with proper MM & Risk control.

ML based strategies works but be clear that it will not make you rich overnight and the results will be inline with index returns but with some additional alpha based on the model's edge used in these strategies.

Frankly at the moment I think it is more suitable for large fund management running hundreds of strategies in parallel, for poor retailers like us using our normal cognitive intelligence to trade mechanically should suffice...:) Also no fun being on the sideline and watch computers having all the excitement right?;)
 
Last edited:
I understand one thing...this market is not run as free or open market and usually run by sentiments...It is also dependent on how few traders trade.
And this is why I build KNN algorithms as these sentiments, if global can be seen in the behavior of other indices. So you are right - I have automated the process of collecting historical data to find the distances - an ofcourse run regression equations on the same.

I was also thinking of using Non-Homogeneous Markov Models to build regression equations - but there are too many complicated factors that can effect the outcome. And usually today, its not about how the company is actually doing but what people feel about this. This is why my algorithm is not building on stock factors mainly but on how humans trade....

For example, as a test I am putting what my model is pumping for today of the top 10 stocks. You can see I have given them ranks based on factors other than ML like bias, sharpe ratio, how it has been performing for years etc, Market Capital etc.

And today NIFTY looks to have a run...It might cross 11750 if it has a run....sorry I can't reduce the image size on phone...This is for model testing....
Thanks

1563334674657.png
 
Last edited by a moderator:
Thanks @ncube , as I said earlier, we have all kinds of experts participating here. Now I hope someone comes up with an image editor for mobile phone :)

I have edited and resized the image in @Arindam Bhattacharjee 's post.
@Arindam Bhattacharjee Good to see you are using Data analytics to make predictive analysis. This is an upcoming field and we are already part of AI based ecosystem and will see lot more in near future.

Few comments based on the information you have shared and for understanding of others:
1. Any particular reason you have used KNN (K-Nearest Neighbor) Algorithm? As this algorithm is more suitable to solve classification problems. For example, in predictive recommendation systems such as those used in online portals (Amazon, Flipkart etc) where you see "you may also like this" kind of recommendations.
2. I presume you are using this algorithm to 1st predict only the probable direction the stock will move based on the 6-7 parameters and then use some regression algorithm to measure the probable extent of that move?
3. Just a suggestion, you may explore using simple ensemble models of Random Forrest along with linear regression algorithm. I have observed they give better results.
4. These ML models just provide static analysis, you can try making it more interesting & efficient by using Reinforcement learning (Deep learning) techniques on top to make it a true AI application which will learn dynamically from its mistakes and successes and take intelligent calls (Similar to how computers play Chess/Games these days and there are lot of research papers available on this topic).

However word of caution based on my experience:
1. Strategies based on these type of predictive models will work best if they are fully automated else if human intervention is involved, he will most likely second guess the trade signals when the strategy enters draw-down period. Hence the models should be clear to understand and not completely black box so that one has confidence in using it.
2. Each ML model is a strategy based on machine learned rules, it does not mean it will not fail in future. It is just like we running one of our successful strategies mechanically, but there will be periods/conditions after which the strategies stop working and we many need to make changes to it. Similarly ML based models/strategies also require regular monitoring/refining/optimization.
3. The edge in these types of models are very small, i.e it cannot scale well and if it is posted on the open forum the edge will be lost much faster. I remember testing a strategy mentioned by @UberMachine in his automation thread and it works but the edge is very small and it cannot scale up well if many trade it and overhead costs are included.
4. Hence best way to apply machine learning based models is to diversify it, i.e select multiple strategies (5-10) which are not co-related and run them all in parallel with proper MM & Risk control.

ML based strategies works but be clear that it will not make you rich overnight and the results will be inline with index returns but with some additional alpha based on the model's edge used in these strategies.

Frankly at the moment I think it is more suitable for large fund management running hundreds of strategies in parallel, for poor retailers like us using our normal cognitive intelligence to trade mechanically should suffice...:) Also no fun being on the sideline and watch computers having all the excitement right?;)
 

iwillwin

Well-Known Member
Sorry didn't read your response....
Anyways as I said, I am using KNN on 7-8 parameters like oil prices, other stock exchanges, dollar values and learning over the pattern for last 45 days....
I am building an inverse covarince matrix - based on the distances that I get for the stock with global parameters...
That is basic AI knowledge- but trick was to do it realtime....there is not much rocket science in this...
And then I use Sharpe Ratios and some indices as weights to the stocks - and look for its tendency to go up....
How does it predict random events like Trump tweets....does it also read human psychology
 
As I said, I am doing this exercise with two intentions...

First - I work on something called smart village. You might have seen Paani Foundations work. So I want to make a trading exchange in the future for Climate Change that impacts changing human behavior - and trading in commodities.

Second - Since Raghuram Rajan has also spoken against these practices of going to debt market. I understand there - they you use complete algorithmic trading and drive market sentiments for their own benefits....There is a fear that like demonitization - these markets would be used to steal money.
But if Indian small-time traders are smart - and can use AI in combination with their brains - I guess this uneven playing grounds could be levelled and we might truely have free market economics....

My training is as Climate Change economists.....
Yes, Trump's tweets are reflected in many stupid trades made....so their psychology is in the data and the distances take care of stocks behavior to such stupid tweets.
But my intention is - if individuals are capable of making good decisions - then Trump might not dare.

I follow the work of CD Deshmukh who has dreamt of market economics without the greed of wall street based on understanding Indian civilization - I am trying to implement this....

Also as I said, market will begin at 11691 - it did reach 11683 in the first 15 minutes of trade. Now if it has a tendency to go up, it might reach 11750 today.

@Arindam Bhattacharjee Good to see you are using Data analytics to make predictive analysis. This is an upcoming field and we are already part of AI based ecosystem and will see lot more in near future.

Few comments based on the information you have shared and for understanding of others:
1. Any particular reason you have used KNN (K-Nearest Neighbor) Algorithm? As this algorithm is more suitable to solve classification problems. For example, in predictive recommendation systems such as those used in online portals (Amazon, Flipkart etc) where you see "you may also like this" kind of recommendations.
2. I presume you are using this algorithm to 1st predict only the probable direction the stock will move based on the 6-7 parameters and then use some regression algorithm to measure the probable extent of that move?
3. Just a suggestion, you may explore using simple ensemble models of Random Forrest along with linear regression algorithm. I have observed they give better results.
4. These ML models just provide static analysis, you can try making it more interesting & efficient by using Reinforcement learning (Deep learning) techniques on top to make it a true AI application which will learn dynamically from its mistakes and successes and take intelligent calls (Similar to how computers play Chess/Games these days and there are lot of research papers available on this topic).

However word of caution based on my experience:
1. Strategies based on these type of predictive models will work best if they are fully automated else if human intervention is involved, he will most likely second guess the trade signals when the strategy enters draw-down period. Hence the models should be clear to understand and not completely black box so that one has confidence in using it.
2. Each ML model is a strategy based on machine learned rules, it does not mean it will not fail in future. It is just like we running one of our successful strategies mechanically, but there will be periods/conditions after which the strategies stop working and we many need to make changes to it. Similarly ML based models/strategies also require regular monitoring/refining/optimization.
3. The edge in these types of models are very small, i.e it cannot scale well and if it is posted on the open forum the edge will be lost much faster. I remember testing a strategy mentioned by @UberMachine in his automation thread and it works but the edge is very small and it cannot scale up well if many trade it and overhead costs are included.
4. Hence best way to apply machine learning based models is to diversify it, i.e select multiple strategies (5-10) which are not co-related and run them all in parallel with proper MM & Risk control.

ML based strategies works but be clear that it will not make you rich overnight and the results will be inline with index returns but with some additional alpha based on the model's edge used in these strategies.

Frankly at the moment I think it is more suitable for large fund management running hundreds of strategies in parallel, for poor retailers like us using our normal cognitive intelligence to trade mechanically should suffice...:) Also no fun being on the sideline and watch computers having all the excitement right?;)
 

Similar threads