Thoughts on "The A to Z of trading career - musings of a professional trader !!"

[madank]

Good discussions by @ncube and @mindgames. Din't i tell you that you guys are much more informed on backtesting parameters?

Anyways, let me try to illustrate what i wanted to say in simpler way. Lets take a Nifty system for example with 5 years of backtested results

System parameters

Year 1 - 1200 points made with 180 points max DD (in that year)
Year 2 – 1400 points made with 165 points max DD
Year 3 – 1000 points with 160 pts max DD
Year 4 – 1850 points with 200 points max DD
Year 5 – 1300 points with 160 points DD

So, average points for 5 years = 1350 and max DD for 5 years = 200 points. What we are interested in is the ratio of average points to max DD for that particular timeframe and it is 1350:200 = 6.75:1 in this example.

@mindgames – we are not considering CAGR here. Hope the above illustration made my point clear. Please let me know your question (if any)

To get 1400 points we need 400 * 3.5 = 1400, which implies the systems should generate 100 * 3.5 = 350 trades in a year.

As our trade risk is 10 points, we can have max 200/10 = 20 consecutive losses.

Hence this system will comply to the 1:7 requirement if it can generate 350 trades in a year and limit max consequtive losses to 20.

When a trader has a system that gives him only 100 trades per year, how do we make it to give 350 trades in a year? Am not able to understand the logic behind this statement. How do we extrapolate the trades from 100 to 350 in any given year? Can you please explain?

 

[ncube]

When a trader has a system that gives him only 100 trades per year, how do we make it to give 350 trades in a year? Am not able to understand the logic behind this statement. How do we extrapolate the trades from 100 to 350 in any given year? Can you please explain?

First I hypothesized a system model by assuming that it will generate 100 trades a year each with 10 risk points. Based on the results I reverse engineered the model to fit the expected Total Return: Max Draw-down ratio of 7:1 to find the total number of trades the system need to generate in a year.

Of course this is just a hypothetical case to prove that it is possible to have a system with Total Return:Max Draw-down ratio of 7:1 or better.

 

[ncube]

Good discussions by @ncube and @mindgames.
System parameters

Year 1 - 1200 points made with 180 points max DD (in that year)
Year 2 – 1400 points made with 165 points max DD
Year 3 – 1000 points with 160 pts max DD
Year 4 – 1850 points with 200 points max DD
Year 5 – 1300 points with 160 points DD

So, average points for 5 years = 1350 and max DD for 5 years = 200 points. What we are interested in is the ratio of average points to max DD for that particular timeframe and it is 1350:200 = 6.75:1 in this example.
?

This calculation will have a problem as we are using time series data the draw-downs can span across years. If we run Monte-Carlo simulation on this time series data sets the results will give higher draw-down possibilities. Since we have 5 year data its better to run the back-testing on entire 5 year data rather than year wise to get more realistic picture.

 

[Happy_Singh]

My friend, max draw-down is not a one time event, it is the worst case scenario (sequence of consecutive losses) for your system as observed in the historical test sample. This will help you make a rough estimation of how your system may perform in future.

I am not saying it should not be done but If you want to use lifetime CAGR return then better compare it with the lifetime average annual draw-down... so that the measurement metrics are at same scale.

Amibroker Backtest presents it as Recovery Factor (=Total Gains Made/Max DD)

Happy

 

[ncube]

On the back-testing topic, I will explain how we can use the machine learning concepts to develop efficient trading systems/models. Also I will share a trick we usually use to boost the overall performance when using multiple systems.

When developing machine learning analytical models we go through these standard processes which we can also apply when developing & testing Trading systems:

1. Data Collection: In this phase we collect the historical data needed for analysis, based on the time frame we wish to work we can collect data accordingly. If daily then we can try to get for about 10 yrs and for lower time frames the historical data could be smaller. However thumb rule is to collect as much data as possible.
2. Data Cleaning: In this phase we try to structure the data by removing outliners and interpolating missing data, in trading terms we may need to adjust splits, error values etc.
3. Feature Engineering: This is the crucial phase and I consider this to be the most important step in designing any system. What we do here is try to understand the domain, the impact of various parameters etc, i.e we try to see and apply our intuition to understand how the system will behave in various market conditions, what will be the impact of indicators values etc, the idea is to hypothesize how we believe the system will behave and develop our concepts by selected the right parameters for our system.
4. Identify Training & Test set: Since we are using time series data, we have limited options as we cannot shuffle data or use powerful cross validation processes. So the standard approach is to split the data in 70:30 ratio. I,e if you have 10 years data, split it into 1st 7 yrs for training and next 3 yrs for testing.
5. Build Analytical models/systems: Based on our understanding from Feature engineering phase we can develop multiple ideas and decide on the system & parameters to be used and also identify the parameters we wish to measure. I have explained in a earlier post the parameters we usually collect such as System expectancy, Draw-down, DD duration, signal frequency & total returns.
6. Model Training: We run all the system models we have identified on the 7 yrs training data set and record the measurement parameters for each of the system. If required we can fine tune the system parameters but need to ensure that we dont over-fit the model to training data. i.e the objective is not to make the parameters highly dependent of training data that it will not work on future data.
7. Model Testing: After training the models we can select the best performing models for testing on out of sample data (3 yr data) and again measure the result parameter values for each of the models.

If we are satisfied with the model results on test data, we can then decide to take it live by forward testing on real market with smaller sizes. Now if we have multiple good models to choose then in machine learning we use a concept called ensembling which helps to boost the overall performance. In ensembling we combine the output from all the system models and define a logic to decide whether to take the trade based on the signals generated by each of the model.

There are 2 methods either by voting or weight-age. From each of the models we use 2 key outputs, one is the trade signal (Buy,Sell,Neutral) and other total return generated by each of the models in test data and give weight-age to each model based on their total returns.

Lets take an example, if we have 3 system models M1, M2 & M3 and their total returns on testing data are as follows:

M1= 10000
M2= 11000
M3= 9000

Based on this we can assign following weight-age M2 > M1> M3

Usage: Now when we get a new trade signal on any of the 3 models we first check the status of signal on all the 3 models, for example M1 = Buy, M2 = Neutral, M3 = Buy, then we go with the majority vote in this case Buy, the benefit here is that this gives us additional confidence to take trades with higher risks. Now if we have even number of models or the signals contradict each other but we still want to take a trade with lower risks then we give priority to the signals from models with higher weight-age.

My apologies if the explanation is more technical I tried my best to keep it as simple as possible...

 

[mindgames]

@ncube i referred to CAGR/MDD because that is a popular stat. its logic (though not very intuitive) is not 18 years in the numerator --> 18 years in the denominator. it is more like "what is the risk you incurred / how much were you willing to lose at one point to generate that performance" over the time period. for the same reason, average DD is wrong.

@madank yes, this sounds good. ratio will definitely be better since we reset the returns computation for every year. also addresses the issue of random distribution of winners and losers.

is there an equivalent measure in Amibroker?

 

[ncube]

@ncube i referred to CAGR/MDD because that is a popular stat. its logic (though not very intuitive) is not 18 years in the numerator --> 18 years in the denominator. it is more like "what is the risk you incurred / how much were you willing to lose at one point to generate that performance" over the time period. for the same reason, average DD is wrong.

Ok My Friend I respect what you are saying as this would be based on your experience and must be correct...I am just a part time trader...

 

[mindgames]

Ok My Friend I respect what you are saying as this would be based on your experience and must be correct...I am just a part time trader...

not at all. i'm a noob. discussing helps to shape views.

cheers

 

[mindgames]

@madank i remember seeing a plan/list of topics you intend to cover but cant find it now. would you also cover something on the lines of - at what stage one must stop optimizing the backtests?

for instance --> let's say our system is a simple 20 MA crossover based. next, we use 'X' MA to filter long and short entries. then we slap another indicator on the chart......when does one stop and move on to work on other factors such as position sizing?

 

[Happy_Singh]

@madank yes, this sounds good. ratio will definitely be better since we reset the returns computation for every year. also addresses the issue of random distribution of winners and losers.

is there an equivalent measure in Amibroker?

Amibroker gives annualized % Return divided by Max. system % drawdown

Exposure % - 'Market exposure of the trading system calculated on bar by bar basis. Sum of bar exposures divided by number of bars. Single bar exposure is the value of open positions divided by portfolio equity.

Net Risk Adjusted Return % - Net profit % divided by Exposure %

Annual Return % - Compounded Annual Return % (CAR)

Risk Adjusted Return % - Annual return % divided by Exposure %

Avg. Profit/Loss, also known as Expectancy ($) - (Profit of winners + Loss of losers)/(number of trades), represents expected dollar gain/loss per trade

Avg. Profit/Loss %, also known as Expectancy (%) - '(% Profit of winners + % Loss of losers)/(number of trades), represents expected percent gain/loss per trade

Avg. Bars Held - sum of bars in trades / number of trades

Max. trade drawdown - The largest peak to valley decline experienced in any single trade. The lower the better

Max. trade % drawdown - The largest peak to valley percentage decline experienced in any single trade. The lower the better

Max. system drawdown - The largest peak to valley decline experienced in portfolio equity. The lower the better

Max. system % drawdown - The largest peak to valley percentage decline experienced in portfolio equity. The lower the better

Recovery Factor - Net profit divided by Max. system drawdown

CAR/MaxDD - Compound Annual % Return divided by Max. system % drawdown. Good if bigger than 2

RAR/MaxDD - Risk Adjusted Return divided by Max. system % drawdown. Good if bigger than 2.

Profit Factor - Profit of winners divided by loss of losers

Payoff Ratio - Ratio average win / average loss

Standard Error - Standard error measures chopiness of equity line. The lower the better.

Risk-Reward Ratio - Measure of the relation between the risk inherent in a trading the system compared to its potential gain. Higher is better. Calculated as slope of equity line (expected annual return) divided by its standard error.

Quoting above from help file most of the ratios we get on the back test . . .
but anything else we need can be easily coded for

Happy