Is my backtest cheating?

#1
Hello all. I was hoping that I could get some opinions on an AFL that I tweaked from one I found on the forums. Is this a data snooping bias? In other words, is it triggering trades on information in the future that it should not yet know? I'm backtesting it and getting so-so results, but before I build on it, I just wanted to make sure. "a" and "b" are just simple crossover criteria not shown here.

---

Buy = a AND (H!=O) AND TimeNum() > 093000 AND TimeNum() < 150000;
Sell = b AND (L!=O) OR TimeNum() >= 155000 AND TimeNum() <= 160000;

BuyPrice = ValueWhen(Buy,O,1);
SellPrice = ValueWhen(Sell,O,1);

Buy = Ref(Buy,-1);
Sell = Ref(Sell,-1);

---

Thank you so much.
 

augubhai

Well-Known Member
#2
Hello all. I was hoping that I could get some opinions on an AFL that I tweaked from one I found on the forums. Is this a data snooping bias? In other words, is it triggering trades on information in the future that it should not yet know? I'm backtesting it and getting so-so results, but before I build on it, I just wanted to make sure. "a" and "b" are just simple crossover criteria not shown here.

---

Buy = a AND (H!=O) AND TimeNum() > 093000 AND TimeNum() < 150000;
Sell = b AND (L!=O) OR TimeNum() >= 155000 AND TimeNum() <= 160000;

BuyPrice = ValueWhen(Buy,O,1);
SellPrice = ValueWhen(Sell,O,1);

Buy = Ref(Buy,-1);
Sell = Ref(Sell,-1);

---

Thank you so much.
Not cheating. It is only looking at data from the past and present. :thumb:
 
#3
Not cheating. It is only looking at data from the past and present. :thumb:
Thanks very much! I clicked Thanks on your post.

That's a relief. Could you clarify some things for me to make sure I am understanding what I'm testing here?

a = crossover formula in the works

Buy = a

BuyPrice = ValueWhen(Buy,O,1);
So, this is looking at the price when the most recent occurrence of when the previously defined Buy signal occurred, right? By setting BuyPrice to this, what is it doing? I guess I don't fully understand BuyPrice's functionality.

Buy = Ref(Buy,-1);
Is this buying the stock at the open of the next bar after a Buy signal triggered?

Thanks again for your help!
 

augubhai

Well-Known Member
#4
Thanks very much! I clicked Thanks on your post.

That's a relief. Could you clarify some things for me to make sure I am understanding what I'm testing here?

a = crossover formula in the works

Buy = a

BuyPrice = ValueWhen(Buy,O,1);
So, this is looking at the price when the most recent occurrence of when the previously defined Buy signal occurred, right? By setting BuyPrice to this, what is it doing? I guess I don't fully understand BuyPrice's functionality.

Buy = Ref(Buy,-1);
Is this buying the stock at the open of the next bar after a Buy signal triggered?

Thanks again for your help!
Sorry, I should have looked at the code more carefully...
Buy = Ref(Buy,-1);
This line is actually shifting the Buy signal to the previous bar... so it is cheating.

Sorry about misleading u :eek:
 
#5
Sorry, I should have looked at the code more carefully...
Buy = Ref(Buy,-1);
This line is actually shifting the Buy signal to the previous bar... so it is cheating.

Sorry about misleading u :eek:
Should the code by Buy = Ref (Buy, 1) if I want to buy at the open of the next bar?
 

augubhai

Well-Known Member
#6
Should the code by Buy = Ref (Buy, 1) if I want to buy at the open of the next bar?
Better to define a new variable to avoid confusion. You could maybe call the new variable as BuySignal. So then

BuySignal = a AND (H!=O) AND TimeNum() > 093000 AND TimeNum() < 150000;
Buy = Ref(BuySignal,-1);

BuyPrice = ValueWhen(Buy,O,1); // this line should appear only after u have defined Buy, else u will get wrong values.
 
#7
Better to define a new variable to avoid confusion. You could maybe call the new variable as BuySignal. So then

BuySignal = a AND (H!=O) AND TimeNum() > 093000 AND TimeNum() < 150000;
Buy = Ref(BuySignal,-1);

BuyPrice = ValueWhen(Buy,O,1); // this line should appear only after u have defined Buy, else u will get wrong values.
Thanks. So even though we have the following, it is not cheating and buying 1 bar ago?

Buy = Ref(BuySignal,-1);
 
#8
Buy = Ref(BuySignal,-1);
Use Trade delays instead. Check Code check & Profile to find future leak. Otherwise your system be robust enough to trade....