Dissecting the Nest Plus Auto Trading API

@Yusi

Was going through your helpful posts now. As I have mentioned elsewhere in TJ, the problems/questions/facts that face a wannabe auto trader using AMI-Nest combination are:

a) repainting of signals (as you mentioned) is problematic for people using gfdl based plugins using Immediate option because the order has been placed at first appearance and if the trend now reverses the signal won't be repeated, leaving an orphan trade.
b)painting of "rogue" signals, such as a buy after a buy, instead of a sell. I can hazard a guess that this is manageable by the afl code but have so far not received any concrete suggestion for a remedy.
c) plug ins/APIs/Bridges to make the OMS and the afl talk to each other in passing variables and parameters back and forth will greatly contain the damages a) and b), but they are not found, at least not easily. There are claims that are not borne out by a demo.
d) is the Nest + (trading plugin) a virtue or a vice? Two developers demonstrated remotely that by NOT using N+ and using a different system I could trade auto even in MCX. It beats me why brokers are stuck (may be they are helpless), and would like to get some illumination on this point.
e) issues that Nest Plus/GDFL has with Windows 8. The afl works fine but not the auto part, giving a COM error (which you mentioned).

Interestingly, one very reputed platform provider agreed that it was a risk that an afl produced an orphan trade due to a missed trade by sudden price movement but they demanded money to add this security protection, calling it a non-essential feature.

10. Simulation mode

There is a particular snag that keeps recurring in one form or another :

» One cannot keep going buy-a-Hindalco, sell-a-Hindalco and get much further. Testing with trades of low value sucks.
» There is undesirable larger risk in testing with a future lot. And a mini-Nifty is no more available.

But this goes much further, far too much has to be done live market. You have to wait for Monday morning to try out an idea. Bug fixes have to be done quickly while market-hours last. Errant trades have to reversed, and you often have to start all over again. Bread and butter trading is affected. Will leave it to others, who have gone live with auto-trading so far, to add to the list and potential trauma of code failing or unexpected trades / situations (not to mention losses).

So why not simulate auto-trade? Some of it should be possible. At the least, the extra buy/sells that occurred during the run of the basic auto-trade would not have caught me off-guard. As things mature, there will be a lot more code between if (AT) { ... } of the auto-trade AFL that will need checking before actually going live. As far as AmiBroker AFLs are concerned, it should be possible to test a significant part of this section. There would be some degree of 'live' paper trading possible.

The test harness attempts to do this with a simulation mode. Here, AutoTrade can be toggled to 'Yes'. As far as Ami is concerned, orders are placed, order numbers increase, etc.

10. Simulation mode ... [2]

To illustrate the simulation mode, the Button Trading code by Barry Scarbrough has been adapted for Plus (and simplified to suit). This, and similar AFLs, were possibly written when AmiBroker introduced the IB plug-in.

A sample screenshot shows this in use for NIFTY intraday trades of 21 Nov. The Properties window shows an additional toggle called 'Simulation'. If Auto trade is Running, and Simulation mode is Yes, orders are actually placed from the AFL, but not executed by the Plus API. When trading live, Auto trade would be Running, and Simulation mode would be No.



As you can see, all trades are winners. Hindsight is a wonderful thing.

The log shows something like:

11. Order Status

Back to basics... once an order is placed, the status needs knowing. This investigation cannot be done in simulation mode; it is back to buy-a-Hindalco, sell-a-Hindalco. As a precursor, what all of us would have observed in the Order Book window in terms of Status:

While there would be more, the key ones to handle seem to be Open, Complete, Trigger Pending, Cancelled, Rejected. To check these out, the following orders were placed as shown in Ami's log window, with CMP of Hindalco @ Rs 122.



~~ Orders 1 & 8 are beyond the circuit and expected to be rejected.
~~ 2 & 7 are typical limit buy / sell orders.
~~ 3 & 6 are typical sell / buy stop-loss orders
~~ Order 4 is a market buy placed as a limit. For some reason, market orders just don't work for me.
~~ 5 is a market sell placed as a limit, but as Nrml; expecting it to be rejected.

11. Order Status ... [2]

The Plus API's GetOrderStatus returns 3 values for a given order number -- status, fill quantity, and fill price. Checking the status of these orders gives:



No surprises there (ignore the time gap between status of order number 1 and the rest; got caught in some code). But certainly a couple of oddities...

For one, the status is all in lowercase. Totally unexpected !

The second is that, for orders that are not complete, the quantity / price in the status is zero. Non-zero values occur only for complete orders. It is an unknown what is returned for partial fills. Somehow, I was expecting that placed values would be returned and that an AFL would not have to keep track but could query -- this would have permitted a fall-back in case it was felt necessary to manually change quantity or price. But this is not a realistic expectation as the order status would have to return two price values in case of SL order.

Some manual order changes would work, and need to be taken into account. For example, Order No 3, the sell SL, was manually modifed to 128.50 and became an Open order. This would be a typical case where the price jumps over the stop.



As an ultimate order life-cycle closure, at EOD, typically near 16:00 hours for NSE/NFO, all open orders are cancelled.

11. Order Status ... [3]

Points to ponder:

► There is a small time lag between placing the order from the Api Generated Orders window (for the retail user), or even from the AFL (for other users). This time lag would be at least the ping time to the OMS / exchange, and it is reasonable to peg it between 0.5 to 1 sec as the norm. Cannot catch the status during this period; it is probably 'Validation pending'. Unsure what a status query during this period would be. This time lag cannot be eliminated for other operations, say Modify / Cancel orders -- there will always be that time gap of unknown, and the AFL must accommodate it.

► How often should one query the status in an AFL? Typical AmiBroker executions are every 1 second. If there are 30 to 50 open orders (Open + Trigger Pending), this will be costly. Essentially, will this solution scale?

► There are situations where the Status query returns an error. A situation that cannot be helped is where the order is not yet user confirmed from the Api Generated Orders window. No Nest/NOW order number has been generated in this case, and the query should be re-performed after some time. Another case, is where the order is rejected due to, say, a wrong parameter or an invalid user ID. In the latter case, no further query need be done. How does one distinguish between the two cases?

► As the day progress, NOW memory usage goes higher. Post lunch, it tends to pulse, and one cannot restart in this case. Since order placement and query are blocking calls, and are not parallel processed by Ami afaik, this is likely to affect auto-trading.

► Most would have encountered a situation where a Reconcile needs to be done. What status is returned if a reconcile is pending?

It still bugs that much of this is not documented, and has to be learnt the hard way. Do not mind the hard way, but the time is better spent on the algo than basics.

Re: 11. Order Status ... [3]

Dear Yusi,

Say I want to automate a SAR based strategy and getting a buy signal in 50 scripts simultaneously.
How long will take to place the orders successfully ?

(When a basket order file is created and uploaded it just takes a second or two to place them.)

TIA.

Hi All,

Am not actively trading nowadays and have lost touch with Excel-Ami-NestPlus..
Have an query . Can we execute trades i.e buy/sell from Excel to Now ?
I do not want to use external Data from anywhere ,nor do I want to use Amibroker , just do some calculation in excel when data is available in Excel via Now-Excel Linking ...

Regards,
Raju

Yes it is possible.

Re: 11. Order Status ... [3]

I would not hazard a guess -- my 30 order baskets have ranged from 2 to 8 seconds. But the gut says 2 to 2.5 times the basket time.

The analytical reasoning is to look at it from TCP's point of view. A 50 order basket would be one packet up, and 50 confirmations down. Whereas, the API version would be 50 orders up, and 50 down. We assume the the SYN/SYN-ACK handshake is not necessary, and also that order packet aggregation does not occur.

Much easier to run an experiment to get a ballpark figure. If required, let me know, can do it the next time I take this up.