Datafeed Vendor Comparison

TradeOptions

Well-Known Member
@CougarTrader and @bpr, please continue the discussion, and do not be discouraged by any comments. You guys have made very valid points and the current situation in India is not very good for retail traders who do Order Flow Analysis etc. and hence require proper tick data for that.

I am uploading the TICK Data from GFDL for yesterday trading session. This is from the tick data email that they send to the subscribers, who have to pay extra for this, other then the normal datafeed. You can see in these tick files that the NIFTY-I Symbol has got 21000+ trade ticks and BANKNIFTY-I has got 19000+ trade ticks respectively. And these are the 2 most active futures symbols that we have. All other symbols have lesser trade ticks compared to them.

I encourage you guys to use these files and do the comparison with the tick data of TrueData and any other such proper source of tick data, so that we can see who is providing what kind of data in the name of tick data.

Thanks and regards

https://www.dropbox.com/s/gtibpx19igqfto4/NIFTY-I.NFO.csv?dl=0
https://www.dropbox.com/s/3wjxprqt87kgcuo/BANKNIFTY-I.NFO.csv?dl=0
 

bpr

Well-Known Member
@CougarTrader and @bpr, please continue the discussion, and do not be discouraged by any comments. You guys have made very valid points and the current situation in India is not very good for retail traders who do Order Flow Analysis etc. and hence require proper tick data for that.

I am uploading the TICK Data from GFDL for yesterday trading session. This is from the tick data email that they send to the subscribers, who have to pay extra for this, other then the normal datafeed. You can see in these tick files that the NIFTY-I Symbol has got 21000+ trade ticks and BANKNIFTY-I has got 19000+ trade ticks respectively. And these are the 2 most active futures symbols that we have. All other symbols have lesser trade ticks compared to them.

I encourage you guys to use these files and do the comparison with the tick data of TrueData and any other such proper source of tick data, so that we can see who is providing what kind of data in the name of tick data.

Thanks and regards

https://www.dropbox.com/s/gtibpx19igqfto4/NIFTY-I.NFO.csv?dl=0
https://www.dropbox.com/s/3wjxprqt87kgcuo/BANKNIFTY-I.NFO.csv?dl=0
bro we did the comparison long back. I don't think anything has changed...it is not true TBT data

here are the truedata files for nifty & banknifty 01-FEB-2019

enjoy comparing
 

Attachments

TradeOptions

Well-Known Member
bro we did the comparison long back. I don't think anything has changed...it is not true TBT data

here are the truedata files for nifty & banknifty 01-FEB-2019

enjoy comparing
Actually I posted the TICK Data, whereas you posted the 1 min data. Please post the TICK Data files.

I am also posting the GFDL 1 min data for both symbols, in case some one wanna compare the Minute Data as well.

Regards
 

Attachments

bpr

Well-Known Member
Actually I posted the TICK Data, whereas you posted the 1 min data. Please post the TICK Data files.

I am also posting the GFDL 1 min data for both symbols, in case some one wanna compare the Minute Data as well.

Regards
my bad here is the tick files. there are some phantom ticks at the start that is a feature of truedata ignore it :D

Looks like GDFL has way more ticks. but does it contain all the ticks of truedata ??

Edit wrong data removed
 
Last edited:

yusi

Well-Known Member
A few observations:

@TradeOptions : The NIFTY-I data from GFDL has 21824 rows. In a full NSE trading day, there are 22500 seconds. Some rows, such as 10:11:01 and 10:13:05, are missing perhaps due to no change in top line data.

About 3820 rows have a LTQ of 0. So, if you ignore the Best Buy/Sell of the topline, there are 18000 rows of trade data. Data compression by 1 second and occurring trade should be that count.

Finally, this is EOD 1 second data. It is very likely that less than these will be received as broadcasts in your charting software.

The F&O bhavcopy does not contain the number of trades for NIFTY-I (unlike the equity bhavcopy), but if it is any indication RELIANCE had 214121 trades on 2019.02.01. That is the volume of ticks that a true TBT data file of that ticker should have (ignoring Best Buy/Sell topline changes).

Do not know which study @bpr is referring to, but a casual comparison of 2014 data for Odin and Trade Tiger is here.
 

yusi

Well-Known Member
Thanks, I get that but a bit confused among so many data providers in today's market. Should I go for TrueData or GDFL or with anyone else hiding in the hood? No need for TBT, only good accurate real-time data is what I am after. Please guide.
Will add my tuppence here: the best you can expect today, either from a 3rd party vendor or terminal based data, is compressed 1 second data. Here, for example, for NIFTY-I you can expect a difference of 18000 quotes to 15000 quotes. @traderniftybull was not joking when he asked if it really made a difference to you; the difference is a beholder perception.

A key difference is in the capture of day Open, and High/Lows (both day and minor). Terminal based captures tend to miss more of these. It is not that 3rd party vendors do not miss these; they do. I have seen posts (cannot trace back to the reference) where GDFL sends a corrective quote every periodic interval, say 1 minute, where the High/Low quote is corrected. Users have said that the chart changes but only for the immediate past. Do not know about TrueData. This would change the last minute bar values. For terminal capture data, this would not be possible without a backfill.

The dice roll is between cost and immediate accuracy.
 

TradeOptions

Well-Known Member
A few observations:

@TradeOptions : The NIFTY-I data from GFDL has 21824 rows. In a full NSE trading day, there are 22500 seconds. Some rows, such as 10:11:01 and 10:13:05, are missing perhaps due to no change in top line data.

About 3820 rows have a LTQ of 0. So, if you ignore the Best Buy/Sell of the topline, there are 18000 rows of trade data. Data compression by 1 second and occurring trade should be that count.

Finally, this is EOD 1 second data. It is very likely that less than these will be received as broadcasts in your charting software.

The F&O bhavcopy does not contain the number of trades for NIFTY-I (unlike the equity bhavcopy), but if it is any indication RELIANCE had 214121 trades on 2019.02.01. That is the volume of ticks that a true TBT data file of that ticker should have (ignoring Best Buy/Sell topline changes).

Do not know which study @bpr is referring to, but a casual comparison of 2014 data for Odin and Trade Tiger is here.
@yusi, thanks for sharing your observations.
 

rmike

Well-Known Member
The culprit(s) for prohibitive cost of authorized data are not the authorized data vendors, but the exchange(s) themselves!!!

The vendors, in fact, are worthy of empathy as they are trying to make ends meet in a very tough situation which is not of their own making!!!

Correctness of data is certainly one of the paramount factors contributory to quality of trading process!!!

BUT this is to be understood in the context of a huge caveat!!!

What process benefits from what aspect of correctness!!!

When a trade takes place (as per the matching algorithm appropriate for the prevalent session - auction OR normal) it generates a tick!!!

A single tick has the following associated markers

- Trade Timestamp
- Traded Price
- Volume (or number of contracts)
- Open Interest (as applicable)

The Bid & Ask for that particular trade time are supplementary markers - applicable for analytics specific to particular trading approaches!!!

Whether this tick will be counted as authorized (or not) will depend upon the filtering algorithm (the sampling algorithm pertains to transmission as per designated scale for client type - this in turn is governed by realities of input cost Vs return for required infrastructure or wherewithal)!!!

A collection of filtered ticks w.r.t a time period will have the following additional attributes

- Open
- High
- Low
& - Close

All other (mathematical) attributes (e.g average, median etc) are merely derivatives

Pay very very close attention to this

do we really need very very accurate data
Because it ought to be a cause for some serious introspection!!! That is, if you were, in fact, paying attention!!!

Apart from latency which would be a universal cause of detriment to ALL types of traders & trading approaches, what are the attributes which would have specific bearing upon your type of trading approach???!!!!

A query for which data is better can not really yield any response of value unless its qualified as pertaining to the type of trading approach & event horizon!!!

With as many approaches (at least, and more in vogue) as there are traders, is it reasonable to expect that what may suit Joe would also be kosher for Jane???!!!

What may suit you is going to be very very personal because it is YOU who is trading the markets as per a particular approach, with a particular strategy, with specific tactics for entry, exit, position sizing and scaling suited to that strategy and specific event horizon!!!

That realization can only dawn when YOU have thought out your trading process through & through!!!

YOU have to decide whether you want to make money with OR without TBT!!!

Just to make what is intended clearer with illustrative examples

- Did grandmothers of yore who made money simply by keeping score with yesterday's close prices from the newspaper and by hand drawn point & figure charts needed one minute (forget tick by tick :) ) data to do it??!!!

- Would an HFT arb firm be able to make do with 30 sec bars??!!!

- Does your system (or YOU) need TBT to differentiate between an uptrend and a downtrend???!!!

- Do YOU need TBT to realize that support & resistance are zones and NOT a singular point???!!!

- If YOU (or your system) trades on a 5 min bar, does it make any appreciable difference whether that bar contains 300 ticks or 900 ticks???!!!!

- Do YOU understand that nuances of period containers being marked by start - end time (different than that of the provided data timestamp) by your specific TA platform has a bearing upon showing 'apparently' different high/ low (or, say, maribozu :) ) for the same period compared candlestick/ bar???!!!

Think about it!!!

As rightly brought out by @yusi that, at times, mostly the high/ low values are the culprit

BUT do you know that except for a few handful, the majority of technical indicators are based on the close!!!

What @yusi kinda missed out on is that high & low are minor juveniles - It is actually volume that is the main offender!!! This is a problem not limited to a particular exchange but extends worldwide including exchanges of even the (so called) developed markets!!! - Correct info about true volume is very, very hard to get!!!!

There is absolutely no point in data comparison between, say, GFDL or TrueData as both are unlikely to have the same sampling and transmission policies (depending on specific wherewithal, mode of transmission and ensuing cost). Furthermore neither of them is an accredited benchmark for NSE data!!!

There is absolutely no point in data comparison between, say, NEST, NOW, Sharekhan and/ or Odin terminals as none of them is an accredited benchmark for NSE data!!!

These exercises are merely placebos to appease a vacant mind and to placate nebulous concern!!!

If at all data comparison has to yield any meaningful inference for RETAIL TRADERS, then its better to compare one minute data from your terminal/ vendor (authorized or otherwise) with TAME charts of NSE - Publically available for all - while accounting for TA platform containerizing nuances!!! NSE can be faulted for many things, but deliberately publishing incorrect data on its associated website is not one of them!!!

Even Yahoo or Google purchase data from the exchanges/ authorized vendors (or under special arrangement - there are NO free lunches :) ) and have nothing to gain (and only much to lose in respect of their reputation) by publishing (deliberately) erroneous data on their sites. These too can be good enough to trade on, specific to your trading approach or event horizon!!!

As brought out by @TradeOptions, if your trading approach utilizes order flow analysis then you will require specific data attributes, which would entail specific provision (however, this is not to imply that you will somehow be magically transformed into a winning trader by doing this :) )!!!!

As it generally happens in any endeavour specific to a particular pursuit, people end up with hallowed beliefs without subjecting them to rational scrutiny. The subject of 'Correct Data' is one such associated with the business of trading and is a favourite whipping boy OR a peg to hang the blame for failure(s) in becoming a successful trader by many!!!

I've said it earlier AND will say it again - The Perfect data is a Chimera that exists only in the innards of the exchange servers - that too at the moment of its creation (trade execution), thereafter it will no longer be perfect if only for no other reason than that of inevitable transmission latency!!!!

YES, the odds are stacked against the retail crowd - BUT nobody promised you an even playing field - The world does not go about its business in a fair manner!!!

A trader, rightly, ought to focus upon how to make money in spite of these limitations!!!

This post is only to aid 'food for thought' for making the suitable data call by YOU!!!
 
Last edited:

Similar threads