EOD Download/ Convert from NSE and BSE

Hi saji

Thanks for immediate feedback. I think exit button on file menu followed earlier logic for exit. It will not affect functioning except appearing as if programme was not willing to quit and didn't believe that a user would want to quit .I will make it behave properly to user command.

I am doing setup now on Inno setup.Some scripting is required which means testing the setup before release.

***** would not accept any exe files even if it is zipped. Therefore mail is bouncing. Thanks for your help.

I hope that this programme serves basic data acquisition requirements.As data is directly from BSE/NSE I believe that reliability question would not be there as in case of ***** and other data feed providers.

Pankaj

Hi skj

welcome to the forum. There are many downloaders available. one of them is satya's Data Extractor 2.0 currently available free of cost.It provides conversion into DHOLCV format. Few others use excel..You can visit Murthygaru's post to find all relevant links to softwares available this forum.
http://www.traderji.com/54514-post1.html

I have also tried my hands at this and you can download EODDownloadConvert ver1.0.5 .Go through previous posts and you would know what to be done, especially link mentioned below.

http://www.traderji.com/56018-post129.html

Pankaj

Hi Pankaj,
Just got some time so I decided to do the update from Download and Convert.

A few observations if I may.
The last time I used it was 5 days ago and I do not remember what files were downloaded or converted or to which directory.
1 Preferably on the first run let the settings be saved automatically and be restored at startup.
2 I tend not to use C:\Program Files\Download and Convert as a data directory as in the event of format,etc all data is lost. App must ask the user his choice of using any folder on any drive on first run and automatically use that folder as default on subsequent runs.
3 While converting, an error message keeps popping up "Error 53, File not found, Line 790...". The app waits for the user to press "OK" to continue.
Choice to turn off error messages reqd.
4 While d/l from NSE equity the app gives an error message "Unable to unzip", but it's a csv file. The error msg will continue 20 times if 20 files are queued.
5 The field sorting listboxes at the lower right can be done away with as Amibroker and MS can import data in ANY sequence as long as Ticker, Date, OHLC, V are provided in the csv.
If you want to give more choices, provide 4-5 most popular sequences you think are reqd in a listbox.
6 The app should remember where it last stopped during every run, either while d/l or converting and not depend on the user. User needs to be able to click download or convert and let it finish while he does something else.
7 Convert date to day so that the app knows Saturday/Sunday data is never available. Certain holidays with fixed dates can also be preprogrammed so that the app knows not to expect data on these dates. eg Independence day or Christmas
8 For 1st run the app will process some 200 or so files, the app must remember what it has finished and what remains in case the user stops before it has completely finished.

Please view this constructive feedback as a stepping stone to improve the app performance and usability and popularity. In case I can help further in any way please let me know.
Thanks for a great program, lets make it better!!

Hi pasha

Thanks for feedback.

I think you have not downloaded the patch-5 which has some of the problems answered.

1. Program saves the important settings on last run for an exchange.So next time same setting will be loaded.For downloading it may not matter much.

2.You want data directory different from program directory?If so , will look into that.

3. & 4. These problems may not occur in patch-5. If you want to switch on /off error messages , you can go to settings page in options menu and tick the box.

5. As settings are save, you may just click on convert button. However, if you list popular formats and requirements I would place it before others to decide.

6.Once you have finished your bulk download , it is a question of daily download.It would require two clicks -Add URLs and Download to get EOD data from both exchanges .I used to skip sat/sun but was advised against it as there is special trading on saturdays or sundays. People would not remember if there was trading on sat three years back.It would produce gaps in data.If file is not available application moves to next file quickly. As regards NSE holidays/BSE holidays users can enter dates through holiday page.

7.Conversion is done for available files.If file is not there the program would not do anything.once a file is converted,it is moved to another directory,so that only available files are converted .If you click stop button in the middle, the program would stop after the file under conversion is finished.It can not convert duplicates unless you provide same data to it.IF inadvertantly you have converted same dates twice it would not affect Date wise EOD as data would be same. For stockwise you would get some duplicate dates. You can run remove duplicates for one or all files. Just select the folder where your stockwise data files are stored.

8.This point is already covered above.

I am not writing anything in registry so that program installation is almost as simple as copying exe file or deleting it. Rest of the files are standard windows system files/IE files and mostly I ahve used functions which would make it compatible with Win98 and winxp.If program asks you to ignore abort or retry, you can safely choose ignore button and program would run.

I have assumed that user would like to have different choices. But if that is not so then I can make it function like two click program-Just download and convert.But then Satya's DE does that elegantly.

My ultimate aim is to put a charting with TA indicators with scripting and portfolio management with its own free source of data for Indian markets and make it available free of cost to others.But then that is a long way off.

Technologies are available but integration is required.Its like you have the cloth but need to get a suit stitched.Tailormade for You

Pankaj

Hi Pankaj,
Thanks for your answers.
My only aim is that it needs to be simple to use and intuitive.
Sorry if I have made any mistakes in use, I had very little time.

>My ultimate aim is to put a charting with TA indicators with scripting and portfolio management with its own free source of data for Indian markets and make it available free of cost to others.<
The charting is simple from VB, the indicators are a different story. I had written a charting program then abandoned it as I found AB was much faster and could do all that I wanted.
VB explorations can be written from within AB so there's no need to learn programming all over again.
I will d/l and use the patch and report again this evening.

Hi pasha

Thanks for your comments.

Of course one can purchase AB or Metastock and a commercial data feed and be done with that.No intention to rival that.VB6's capabilities are much more than what I have exploited so far.Infact proper charting needs a different approach in vb6 than the simple mscharts.Indicators are damn easy provided you know the scripting and programmed for it.

Unfortunately , its in the nature of mankind to keep reinventing wheels which leads to redundancies in the system. Any system which has redundancies built in tends to be stable.

With DE2.0 your job is actually done.There is nothing more to add to that beautiful programme, yet someone might like to experiment with a complex system.May be like wheels fitted with bearing balls on the outer surface. It will rotate but not budge an inch, theoretically at least.

DO give your feedback after use. rest assured all feedback goes into task list and appropriately attended to.

Pankaj

Hi All

EODDownloadConvert ver 1.0.6 i.e. patch-6 is released.

It is in the nature of maintenance release based on the feedback received.

Release Note for Patch-6


Rectified NSE data conversion issue.
Internet checking streamlined.
Converter form was made default.
yyyymmdd date format was made default.
EQ, BE or Both Checking introduced.

Those , who are using previous version, need to download only patch-6 and extract files in the program folder.

New users can download Setup files and Upgrade files. After running setup just extract upgrade files and place in the programme folder.Symbollist.txt file is to be placed in BIN folder. which will be present or created on first run.

Winxp users can select ignore button for error message
the end of the installation process :

An error occurred while registering the file "C:\WINDOWS\system32\shfolder.dll"
Win98 will not get this message.

Users can use Send mail facility to send error message at *EODDLC@*****.com


Those who want to be on mailing list may send message to the above address and they will be included in the mailing list for patch and upgrades.

Download Link for the Patch is
http://briefcase.rediff.com/bn/br/pj...atch-6.zip?d=0

For upgrade it is
http://briefcase.rediff.com/bn/br/pj...pdrade.zip?d=0

The output is compatible with metastock, FCharts and Fibotrader in default mode.

For AMIBroker, select "use 9 char Symbol(*****)" check boxes.

For other functions do read help file.

Do give your feedback as it helps in finding bugs and rectifying it.


Pankaj

Hi Pankaj,
A few problems while using it. I d/l patch 6 and copied the exe to the DC directory. While d/l last weeks files from BSE, got the message "Sorry! cannot unzip. go to temp and unzip manually".
This does not happen if I d/l only 1 file.

Since I wanted to test it from scratch, I uninstalled and reinstalled the program and copied the updated exe. While uninstalling, it asked whether I wanted to keep certain system files.
On 1st run the message came "This is the first run, creating files and folders ", or something like that. On clicking Ok, a message box appears "runtime error 76, Path not found" and on clicking ok, the app does not terminate.
It has to be shut with Ctrl-Alt-Del.

Deleted all files and folders and reinstalled, but it does not start at all now.
Gives the same message of "runtime error 76, Path not found".
VB6 runtimes are installed, I'm using Win Me.

My suggestion is not to use an installer. Advise users it will not work unless VB6 runtime is installed.
Let the app run as a standalone exe from anywhere and save settings in an ini in the same folder as the app. No need to waste effort on finding an installer is an added plus.

Hi pasha

May be this is problem specific to WIN Me as I have not tested on it. If app is not terminating, does it not work. Mostly path will get created during this process. For any runtime error application gets terminated is it can not recover. But obviously it appears to function. Patch-6 is meant to replace only the exe file. There is no need to do everything from the scratch as you would not get any additional files but only system files will get reinstalled. That you have already done once you installed. Installer is supposed to do that much only. If you get error message, better course would be to ignore as it would not copy files which are already in your system. That is mostly seems to be the problem.

So the best course would have been to copy the patch to the programme folder.

If possible follow the steps given below
Copy the patch to your application folder. Run by clicking on it. If it gives error 76 then check if it runs afterwards. Else check if the following folders are created or not.

"\DATA\BSE"
"\DATA\NSE"
"\DATA\Others"
"\DownloadedFiles"
"\BIN"
"\Indices"

"\Temp"
"\Help"




Which ever is missing Just create that folder manually.

Run the programme again. There should be not problem.

Thanks for giving feedback I will go through it and check what could be the problem and rectify it.

Pankaj

Hi All

Today while discussing with satya , he informed that EOD bhavcopies given by NSE contains only 554 scrips and it is partial.He downloaded dbf file and updated bhav copy and requested me to post it in forum for the benefit of members as He was too tired and going to sleep
So here is the file. members do update it.It has no header.
Pankaj