How to delete intraday data quotes ?

#1
Hello Amibroker Experts,

To keep the data size just to "what is required" so that Amibroker loads faster; I would like to remove intraday data quotes from all symbols in one go while keeping the EOD data intact.

Is there a way out other than going through each symbol in the quotes editor and deleting them manually ?

Thanks !
 

KAL.YUG

Well-Known Member
#3
.

Most Important : Back Up your Original Data First

I wish to do the same. Anyone any help?
Code:
Filter=1;
AddColumn(O,"Open");
AddColumn(H,"High");
AddColumn(L,"Low");
AddColumn(C,"Close");
AddColumn(V,"Volume",1.0);
AddColumn(OI,"Open Interest",1.0);

Save/Use as explorer/Analysis.
In Analysis > Settings select periodicity to Daily
Select symbols that need to be converted, select "Range" - 'All Quotes' and hit explore.
Save -- File > Export HTML/CSV and save as csv.

(Remember : Back Up your Original Data)
Delete IEOD symbols that you needed conversion.
Use the saved EOD csv file to import back (Import Wizard)

:)
 
Last edited:
#4
If one has EOD data for the scrips, then a method is
- backup original data
- open a new database folder and upload/import EOD data and save it
- delete previous database folder

but do not use this method, if you do not have EOD data files.
 

KAL.YUG

Well-Known Member
#5
If one has EOD data for the scrips, then a method is
- backup original data
- open a new database folder and upload/import EOD data and save it
- delete previous database folder

but do not use this method, if you do not have EOD data files.

Yes .. if you intend to convert all symbols to EOD one can back up database, delete the IEOD all data from that folderand import the new EOD csv.

Make sure you also save the default (or any other) layout/template...just in case.
 
Last edited:

cloudTrader

Well-Known Member
#7
Thank you very much friends, but these processes are bit manual and cumbersome at my end since I might be required to perform it Daily to keep my Database clean. Any other way? I am searching too...
Josh1 utility helps to delete ticks from symbols automatically for current day or selected period. You can run the utility after trading hours and delete ticks with the help of that utility. So your work will be just to run the utility to keep your database clean.
 
#8
Thank you very much friends, but these processes are bit manual and cumbersome at my end since I might be required to perform it Daily to keep my Database clean. Any other way? I am searching too...
Then you need to automate the Task. Before running the below scripts, backup of the database is necessary to avoid any unexpected data loss.

Here List 0.tls is the name of the watchlist and DB happens to be the name of the database. "filespec" and "DataDir" variables needs to be specified accordingly. Also "DeleteFrom" and "DeleteTo" need to specified in the same format as written.

If you want to remove intraday quotes but keep EOD, then Run this JavaScript after saving it as Anything.js from anywhere.
JavaScript:
Date.prototype.yyyymmddhhmmss = function()
{
     //Modified the code from https://www.yijunma.com/tool/code-snippet/format-date-yyyymmdd-hhmmss/
     var mm = this.getMonth() + 1; // getMonth() is zero-based
     var dd = this.getDate();
     var hh = this.getHours();
     var mi = this.getMinutes();
     var ss = this.getSeconds();
     var yyyymmdd = [ this.getFullYear(), ( mm > 9 ? '' : '0' ) + mm, (dd > 9 ? '' : '0') + dd ].join('');
     var hhmmss = [ ( hh > 9 ? '' : '0' ) + hh, ( mi > 9 ? '' : '0' ) + mi, ( ss > 9 ? '' : '0' ) + ss, ].join('');
     return yyyymmdd + '' + hhmmss;
};

filespec = "C:/TradingApps/Amibroker/DB/WatchLists/List 0.tls";
DataDir = "C:\\TradingApps\\Amibroker\\DB";

var fso, watchlist, s, ForReading;
ForReading = 1, ForWriting = 2, s = "";
fso = new ActiveXObject( "Scripting.FileSystemObject" );
watchlist = fso.OpenTextFile( filespec, ForReading, false );

var oAB = new ActiveXObject( "Broker.Application" );
oAB.LoadDatabase( DataDir );
var oStocks = oAB.Stocks;
var Qty = oStocks.Count;

var DeleteFrom = new Date( "December 13, 2018 00:00:00" ).yyyymmddhhmmss();
var DeleteTo = new Date( "December 14, 2018 23:59:59" ).yyyymmddhhmmss();

file = fso.OpenTextFile( "_remowe_xdays.log", ForWriting, true );
file.WriteLine( "Starting delete quotes from date:" + DeleteFrom );
file.WriteLine( "" );

while( !watchlist.AtEndOfStream )
{
     s = watchlist.ReadLine();
     for( i = 0; i < Qty; i++ )
     {
         oStock = oStocks( i );
         if( s == oStock.Ticker )
         {
             file.Write( i + ". " + oStock.Ticker + "=" );
           
             for ( j = oStock.Quotations.Count - 1; j >= 0; j-- )
             {
                 var tmpDateNum = new Date( oStock.Quotations( j ).Date ).yyyymmddhhmmss();                
                 if( ( tmpDateNum % 1000000 != 0 ) && ( tmpDateNum >= DeleteFrom ) && ( tmpDateNum <= DeleteTo ) )
                 {
                     oStock.Quotations.Remove( j );
                 }
             }
             file.WriteLine( "OK" );
         }
     }
}
watchlist.Close();
oAB.RefreshAll();
oAB.SaveDatabase();

WScript.Echo( "Done!" );
Now if you want to remove all quotes including both Intraday and EOD, then Run the below JavaScript after saving it as Anything.js from anywhere as you wish.
JavaScript:
Date.prototype.yyyymmddhhmmss = function()
{
     //Modified the code from https://www.yijunma.com/tool/code-snippet/format-date-yyyymmdd-hhmmss/
     var mm = this.getMonth() + 1; // getMonth() is zero-based
     var dd = this.getDate();
     var hh = this.getHours();
     var mi = this.getMinutes();
     var ss = this.getSeconds();
     var yyyymmdd = [ this.getFullYear(), ( mm > 9 ? '' : '0' ) + mm, (dd > 9 ? '' : '0') + dd ].join('');
     var hhmmss = [ ( hh > 9 ? '' : '0' ) + hh, ( mi > 9 ? '' : '0' ) + mi, ( ss > 9 ? '' : '0' ) + ss, ].join('');
     return yyyymmdd + '' + hhmmss;
};

filespec = "C:/TradingApps/Amibroker/DB/WatchLists/List 0.tls";
DataDir = "C:\\TradingApps\\Amibroker\\DB";

var fso, watchlist, s, ForReading;
ForReading = 1, ForWriting = 2, s = "";
fso = new ActiveXObject( "Scripting.FileSystemObject" );
watchlist = fso.OpenTextFile( filespec, ForReading, false );

var oAB = new ActiveXObject( "Broker.Application" );
oAB.LoadDatabase( DataDir );
var oStocks = oAB.Stocks;
var Qty = oStocks.Count;

var DeleteFrom = new Date( "December 13, 2018 00:00:00" ).yyyymmddhhmmss();
var DeleteTo = new Date( "December 14, 2018 23:59:59" ).yyyymmddhhmmss();

file = fso.OpenTextFile( "_remowe_xdays.log", ForWriting, true );
file.WriteLine( "Starting delete quotes from date:" + DeleteFrom );
file.WriteLine( "" );

while( !watchlist.AtEndOfStream )
{
     s = watchlist.ReadLine();
     for( i = 0; i < Qty; i++ )
     {
         oStock = oStocks( i );
         if( s == oStock.Ticker )
         {
             file.Write( i + ". " + oStock.Ticker + "=" );
           
             for ( j = oStock.Quotations.Count - 1; j >= 0; j-- )
             {
                 var tmpDateNum = new Date( oStock.Quotations( j ).Date ).yyyymmddhhmmss();                
                 if( ( tmpDateNum >= DeleteFrom ) && ( tmpDateNum <= DeleteTo ) )
                 {
                     oStock.Quotations.Remove( j );
                 }
             }
             file.WriteLine( "OK" );
         }
     }
}
watchlist.Close();
oAB.RefreshAll();
oAB.SaveDatabase();

WScript.Echo( "Done!" );
 
Last edited:

CougarTrader

Well-Known Member
#9
Then you need to automate the Task. Before running the below scripts, backup of the database is necessary to avoid any unexpected data loss.

Here List 0.tls is the name of the watchlist and DB happens to be the name of the database. "filespec" and "DataDir" variables needs to be specified accordingly. Also "DeleteFrom" and "DeleteTo" need to specified in the same format as written.

If you want to remove intraday quotes but keep EOD, then Run this JavaScript after saving it as Anything.js from anywhere.
JavaScript:
Date.prototype.yyyymmddhhmmss = function()
{
     //Modified the code from https://www.yijunma.com/tool/code-snippet/format-date-yyyymmdd-hhmmss/
     var mm = this.getMonth() + 1; // getMonth() is zero-based
     var dd = this.getDate();
     var hh = this.getHours();
     var mi = this.getMinutes();
     var ss = this.getSeconds();
     var yyyymmdd = [ this.getFullYear(), ( mm > 9 ? '' : '0' ) + mm, (dd > 9 ? '' : '0') + dd ].join('');
     var hhmmss = [ ( hh > 9 ? '' : '0' ) + hh, ( mi > 9 ? '' : '0' ) + mi, ( ss > 9 ? '' : '0' ) + ss, ].join('');
     return yyyymmdd + '' + hhmmss;
};

filespec = "C:/TradingApps/Amibroker/Nifty DB/WatchLists/List 0.tls";
DataDir = "C:\\TradingApps\\Amibroker\\DB";

var fso, watchlist, s, ForReading;
ForReading = 1, ForWriting = 2, s = "";
fso = new ActiveXObject( "Scripting.FileSystemObject" );
watchlist = fso.OpenTextFile( filespec, ForReading, false );

var oAB = new ActiveXObject( "Broker.Application" );
oAB.LoadDatabase( DataDir );
var oStocks = oAB.Stocks;
var Qty = oStocks.Count;

var DeleteFrom = new Date( "December 13, 2018 00:00:00" ).yyyymmddhhmmss();
var DeleteTo = new Date( "December 14, 2018 23:59:59" ).yyyymmddhhmmss();

file = fso.OpenTextFile( "_remowe_xdays.log", ForWriting, true );
file.WriteLine( "Starting delete quotes from date:" + DeleteFrom );
file.WriteLine( "" );

while( !watchlist.AtEndOfStream )
{
     s = watchlist.ReadLine();
     for( i = 0; i < Qty; i++ )
     {
         oStock = oStocks( i );
         if( s == oStock.Ticker )
         {
             file.Write( i + ". " + oStock.Ticker + "=" );
           
             for ( j = oStock.Quotations.Count - 1; j >= 0; j-- )
             {
                 var tmpDateNum = new Date( oStock.Quotations( j ).Date ).yyyymmddhhmmss();                
                 if( ( tmpDateNum % 1000000 != 0 ) && ( tmpDateNum >= DeleteFrom ) && ( tmpDateNum <= DeleteTo ) )
                 {
                     oStock.Quotations.Remove( j );
                 }
             }
             file.WriteLine( "OK" );
         }
     }
}
watchlist.Close();
oAB.RefreshAll();
oAB.SaveDatabase();

WScript.Echo( "Done!" );
Now if you want to remove all quotes including both Intraday and EOD, then Run the below JavaScript after saving it as Anything.js from anywhere as you wish.
JavaScript:
Date.prototype.yyyymmddhhmmss = function()
{
     //Modified the code from https://www.yijunma.com/tool/code-snippet/format-date-yyyymmdd-hhmmss/
     var mm = this.getMonth() + 1; // getMonth() is zero-based
     var dd = this.getDate();
     var hh = this.getHours();
     var mi = this.getMinutes();
     var ss = this.getSeconds();
     var yyyymmdd = [ this.getFullYear(), ( mm > 9 ? '' : '0' ) + mm, (dd > 9 ? '' : '0') + dd ].join('');
     var hhmmss = [ ( hh > 9 ? '' : '0' ) + hh, ( mi > 9 ? '' : '0' ) + mi, ( ss > 9 ? '' : '0' ) + ss, ].join('');
     return yyyymmdd + '' + hhmmss;
};

filespec = "C:/TradingApps/Amibroker/Nifty DB/WatchLists/List 0.tls";
DataDir = "C:\\TradingApps\\Amibroker\\Nifty DB";

var fso, watchlist, s, ForReading;
ForReading = 1, ForWriting = 2, s = "";
fso = new ActiveXObject( "Scripting.FileSystemObject" );
watchlist = fso.OpenTextFile( filespec, ForReading, false );

var oAB = new ActiveXObject( "Broker.Application" );
oAB.LoadDatabase( DataDir );
var oStocks = oAB.Stocks;
var Qty = oStocks.Count;

var DeleteFrom = new Date( "December 13, 2018 00:00:00" ).yyyymmddhhmmss();
var DeleteTo = new Date( "December 14, 2018 23:59:59" ).yyyymmddhhmmss();

file = fso.OpenTextFile( "_remowe_xdays.log", ForWriting, true );
file.WriteLine( "Starting delete quotes from date:" + DeleteFrom );
file.WriteLine( "" );

while( !watchlist.AtEndOfStream )
{
     s = watchlist.ReadLine();
     for( i = 0; i < Qty; i++ )
     {
         oStock = oStocks( i );
         if( s == oStock.Ticker )
         {
             file.Write( i + ". " + oStock.Ticker + "=" );
           
             for ( j = oStock.Quotations.Count - 1; j >= 0; j-- )
             {
                 var tmpDateNum = new Date( oStock.Quotations( j ).Date ).yyyymmddhhmmss();                
                 if( ( tmpDateNum >= DeleteFrom ) && ( tmpDateNum <= DeleteTo ) )
                 {
                     oStock.Quotations.Remove( j );
                 }
             }
             file.WriteLine( "OK" );
         }
     }
}
watchlist.Close();
oAB.RefreshAll();
oAB.SaveDatabase();

WScript.Echo( "Done!" );
This is exactly what I was looking for. Thank you very much.
 
#10
anything .js not worked with me,
I changed DB, but .tls file not in DB folder on" watchlist" sub folder don't know where .tls is there,
changed valid date for delete ticks..

run the .js ,shows path not found..
 

Attachments

Last edited:

Similar threads