Developing Python Library for Trading Automation

#31
You doing all this alone and lucky enough to be married atleast you don't have to survive on (Apna Haath Jagannath).It is pretty difficult.It's just community service right now and ideally this work should be done by a team with some financial backing at their disposal.Now thanks to you it is open source like a mandir ka ghanta anyone can come and bang it.

But still solid work saw the code pretty clean.I will try it sometime on real account in Zerodha.The problem is to make it a fully automated solution I have to add some code with trading logic and risk management and portfolio optimization codes as well like
a) Stop trading when equity falls below certain level
b) Trailing stoploss
c) historical data analysis
d) Fundamental research

Most of them I had done on www.quantopian.com because it has lot of customization possibilities and python is easy simple.But the main problem is to continue such financial backing is required without which it is just a random hobby.I have stopped but if you can continue hats off to you
 
#32
Finally some development over the weekend, not exactly what you would expect though :p
Updated the library NSEpy. NSEpy is a library to get the stock, index, derivative historical data series into python in pandas dataframes, ready for analysis. Important part is it now has support for downloading derivative data ;)

check out my github page https://github.com/swapniljariwala/nsepy for full instructions on how to use it.

Please report any bugs, issues or whislist on
https://github.com/swapniljariwala/nsepy/issues
 
#34
OK so I understand that lot of people here are doing algo-trading (semi-automatic). I was thinking that just for the sake of demonstration and a proof of concept, I can implement some simple strategy in back-testing and in real-time with my own money (very small cash in intraday stock) using my python library for Zerodha (a strategy that is already proven on some other trading/charting software and is being used in live trading). I'll share the results here for analysis and study.

I'd some questions though.
1. Do people use algo-trading in intraday?
2. If they do? is the basis for the strategy news/information based or purely pattern based? (because I can program for levels/patterns but not for news/information)
3. How are intraday strategies are backtested? where do you get historical intraday minute level data for backtesting?
4. If you can provide some strategy (should be defined by Math and Logic so that it can be coded) with results, I'll try to implement using the python library in fully automatic mode.

I can scrape some intraday data from the web or can even closely simulate the intraday data if needed.

Just on a side note, would it be beneficial for everyone if I can manage and share actual minute level data for some important securities in .csv format for backtesting purpose?
 
Last edited:
#35
Hello Swapnil

Following are some of the answers to the questions you asked ?

1. Yes algotrading is useful in intraday and many use it for HFT but HFT requires the execution time of the code to be also low.Python can be used for Low frequency trading , intraday trading on a minute based data or seconds also but for nano second and millisecond execution time not the best.Either C++ or Scala will have to be used

2. Need third party news api provider for that.

3. Historical intraday minute level data(real time feeds) have to be purchased from authorized data vendors of NSE like Global Data feeds

4. Here is one strategy written in Python code functioning is explained in comments.It was written in quantopian.If you can try to convert the equivalent code.There are comments in the code denoted by "#".

#Import packages numpy , scipy and pandas dataframe
import numpy as np
from scipy import optimize
import pandas as pd
#Initialize function will run only once when the algorithm starts running.
def initialize(context):
#Define a universe of stocks that the algorithm will trade.Include all Finance , Technology , Energy , Healthcare , Industrial and
context.stocks = [ sid(8554),sid(42950),sid(700),sid(24),sid(36628),sid(5061)]
''' sid(19662), # XLY Consumer Discrectionary SPDR Fund
sid(19656), # XLF Financial SPDR Fund
sid(19658), # XLK Technology SPDR Fund
sid(19655), # XLE Energy SPDR Fund
sid(19661), # XLV Health Care SPRD Fund
sid(19657), # XLI Industrial SPDR Fund] # XLU Utilities SPRD Fund'''
#Define maximum leverage to which algorithm can accelerate
context.leverage_buffer = 2.45
#Define number of days to lookback which retrieving price and volume history
context.lookback = 30
#Stop the algorithm when loss > value*100 in percentage
context.stop_portfolio = 0.10
#Define a dictionary to store whether a particular instrument has to be bought or sold
context.B_OR_S = {}
#Store true if any position is taken in a particular stock or not false means no positions
context.invested = {}
#Store
context.stop_price = {}
context.stop_pct = 0.95
#This is a recurring function call which runs the function "monthly_rebal" every week start 3 minutes before market close
schedule_function(monthly_rebal,
date_rules.week_start(),
time_rules.market_close(minutes=3))
#Run a loop which sets default values of Buy or Sell , invested or not and stop price
for stock in context.stocks:
context.B_OR_S[stock] = False
context.invested[stock] = False
context.stop_price[stock] = 0.0

#This function runs every time the datafeed comes.If the datafeed is daily then function is called daily at market open
def handle_data(context, data):
leverage = context.account.leverage
record(levrage = context.account.leverage)
record(equity = context.portfolio.cash)
#Check existing open positions everytime datafeed comes to set a trailing check for the positions.Exit positions
set_trailing_stop(context, data)
#Iterate stock list to check existing positions and exiting sell positions if leverage greater than max leverage
for stock in context.stocks:
if stock in data:
if leverage > context.leverage_buffer:
if context.invested[stock] == True and context.B_OR_S[stock] == False:
log.info("Liquidating position %s" % (stock))
order_target_percent(stock, 0)
context.invested[stock] = False
return

def monthly_rebal(context, data):
#Store no of stocks in the universe
l = len(context.stocks)
#If no stocks exit
if l == 0:
return
#else if there are stocks calculate weight of each stock by dividing by number of stocks
else:
weight = 1.0/l
#Traverse list of all stocks
for stock in context.stocks:
if stock in data:
#Fetch close prices of the particular stock denoted by "stock" with number of days as context.lookback value
close = history(context.lookback,'1d', field='close_price')[stock]
#Store the maximum close price of the stock in the specified period
max_high = np.max(close)
#Store the minimum close price of the stock in the specified period
min_low = np.min(close)
#Store the current price of stock in variable
cur_p = data[stock].price
#Calculate the mid point of maximum high close and minumum low close
mid_range = (max_high+min_low)/2
#Calculate the mean of mid range and maximum high
upp_limit = (max_high+mid_range)/2
#Calculate the mean of mid range and minimum low
low_limit = (min_low+mid_range)/2
#If current price of instrument > upper limit check cash available for trade and sell
if cur_p > upp_limit:
portfolio_check(context,data,stock,weight,1)
#If current price of instrument < lower limit check cash available for trade and buy
elif cur_p < low_limit:
portfolio_check(context,data,stock,weight,0)

def portfolio_check(context,data,stock,weight,trig):
#Store available free cash to place orders
equity = context.portfolio.cash
#Starting portfolio value
portfolio_size = context.portfolio.portfolio_value
#Portfolio value at which the algorithm will stop if free cash reaches below it
stop_portfolio = (1.0 - context.stop_portfolio)*portfolio_size
#Stop if equity reaches portfolio value is lower is than stop_portfolio
if equity < stop_portfolio:
return
#If there is no position in the stock and sell signal generated place buy order
if context.invested[stock] == False and trig == 1:
# Buy stock with weight by allocating 1/len(stock_universe) portfolio in percent.
# If there are 10 stocks in the universe allocate 1/10 th of the portfolio to buy the stock
order_target_percent(stock, -weight)
# Set invested or not to true
context.invested[stock] = True
#If there is no position in the stock and buy signal generated place sell order
elif context.invested[stock] == False and trig == 0:
order_target_percent(stock, weight)
# Set invested or not to true
context.invested[stock] = True

def set_trailing_stop(context, data):
# retrieve all the open orders and log the total open amount
# Add this guard to prevent over-ordering
open_orders = get_open_orders()
# open_orders is a dictionary keyed by sid, with values that are lists of orders.
if open_orders:
# Traverse all the open orders every time a new data feed comes
for security, orders in open_orders.iteritems():
# Traverse the open orders each specific open order denoted by oo
for oo in orders:
#Store the price at which open order was executed
p = oo.price
#Store the amount of stock bought or sold
am = oo.amount
#Get the symbol of the stock of which open order is placed
stock = security.sid
#Get the current price of stock in open order
cur_price = data[stock].price
#Trailing stoploss 0.1 indicates 10 percent
trailing_stop = context.trailing_stop
#If amount of stock > 0 it is bought and if current price is also greater than 1.1*price of purchase exit positions
if am > 0 and cur_price > (p+p*trailing_stop):
order_target_percent(stock,0)
#If amount of stock < 0 it is sold and if current price is lesser than 0.9*selling price exit positions
elif am < 0 and cur_price < (p-p*trailing_stop):
order_target_percent(stock,0)
 
#36
Finally some development over the weekend, not exactly what you would expect though :p
Updated the library NSEpy. NSEpy is a library to get the stock, index, derivative historical data series into python in pandas dataframes, ready for analysis. Important part is it now has support for downloading derivative data ;)
sjerry4u

Great work. I am a Python newbie. Am planning to test out your code sometime shortly.

I was planning trading infrastructure for myself...presently use Ami+Manual Trades. One of the major requirements was historical data. I am looking to build a historical data server for backtesting...it is too much work managing Ami data files & updating them.

It was in that context that I was searching for quote downloading libraries in python, when I came across your post.

Have you thought about best method to store historical data on scale? What is the best method of storing time series data so that quote streaming is low resource? Obviously rdbms are out, though I did think of storing daily feed in blob and them processing it at the end of day to make it one blob per symbol. That way, all you do is pick a blob n stream it for a day. But, somehow, I am still doubtful about how to proceed on storage. Any ideas?

I would be willing to test on zerodha live, if you can compute the daily support & resistance of lets say ICICI (my favourite) and buy x points from support & sell y points from resistance. I am routinely having abt 1000 share portfolio of icici and trading 10 shares or so on an automated script won't dent the position much. We can see if buying selling breaks anywhere, the strategy of course is not of much use to anyone except a long term holder of a given scrip.

I am looking to script the pressing of the semi-automation buttons of Pi using python code or autoit, so script should be able to run unattended and any bugs may be found that way.

Let me know what you think.

With best regards.
Sanjay.
 
Last edited:

TradeOptions

Well-Known Member
#37
sjerry4u

Great work. I am a Python newbie. Am planning to test out your code sometime shortly.

I was planning trading infrastructure for myself...presently use Ami+Manual Trades. One of the major requirements was historical data. I am looking to build a historical data server for backtesting...it is too much work managing Ami data files & updating them.

It was in that context that I was searching for quote downloading libraries in python, when I came across your post.

Have you thought about best method to store historical data on scale? What is the best method of storing time series data so that quote streaming is low resource? Obviously rdbms are out, though I did think of storing daily feed in blob and them processing it at the end of day to make it one blob per symbol. That way, all you do is pick a blob n stream it for a day. But, somehow, I am still doubtful about how to proceed on storage. Any ideas?

I would be willing to test on zerodha live, if you can compute the daily support & resistance of lets say ICICI (my favourite) and buy x points from support & sell y points from resistance. I am routinely having abt 1000 share portfolio of icici and trading 10 shares or so on an automated script won't dent the position much. We can see if buying selling breaks anywhere, the strategy of course is not of much use to anyone except a long term holder of a given scrip.

I am looking to script the pressing of the semi-automation buttons of Pi using python code or autoit, so script should be able to run unattended and any bugs may be found that way.

Let me know what you think.

With best regards.
Sanjay.
Hi Sanjay

Checkout this link, you might get some ideas -

James Blackburn - Python and MongoDB as a Platform for Financial Market Data

https://www.youtube.com/watch?v=FVyIxdxsyok

Thanks and regards
 
#39
Hi. For a complete noob whos just discovering Python. Can you show how you have a set up for placing orders into icicidirect.

Thanks
 
#40
HI swapnil,
Great work man. I am a positional trader but wants to automate my buying,selling and risk management per stock. Did you looked into zerodhas recent KITE API? I am more of a R programmer but your libraries will force me to learn Python :). I have seen your blog and your work in aws. I am interested in aws do you have any idea of the cost (rupees) and have you tried microsoft azure. Did you implemented any algo trading in aws?