To automate or not to automate?

If you don't know where to start, start here! Don't be afraid to ask questions.

Moderator: moderators

greaterreturn
rank: 50+ posts
rank: 50+ posts
Posts: 64
Joined: Mon Feb 11, 2008 2:57 am
Reputation: 0
Gender: None specified

Postby greaterreturn » Tue Apr 15, 2008 1:09 am

Right Michal, I'm working on my own algorithm for filtering. Quite complex but basically, when I receive a tick more than X % away from the last tick, I will start a millisecond timer of Y duration, if any tick arrives back in the acceptable range, I'll dump the bad ones and move on.

But what do I do if the Y duration timer expires and the ticks are still "Bad". Do I assume they're good? Very bizarre. The human mind can filter them out so OBVIOUSLY.

It seemed like such a simple issue are first. Plus, how in the world do you test to make sure your filter works??? Who can predict what bad ticks will come and how bad they'll be? I'll figure it out, of course. But it takes tons of research and time.

Sincerely,
Wayne

Please add www.kreslik.com to your ad blocker white list.
Thank you for your support.

peter_77_peter
rank: <50 posts
rank: <50 posts
Posts: 41
Joined: Fri Feb 02, 2007 3:36 pm
Reputation: 0
Location: a beautiful pirate island in the mediteranean sea
Gender: Male

incorporating GA

Postby peter_77_peter » Tue Apr 15, 2008 10:08 pm

Hi Michal, Ed, and all the others

michal.kreslik wrote:Hello, Peter,

in the field of automated trading, I'd say I can usually tell the author who knows what is he talking about from the one who doesn't in the first three sentences of his article. Then I stop reading if it's the latter case.

I've read your post in one shot. It shows that you've really done your hard homework and you're heading towards becoming one of the 0.01% of the people who make it as automated traders. I can put my signature to almost everything that you said in your post. Thanks for being a kreslik.com member and posting your thoughts here.


Thank you so much for providing kreslik.com , and thank you all for posting and sharing. There´s a lot of inspiring discussions in this forum that contributed to our development of trading know-how! And thank you so much for you encouraging words!

peter_77_peter wrote:the distribution of trades is not reliable. But the distribution of the trades is crucial to the appearance of loosing streaks


michal.kreslik wrote:You are probably talking about the time/sequence distribution of the trades, not about the trade outcome distribution itself. Sure, the particular sequence of trades' outcomes in testing (and in trading as well) is random. Otherwise you could only open the trades that you know will be profitable. Obviously, that would be too easy.


Exactly!

michal.kreslik wrote:With multiple degrees of freedom as your trading system inputs, the best approach is to use some non-black-box AI algorithm like GA. GA cuts on the optimization time considerably and if set up properly, it's not generally prone to converging towards local maxima on the fitness landscape. You already know that the fitness function should be closely related to the linearity of the equity curve, measured with no position sizing. Under certain circumstances, the fitness function should be constructed in a different way, but that would be a topic on its own.


Actually, at the moment, we are doing all testing "manually". That is very time consuming, but, at least in the beginning, probably the best process to

a) reflect every single part of the strategy and the overall behaviour all over again
b) acquiring data and learn from that
c) get a feeling for the robustness achieved so far

Of course the only reliable data is "real world" data, but if you know exactly about the limitations of backtesting and deal with that, there´s pretty much to learn. We´ve written down a lengthy documentation of all inputs, dependencies, effects (expected vs. measured), limitations and fitting traps we face, as well as plausible routines to breed mutations of our strategies that reflect our intentions / logic, and try to learn as much and as efficient as we can from our backtesting. (and of course we´ve tried to keep everything as adaptive and relative by concept/design as possible before we´ve entered backtesting.)

At the moment we are about to "breed" 5-10 mutations of our system / adjustments to our favouirte stocks and want to put them into "real world trading" for some months, to acquire realiable data and learn from that.

But, actually I´m pretty much interested in learning about supporting / replacing our optimization process incorporating GA. Do you still sell your TS.GO Licence? If not, are there any other products that anybody here can recommend? It would be definitively great to experiment with using GA to further formalize our intentions considering linearity as well as automate the process of breeding mutations that are "fit" with respect to that....and learn from that.

By the way: Your point about measuring linearity without position sizing is quite good. (We´ve gotten there as well on our own.)

michal.kreslik wrote:You don't need to use Monte Carlo random sampling for determining your risk/position sizing values. It's actually much simpler than that. You just need to ask yourself two questions:
  • what is the maximum loss in terms of the trading capital percentage before I stop trading this system?
  • what I want the probability of such a loss be?
You can determine both of these values at your discretion and risk appetite based on the results of your testing. I'm sure you know how to calculate it.


Actually we´ve incorporated MC using the MSA Tool from Adaptrade and feel quite comfortable about this (thank you Ed for your input on that). Of course this is not really necessary, because, as you´ve put it completely right, you can calculate the probabilities of worst case loosing streaks using some of the data you get from back testing, and adapt the risk parameters, you use to derive your position sizing corresponding with the exit strategies, to that simple math.

Eitherways it´s very important to do your homework here!

any feedback appreciated,
all the best from spain,
Peter

User avatar
michal.kreslik
rank: 1000+ posts
rank: 1000+ posts
Posts: 1047
Joined: Sat May 13, 2006 2:40 am
Reputation: 36
Location: Monte Carlo, Monaco
Real name: Michal Kreslik
Gender: Male

Re: incorporating GA

Postby michal.kreslik » Tue Apr 15, 2008 11:45 pm

peter_77_peter wrote:But, actually I´m pretty much interested in learning about supporting / replacing our optimization process incorporating GA. Do you still sell your TS.GO Licence? If not, are there any other products that anybody here can recommend?

It only makes sense to use GA if the number of combinations you are testing becomes too high to optimize the system using the exhaustive optimization. You should make sure that you're not optimizing water, i.e. a change in the input parameters should deliver a measurable/significant change in the output properties of the system. Also, it's not just the significance of the parameter itself, but the granularity of the tested values as well that can make a big difference in the time and resources needed to optimize the system. For some inputs, it makes sense to use logarithmic sampling as opposed to the linear one.

I was using Grail GO in the old days when I was using Tradestation. Fortunately, these days are long gone :) I sold my GGO license long time ago. There are publicly available GA libraries on sourceforge.net and elsewhere or you can write your own genetic library.

peter_77_peter wrote:all the best from spain,
Peter

I've just returned from Spain and Andorra a couple of weeks ago. Sagrada Familia in Barcelona is fantastic :)

Michal

peter_77_peter
rank: <50 posts
rank: <50 posts
Posts: 41
Joined: Fri Feb 02, 2007 3:36 pm
Reputation: 0
Location: a beautiful pirate island in the mediteranean sea
Gender: Male

Re: incorporating GA

Postby peter_77_peter » Wed Apr 16, 2008 11:11 pm

michal.kreslik wrote:It only makes sense to use GA if the number of combinations you are testing becomes too high to optimize the system using the exhaustive optimization. You should make sure that you're not optimizing water, i.e. a change in the input parameters should deliver a measurable/significant change in the output properties of the system. Also, it's not just the significance of the parameter itself, but the granularity of the tested values as well that can make a big difference in the time and resources needed to optimize the system. For some inputs, it makes sense to use logarithmic sampling as opposed to the linear one.


yes exactly. Thats why it takes us about 1.5 days per to adapt our strategies on one single stock. Doing optimization "by hand" on basis of 3 years, doing Walk Forward/Out of sample testing as well as Monte Carlo Analysis. We´ve got about 20-30 parameters that are worth a closer look. Some of them are kind of logical switches, that are iterating different circuitry of i.e. the trigger signals we use. Some are "classical parameters" i.e. to adjust the permeability of our trend filters. Some are specifying the adaptiveness off our exits to volatility, some are specifying the effect of additional protection logic, ...., probably the use of a GO would make perfect sense, because "brute force" is to slow to handle it, so what we basically did is: writing a "parameter book", describing each and every functionality, all dependencies and effects, of all parameters, resulting in some kind of state machine, picturing this we tried to figure out "logical paths" that we further investigate by optimization. On the one hand side this heuristic is necessary to deal with a overhelming lot of possibilities of different circuitries/filtering out a part of the population that we can handle by applying analytical thinking. On the other hand we as well think that this helps us to avoid fitting. That is: We want to achieve fitness with respect to linearity and relative performance parameters with settings that we as well consider rational, that is reflecting the logic of the ideas / solutions we´ve thought out during the development.

Of course we´ve tested out every module on its own during development, to get some isolated "proof of concept" for every building block, but now we are testing the "everything put together toolkit" and its nice to see that the parameter settings that we seem to filter out as "fit" within our optimization process are pretty close to what we´ve thought they should be as well as robust.

The advantage of manually following our own heuristic during optimization is, that we are actually modelling the outcomes following logical paths, that is, i.e. when we first set up an approximate exit chanel, then adding its further features on at a time. That way, we "have a goal" with every parameter, some kind of hypothesis that we can test, comparing the effects it should have, and what can be seen within the optimization results.

But actually, we are doing this for some weeks now and it would be good to get some fresh blood into our process, maybe we are missing some fit subset of the population with our "heuristic", maybe the results of running a GO would be inspiring, maybe they would basically confirm the mutations that we breed, maybe its outcome would come up with some new twists.

Besides that it would be a necessary increase of our effeciency. I think we should try this out!

Any help / thoughts about that appreciated as usual!

michal.kreslik wrote:I've just returned from Spain and Andorra a couple of weeks ago. Sagrada Familia in Barcelona is fantastic :)

Michal


Actually I´ve been several times to barcelona but never made it to sagrada familia. But I love bario gothico and the general vibe of that vibrant city. If you ever make it to the balears, tell me, and I can recommend you some great places to spend some time at the sea.....

Peter

peter_77_peter
rank: <50 posts
rank: <50 posts
Posts: 41
Joined: Fri Feb 02, 2007 3:36 pm
Reputation: 0
Location: a beautiful pirate island in the mediteranean sea
Gender: Male

help needed concerning Brokerages and Ninja TS OrderRouting

Postby peter_77_peter » Tue Jul 08, 2008 8:20 pm

Dear Pirates,

So, we are finally live and auto trade 6 symbols with real money since 17th of June still using TS (with our some thousand lines of code trend following strategy mentioned in the posts above / before). Actually our online-monitoring .dll ASPX Tool is still a little buggy, but, just clicking it, it says that we are f.e. shorting NYX (since 3th of July with open profit 317$ and real capital invested in this stock: 7k$).

Actually we´re doing somehow as expected, a couple of small losses, overcompensated by some really good trades, i.e. shorting GM from 18.6. till 24.6. or shorting BLK from 24.6. till 30.6. Though we are based in europe, I´ve had a pretty nice day when I´ve seen that we are invested short on four stocks on some evening, reading the mid days news the next day, that the nyse was getting down, especially one of the stock that we´ve been invested short. All in all we would have been slightly positive right from the start IF and now comes our big bad surprise:

If we would have known that TS maintains a pretty much hidden easy-to-borrow list, which is daily updated, quoting the stocks that can be shorted, implying that all others cannot be shorted. Literally this means, you don´t know in advance whether you get filled or not. Actually only about half of our short trades got filled. Some good ones came through, but our best trade, the GM short, was not filled.

So, we resume that our strategy is doing something in between "not bad" , "quite okay" and "somehow good", the results of the first 3 weeks are slightly positive and seem to reflect the back testing results. Right from the beginning of going online we knew that we´ll have to pay for acquiring "real data" and mastering this essential part of our learning curve. So, actually we´ve paid some dollars, for now knowing, that it seems that we don´t have a plattform or strategy problem in principle, but we´ve got a brokerage problem. That´s not that bad, and can possibly be fixed in a reasonable way throughout the following.

To get reassured by all you traders, whether doing the right thing, my question is:

Do you have any Broker suggestions, in particular concerning stock trading and availability for shortable stocks. I´ve checked IB on that, 5 out of 6 stocks that we trade seem pretty much shortable there, 1 out of 6 could have minor liquidity problems with shorting, but actually, this would be way much better than TS.

So, if I did my research right, then, it might be the best workaround to:

a) as a start, continue with our strategy in TS, routing our automated trading to IB via Ninjatrader: http://www.ninjatrader.com/webnew/tradi ... rading.htm

This would be a pretty nice step, because,

b) we can fix it soon, without major expenses in time and cash. With the exception of transfering the TS account to some other Broker. Would you recommend IB for stocks / especially shorting? Besides that, we would open up to other markets / brokerages, with this step, and, if that works out fine, we might start recoding our EL Code to C# based Ninja Script, to get more and more indepentend from TS.

We might as well give the EL compatibility of MultiCharts a try, but, I think, as a start, Ninja Trader is providing everything we need and a lot of flexibility. And if our prototype System will result in a positive track record within the next 6 months, then we will recode all the way, migrating to Neo Ticker, or whatever else seems most appropriate. From the beginning we thought that TS is quite nice for prototyping, and that we´ll have to recode everything if we can prove that our prototype is doing good.

Another question I have is somehow stupid, if our internet connection is set back from the provider, our computer system manages to reconnect, but, in some cases, tradestation does not recognize that the connection is back again, and stays "offline". Is there any known workaround for this?

Any more suggestions with respect to brokerages availability for shorting stocks, connecting TS to IB using NinjaTrader, Ninja Trader as a Plattform and IB as a broker in general, Multi Charts EL compatibility (Multi Data Charts? .dlls?), programming to the IB (or other brokerage) API directly, and whatever else comes to your mind that might be of good advice is pretty much appreciated......

best regards,
Peter

Please add www.kreslik.com to your ad blocker white list.
Thank you for your support.

User avatar
michal.kreslik
rank: 1000+ posts
rank: 1000+ posts
Posts: 1047
Joined: Sat May 13, 2006 2:40 am
Reputation: 36
Location: Monte Carlo, Monaco
Real name: Michal Kreslik
Gender: Male

Re: help needed concerning Brokerages and Ninja TS OrderRout

Postby michal.kreslik » Wed Jul 09, 2008 8:13 am

Hello, Peter,

it's a pleasure for me to reflect briefly on your post. I always like to talk to people who do their homework first before they start the discussion, as you do.

  1. after resolving myriads of problems that relate to setting up a fully automated system, you inevitably hit the snag of not having a good market access. There's no way around it: the next step in your quest is to get as direct access to the market as possible. The risks and drawbacks of dealing through brokers (as opposed to having a direct market access) are, among other things:
    • some employee at the broker can try to copy cat your system
    • your costs are always higher when trading through a broker
    • brokers inevitably add another layer of unreliability to the system
  2. I can't comment on stocks, but as far as Forex is concerned, IB's got the lowest commissions in the retail business and they've got very reliable filling, but their customer support is virtually non-existent. So be prepared for solving all your problems with IB platform on your own. There's no one out there to help you out from IB.
  3. the best solution in terms of programmatic interoperability and flexibility in the long run is to strive for having as few black boxes between you and the market. That is, forget about TS, forget about NeoTicker, forget about Ninja. Connect directly to the market via your market access provider's API and with your own code.

I wish you all trading success!

Michal

User avatar
TheRumpledOne
rank: 10000+ posts
rank: 10000+ posts
Posts: 15544
Joined: Sun May 14, 2006 9:31 pm
Reputation: 3035
Location: Oregon
Real name: Avery T. Horton, Jr.
Gender: None specified
Contact:

Postby TheRumpledOne » Wed Jul 09, 2008 4:33 pm

That is if they allow you access to their API!!
IT'S NOT WHAT YOU TRADE, IT'S HOW YOU TRADE IT!

Please do NOT PM me with trading or coding questions, post them in a thread.

androfx
rank: <50 posts
rank: <50 posts
Posts: 20
Joined: Thu Oct 26, 2006 3:27 pm
Reputation: 0
Location: Europe, Czech Republic... sometimes
Gender: None specified

Postby androfx » Wed Jul 09, 2008 6:49 pm

Just happened today: IB TWS stop to update all bid related information for one currency, just been reporting the same old tick value and volume information through. Restart of TWS corrected the problem.... :-(

peter_77_peter
rank: <50 posts
rank: <50 posts
Posts: 41
Joined: Fri Feb 02, 2007 3:36 pm
Reputation: 0
Location: a beautiful pirate island in the mediteranean sea
Gender: Male

thanks for the advice

Postby peter_77_peter » Thu Jul 10, 2008 10:44 pm

Dear Michal,
it´s always a pleasure to:

1. post all kinds of reflections on the problems we are facing on our quest towards a profitable automated trend following system on www.kreslik.com

and

2. always receive a handful of hands-on, inspiring as well as experienced answers within some hours.

That´s simply great!

And your post puts it precisely to the point, as usual:

1. There's no way around it: the next step in our quest is to get as direct access to the market as possible.

2. Apart from their customer support, everybody is talking good about IB.

3. The best solution in terms of programmatic interoperability and flexibility in the long run is to connect directly to the market via our market access provider's API and with our own code.

So actually, at the moment we are in prototype phase, and we want to keep this running for at least half a year to get a first track record and real data and try to learn from that. So, to recode the whole system is on our list, but thats long term, that is, after we are convinced that our prototype track record is worth recoding. TS Data is quite okay for a start, and our some thousand lines of code prototype is running on TS. The stability encountered so far is quite acceptable, we are only trading on 6 stocks with some different time frames each. TS Brokerage is not okay, so we want to connect to IB.

So, first, we will play with Ninja Trader, just to see, whether it is basically working: having our TS System generating our signals, routing the trading through Ninja Trader to IB. Okay, this is just to get a "quick and dirty" start, because, TS with some .DLLs, on a server running in spain, interfacing Ninja, routing through IB, that is very likely to produce some hard to track errors, that cost time and money. But on the other hand side, within our prototype phase, our trades are about 3-5 days, and, we are not into ultra fast trading, some minutes +- can be tolerated in the beginning, and will be further improved later.

So, what we really like to do and hope to achieve in July is:
working ourselves through:
http://individuals.interactivebrokers.c ... entity=llc
http://individuals.interactivebrokers.c ... iguide.htm
http://individuals.interactivebrokers.c ... Iintro.swf
http://individuals.interactivebrokers.c ... lights.php
and the following.

And then writing our own code ASAP to interface the IB API and get better filled with our short trades. Apart from order routing to IB we´ll keep things (what is left: signalproduction from market data) running on TS for the Prototype stage, and then recode everything from scratch, if we believe in our prototype stage track record half a year from now. We then get better control, better interoperability, less black boxes, and throughout that a lot of possibilities open up, second to none being more directly connected to more markets.

So, as usually, in the beginning there is a lot of work to do.
thank you for your advice!
all the best,
peter

peter_77_peter
rank: <50 posts
rank: <50 posts
Posts: 41
Joined: Fri Feb 02, 2007 3:36 pm
Reputation: 0
Location: a beautiful pirate island in the mediteranean sea
Gender: Male

to automate is not that bad....

Postby peter_77_peter » Mon Jul 21, 2008 9:59 pm

well, as the subject of this topic is: "to automate or not to automate", I´d like to put my vote to rename it, "to automate is not that bad".

Actually, it took us approximately 20 Month to get from: starting from scratch, to, finally beeing real money, real market with our automated trading system. But, actually, we´ve did well within the runup of the last days and our statistics says, that we have made roundabout 3% of plus within about 38 trades since 18th of june.

Well, one month has no significance, but, if we compare the results with the "back and forward testing curves", then our best reasoning is: that what we see is neither fitting data nor statistically abnormity, it seems that the curves are simply continuing to do what they did within the tests. And if you look at the dow, the times when we did our profits were between 6/20 and 6/24 ; 6/26 and 6/30 as well as 7/16 and 7/18. These are about the phases that could have been traded by such a system. The other phases have simply not costed us too much. So thats how we made our first 3%, about the way we´ve did our design.

so, if our system continues to produce such trades and numbers, then to automate cannot be that bad, so, lets see how our system will manage the next some months.....

all the best for your trading,
peter

Please add www.kreslik.com to your ad blocker white list.
Thank you for your support.


Return to “beginners forum”