Page 1 of 7

To automate or not to automate?

Posted: Mon Feb 11, 2008 4:55 pm
by Aldo
Hello, hoping for some help and advice here on automated trading. I've been looking at a forex strategy documented on a number of forums on the web (3 ducks) and have spent some enough time looking at the charts to be convinced that it is a relatively simple strategy that is reasonably successful.

The problem I have with it is that it's essentially an intraday strategy and I work full time. Although I have time to glance at charts during the day I can't (for obvious reasons) spend a great deal of time on this so essentially I am excluded from the action.

I've done a bit of coding in the past (Java, PHP, & some web stuff) so I'm more than comfortable writing code and wondered whether this strategy could be automated. So I spent quite a while googling automation and it seemed that MetaTrader would fit my bill as a platform enabling automation. Now it doesn't seem the best platform, but it does seem quite widely used and fairly popular ... and while I'm more than happy to pay for automating my trading, I wanted to determine whether it was going to be worth my while first. So with that in mind I downloaded MetaTrader and tried to write my first EA using some free language reference guide I found on another forum. I got something basic working but backtesting showed flaws in the entry/exit/money management strategies etc etc so I'm a long way from getting it right.

So I guess my questions are the following ...

1) Is it worth my while trying to automate aspects of my trading, esp if I have no way of trading some strategies otherwise?
2) If the answer to above is yes, what's the best and most cost-effective platform (not necessarily free) for me to look at? In terms of coding I'm not that bothered about the platform but good documentation with examples would be useful.
3) Any other help or advice welcome ...

Have chosen this site to ask the question as it seems the most professional and knowledgeable of the forums I've looked at in the past.

Many thanks,
Allan

Posted: Mon Feb 11, 2008 10:05 pm
by eudamonia
Allan,

I'm sure some other folks will chime in here at some point so I'll state a few of my opinions regarding backtesting and automation.

1) If you cannot trade the methodology successfully (i.e. you are incapable or are not sure if you're capable) I think you will find that automating a methodology will be very difficult. There is more to a trade than entering and exiting and there are numerous nuances that a discretionary trader uses to manage their trade that account for their success.
2) Since this is the case I will go so far to state that I have NEVER met a successful automated trader who could not also trade competently without automation. This applies doubly so for any intraday trading. The smaller the timeframe the more precise you must be.
3) Backtesting is the DEVIL and here is why: a. few backtesting platforms have the EXACT same data and precision of data computation as a live trade b. few programmers are capable of dealing effectively with LIMIT orders which are the bread and butter of short term traders (ie. how do you deal with partial fills, how do you know when you got filled etc.) c. there are numerous assumptions that programmers must overcome and often the only real way to test them is with REAL live market data and REAL trading $.
4) Lastly, there are very few programs capable of automation/backtesting in a satisfactory fashion. NeoTicker and SmartQuant are two that seem to be acceptable. However, you had better have a Ph.D. in computer programming to utilize them effectively. Knowledge of "some code" won't cut it. You had better be able to program at the highest level to have any hope of creating a satisfactory automated system, debugging it, testing it, and most importantly UNDERSTANDING how every little assumption you make will affect your results.

By the way if these items seem astounding they shouldn't. 99% of the guys/gals on these boards have 1-2 years or more of programming "experience". Nearly all of them would love to automate their trading. 99% of them will never successfully automate their system. The 1% that know what they are doing are more secretive than a CIA agent. So if you think you have what it takes be my guest.

Edward

P.S. I'm a die-hard discretionary trader who wasted several years trying to automate until I met one of the 1%. I never learned any great programming secrets - except this - the guys who really know what's up write their OWN platforms and sleep, eat and otherwise live programming.

Posted: Tue Feb 12, 2008 11:28 am
by Aldo
Thanks for your comments, it's very useful to get feedback from experienced people that have been doing this for a while (and lived to tell the tale).
I'd like to query some of your points to try and generate a discussion on this as I'm interested to get a few opinions.
1) One could argue that some people are perfectly capable of setting out a clear strategy with well defined entry and exit points, but when it comes to trading that plan it all falls apart. Now I don't believe I'm one of these people but one could argue that discretionary trading is not for them and they could benefit from automating their strategy to take the emotion out of it.
2) Interesting, but I guess it makes sense.
3) Indeed, I've read this so often over the last week or so. Would you say it has no relevance at all, or just very little?
4) I wouldn't say I'm in the top 1% by any stretch of the imagination but I do have a Masters with distinction in software & systems and while I don't write code at all anymore in my line of work I thought that with a lot of work and testing (backtest/demo/live with small amounts) it would be possible to achieve a level of automation for certain aspects of trading (i.e. a small percentage of strategy). Interesting to hear your views on this, is this the general consensus on automation?

cheers
Allan,

Posted: Tue Feb 12, 2008 4:15 pm
by eudamonia
Allan,

1) I hear this fallacious argument all the time, "automating my trading method would eliminate emotions from my trading". I disagree - ultimately a human must turn on the switch and let the machine do it's job. If emotions would be a factor in your manual trading they would affect you in automation as well.
3) Please don't get me wrong. Backtesting done properly is quiet relevant (much like brain surgery done properly), however, done poorly leads to serious account (i.e. brain) damage.
4) What I guess I'm trying to say is that many many reasonably intelligent (or more so) individuals have walked down your path. If you feel that you have some edge over these people or are prepared to work harder than them then you may succeed. This doesn't mean it's impossible - just much more difficult than the would be automated trader believes. Also I don't think you'll find a general consensus on much in trading. To be fair I have very strong opinions on the matter but then again I'm not the only one.

Edward

Re: To automate or not to automate?

Posted: Tue Feb 12, 2008 9:34 pm
by obx
What Eudo said about If you cannot trade the methodology successfully manually is correct but incomplete. The criteria for whether to automate a system is not whether you can code up the entry and exit skills of a method but whether you can code up any necessary context / background / market conditions that support the system.

Lets pick on you know who for a minute on this one. All you need to trade is one line seems like an easy system to automate. But as you progress with the development, you realize the levels of unconscious competence and wet ware inner indicators and execution skills actually involved in trading a one line system and if you scroll around you start finding cockpit instrumentation-like representations of regular squiggly line indicators ( moving averages, wave compasses, etc.). Context! context used for setting targets / staying in or exiting decisions in real time and sizing decisions is what really makes a system thrive. And this example is a simple system with simple contexts. If you can program context ie program how you want your system to respond to various contexts and market conditions, then procede with confidence and persistence in automation. If not, do it by hand. And if you dont see any need for programming context / market conditions that support a system, then please dont waste your time trying to automate.

hth

do it by hand - this somehow reminds of that old joke about the dumbest neck saying to his dim buddy lets go to Nevada and open up a whore house. We could work it by hand till we get some help :twisted:

pps why will this forum not take apostrophe's and dashes and dot dot dots etc ????

Re: To automate or not to automate?

Posted: Tue Feb 12, 2008 9:57 pm
by obx
More
Allan, your statement One could argue that some people are perfectly capable of setting out a clear strategy with well defined entry and exit points, but when it comes to trading that plan it all falls apart illustrates precisely what Im talking about. The agent of execution (human or machine) is not flawed. It is operating with an insufficiently contexted strategy, an incomplete plan,

That can be extremely positive knowledge ie Im not being critical or negative at all . It acknowledges that most systems have unconscious rules beyond the clear set out strategy part. It is not the inability to execute a plan that frustrates and fails. It is the incompleteness of the plan that renders it totally unacceptable for real time conditions. So basically, do the screen time necessary to very explicitly include context in strategy and the capacity to perform and execute becomes non issue whether the system is automated or not.

its sorta like dont hate the playa hate the game

Posted: Tue Feb 12, 2008 11:52 pm
by eudamonia
obx,

Good points. I agree that understanding of context and the mechanics of execution are much more important than the exact "methodology" of your trading setup.

What I noticed when I traded live with Avery for a week in Mexico is that he could trade successfully with only price on the screen or any number of indicators. It really didn't matter. Avery was the "secret" behind his success.

Edward

Posted: Wed Feb 13, 2008 8:57 pm
by Aldo
Thanks for the points obx. This was kind of where I was getting at ... I realise context and execution are key to this. And I understand that if I can't trade it manually then there's no point trading it automatically (unless I want to lose money quicker!).

So going back to my original question ... let's say I can trade my strategy in my sleep but it's intraday and so my day job prohibits me from doing this. Therefore I'm out of the action. I see two options here ... either I try and automate it and even if it's not as effective as manually trading as long as it makes money it's better than not being in the game ...
Or I take an approach along the lines of the thread " Craigaudio's ER2 Buyzone Strategy " and make some trades early morning/late at night and leave it at that. My timezone is GMT for info.

One key point could be that even if I didn't work all day I'm not sure I'd want to sit in front of a flashing price screen all day anyway.

Anyone have any more thoughts on this?
cheers,
Al

Posted: Fri Feb 15, 2008 2:59 pm
by obx
re: "two options here ... either I try and automate it ... Or I make some trades early morning/late at night" OR both!

For the automation: If system does meet the criterea for automation AND you can check up on it during your work day (ie while it doesn't require constant supervision like a toddler, it's sorta like 'do you know where your teenager is?') then code it up and forward test it with very light exposure and learn and adjust from there going forward.

For the early or late times: Pick one. Bring your time frames down and learn and adapt from there going forward - also with light exposure.

hth

Posted: Sat Feb 16, 2008 6:39 pm
by michal.kreslik
Aldo et al.,

from my trader's perspective, there is nothing more rewarding than observing my strategy generating equity and being automatically traded on a remote server without me touching it. This is what I call a true freedom in life.

However, before you can experience that freedom, you need to devote years of hard work to it. Sorry, there are no shortcuts to achieving that state as well as there is no shortcut to playing Chopin well. And no, you can't do it part time.

Technically, there is no difference between a good discretionary trader and a good automated trader - both of them are following the rules that they found to be profitable in the long run. They found these rules by tireless testing. If you consider yourself a professional trader (= an individual that is making the majority of his income by trading), you can't trade a system without properly testing it first. There are many ways on how to test the trading system incorrectly and that's probably why many amateurs are saying that backtesting doesn't work. I could probably write a book on this theme, but let's just simplify it and say here that sloppy backtesting is worthless indeed.

As Ed correctly put it, both the discretionary trader and the automated trader are exposed to the emotional pressure - the automated trader has exactly the same free will in not following the rules that he had found to be profitable as the discretionary trader has. For the automated trader it's a little bit easier to follow the rules though as he does not have to be present at every trading decision as is the case for the discretionary trader. Also, the discretionary trader can't react as quickly as the automated system can.

However, as I see it, one of the main reasons for automating the trading is to be able to trade more markets at the same time. Even the most tenacious discretionary trader simply can't follow 20 forex pairs and place 500 trades round the clock, for instance. Another reason for going auto is that the discretionary trader has to concentrate on trading while he is trading whereas the automated trader can use this spare time for further development or simply for lying on the beach and rating the bikini-clad ladies around :)

For some traders, the thrill of placing the trades personally and being individually present in the market is an important part of the business. That may be one of the reasons why they are not interested in automating their trading.

Some traders are not able to automate their systems because they don't actually know what exact input information leads them to the specific trading decisions.

Unlike the discretionary trader, the automated trader is forced to specify all rules for the trading system down to the slightest possible detail. You wouldn't believe how incredibly complicated is to explain a simple task to a dumb computer. For complex tasks, this is actually not possible at all yet (computers can't drive a car yet). In a sense, the automated trader has to explain to the brainless computer how to drive a "car" of his automated trading system. And this is not just a question of the trading system rules themselves, but the whole intricate IT infrastructure that surrounds it as well.

I wish you all success,
Michal