## regression fitness function - smoothing out the equity curve

trading strategy optimization using genetic algorithms

Moderator: moderators

michal.kreslik
rank: 1000+ posts
Posts: 1034
Joined: Sat May 13, 2006 2:40 am
Reputation: 3
Location: Monte Carlo, Monaco
Real name: Michal Kreslik
Gender:
Contact:

### regression fitness function - smoothing out the equity curve

Hello,

in order to arrive at as linear equity curve as possible, I am using my own fitness function. I feed the resulting value of this fitness function to the genetic optimizer (I am using Grail GO) and GGO then tries to maximize the value.

This way, I am not interested in absolute NetProfit, but intead in a smooth, linear equity curve which lends itself to applying position sizing algorithms more easily.

I am not using the widely used least-squares linear regression calculation since that would consume too much a computing time with EasyLanguage. Instead, I am using a very similar calculation which is way faster:

1. first I calculate the average equity growth for 1 bar (total profit / number of bars) (vAvgBarEqGrowth, see below)

2. then I subtract this average value at that particular bar (for bar 100, it will be 100*average equity growth for one bar) from the actual equity value at that bar - I do this on each bar and I sum up all the differences into one temporary variable (tDiffSum)

3. this variable divided by the number of bars is the average shift of the average equity growth regression line from the ideal average growth equity line (vAvgLinShift, see below)

4. then I simply calculate the ideal growth equity line by adding the average shift (either positive or negative) to the average equity growth line (vLinEquity + vAvgLinShift, see below)

5. then I calculate the maximum deviation of the actual equity value from the ideal growth equity line by going thru all bars (vMaxDev = MaxList(vMaxDev, AbsValue(aCumEquity[tCounter] - (vLinEquity + vAvgLinShift))), see below)

6. the fitness function equals to total profit / maximum deviation (or error). GGO will try to maximize this value, thus producing very smooth equity curve (FitnessVal = aCumEquity[vTotalBars] / vMaxDev, see below).

Alternative to the total profit / max error fitness might be total profit / standard deviation. Alas, this would take up more time to calculate. The latter fitness function (TotalProfit/StdDev) would theoretically favorize solutions with well-proportioned errors from the linear equity line with only occasional outliers over jagged equity curve with a maximum outlier equal to max error. Given little difference between the two methods (I did some comparisons in excel) and faster execution of determining max error, I did chose total profit / maximum deviation (as revealed by the following code).

In the strategy code, I record the bar-by-bar equity values needed to perform the final calculation into the dynamic array aCumEquity[]:

Code: Select all

`vars:   vTotalBars(0),      vAvgBarEqGrowth(0),   vAvgLinShift(0),   vLinEquity(0),   vMaxDev(0),      tDiffSum(0),   tCounter(0);arrays:   aCumEquity[](0);if BarNumber >= 1 then   begin      Array_SetMaxIndex(aCumEquity, BarNumber);      aCumEquity[BarNumber] = NetProfit + OpenPositionProfit;      vTotalBars = BarNumber;   end;`

In GGO, I select any fitness function (for example, 2) and replace the code for that function by this one:

Code: Select all

`if FitnessType = 2 then   if (winners + losers) > 0 then      begin         vAvgBarEqGrowth = aCumEquity[vTotalBars] / vTotalBars;         vLinEquity = vAvgBarEqGrowth; { linear equity at bar 1            will be equal to the average bar equity growth value}         for tCounter = 1 to vTotalBars            begin               tDiffSum = tDiffSum + (aCumEquity[tCounter] - vLinEquity);               vLinEquity = vLinEquity + vAvgBarEqGrowth;            end;         vAvgLinShift = tDiffSum / vTotalBars; { average deviation            from the perfectly linear, though shifted equity curve}         vLinEquity = vAvgBarEqGrowth;         for tCounter = 1 to vTotalBars            begin               vMaxDev = MaxList(vMaxDev,                  AbsValue(aCumEquity[tCounter] - (vLinEquity + vAvgLinShift)));               vLinEquity = vLinEquity + vAvgBarEqGrowth;            end;         FitnessVal = aCumEquity[vTotalBars] / vMaxDev;      end   else      FitnessVal = -99999;`

Real-life example of linear regression-fitness-optimized equity curve of one of my strategies:
Attachments
linear-EQ.gif (18.93 KiB) Viewed 19032 times
Last edited by michal.kreslik on Thu Oct 26, 2006 1:45 pm, edited 1 time in total.

JPT
rank: <50 posts
Posts: 49
Joined: Mon May 22, 2006 3:26 pm
Reputation: 0
Location: Monterey, CA USA
Gender:
Michal:

Thanks for this code. I will try it on my Grail program

michal.kreslik
rank: 1000+ posts
Posts: 1034
Joined: Sat May 13, 2006 2:40 am
Reputation: 3
Location: Monte Carlo, Monaco
Real name: Michal Kreslik
Gender:
Contact:
JPT wrote:Michal:

Thanks for this code. I will try it on my Grail program

You're welcome, Jim,

I communicated with Wouter of Grail Systems, he said he may include this fitness function right within the Grail GO.

JPT
rank: <50 posts
Posts: 49
Joined: Mon May 22, 2006 3:26 pm
Reputation: 0
Location: Monterey, CA USA
Gender:
Michal:

Since I am new to arrays. It has been over 30 years since my last calculus class. Could you answer a few questions?

The first set of code sets up the array. I think I understand it but let me ask some ignorent questions:

The array_setMaxIndex instruction sets up the array size. Is that correct? Then for each bar you store the net profit + open position profit on that bar.

I guess I am confused on where you get the total profit of your position? Is that done in the next section where you calculate the fitness value?

Sorry for being so dense but ..... you are obviously a much better programmer than I

JPT
rank: <50 posts
Posts: 49
Joined: Mon May 22, 2006 3:26 pm
Reputation: 0
Location: Monterey, CA USA
Gender:
Michal:

Could you also explain your mathematical approximation for the least squares expression. I am interested in speeding up my code. I use the slope of the least squares line. Any hints would be apprciated

michal.kreslik
rank: 1000+ posts
Posts: 1034
Joined: Sat May 13, 2006 2:40 am
Reputation: 3
Location: Monte Carlo, Monaco
Real name: Michal Kreslik
Gender:
Contact:
JPT wrote:The array_setMaxIndex instruction sets up the array size. Is that correct?

Yes, Array_SetMaxIndex sets the size of dynamic array. Since we don't know beforehand how many bars there are on the chart, we need dynamic array for that.

JPT wrote:Then for each bar you store the net profit + open position profit on that bar.

I guess I am confused on where you get the total profit of your position? Is that done in the next section where you calculate the fitness value?

We are interested in bar by bar open profit, not the individual trades' profit.

michal.kreslik
rank: 1000+ posts
Posts: 1034
Joined: Sat May 13, 2006 2:40 am
Reputation: 3
Location: Monte Carlo, Monaco
Real name: Michal Kreslik
Gender:
Contact:
JPT wrote:Michal:

Could you also explain your mathematical approximation for the least squares expression. I am interested in speeding up my code. I use the slope of the least squares line. Any hints would be apprciated

You may apply the points 1 to 4 from my original post in this thread to get the least-squares-regression-resembling calculation to just about any time series data.

michal.kreslik
rank: 1000+ posts
Posts: 1034
Joined: Sat May 13, 2006 2:40 am
Reputation: 3
Location: Monte Carlo, Monaco
Real name: Michal Kreslik
Gender:
Contact:

I post here the Detailed equity curve of the linear-regression-optimized strategy, too:
Attachments
EC_detailed.jpg (63.28 KiB) Viewed 3155 times

JPT
rank: <50 posts
Posts: 49
Joined: Mon May 22, 2006 3:26 pm
Reputation: 0
Location: Monterey, CA USA
Gender:
Michal:

I saw the post. Good for you. It looks great. I just got caught in a very stupid trade today with the holiday week coming up. Well, I will learn eventually.

michal.kreslik
rank: 1000+ posts
Posts: 1034
Joined: Sat May 13, 2006 2:40 am
Reputation: 3
Location: Monte Carlo, Monaco
Real name: Michal Kreslik
Gender:
Contact:
Today I will test a slightly modified formula for this fitness.

The resulting general fitness value will be NetProfit/(MaxError squared)*TotalTrades instead of NetProfit/MaxError.