regression fitness function - smoothing out the equity curve

trading strategy optimization using genetic algorithms

Moderator: moderators

User avatar
michal.kreslik
rank: 1000+ posts
rank: 1000+ posts
Posts: 1032
Joined: Sat May 13, 2006 2:40 am
Reputation: 0
Location: Monte Carlo, Monaco
Real name: Michal Kreslik
Gender: Male
Contact:

regression fitness function - smoothing out the equity curve

Postby michal.kreslik » Thu May 18, 2006 10:53 pm

Hello,

in order to arrive at as linear equity curve as possible, I am using my own fitness function. I feed the resulting value of this fitness function to the genetic optimizer (I am using Grail GO) and GGO then tries to maximize the value.

This way, I am not interested in absolute NetProfit, but intead in a smooth, linear equity curve which lends itself to applying position sizing algorithms more easily.

I am not using the widely used least-squares linear regression calculation since that would consume too much a computing time with EasyLanguage. Instead, I am using a very similar calculation which is way faster:

1. first I calculate the average equity growth for 1 bar (total profit / number of bars) (vAvgBarEqGrowth, see below)

2. then I subtract this average value at that particular bar (for bar 100, it will be 100*average equity growth for one bar) from the actual equity value at that bar - I do this on each bar and I sum up all the differences into one temporary variable (tDiffSum)

3. this variable divided by the number of bars is the average shift of the average equity growth regression line from the ideal average growth equity line (vAvgLinShift, see below)

4. then I simply calculate the ideal growth equity line by adding the average shift (either positive or negative) to the average equity growth line (vLinEquity + vAvgLinShift, see below)

5. then I calculate the maximum deviation of the actual equity value from the ideal growth equity line by going thru all bars (vMaxDev = MaxList(vMaxDev, AbsValue(aCumEquity[tCounter] - (vLinEquity + vAvgLinShift))), see below)

6. the fitness function equals to total profit / maximum deviation (or error). GGO will try to maximize this value, thus producing very smooth equity curve (FitnessVal = aCumEquity[vTotalBars] / vMaxDev, see below).

Alternative to the total profit / max error fitness might be total profit / standard deviation. Alas, this would take up more time to calculate. The latter fitness function (TotalProfit/StdDev) would theoretically favorize solutions with well-proportioned errors from the linear equity line with only occasional outliers over jagged equity curve with a maximum outlier equal to max error. Given little difference between the two methods (I did some comparisons in excel) and faster execution of determining max error, I did chose total profit / maximum deviation (as revealed by the following code).

In the strategy code, I record the bar-by-bar equity values needed to perform the final calculation into the dynamic array aCumEquity[]:

Code: Select all

vars:
   vTotalBars(0),
   
   vAvgBarEqGrowth(0),
   vAvgLinShift(0),
   vLinEquity(0),
   vMaxDev(0),
   
   tDiffSum(0),
   tCounter(0);

arrays:
   aCumEquity[](0);

if BarNumber >= 1 then
   begin
      Array_SetMaxIndex(aCumEquity, BarNumber);
      aCumEquity[BarNumber] = NetProfit + OpenPositionProfit;
      vTotalBars = BarNumber;
   end;


In GGO, I select any fitness function (for example, 2) and replace the code for that function by this one:

Code: Select all

if FitnessType = 2 then
   if (winners + losers) > 0 then
      begin
         vAvgBarEqGrowth = aCumEquity[vTotalBars] / vTotalBars;
         vLinEquity = vAvgBarEqGrowth; { linear equity at bar 1
            will be equal to the average bar equity growth value}

         for tCounter = 1 to vTotalBars
            begin
               tDiffSum = tDiffSum + (aCumEquity[tCounter] - vLinEquity);
               vLinEquity = vLinEquity + vAvgBarEqGrowth;
            end;

         vAvgLinShift = tDiffSum / vTotalBars; { average deviation
            from the perfectly linear, though shifted equity curve}
         vLinEquity = vAvgBarEqGrowth;

         for tCounter = 1 to vTotalBars
            begin
               vMaxDev = MaxList(vMaxDev,
                  AbsValue(aCumEquity[tCounter] - (vLinEquity + vAvgLinShift)));
               vLinEquity = vLinEquity + vAvgBarEqGrowth;
            end;

         FitnessVal = aCumEquity[vTotalBars] / vMaxDev;
      end
   else
      FitnessVal = -99999;


Real-life example of linear regression-fitness-optimized equity curve of one of my strategies:
Attachments
linear-EQ.gif
linear-EQ.gif (18.93 KiB) Viewed 18103 times
Last edited by michal.kreslik on Thu Oct 26, 2006 1:45 pm, edited 1 time in total.

Please add www.kreslik.com to your ad blocker white list.
Thank you for your support.

JPT
rank: <50 posts
rank: <50 posts
Posts: 49
Joined: Mon May 22, 2006 3:26 pm
Reputation: 0
Location: Monterey, CA USA
Gender: None specified

Postby JPT » Thu May 25, 2006 4:46 pm

Michal:

Thanks for this code. I will try it on my Grail program

User avatar
michal.kreslik
rank: 1000+ posts
rank: 1000+ posts
Posts: 1032
Joined: Sat May 13, 2006 2:40 am
Reputation: 0
Location: Monte Carlo, Monaco
Real name: Michal Kreslik
Gender: Male
Contact:

Postby michal.kreslik » Thu May 25, 2006 5:01 pm

JPT wrote:Michal:

Thanks for this code. I will try it on my Grail program


You're welcome, Jim,

I communicated with Wouter of Grail Systems, he said he may include this fitness function right within the Grail GO.

JPT
rank: <50 posts
rank: <50 posts
Posts: 49
Joined: Mon May 22, 2006 3:26 pm
Reputation: 0
Location: Monterey, CA USA
Gender: None specified

Postby JPT » Thu May 25, 2006 5:07 pm

Michal:

Since I am new to arrays. It has been over 30 years since my last calculus class. Could you answer a few questions?

The first set of code sets up the array. I think I understand it but let me ask some ignorent questions:

The array_setMaxIndex instruction sets up the array size. Is that correct? Then for each bar you store the net profit + open position profit on that bar.

I guess I am confused on where you get the total profit of your position? Is that done in the next section where you calculate the fitness value?

Sorry for being so dense but ..... you are obviously a much better programmer than I

JPT
rank: <50 posts
rank: <50 posts
Posts: 49
Joined: Mon May 22, 2006 3:26 pm
Reputation: 0
Location: Monterey, CA USA
Gender: None specified

Postby JPT » Thu May 25, 2006 5:09 pm

Michal:

Could you also explain your mathematical approximation for the least squares expression. I am interested in speeding up my code. I use the slope of the least squares line. Any hints would be apprciated

Please add www.kreslik.com to your ad blocker white list.
Thank you for your support.

User avatar
michal.kreslik
rank: 1000+ posts
rank: 1000+ posts
Posts: 1032
Joined: Sat May 13, 2006 2:40 am
Reputation: 0
Location: Monte Carlo, Monaco
Real name: Michal Kreslik
Gender: Male
Contact:

Postby michal.kreslik » Thu May 25, 2006 11:28 pm

JPT wrote:The array_setMaxIndex instruction sets up the array size. Is that correct?


Yes, Array_SetMaxIndex sets the size of dynamic array. Since we don't know beforehand how many bars there are on the chart, we need dynamic array for that.

JPT wrote:Then for each bar you store the net profit + open position profit on that bar.

I guess I am confused on where you get the total profit of your position? Is that done in the next section where you calculate the fitness value?


We are interested in bar by bar open profit, not the individual trades' profit.

User avatar
michal.kreslik
rank: 1000+ posts
rank: 1000+ posts
Posts: 1032
Joined: Sat May 13, 2006 2:40 am
Reputation: 0
Location: Monte Carlo, Monaco
Real name: Michal Kreslik
Gender: Male
Contact:

Postby michal.kreslik » Thu May 25, 2006 11:33 pm

JPT wrote:Michal:

Could you also explain your mathematical approximation for the least squares expression. I am interested in speeding up my code. I use the slope of the least squares line. Any hints would be apprciated


You may apply the points 1 to 4 from my original post in this thread to get the least-squares-regression-resembling calculation to just about any time series data.

User avatar
michal.kreslik
rank: 1000+ posts
rank: 1000+ posts
Posts: 1032
Joined: Sat May 13, 2006 2:40 am
Reputation: 0
Location: Monte Carlo, Monaco
Real name: Michal Kreslik
Gender: Male
Contact:

Postby michal.kreslik » Fri May 26, 2006 3:31 pm

As a reaction to the Tradestation forums thread:
https://www.tradestation.com/Discussion ... c_ID=52568

I post here the Detailed equity curve of the linear-regression-optimized strategy, too:
Attachments
EC_detailed.jpg
EC_detailed.jpg (63.28 KiB) Viewed 2226 times

JPT
rank: <50 posts
rank: <50 posts
Posts: 49
Joined: Mon May 22, 2006 3:26 pm
Reputation: 0
Location: Monterey, CA USA
Gender: None specified

Postby JPT » Fri May 26, 2006 3:44 pm

Michal:

I saw the post. Good for you. It looks great. I just got caught in a very stupid trade today with the holiday week coming up. Well, I will learn eventually.

User avatar
michal.kreslik
rank: 1000+ posts
rank: 1000+ posts
Posts: 1032
Joined: Sat May 13, 2006 2:40 am
Reputation: 0
Location: Monte Carlo, Monaco
Real name: Michal Kreslik
Gender: Male
Contact:

Postby michal.kreslik » Mon Jun 05, 2006 1:35 pm

Today I will test a slightly modified formula for this fitness.

The resulting general fitness value will be NetProfit/(MaxError squared)*TotalTrades instead of NetProfit/MaxError.

Specifically for Tradestation & GGO:

FitnessVal = aCumEquity[vTotalBars] / power(vMaxDev, 2) * (winners + losers);

This way, the genetic algorithm should favorize candidate solutions with higher statistical significance - the more observations (trades), the more certainty that the results (trading system behavior) are not caused by a chance alone. Also, the assessment whether the equity curve is linear wil be done exponentially (the squared MaxError in the formula).

Please add www.kreslik.com to your ad blocker white list.
Thank you for your support.


Return to “genetic optimization”