Kreslik.com - traders community since 2006

Mikal and others:

What types of programs/procedures do you use for data mining a.k.a finding statistically relevant information out of historical data?

This is something I've been approaching in a very haphazard way and would like to improve.

Edward

Michal,

Here's another good reason http://www.smartquant.com/. I've got my C# books now and I'll be sharpening the pencils for awhile myself.

Edward

The Hueristic Lab is a GUI interface program written in C# (.Net compatible) and is designed to test various problems with a variety of heuristic methods including: GA, simulated annealing, tabu, etc. Obviously this is primarily an academic tool but could be converted to financial analysis quite eas...

Michal, Its a .pdf and should be working, but if not here is the page its listed on. http://www-illigal.ge.uiuc.edu/techreps.php3. Look for the following document: Yu, T.-L., Sastry, K., Goldberg, D.E (2005) Online Population Size Adjusting Using Noise and Substructural Measurements Obviously the Il...

Michal, Have you seen this journal article from IlliGAL (Illinois GA lab)? ftp://ftp-illigal.ge.uiuc.edu/pub/papers/IlliGALs/2005017.pdf The article discusses population sizing adjustments on the fly and also discusses various ways to determine appropriate population size from a set of parameters ba...

Forest,

Your right I hadn't noticed that Neuroshell Trader Pro had GA capabilities. Please keep us posted on how you like Neuroshell, since I've considered looking into them.

Edward

JPT, Let's try a couple of hypotheticals to get to the bottom of the problem. Say in real time you've been making money with this strategy for about a month. And then you test the strategy in backtesting for that same month and it turns out to be a loser. If this is the case then clearly the problem...

Michal,

That's very helpful, thank you. What types of side effects can you encounter when the mutation rate is too high (and how much is too high)?

Edward

jhtumblin, Historical drawdowns are not necessarily a worst case scenario. The markets a driven by non-linear statistics producing fat-tails. What that means is that your worst case scenario could be a whole lot worse than your historical testing suggests. Sometimes by a factor of 10 or more. I'd sa...

Michal, I like the idea of examining how many combinations I'm searching. And my combinations definately look like the U.S. deficit. Thanks for the spreadsheet. I guess my question is how do I use mutation probability and population size in correlation to the number of combinations I want to look at...