## probability: statistics & regression to the mean

If you don't know where to start, start here! Don't be afraid to ask questions.

Moderator: moderators

bazmonaut
rank: <50 posts
Posts: 18
Joined: Fri Jul 03, 2009 6:48 am
Reputation: 0
Location: Devon, United Kingdom
Gender:

### probability: statistics & regression to the mean

I've been doing a lot of reading (like Patch), especially the 'Never Lose Again' thread (babypips), and the embedded thread from jjrvat on price analysis. (If you haven't read these, do).

I have learned:
- that indicators are just an expression of price
- EVERYTHING in trading is built on price action
- many successful traders DON'T USE INDICATORS (eh?)
- testing strategies (e.g. EAs) is LESS IMPORTANT than understanding the market and price action
- an understanding the market and price action can be gained through analysis of STATISTICS and OBSERVATION of price action

I am slowly coming to the realisation that I have been approaching this arse-about. I have been coding/strategizing my little fingers off trying to get profitable results. I think now that this is the END RESULT and not the FIRST STEP.

The FIRST STEP is understanding basic price analysis and the market. Seems like a big ask for a relative newbie, to match the street smarts and experience of traders with 30+ years... "It's easy when you know how", or rather, it's easy when you have a good understanding of the market and price action.

TRO says over and again, 'anything can happen'. Yet he KNOWS what will PROBABLY happen next. The same for other successful traders. They know where price has to go next (probably). So it seems to me that indicators are just a GUIDE to determining the PROBABILITY of what will happen next.

Statement: an understanding of what will PROBABLY happen next is key to trading successfully.

WHAT DO YOU THINK?

What's my grade? Have I just been appointed 'Minister for Stating the Bleeding Obvious'?

More to follow...

bazmo
becoming stranger in a strange land

monolisa
rank: 150+ posts
Posts: 351
Joined: Fri Jun 12, 2009 11:38 am
Reputation: 0
Location: All Quiet on the Western Front
Gender:
Hi bazmo,

I would think an understanding of what is (also a bit of what has been) happening is part of the key to profitable trading. Another part is your money/risk management.

If you have time have a look at the NLA thread here in kreslik.

Lis
"Know your enemy and know yourself, find naught in fear for 100 battles. Know yourself but not your enemy, find level of loss and victory. Know neither your enemy or yourself, wallow in defeat every time." - Sun Tzu

bazmonaut
rank: <50 posts
Posts: 18
Joined: Fri Jul 03, 2009 6:48 am
Reputation: 0
Location: Devon, United Kingdom
Gender:
(Shamelessly stealing/restating the ideas and hard-won knowledge of others, thanks guys!)

...mmm so...

I also think that PROBABILITY is closely aligned with what we might call COMMON SENSE.

I've done some basic high school statistics and probability but I'm no maths whizz. Seeing this as COMMON SENSE helps me to understand it. If the definition of insanity is 'doing the same thing over and again while expecting a different result each time', then let's call the definition of COMMON SENSE as 'doing the same thing over and again and expecting the same result each time'. It's common sense because 'it' happened last time, so it makes sense that 'it' will happen again this time. However, we're not surprised if exactly the same thing DOESN'T happen, we just 'know' that it's quite likely to happen.

A trading example: if price is moving UP then COMMON SENSE says that it will most likely move UP more. It is less likely to move down. It MIGHT move down, but really we should expect it to move up... Getting warmer...

That trading example 'makes sense' to me. I get it. What I don't have is any PROOF that this is actually the case. I don't like stating things as fact if I don't REALLY KNOW. It leads to the spread of ignorance, prejudice etc etc. Also, I want to know exactly HOW LIKELY this is to happen. Sure we can expect it to happen most of the time, but what is 'most' anyway? Two out of every three times? Six out of ten? Fifty-one out of a hundred?

Are you interested in finding out what the REAL figure is? (I am!) Do you want to KNOW for yourself rather than trusting the 'facts' as presented by others (no disrespect intended to the 'others')? (I do!)

WARNING: I am making this up as I go along! Woohoo!

Note: the section, 'how to collect price data & do your own probability analysis' has been moved to another post (today). This other post has all the details on how I collected/analysed this data to find the result...

My test in English:
IF the colour of the current candle is green (up), then how often is the colour of the NEXT candle also green? Ditto for red candles... I have also added a 'doji' label, if open price = close price... OK, it's not perfect, but as I said earlier, we should be able to draw some conclusions from our analysis.

The result is that in 47% of cases (excl. dojis) a green candle will follow a green candle, or a red will follow a red...

That's NOT what I thought at the start of this post! COMMON SENSE has taken a beating at the hands of mean old statistics! BOO... But, do I have enough data? Is my data correct? Is my analysis correct? Is my analysis statistically significant (i.e. is 47% close enough to 50% to make no difference?)...

But wait, all is not lost. Maybe we are ASKING THE WRONG QUESTIONS. I can't imagine that anyone would base a strategy solely on the colour of the previous candle... at least, now I can see that this factor alone is not going to make me profitable...

- how often does a doji REALLY signal a change in trend direction?
- how often does a close off the low result in the following candle being red?
- how often does a candle reach it's average range? (... between 08:00 and 12:00 GMT)
- what is the maximum and average deviation I can expect from average candle range?

So, what ARE the RIGHT questions to ask?

What do YOU think?

More to come, but I think that's enough for now...

bazmo
Attachments
DataCollector v1 - bazmonaut.zip
Last edited by bazmonaut on Tue Jul 14, 2009 11:28 am, edited 1 time in total.
becoming stranger in a strange land

monolisa
rank: 150+ posts
Posts: 351
Joined: Fri Jun 12, 2009 11:38 am
Reputation: 0
Location: All Quiet on the Western Front
Gender:
Interesting... I suppose if you collect all available data the prob would be close to 50%? There is only one way out of the two (up or down) if you exclude same open/close price.

I would be cautious with "common sense". The smart money know what "common sense" is and take advantage of it to the max.

My question is: Why the current bar go up (or down)?

Lis
"Know your enemy and know yourself, find naught in fear for 100 battles. Know yourself but not your enemy, find level of loss and victory. Know neither your enemy or yourself, wallow in defeat every time." - Sun Tzu

bazmonaut
rank: <50 posts
Posts: 18
Joined: Fri Jul 03, 2009 6:48 am
Reputation: 0
Location: Devon, United Kingdom
Gender:
Hi monolisa

For a newbie, the 'why' is the most difficult thing to find out... I'm trying to break it down a little in this post, just looking at probability.

Agree totally that money management is also a key factor - I just go for 5pts, when I'm winning consistently then I'll look at partial exits and longer trades. In all my tests, win/loss ratio was THE deciding profitability factor - adjusting SL/TP of course important, but if you're not winning consistently then this is academic IMHO...

bazmo
becoming stranger in a strange land

bazmonaut
rank: <50 posts
Posts: 18
Joined: Fri Jul 03, 2009 6:48 am
Reputation: 0
Location: Devon, United Kingdom
Gender:
I found this:

I notice that average range is a recurring theme in BuyZone and DTB... ATR is just an expression of the relative size of the green and red objects on my chart, it's something that (with practice) I should be able to see on the chart without any indicators. But for now, I'd like to really understand how this works, prove it to myself...

I'm also very interested in exploring the concept of 'regression to the mean'. Statistics again...

Can anyone recommend a good book on stats?

My next step: collect some data and perform some analysis to based on ATR and the strategy outlined in the above link.

Anyone else got ideas?
becoming stranger in a strange land

noone22
rank: 150+ posts
Posts: 390
Joined: Tue Aug 05, 2008 4:50 am
Reputation: 0
Gender:
bazmonaut wrote:Can anyone recommend a good book on stats?

Noone could recommend this good book
508 pages cover a lot
"Probability And Statistics Textbook"
Forex_in_the_Past

I'm not going into discussion about stats & probabilities
with you or others, but my opinion is - your initial
approach and questions are very general and won't lead
you anywhere (in practice).
Even if you know (for sure) from stats or any other source,
that probability of appearing of next candle to be green is 47.48% -

To succeed, you need to find more probable scenario
(75% probability at least) with much more strict and limited condition.

Something like this
(note, this is absolutely artificial rule,
I'm writing it just for example
and I've created it from scratch 5 minutes ago
- so don't attempt to trade it):

GBPUSD green candle is exceeding 10 pips in 78.79% cases
during opening hour of London 8am GMT
and probability of such candle to be green is 75.76%,
if ATR > 12 and MACD is rising.

- this may be of some value.