Let's Compare Strategies

[Adamdetx]

Marco, I appreciate the advice, I've got it running this morning on a sim account in real time, and with no slippage, slippage set to 1, and slippage set to 2 there is no significant difference. I'll post more results as I have them.

[marcow]

No slippage, I am using limit orders

No slippage, really ? In live trading your limit order will not be filled if the price touches it, because there are hundreds of orders in the orderbook which have to be filled first before your order gets filled. NT sim doesn't account for that. In my opinion you should use a slippage of 2 in sim to get somewhat reliable results on a strategy with tight stops and targets.

Did you also test the strategy in market replay ? First run 1 day without slippage, then the same day with slippage 1 and see what happens. Do you see what I mean ?

Again, I'm not criticizing your strategy, and I hope for you that it is profitable in live trading, I'm just pointing to the errors I made myself when I thought I had a profitable strategy.

The performance dates in this image are january 1st to 3rd 2012, that's not last week ...
Would like to see the results from last week.


Marco

[Adamdetx]

Performance results

No slippage, I am using limit orders. The performance is in the attached screenshot.

[marcow]

I started this running at the end of last week in sim mode on a live connection, and it has been running in real time since then.

What are the performance results for this time period ? How many ticks of slippage did you use ?

[Adamdetx]

I've found the same problem. About a year ago I blew out an account learning about false positives in backtests and optimizations. Now I don't run anything unless it runs well playing forward either on a replay, or real time on a sim account, or both. This strategy is really data intensive, so I can only run the replay for about a day at a time, but the individual days all support this data. I started this running at the end of last week in sim mode on a live connection, and it has been running in real time since then.

[jsktong]

BT meaning back testing.
Just to clarify, I am not by means expert. I am going through the learning phase as well..... I had a strategy which i set the stop too tight. So when I back tested, the result just shined on me. something like profit factor of close to 4. then I looked at the chart for entries and exits. almost 99% were perfect exits. meaning say I sold 100, bought back at 99.x. Everyone is a winner.
It turned out when you optimize or back tests with historical data, NT strategy analyzers doesn't know which price came first. so in fact, on the same trade in real time, it could've been selling 99.x before buying 100. So results are not real. Unless you run everything on tick basis, then it gives much more accurate results.

[Adamdetx]

I hate to sound terribly ignorant, but what do you mean by BT? It does have a very tight target and stop. As far as working with real money, on my other strategies I have found that when I can run them in sim they produce fairly similar results with real money. I expect this to be the same.

[jsktong]

Hey guys, I hope it's not to late to post my latest. Normally I go for longer term bots, but here is my latest scalper.

Profit Factor 2.76
Net Profit $5210
Percent Profitable 91.33%

It only has a few days of data in the screen shot since it a really data intensive strategy.
I have been able to take some random sampling of days in Dec 2011 and Jan 2012 and the results are congruent with what I'm seeing in Sim now. By the way the screen shot is from running sim in real time. If I can just scrape some money together now I can run this thing.

Question, this is a 2 day sampling. It trades so many times and each trade's pnl is quite small. did you have very tight stop/target? I don't think NT's BT is that real on such tight trades. If entries and target are within same bar, then this is prob not going to work in real money.

[Adamdetx]

My Strategy

Hey guys, I hope it's not to late to post my latest. Normally I go for longer term bots, but here is my latest scalper.

Profit Factor 2.76
Net Profit $5210
Percent Profitable 91.33%

It only has a few days of data in the screen shot since it a really data intensive strategy.
I have been able to take some random sampling of days in Dec 2011 and Jan 2012 and the results are congruent with what I'm seeing in Sim now. By the way the screen shot is from running sim in real time. If I can just scrape some money together now I can run this thing.

[sledge]

ES 06-12

Market Replay

This time the short side was strong than the long side. hmmm

ES 03-12 next.

[sledge]

I am going to start working on one now that you say it can be done.

I haven't tried it across weekends, not sure if Saturday would throw an error.

I record all my data, so haven't had any need to run it.

[koganam]

Profit Factor: 2.79
# of Trades: 376
# of Months tested: 11
Cum. Profit: 69%
Percent Profitable: 57.18%
Sharpe Ratio: 1.85
Ratio W/L: 2.11
Commission: True
Slippage: None
Max Drawdown: -1.31%

I have been optimizing this strategy since starting systems trading in August of this year. I have to laugh when I compare my original trading strategy to this one. I am not sure what I enjoy more writing code to increase the numbers above or actually trading. Live trading results have been similar to the numbers above - so far so good.

Since I am new to systems trading I am curious to see the honest backtest results of the other members as well as their comments of course.

Looking forward to your replies.

Well, here is one set of results.

[:::grimReaper:::]

However, if I am optimizing, lets say from 11/1/11 to 11/30/12 to be used for Dec 12 then am I not in good shape (albeit no one can predict the future) if for the last 12 months I am end up with the numbers I presented below. I need to understand how walking forward is more valuable than consistent profits over a long period.

Optimizer is just providing the optimal parameter settings in hindsight. You need to use a walk-forward to get a better idea of how it'll perform in the future. Read these links then ask if you have questions:

http://en.wikipedia.org/wiki/Walk_forward_optimization

http://en.wikipedia.org/wiki/Overfitting

If you don' understand these concepts, you will lose a lot of $. Good thing you did't go live yet.

[sledge]

What you are describing here is a walk-forward test, albeit done manually. Not to be talking down at you, just reminding us all, a walk forward test just means that you optimize over a period, then use the optimized parameters in a later period to see if you obtain similar results.

Personally, once I do that, I am at least going to run MarketReplay to see if the results are any good in quasi-live trading. If I have good results, only then will I consider live trading, first on Sim, then ...

This is probably "general news" for the people who have been around a while..

I was in a 'trading room' this past week, and was introduced to this kind of a chart.

NT does not provide this chart, yet.

I have a nice screen capture of the chart from the trading room, (which has copyrights from 2006-7 from some website) but this will suffice.

http://forum.mql4.com/35483/page2

It's the second image in the post.

replace the left side with NT's WIN/LOSS ratio, and the bottom with NT's WIN %.

to the right of the line would be "profitable". the further to the right and upper quadrant would be most excellent.

left of the line = keep trying.

This would be an easy chart for NT to produce in their Graphs section.

[koganam]

I appreciate the advice on walking forward and I will try to put more emphasis into using it as I better understand its value. However, if I am optimizing, lets say from 11/1/11 to 11/30/12 to be used for Dec 12 then am I not in good shape (albeit no one can predict the future) if for the last 12 months I am end up with the numbers I presented below. I need to understand how walking forward is more valuable than consistent profits over a long period. Originally I tried optimizing for 3 weeks but the results were skewed in that I can easily curve fit my parameters to any given 3-4 weeks.

I am going to play with walking forward and hopefully post some results.Again thanks for the advice.

Other than Sledge I still do not see anyone post their numbers so that we can compare and have this educational conversation.

What you are describing here is a walk-forward test, albeit done manually. Not to be talking down at you, just reminding us all, a walk forward test just means that you optimize over a period, then use the optimized parameters in a later period to see if you obtain similar results.

Personally, once I do that, I am at least going to run MarketReplay to see if the results are any good in quasi-live trading. If I have good results, only then will I consider live trading, first on Sim, then ...