Transaction by transaction backtesting/replay

[dom993]

Hi there,

I understand Ninja's Backtest feature always uses CalculateBarOnClose = True for the strategy.

This is perfect for strategies intended to be used live with CalculateBarOnClose = True, but for strategies intended to be used live on a tick-by-tick basis, this doesn't provide a mean to test/verify the real-time behavior of the software.

So far, I have identified that Market Replay provides that transaction by transaction replay capability, however it seem to require "Market Replay data", which can be either captured in real-time, or downloaded from the Ninja server.

I want to do several years of market replay, the real-time capture is no use to me at this point, and the Ninja server doesn't seem to have more than a week (or two) of market replay data available.

I do not need Level-1 for my replays, just replaying the transaction history (transaction by transaction) would do, I have 3 years of tick-data available in Ninja, how do I get Ninja to use that tick-data for Market Replay?

Thanks in advance

[NinjaTrader_AdamP]

dom,

Its also possible to back test with something like CalculateOnBarClose = false. This is done by adding intrabar granularity to your code. Below is a reference sample for this.

http://www.ninjatrader.com/support/forum/showthread.php?t=6652

As far as getting that much market replay data, it may be unavailable. I am aware of a 3rd party tool that can reply tick data through the external feed but I can't remember what it was called. Apologies for that, hopefully you would be able to find it. I seem to remember it in this forum. I will look around and see if I can find it.

[dom993]

Adam,

I am getting very odd "results" ... my main chart is a 100-Volume CL chart, I add in the strategy a 2nd timeframe using 10-Volume, and in this simple test I count the number of "intrabars" inside each main bar.

When I load the strategy, apparently some "main" bars can count at least 33 intrabars ?!!
(see attached)

#region Using declarations
using System;
using System.ComponentModel;
using System.Diagnostics;
using System.Drawing;
using System.Drawing.Drawing2D;
using System.Xml.Serialization;
using NinjaTrader.Cbi;
using NinjaTrader.Data;
using NinjaTrader.Indicator;
using NinjaTrader.Gui.Chart;
using NinjaTrader.Strategy;
#endregion

// This namespace holds all strategies and is required. Do not change it.
namespace NinjaTrader.Strategy
{
/// <summary>
/// Enter the description of your strategy here
/// </summary>
[Description("Enter the description of your strategy here")]
public class TestDualTF : Strategy
{
#region Variables
// Wizard generated variables
// User defined variables (add any user defined variables below)
int _MainBarCount;
int _IntraBarCount;
#endregion

/// <summary>
/// This method is used to configure the strategy and is called once before any strategy method is called.
/// </summary>
protected override void Initialize()
{
// Add 2nd timeframe
Add( PeriodType.Volume, 10);

_IntraBarCount = 0;
_MainBarCount = 0;

CalculateOnBarClose = true;
}

/// <summary>
/// Called on each bar update event (incoming tick)
/// </summary>
protected override void OnBarUpdate()
{
if (BarsInProgress == 0)
{
if (_MainBarCount == 0)
_MainBarCount = CurrentBars[0];
else
_MainBarCount += 1;
Print("TestDualTF main bar# " + CurrentBars[0] + " check : " + _MainBarCount);
_IntraBarCount = 0;
}
else
{
_IntraBarCount += 1;
Print("Main Bar# : " + CurrentBars[0] + " - check : " + _MainBarCount + " - Sub Bar# : " + CurrentBars[1] + " - IntraBarCount : " + _IntraBarCount);
}
} // OnBarUpdate()

#region Properties
#endregion
}
}

[NinjaTrader_AdamP]

dom,

This is to be expected on tick-based charts. There are a variable number of ticks per bar. Tick-based charts like volume, range, tick, etc. are time independent so they will not always match up exactly within some set time frame.

[dom993]

So basically, you were suggesting me to use a "feature" you know doesn't work ?!

Let's reset then ... this IntraBar granularity doesn't work for my application, and I still need to do heavy transaction by transaction replay to verify the real-time behavior of my strategy.

I have 3 years of tick-data, I am using 100-Volume as my strategy timeframe, how do I get Ninja to perform a transaction by transaction replay of the 3 years of tick-data?

OR

How do I convert 3 years of tick data into 3 years of market replay data ?

[NinjaTrader_AdamP]

dom,

I am not sure what you mean here. Could you clarify what your expectations are so I may comment further?

Ticks come in sporadically, and there will be a different number of ticks per minute for example. One minute there may be 10 the next there may be 35. This has to do with the exchange, and how humans at the exchange are placing orders more or less at random times. A tick represents when the price changed due to change in supply/demand based on the activity at the exchange.

In real time trading you would be getting the same behavior as the intrabar granularity example provides to you in backtesting.

[dom993]

Adam,

I mentioned that my main chart is 100-Volume. For the "IntraBar" I use 10-Volume, there is no question there should be 10 * 10-Volume bars per 100-Volume bar.

You then wrote "This is to be expected on tick-based charts. There are a variable number of ticks per bar. Tick-based charts like volume, range, tick, etc. are time independent so they will not always match up exactly within some set time frame.", which means you know Ninja won't work the way I am expecting it to do.

Your last answer doesn't explain anything. In real-time like in reading the tick database, there should always be 10 * 10-volume bars to form one 100-volume bar. This is not random, basic mathematics.

Fine. I am already tired of this. But I need a solution. So let me ask you again:


I have 3 years of tick-data, I am using 100-Volume as my strategy timeframe, how do I get Ninja to perform a transaction by transaction replay of the 3 years of tick-data?

OR

How do I convert 3 years of tick data into 3 years of market replay data ?

[NinjaTrader_AdamP]

dom,

I have 3 years of tick-data, I am using 100-Volume as my strategy timeframe, how do I get Ninja to perform a transaction by transaction replay of the 3 years of tick-data?

If you want tick by tick back testing, what I sent you as a reference sample is recommended. You can add the following :

Add(PeriodType.Tick, 1);

This is transaction by transaction, as its tick data. It would call OnBarUpdate() for each tick, which is the same as COBC = false on a live chart. If you intend to backtest similar to COBC = false, this is the only way to do it. Adding volume bars for intra-bar granularity would not be the same, and my comments were assuming you were using Add(PeriodType.Tick, 1);, so apologies for misreading you.

This volume bar issue I would need to replicate on my end as what you said makes sense.

How do I convert 3 years of tick data into 3 years of market replay data ?

Unfortunately there are no tools to do this currently, however it would be possible to have a tool that connects to the external data feed which would replay historical data files.

http://www.ninjatrader.com/partners#Third-Party-Add-Ons

[dom993]

dom,

If you want tick by tick back testing, what I sent you as a reference sample is recommended. You can add the following :

Add(PeriodType.Tick, 1);

Sure ... I added a means to keep track of the IntraBarVolume, and guess what? It doesn't add-up to 100 ... never.

Please, run this on a 100-Volume chart & see for yourself.


// This namespace holds all strategies and is required. Do not change it.
namespace NinjaTrader.Strategy
{
/// <summary>
/// Enter the description of your strategy here
/// </summary>
[Description("Enter the description of your strategy here")]
public class TestDualTF : Strategy
{
#region Variables
// Wizard generated variables
// User defined variables (add any user defined variables below)
int _MainBarCount;
int _IntraBarCount;
double _IntraBarVolume;
#endregion

/// <summary>
/// This method is used to configure the strategy and is called once before any strategy method is called.
/// </summary>
protected override void Initialize()
{
// Add 2nd timeframe
Add( PeriodType.Tick, 1);

_IntraBarCount = 0;
_IntraBarVolume = 0;
_MainBarCount = 0;

CalculateOnBarClose = true;
}

/// <summary>
/// Called on each bar update event (incoming tick)
/// </summary>
protected override void OnBarUpdate()
{
if (BarsInProgress == 0)
{
if (_MainBarCount == 0)
_MainBarCount = CurrentBars[0];
else
_MainBarCount += 1;
Print("TestDualTF main bar# " + CurrentBars[0] + " check : " + _MainBarCount + " - MainBarVolume : " + Volumes[0][0]);
_IntraBarCount = 0;
_IntraBarVolume = 0;
}
else
{
_IntraBarCount += 1;
_IntraBarVolume += Volumes[1][0];
Print("Main Bar# : " + CurrentBars[0] + " - check : " + _MainBarCount + " - Sub Bar# : " + CurrentBars[1] + " - IntraBarCount : " + _IntraBarCount + " - IntraBarVolume : " + _IntraBarVolume);
}
} // OnBarUpdate()

#region Properties
#endregion
}
}

[NinjaTrader_AdamP]

dom,

I will test on my end and get back to you as soon as I can.

[NinjaTrader_AdamP]

Dom,

This behavior has something to do with the way volume bars are constructed as well as how ticks are handled. They aren't synced based on volume so its tough to compare two charts here. I would just suggest using the tick data series so that in backtesting it just plays back tick-by-tick, as this would be the most realistic simulation of how market data would work in real time trading.

Usually volume bars are filled by an overflowing volume tick. For example, if I have 3 ticks in sequence and each tick has 4 volume reported. Then it totals 12 volume. The 2 extra volume would go into calculating the next bar on a 10 volume chart. Since the series are synced by timestamp down to a 1-second granularity you can run into an issue like you are running into now. There are suggestions for how to get around this if you are submitting orders based on two different time frames however for the purposes of backtesting, the most accurate is to use a 1 tick series. This same behavior wouldn't occur in time-dependent charts, its just a currently limitation of multi-series time independent indicators/strategies.

Basically, this is on our list to check into further for future releases and is currently expected behavior.

[dom993]

Adam,

I just ran the exact same strategy on a 100-ticks chart, and I get the IntraBarCount getting as low as 1, and as high as 632 ... it doesn't work any better.

[dom993]

Adam,

I think I now understand what the current behavior is - your statement "the series are synced by timestamp down to a 1-second granularity" triggered me to analyze the flow of calls to OnBarUpdate() for both timeframes, focusing on timestamps. The following is my current understanding of NinjaTrader's behavior, both for you to confirm, and for others who might benefit from it:

- Ninja accumulates transactions (aka ticks) for the current second ... if at any point the base timeframe completes building a bar, OnBarUpdate() is called for BarsInProgress == 0

- When the most recent transaction is the first of a new second, Ninja process all the transactions accumulated in the prior second interval for the additional timeframes.

As a result, some transactions that were actually part of the last base-timeframe completed bar can "appear" later in an additional timeframe (after OnBarUpdate() has been called for the base-timeframe bar) .

As an example, using 5-min as base timeframe and 1-sec as 2nd timeframe, we would see OnBarUpdate() called in the following sequence:

..
- 2nd timeframe 8:59:59

- Base timeframe 9:00:00
- 2nd timeframe 9:00:00

- 2nd timeframe 9:00:01
- 2nd timeframe 9:00:02
- 2nd timeframe 9:00:03
..
- 2nd timeframe 9:04:59

- Base timeframe 9:05:00
- 2nd timeframe 9:05:00
- 2nd timeframe 9:05:01
- 2nd timeframe 9:05:02
..
- 2nd timeframe 9:09:59

- Base timeframe 9:10:00
- 2nd timeframe 9:10:00

- 2nd timeframe 9:10:01
- ...


I hope I get it right! if it's the case, I can get this to work for my existing systems.

[NinjaTrader_Brett]

Hello,

This is exactly the case:

What you can also find that due to this MTF has a chance of "peeking" into the future as well.

Second granularity is a common theme in NinjaTrader 7. Furthermore, in our next major release we are looking into .net TICK timeframes and their effect on performance for next major release. Next major release does not have a release date at this time.

-Brett

[levesque]

I've converted raw tick data to Market Replay format using MS Access. I can either point you in the right direction or possibly do the conversion for you if you're still interested.