Multi-timeframe, no data backfill

[adrian]

In a strategy which runs on the 1min bars, I call:

Add(PeriodType.Day, 1);

during Initialize(). Only problem is, I no longer get any calls to OnBarUpdate! When I remove this call, OnBarUpdate() is invoked many times. I'm guessing that Ninja is looking to accumulate some number of daily bars before it will send any OnBarUpdate events. When running a backtest, I set the number of bars required to 1 and even 0, but this isn't helping. I'm connected to IQFeed so if it needs more data, Ninja can get it, but it doesn't even seem to try. I ran the backtest starting an extra day early so that the daily series could build up the extra data, but even this didn't help.

What am I doing wrong? How do I work around this?

thanks

[NinjaTrader_Dierk]

Expected. Reason: MinBarBars is set to 20 default. Meaning you would need to process the of 20 bars o both series (=20 bars of daily data) before OnBarUpdate is triggered in either series.

[adrian]

Expected. Reason: MinBarBars is set to 20 default. Meaning you would need to process the of 20 bars o both series (=20 bars of daily data) before OnBarUpdate is triggered in either series.

No, I set that to 1 and even to 0, but neither helped.

I had asked a similar question earlier and was told that Ninja would load the necessary data before starting the strategy, but it seems like that's not happening either.

[NinjaTrader_Dierk]

Then there is no data available on one of the series. Please make sure you are connected to a provider which supports daily backfill data (Zen e.g. does not).

[adrian]

Then there is no data available on one of the series. Please make sure you are connected to a provider which supports daily backfill data (Zen e.g. does not).

I'm connected to IQFeed and out of desperation and to verify the connection, I loaded a daily, 5min and 1min chart for the symbol (EXM) and the data loaded just fine. After pre-loading the data, I re-ran the backtest and I didn't get any OnBarUpdate() events with Add(PeriodType.Day, 1) but with it commented out, everything ran fine.

I checked the log and there aren't any errors.

I did have some good news. After all this, when I re-ran with MinBars = 0, and I start the backtest with a 4 day range, I start to get events. 3 day range, nothing, but 4 days, yes. (No idea what's going on, there.)

So I've got a work-around. Not ideal, but it's something. This is all leading up to a bigger and thornier question: I want to have 20 days of daily data available when the strategy goes live, but I don't want to have to run it for 20 days before I start getting my 1min bars! When I go live, is it going to act in this same way, or will it have the smarts to properly load the data?

(And if it can load the data properly when going live, why can't it do that when running backtests?)

[NinjaTrader_JoshP]

Hi adrian,

I think this is the issue you are running into. When you tell it 20 Min Bars Required and have a bars series of Daily you need at least 20 actual trading days on your chart already. This means you need to set your chart's Format Data Series to contain at least 20 actual trading days. Setting it to just 20 will not suffice since weekends subtract from the 20 days and you end up with not enough data in that bar series.

Solution: Try a larger number on your chart. Try something like 40 days back with "Min Bars Required" set to 20.

[adrian]

I think this is the issue you are running into. When you tell it 20 Min Bars Required and have a bars series of Daily you need at least 20 actual trading days on your chart already. This means you need to set your chart's Format Data Series to contain at least 20 actual trading days. Setting it to just 20 will not suffice since weekends subtract from the 20 days and you end up with not enough data in that bar series.

Solution: Try a larger number on your chart. Try something like 40 days back with "Min Bars Required" set to 20.

When you talk about 'charts', what do you mean? I'm just doing Strategy development and backtesting and there doesn't seem to be any setting on the charts to do this. I did open up a daily chart out of desperation in order to pre-load the data, and I set the "bars back" to 100, so that shouldn't be a problem.

I'll also remind you that I've got Min Bars set to '1' or '0' and not '20', so the number of days required is very low. I am running with my end day on Monday the 21st, start day on Friday the 18th (and 17th), and neither work despite giving a day or two to build up the required daily bars, yet no OnBarEvents are fired.

To test the 21st with MinBars=1, I need to start on the 16th, a full three trading days before. And this is despite the fact that all the necessary data is in the Ninja database.

Edit: Setting MinBars=0 has no effect. Starting on the 17th with MinBars = 0 is the same as with MinBars = 1. Wether MinBars = 0 or 1, both run but only when starting on the 16th.

[NinjaTrader_Dierk]

Josh will follow up shortly

[NinjaTrader_Dierk]

There are a few guys working on the same issue. I suggest focusing further posts on this thread:

http://www.ninjatrader-support.com/vb/showthread.php?p=37180

[NinjaTrader_Ray]

Please try this -

- Import attached strategy
- Connect
- Open Strategy Analyzer
- Open output window
- Backtest the strategy with settings in image
- Your should see several lines in the Output window

[adrian]

Please try this -

- Import attached strategy
- Connect
- Open Strategy Analyzer
- Open output window
- Backtest the strategy with settings in image
- Your should see several lines in the Output window

Ray,

Yes, if I go back far enough, eventually I get OnBarUpdate (and I did get output running this test). Let's try cinching those dates up a bit.


I first tried running the backtest
type: Minute
Value: 1
from 4/18/2008
to 4/21/2008
Exclude weekend: true
begins: 6:30 am
ends: 1:00 pm
min bars required: 0


I got nothing in the Output Window, even though it ran through two full days. When I ran from 4/17/2008 to 4/21/2008, I got:

4/18/2008 12:00:00 AM

in the Output Window.

Now here's the very weird thing. I changed OnBarUpdate to:

        protected override void OnBarUpdate()
        {
            if (BarsInProgress == 1)
                Print(Time[0]);
            if( BarsInProgress == 0)
                Print(Time[0]);
        }

So that it would print the 1-min bars as well. When I ran from 4/18/2008 to 4/21/2008 (Exclude weekends=True), I got nothing in the output window, even though it ran for two days. No daily bars, no 1-min bars, nothing.

Then, and this blows my mind, I ran from 4/17/2008 to 4/21/2008 (just one more day) and I got two days of output! Adding one more day goes from no output, to two days of output:

// ...
4/18/2008 12:54:00 PM
4/18/2008 12:55:00 PM
4/18/2008 12:56:00 PM
4/18/2008 12:57:00 PM
4/18/2008 12:58:00 PM
4/18/2008 12:59:00 PM
4/18/2008 1:00:00 PM
4/21/2008 6:31:00 AM
4/21/2008 6:32:00 AM
4/21/2008 6:33:00 AM
4/21/2008 6:34:00 AM
4/21/2008 6:35:00 AM
4/21/2008 6:36:00 AM
// ...

What the...?

[NinjaTrader_Ray]

Then you are able to get it to work. I guess if you start on the 17th, the 1st OnBarUpdate() for the daily bar is the 18th.

[adrian]

Then you are able to get it to work. I guess if you start on the 17th, the 1st OnBarUpdate() for the daily bar is the 18th.

Ray,

Yes, I think there's a bug in here somewhere but since I can get it to work the urgency/severity is lower.

When I run from the 18th - 21st, there's no output. From the 17th-21st, the only daily bar printed is the 18th (though there are two days of minute bars).

[bridenour]

I am having this exact same problem. If i use daily bars in my strategy, it will not process the current days signals. If i run a backtest, i have to run it several days back to get any signals from the current day.

if i run the strategy live in the morning, it simply won't process anythign for the current day. If I take out the day bars, the problem goes away.

unfortunately, day bars are useful for some things in strategy development. I read the entire thread below but there does not seem to be a resolution.

Any ideas about this?:

thanks

[NinjaTrader_Ray]

When running your strategy, can you try setting the paramter "Min. bars required" to zero. By default, it is 20 which means it needs 20 daily bars before OnBarUpdate() is called.