indicator values for non-daily bars not consistent with daily bars?

[Harry]

Harry,
My apologies because I have not yet addressed the code and ideas you sent to me. Further, I probably won’t for another few weeks because of vacation, then personal business trip, then helping to repair houses of destitute people in Appalachia with a church team [even though I am not a church guy].
I really appreciate your time and suggestions, and absolutely will respond.

Just contact me via private message, if you need some further guidance. We can also set up a Skype session, if necessary.

[joemiller]

Harry,
My apologies because I have not yet addressed the code and ideas you sent to me. Further, I probably won’t for another few weeks because of vacation, then personal business trip, then helping to repair houses of destitute people in Appalachia with a church team [even though I am not a church guy].
I really appreciate your time and suggestions, and absolutely will respond.

[joemiller]

turns out I am using session template Forex, see attached.

thanks for the indicator. I will get on it later this morning.

[Harry]

You said in post72 “I have read the entire thread…..” so I assumed you knew “red line”.
The red line is the visual representation of the indicator value.
The indicator is a function of yesterday’s day bar OHLC.
If red line [the indicator] is different for day and non-day, then indicator values are not equal and red lines are not equal.
I am using hourly bars for debugging, it is less cumbersome. Minute bars, to be used eventually, result in an even wider discrepancy between results for day and non-day bars.

I have directly checked input data many times using detailed “print to output window” debug coding statements. All that is buried in previous posts [which you have read]. Of course there is no way around checking input data and I have done that, many times.

“If the indicator has been correctly coded” is paramount. I will not be surprised if it has not been correctly coded. I will be surprised if it has.
The session template is 24 hours x 5 days …. I will check on that.

presently I am starting to work on the “add a daily data series via the Add() method” suggestion.

I am fortunate to have your help. Much appreciated.

3:32am? Where are you?

If you use a template 24/5, you will never make it anywhere, as Bars.GetDayBar() will only produce nonsense bars. It is absolutely necessary that you use the session template FOREX.

Please use the test indicator attached and compare the values displayed for yesterday's HLC with the values for the daily bars. The test indicator does not show the open, but you can add it, if needed.

Minute bars and hourly bars should produce the same result with the correct session template.

I am 6 hours ahead of you in Central Europe.

[joemiller]

You said in post72 “I have read the entire thread…..” so I assumed you knew “red line”.
The red line is the visual representation of the indicator value.
The indicator is a function of yesterday’s day bar OHLC.
If red line [the indicator] is different for day and non-day, then indicator values are not equal and red lines are not equal.
I am using hourly bars for debugging, it is less cumbersome. Minute bars, to be used eventually, result in an even wider discrepancy between results for day and non-day bars.

I have directly checked input data many times using detailed “print to output window” debug coding statements. All that is buried in previous posts [which you have read]. Of course there is no way around checking input data and I have done that, many times.

“If the indicator has been correctly coded” is paramount. I will not be surprised if it has not been correctly coded. I will be surprised if it has.
The session template is 24 hours x 5 days …. I will check on that.

presently I am starting to work on the “add a daily data series via the Add() method” suggestion.

I am fortunate to have your help. Much appreciated.

3:32am? Where are you?

[Harry]

Red line indicator differences are shown in the attachments. I am having a heck of a time trying to get set up to back test the modified indicator file from post#62 [as I said I would do in my previous post]. I don’t understand the error descriptions I am getting so I just dig myself deeper into confusion. I am going to bag it for now and try the “add a daily data series via the Add() method” suggestion. Based on my foggy understanding of what that will do I’m thinking it will work simply and cleanly. Stand by, I may need some help [for which I will be eternally grateful].
The attachments show red line indicators to be
date 1 hr bars 1 day bars
8/29 1.3374 1.3363
8/28 1.3369 1.3369
8/27 1.3375 1.3367
8/26 1.3361 1.3362

I do not know what a red line indicator is. If we want to find out about the differences, we need to know the session template and then directly check the input data, not the indicator values.

There is no way around checking input data.

If the indicator has been correctly coded, indicator values will be identical as long as input data is identical.

[joemiller]

Red line indicator differences are shown in the attachments. I am having a heck of a time trying to get set up to back test the modified indicator file from post#62 [as I said I would do in my previous post]. I don’t understand the error descriptions I am getting so I just dig myself deeper into confusion. I am going to bag it for now and try the “add a daily data series via the Add() method” suggestion. Based on my foggy understanding of what that will do I’m thinking it will work simply and cleanly. Stand by, I may need some help [for which I will be eternally grateful].
The attachments show red line indicators to be
date 1 hr bars 1 day bars
8/29 1.3374 1.3363
8/28 1.3369 1.3369
8/27 1.3375 1.3367
8/26 1.3361 1.3362

[Harry]

Hi Joe,

I think the small differences you are seeing between the real daily bars and the virtual daily bars are due to how the FOREX markets work and how the data provider collect the data.

Why daily data can be different

Futures are traded on a centralized exchange, so for all data providers you should theoretically see the same open, high, low and close to the tick. However, in reality this is not the case, as

- some data providers build daily bars from the regular session only (Zenfire CQG)
- some data providers build daily bars from the full session with the close (Broker data)
- some data providers build daily bars from the full session with the settlement price (DTN/IQ, Kinetick)

In fact each data provider has its own rules to build daily bars. Those daily bars are directly imported by NinjaTrader and not treatedin anyway. So what you get as a daily bar essentially depends on your data provider.


The specific case of FOREX markets

For FOREX things are even worse.

FOREX is not traded on a centralized exchange, but across various electronic communication networks. This means that at each moment there are various prices traded. Also for FOREX there is no official session. Most of the US brokers use the session from 5:00 PM EST to 5:00 PM EST, but not all London FOREX brokers follow this habit, Their trading day may start two hours later at 0:00 GMT.

Some data providers also have daily data, which comes averaged over several communication networks and does not exactly match the intraday data of the ECN that you traded. Don#t be astonished that there are a few ticks difference.

Now, if you trade a real account, and if you want to set up a backtest prior to trading that real account, it is important

- that you use data from the ECN that you use for trading (if you trade with Interactive Brokers you cannot use FOREX data from FXCM or Tenfore)
- that you use data which shows the actual trades that were executed on your ECN and not something else


For backtesting use Bars.GetDayBar()

The data you want to use is the data that can be retrieved from your virtual daily bars, as those bars show the first and last tick traded over your trading day (as defined per session template) and they also show the highest and lowest trade executed during the trading day for that network!

In fact, you do not want to use the daily bars that are supplied from your data provider, but for your backtest it is preferable to use exactly those virtual bars returned by Bars.GetDayBar()! Only pay attention to the session template that you use. Then simply ignore the daily data supplied by your data provider.

[joemiller]

Dearest Harry,

OK, I will get set up to run the latest indicator version sent to me in post# 62 and will report results whatever they are. I earnestly hope you are correct in everything you say, then I can do what I started out to do before start of this dialogue four weeks ago. My time also has been occupied for those 4 weeks [since Aug2]. I kept pursuing this , not to be a pain in the rear end, but because I was able to consistently demonstrate wrong previous day OHLC.

I simply need VALID/TRUE/CORRECT previous day OHLC values to be presented to me when using minute bars. What part of that is so hard to understand? I am clearly aware that The Bars.GetDayBar() function is working exactly as described in the user manual … but it is NOT perfectly fit for my purpose because it is not doing what I need it to do. I have clearly demonstrated that it is not doing what I need. I will attempt to demonstrate that again and will report results, whatever they are. What you call complaining I call pursuit of an answer.

I hope accessing daily historical data by adding a daily data series via the Add() method solves the problem if necessary. I will try that if necessary after doing the test described above.

I don’t see that I have burdened NT with ‘programming services’. The only ‘programming services’ ever rendered have been the few lines of code suggested multiple times to alter indicators and strategies created by me [ie] ‘to try this and see if it works’.

I should have said thanks for the offer of help. However I ignored the offer because posting more charts and session template was just more of the same old same. At that point I had decided to forget all this and move on. However, per the test I am going to do described above, I will be sending results one more time.

[Harry]

This has been so typical [ie] try this, try that, do this, do that, test this, test that, send me this, send me that. I do appreciate your response, but I just plain can’t mess with this anymore. Please look at the extensive evidence I have presented in this thread demonstrating that Bars.GetDayBar() does not work as I expect it to work. At one point in this thread I got the agreement that, yes, it makes sense to request NT programming development to create a Bars.GetDayBar() function that will plainly, simply and cleanly do what I expect. I don’t know why that crashed and burned, but it did. Thanks again for your response.

I have read the entire thread. The Bars.GetDayBar() function is exactly working as described in the user manual and it is perfectly fit for your purpose.

You can also access daily historical data by adding a daily data series via the Add() method.

You have occupied at least four or five staff over the last days, who have tried to help you by explaining how the method works and by suggesting you an alternative method to retrieve daily bars from daily data. I have also tried to help you and asked you to post charts and session template.

But probably nobody can solve your problem, as you complain all the time, but do not listen to anybody.

[joemiller]

This has been so typical [ie] try this, try that, do this, do that, test this, test that, send me this, send me that. I do appreciate your response, but I just plain can’t mess with this anymore. Please look at the extensive evidence I have presented in this thread demonstrating that Bars.GetDayBar() does not work as I expect it to work. At one point in this thread I got the agreement that, yes, it makes sense to request NT programming development to create a Bars.GetDayBar() function that will plainly, simply and cleanly do what I expect. I don’t know why that crashed and burned, but it did. Thanks again for your response.

[Harry]

I have tested the NinjaScript method Bars.GetDayBar() with a specific indicator and it works without any problems. Bars.GetDayBar() produces a virtual bar created from intraday data by applying the session template that you have selected. The values for open, high, low and close therefore depend on that session template.

The daily bars in your historical data base may show entirely different values

-> because they are supplied from a different market / data supplier
-> because your data provider has used a different session definition to create them

This has nothing to do with the code.


Stop using the session template 24/5, it cannot be applied to FOREX

What you need to do:

(1) Check your session template and post it here. If you trade FOREX instruments Bars.GetDayBar() will only return correct values, if you use the correct session template. The correct session template is labeled FOREX and preset under instrument settings. Therefore you can open a FOREX chart

-> either with the FOREX session template
-> or with the <instrument settings> template

For all other session templates, Bars.GetDayBar() will return false values, as it will create virtual daily bars based on a false template.


Check the data via the Historical Data Manager or simply open a daily and a minute chart.

(2) Once you have checked your template please

-> open a daily chart, also open the data box and write down the OHLC for yesterday's (Thursday's) daily bar
-> then open a 1-minute chart and write down the open of the Wednesday 5:16 PM EST bar, the close of the Thursday 5:00 PM EST bar and the high and low of the period between session open and session close.

The daily values and the intraday values should match, at least if you have used the same (US) data provider. In case that they do not match, you need to find out why. But again this has nothing to do with NinjaTrader or any NinjaScript methods. It is just a data problem that you need to fix.

If you post a screenshot of your session template and both your daily and intraday chart, it will be easy to comment on the issue.

[joemiller]

I simply cannot get thru to you guys that I never asked for or expected or wanted custom programming. I just wanted getdaybar to return yesterday’s OHLC which still seems to me like a not unreasonable expectation. It was post#55 before I even got the response “oh, what you want is the daily bar OHLC from yesterday returned by getdaybar, fancy that!, you want what the function name implies". Thru out I have received detailed explanations of why getdaybar doesn’t really get yesterday’s OHLC ..... but I just simply don’t care. I just want it to work in a quite logical way [ie] I would like getdaybar to do what its name implies that it will do. I was hoping NT would allow me to do the minute bar analysis I have never been able to do. Read thru the 68 posts in this thread to get a sense of my frustration in trying to explain what I need. But no problem, I have set this issue to rest. I will continue to use NT. it is the best damn program/organization available, by far. I'm not going to check out the modified code sent to me. I have run out of time.
Cheers & Peace

[NinjaTrader_James]

Hello joemiller,

Thank you for acknowledging that we originally stepped outside our normal support policy in regard to providing you custom programming and that we are not able to do so any further. Unfortunately, providing comprehensive and complimentary custom programming and debugging services to our 40k users would be virtually impossible.

If other support forum members are unable to assist you further, you may absolutely go to one of our approved NinjaScript consultants for further assistance, as you now have a foundation of code to bring to them. A foundation which was meant as a starting point, not a complete solution as we do not provide custom programming for our members.

[joemiller]

NT policy has been misapplied here. makes no sense to me. the code has already been modified and sent to me. I merely want NT to take one more small step in a long journey and identify where those code lines are located. also, how could other users know where the modified lines of code are? that's an empty suggestion . it's NT's service group. so be it, we play it by NT's rules. .