BidAsk Historical...

[NinjaTrader_JoshP]

Not all of the data providers NT supports offers millisecond timestamping.

[dnoff]

Why is NT data only timestamped down to the second? Clearly in any type of normal instrument, the bid ask data synchronized to this level has zero chance of being useful if is not able to be sequenced? Is this a data provider limitation or a NT limitation?

If it is an NT limitation, this design decision does not make sense to as this implementation is not all going to help those who are looking to historically test or even reload indicators on charts that use onMarketData(). As a result I do not see the benefit of adding this to NT 7 in this manner???

(I may be misunderstanding your comments above Dierk, but I thought you mentioned that bid ask data on historical data is only synchronized by timestamp?)

[NinjaTrader_Dierk]

Sorry I don't follow.

[funk101]

Hmm, ok, but what about plotting based on price, vertically as it were? Same does not apply it would seem.

[NinjaTrader_Dierk]

Guys,

Let me outline the opportunities and limitations of the new historical bid/ask feature.
- you could have multiple bid/ask/last series on your indicator/strategy
- the historical bid/ask series would hold ALL bid/ask events sent out by the exchange. Thus, it would NOT be the bid/ask at the time a trade went off
- on the historical part they would be replayed by OnBarUpdate
- OnMarketData only would be triggered realtime
- on historical replay the series are synced by their timestamp
- timestamps in NT are down to 1 second granularity
- on realtime obviously the OnBarUpdate events on multiple series come in at the sequence as sent out by the exchange

So:
- the EXACT sequence of a mix of bid/ask/last events will NOT be maintained while processing the historical data from multiple series
- the EXACT sequence of the bid/ask/last events by themselves WILL be maintained while processing the historical data from multiple series

However, on events with DIFFERENT timestamps the sequence always IS maintained.

Hope this sheds some light.

[darthtrader]

darthtrader,

It is possible to have many ticks with the same time stamp.

true, i soppose i was thinking about a thinner market earlier...
I was just kind of throwing that out there to hopefully get the ball rolling.
I can totally understand the reason for not messing with onmarketdata()...
breaking alot of other uses of it wouldn't make sense just to backfill this kind of stuff...

It just seems to me we do have the tools now to do this, just need to figure out how...I mean don't we basically have something like this if you were to visualize the code gomi posted as far as the arrays:
A - ask
B - bid
L - last

a a a aaa aa a a a aa a aaaa a
L L L L L L L L L L
bb b b bbb b bbbb b b bbbbb b

The problem is the index of the arrays will not line up but we do have the data we need to figure this out.
I could see how streaming from a file could work with onmarketdata but i would think we should be able to do this with just on bar update.

[funk101]

Ok, how can one accomplish "retrieving historical bid ask"? I asked before, is stream reading/writing from a serialized file the best way to achieve this? Is this a painfully slow approach? When a user refreshes the chart, the data get's loaded back in? What are the options for trying to implement this?

[rt-trader]

OK - thankyou for the reply Ray.

[NinjaTrader_Ray]

I imagine there will be a considerable number of 6.5 users waiting for this Historical Bid/Ask capability to build historically what is now possible live via OnMarketData.

From these early comments it seems this may not be straightforward. Is it possible for some sample code on how this might be accomplished be posted?

Or better yet - an enhancement to (or derivative of) OnMarketData which will allow this to be more easily?

Thanks

PS I dont have NT7 yet so can not experiment myself.

Unfortunately there is no reference sample forthcoming anytime soon. This is strictly based on bandwidth and priority of other items however, this is one of those things on the list.

[rt-trader]

I imagine there will be a considerable number of 6.5 users waiting for this Historical Bid/Ask capability to build historically what is now possible live via OnMarketData.

From these early comments it seems this may not be straightforward. Is it possible for some sample code on how this might be accomplished be posted?

Or better yet - an enhancement to (or derivative of) OnMarketData which will allow this to be more easily?

Thanks

PS I dont have NT7 yet so can not experiment myself.

[NinjaTrader_JoshP]

darthtrader,

It is possible to have many ticks with the same time stamp.

[darthtrader]

Ok that works, but then there is no way to link the bid/ask values to the price values. They are just other charts. So in that case I suppose there would be no way to compute the historical delta volume....

I don't have the beta to mess around with yet but doesn't this depend on how these are updating:
Add(Instrument.FullName,PeriodType.Tick,1,MarketDa taType.Ask);
Add(Instrument.FullName,PeriodType.Tick,1,MarketDa taType.Bid);

I take it since you mentioned that the bid/ask bar array is very large, these update on each change of the bid/ask and are not just a snapshot of what the bid/ask was when last goes off...
Wouldn't a way to get historic delta volume then be to compare the timestamp of last with the timestamp of the bid and ask and see if its at/above/below/between?

I soppose I was expecting 7 to have a historic version of onmarketdata(), but with storing bid/ask this seems even more thorough, more a question of filtering out what we don't need.

[gomifromparis]

What would be cool would be to publish the spec of the ntd files.... Then we could read them by custom programming...

[funk101]

What about writing/reading from a filestream? Would that be a viable solution?

[gomifromparis]

I tried to reconstruct historical data using on chart by adding tick data bars.

    
        protected override void Initialize()
        {
            Add(new Plot(Color.FromKnownColor(KnownColor.Orange), PlotStyle.Line, "P1"));
            Add(new Plot(Color.FromKnownColor(KnownColor.Red), PlotStyle.Line, "P2"));
            Add(new Plot(Color.FromKnownColor(KnownColor.Orange), PlotStyle.Line, "P3"));

            Add(Instrument.FullName,PeriodType.Tick,1,MarketDataType.Last);
            Add(Instrument.FullName,PeriodType.Tick,1,MarketDataType.Ask);
            Add(Instrument.FullName,PeriodType.Tick,1,MarketDataType.Bid);

            
        }

        protected override void OnBarUpdate()
        {
    
           P1.Set(Closes[0][0]);
           P2.Set(Closes[1][0]);
           P3.Set(Closes[2][0]);
        }

But the fact is that it uses a lot of time and memory, because the bid/ask bar array is very large, even for a 5 days history. So I don't think it will be easily usable , except maybe for intraday memory.