Market Replay - Getting historic data?

[DaveDave]

Hi
OK, so to use Replay I need some data, I don't have any NT Replay data. But I do have a stack of ES Tick downloads from eSignal (time price trade/ quote size as stuff as text).
Is there anyway I could somehow use the External Data Feed Connection and a bit of code to get NT to "convert" them to Replayable data, or is there some other way even?

And one other side-issue or two.
I assume that to get NT to do the normal "recording" it is necessary to have, what - an open chart or Market Analyser or dome referencing each symbol that might be of interest?
And that it is tick data that will be recorded, regardless of what type of chart interval I have?

Thanks

[NinjaTrader_Ray]

No conversion available. Replay files are every tick/change in level I and level II market data so chart interval is irrelevant.

[VitaminC]

NTDataFeeder

Take a look at here

http://www.ninjatrader-support.com/vb/showthread.php?t=6147

I have developed an utility to feed eSignal EPF to NT. It also works with NT exported files.

[toyogo00]

Hi VitamicC,

Great job!!! I've tested v1.1. Few questions:

1. I'm not able to use bid/ask for tick data file.

standard NT format for tick :
20001001 190056;1.4743;218199
20001001 191605;1.4749;218208
20001001 191809;1.4748;218217
20001001 192016;1.4743;218229

With the bid/ask, I assume it is as belowpls confirm)
20001001 190056;1.4743;1.4748;218199
20001001 191605;1.4749;1.4754;218208
20001001 191809;1.4748;1.4753;218217
20001001 192016;1.4743;1.4748;218229

I cannot even open the file with second format. However, the std format work very well for tick data.


2. Pls explain the "Ignore Timing" check as I really cannot figure out its use.

3. When I use tick datafeed, it only works for range bar chart which is very good if I trade Woodie CCI method. However, it doesn't derive the min/hour/daily/weekly chart from tick datafeed. Pls confirm it.

4. Thus I've to use minute datafeed in order to get min/daily/weekly chart. Do I need to use different period feed for each type of period chart or if I use 1 minute datafeed, I should get all other period chart (5, 10, 30, 60, 240... daily, weekly chart derived automatically.

5. For 3, and 4. It means I can either get a tick replay or period replay but not both. Is it right? Thus I've to delete the instrument historical data if I want to switch between the two. Is it right?

Regards,
Toyogo

[VitaminC]

Hi Toyogo,

1) Your second format is not supported because it is not NinjaTrader export format. For the first format, 1 bid and 1 ask will be submitted for every tick.
If u want to have a better control of bid and ask data, u may use eSignal playback file format. The file format is like below, Q is bid/ask data, T is trade data.

Q,080205,085915,2.7,2.71,34464,1000,SES ,SES
Q,080205,090001,2.7,2.71,34464,11000,SES ,SES
Q,080205,090001,2.7,2.71,34464,1000,SES ,SES
T,080205,090001,2.71,10000,SES
Q,080205,090003,2.7,2.71,53464,1000,SES ,SES
T,080205,090004,2.71,1000,SES
Q,080205,090004,2.7,2.72,53464,18000,SES ,SES

2) If you checked "Ignore Timing", timing data in the file will be ignored. Else if you uncheck it and select "1x" for the speed, the application will try wait according to the duration between the tick data before submitting the data.

3) NinjaTrader builds min and hour charts from min data, weekly charts from daily data.

4) Yes, if u have 1 minute data, then NT will builds 5 min, 10 min etc from it. For weekly charts u'll need daily data.

5) No, NinjaTrader stores Tick data, min and daily data seperately, u do not need to delete tick data to get minute data.

Btw, in case u r interested, I've release NTDataFeeder v1.2, u may download it from

http://www.driveway.com/k2g4t5j5a3

Note: Feeding min and daily data still doesn't work correctly as NT ignored the timestamp that the app is submitting

Regards,
VC

[toyogo00]

Hi VitamicC,

Thank you for your quick reply and all the relevant answers. Unfortunately, I don't have eSignal a/c so I cannot get the format unless there is a historical EPF download site available.

The key here is the tick data feed is only good to test tick / range bar related strategy such as Woodie CCI. As you have pointed out that Feeding min and daily data still doesn't work correctly as NT ignored the timestamp that the app is submitting... This is the biggest draw back if one would like to test the period based strategy... Do you know of other package offering that besides NT?

Regards,
Toyogo

[VitaminC]

Hi Toyogo,
Please take a look at this thread

http://www.ninjatrader-support.com/vb/showthread.php?p=35758

You can download the replay data from

http://ninja.korthrun.net/

then replay with NT built in market replay control, this is a more powerful (and supported by NT) way of replaying data as it supports multiple instruments and multiple timeframes.

My app is only a workaround because i couldn't find replay data for some of the instruments that i want to test and recording market replay data is a time consuming task.

Regards,
VC

[toyogo00]

Hi VC,

Thanks. I am aware of such link but I'm just like you looking for other instrument - GBPUSD replay...

I would like to see a tool where one can re-feed the data just like the real-time feeding at any one time - for example, if I want to replay from 1 Mar 2006 onward, then the tool will be able to reload the historical data from 10 yrs till 1 Mar 2006, then start feeding the data from 0000Hr 1 Mar 2006. This will be an ideal tool for market replay as one can really practice the strategy almost real-time simulation but with speedly forward playing.

Now it takes me months to test the resistance/support strategy on 4hr chart for real-time simulation trade. A lot of times is just waiting.....

Regards,

Toyogo.

[VitaminC]

Hi Toyogo,

Yes, i would like to see that feature in future version of NinjaTrader too

Regards
VC

[toyogo00]

Hi VitaminC,

I've downloaded years of forex data from gain capital in zip files - http://ratedata.gaincapital.com/2008/05%20May/ It's a lot of work to convert it into NT import format using Excel import/export function.

Do you have a program to convert the USD_JPY_Week5.zip directly into NT import file format ?

Regards,
Toyogo

[VitaminC]

Hi Toyogo,

Sorry for late reply, I haven't checked email for quite some time.

I have updated the program to support gain rate data.

Url:

http://www.driveway.com/c8a9g7g9n4

Command line syntax:
NTDataConverter.exe "USD_JPY_Week5.csv" "USD_JPY_Week5.txt" /f GC

Take note that you need to rename the file to $USDJPY.txt before it can be imported by NinjaTrader.

Regards,
VC

[thariman]

Hi VitaminC I tried running your NTDataFeeder but it keep crashing on my Windows Vista 64. I tried changing the compatibility to winxp and run as administrator same result. On the problem details it said System.IO.FileLoadException
I'm running your 1.2 version.

[VitaminC]

Hi VitaminC I tried running your NTDataFeeder but it keep crashing on my Windows Vista 64. I tried changing the compatibility to winxp and run as administrator same result. On the problem details it said System.IO.FileLoadException
I'm running your 1.2 version.

Can you post the input file that you want to feed to NinjaTrader to me? I suspect something problem with the input file.

[thariman]

Can you post the input file that you want to feed to NinjaTrader to me? I suspect something problem with the input file.

It worked if I don't copy the NinjaTrader.Client.dll from the bin dir.


Now if I unselect Ignore Timing or Submit Bid/Ask it stop feeding the data with

Error :System.NullReferenceException: Object reference not set to an instance of an object.
at NTDataFeeder.MainForm.playerTimer_Tick(Object sender, EventArgs e)

I'm using EPF file.

; Symbol=YM #F
; Date=07/28/08-08/08/08
T,080728,000029,11307,1,
T,080728,000131,11308,1,
T,080728,000542,11309,1,
T,080728,000922,11309,1,
T,080728,001653,11309,1,
T,080728,001910,11311,1,
T,080728,001934,11310,1,

[Red Fish]

Problem with the programs

Hi Vitamin C, thanks for trying to write a program to replay historical data. I tried to run both NTDataFeeder and NTDataConverter, neither of them work on my computer, I just get a quick flash of a small black screen and then nothing happens when I try and run NT Data Converter and when I try and run NTDataFeeder I get the windows message 'NinjaTrader Data Feeder has stopped working'. I have Vista, and even if I try and run as administrator it won't run. This is the error report I get for NTDataFeeder
Problem
Stopped working
Problem signature
Problem Event Name: CLR20r3
Problem Signature 01: ntdatafeeder.exe
Problem Signature 02: 1.0.0.0
Problem Signature 03: 47d79299
Problem Signature 04: NTDataFeeder
Problem Signature 05: 1.0.0.0
Problem Signature 06: 47d79299
Problem Signature 07: 49
Problem Signature 08: 5
Problem Signature 09: System.IO.FileNotFoundException
OS Version: 6.0.6001.2.1.0.768.2
Locale ID: 2057
Extra information about the problem
Bucket ID: 307723637


Can you help?

Thanks!