Historical backfill import issue with External Data Feed

[yusibi]

I'm using a utility to feed quotes for Indian stock markets using an external data feed. For context, a screenshot of live NSE feed to NT from Odin.

During backfill using Historical Data Manager Import, an error that is occurring most of the time is:



The import is from the [user]\Documents\NinjaTrader 7\import folder. Some observations related to the error:

• The error only occurs if importing while the External Data Feed (EDF) is connected. All files import successfully if the EDF is disconnected.
• When EDF is connected, most files import successfully if selected individually or in small groups.
• The error typically occurs for the file BANKNIFTY_1.Last.txt. There is nothing special that I can see about this file, it is neither generated first nor last, except that it happens to be alphabetically the first one.

Am clueless as to what to check, and would appreciate some help.

[NinjaTrader_JC]

Hello yusibi,

Connecting to the External Data should not interfere or change anything with Importing Historical Data.

As for the link that you have inside of your post for the NSE, I was not able to view that as well.

With that said, I believe I know what is going on with your importing Historical Data. It is most likely due to the formatting and name of the file that you are trying to import.

As an example "BANKNIFTY_1.Last.txt" would not be able to be imported properly because it does not have the correct Contract Expiry to have NinjaTrader import it. You would have to change the file name to the expiry that has all of the data for it like for example "BANKNIFTY 07-13.Last.txt".

Note that if you have multiple contract expires you will have two different options. 1. Separate the file into each contract expiry. 2. Use a continuous contract expirty "##-##", but this will only make the Historical Data show up if you use the continuous contract as well.

Here is a link to our Help Guide that goes the different file and data formats you may import.

http://www.ninjatrader.com/support/helpGuides/nt7/index.html?importing.htm

Happy to be of further assistance.

[yusibi]

JC, thank you for your response.

I've corrected the link to the screenshot. (Are .png files, or links to them, not accepted in forum posts? Okay, even the text .g i f gets replaced )

Since all the instruments (including BANKNIFTY_1 and NIFTY_1) have been configured as Stock types, and not as Future, your indication that it may be due to expiry related issues may not apply.

As has been mentioned, all files import correctly when External Data Connection is disconnected. When connected, the other 6-7 symbols import give a "Historical data imported successfully" dialog. I've further checked by adding / changing to new symbols. Right now, since NSE is closed, no new quotes are being fed, and all files import successfully, and this is consistently so. Checking, during live market today, it was only the BANKNIFTY import that continued to give the error shown in the first post.

[NinjaTrader_JC]

Hello yusibi,

They are accepted inside of the forum, when you attach them you just have to make sure you press the Upload button after you have selected to file to upload it. Note that it may restrict you if the file is very large but not due to the file type of .txt. or .png.

It may potentially be how the 3rd Party Program is giving data into NinjaTrader, but I would not think it would interfere with it.

Could you post the "BANKNIFTY" file that you were trying to import?

Also, if the market is closed, can you be connected to the External Data Feed and import the file without the error this time without an error?

[yusibi]

JC, again thank you.

BANKNIFTY_1.Last.txt is attached.

I've tried to import the file 3 ways. In all cases, I deleted today's data for the instrument using the Historical Data Manager before the import, and checked it after. The NSE market is currently closed.

1. External Data Feed not connected - successful import
2. External Data Feed connected, utility not running - successful import.
3. External Data Feed connected, utility running - successful import.

[NinjaTrader_JC]

Hello yusibi,

Thanks for the file.

I can see that if you added the Instrument "BANKNIFTY_1" which it looks like you did, everything will be in the correct format.

Looking into the Odin Feeder a little bit it looks like it may have the option to write a file that you may import into NinjaTrader the data. Depending on how it is written, the Odin program maybe still writing to the file or accessing it which is causing all of the errors to occur.

So going forward you may want to make sure that either that program is not writing or accessing the file before you start to Import the data into NinjaTrader as the demonstration video shows.

http://www.volumedigger.com/Software/Odin_Feeder_Help.aspx

Happy to be of further assistance.

[yusibi]

JC, thank you for the directions.

Having ruled out that anything is wrong with the file format, and that is something I double-checked before writing to you, one needs to look elsewhere. You possibly guessed that the file was still being written based on the fact that NSE closes at 15:30 while the last record in the file is 15:13; but that just happens to be the last time it was generated.

Also, in this case, BANKNIFTY_1.Last.txt happens to be the 4th in sequence of the generated files, out of the 10. I happen to be the author of the utility. It is not JDM's Data Feeder, as Joydeep himself will tell you if you asked him.

Can I request you to indicate under what condition the error dialog of the first post is shown. It is a very identifiable message box in that the message text is written in the caption/title parameter, while the message area is blank. Also, after the message box shows, the entire Historical Data Manager window greys out with the only option to close it. If it happens to be a catch-all, well...

[NinjaTrader_JC]

Hello yusibi,

From the image that you sent over, I cannot see the whole item but it does state that "The process cannot access the file..." so that plus watching the videos that shows the files being created leads me to believe that the file is either being written to or (forgot to mention in last post) accessed by another program which is preventing NinjaTrader from being able to read it.

If you would like to send me your log and trace files, I can try to see if there is any more information to know for sure that is going on but it does not appear to be something with NinjaTrader.

You can do this by going to the Control Center-> Help-> Mail to Support.

Please reference the following thread in the body of the email: http://www.ninjatrader.com/support/f...ad.php?t=59650

and in the subject line put ATTN: JC.

Happy to be of further assistance.

[yusibi]

JC, thank you for your continued assistance.

Perhaps, it would be best to let the matter rest; and maybe revisit after some time. If there was an identifiable problem / solution, it would have emerged by now.

To answer your questions, I cannot see the whole message in the NT error dialog either, but I guess it reads : The process cannot access the file [user]\Documents\NinjaTrader 7\import folder\...

No, there is no other process accessing this file (including virus checkers). The file has not been left open, nor is it being written to at the time the NT Historical Data Import is done. Also, the error only occurs when External Data Feed is connected, and not otherwise. The file generation is a batch process, and BANKNIFTY is neither the first or last file generated. A screenshot of the import file generation screen if it helps. Since I have not used Joydeep's Odin Feeder, and unlikely to do so, I'm not sure what you saw in the video there.

Just an FYI, a screenshot of odd behavior where some of the ticks of 11:xx show in 10:xx group. The DB has been reset subsequently, and I've not noticed anything similar since then.