FXCM weekend data missing

[bennyforlan]

Guys there is a major problem with FXCM connection

After reloading the chart of any given instrument ex. EURUSD, weekend data is missing.

What is interesting, is that the data of current trading week is not affected, the charts are updating right from the market opening on Sunday. (Ninja shows exactly the same sunday chart as FXCM webtrader)

But when the trading week is over and a new one begins, the historic data of the previous week opening is lost after you reload a chart.

For better illustration of the issue check the pictures attached. Compare correct FXCM webtrader charts with Ninja charts after reloading the data

The problem affects only weekend data and seems to be related with weekend maintenance.

[NinjaTrader_Jason]

Hello bennyforlan,

Can you please tell me what source is used for historical data. Go to Control Center-->Tools-->Account Connections, highlight your FXCM connection and click Change. Click Next and check what is selected for historical data - NinjaTrader or FXCM servers.

Do you see the missing historical minute data at Control Center-->Tools-->Historical Data Manager-->Edit-tab?

Try the following. Please shut down NinjaTrader and navigate to the folder below.

C:\-->Users-->*Windows user account*-->Documents-->NinjaTrader 7-->db

Delete the "cache" folder.

The db-->cache folder is used to quickly load data in charts, so the first time you load chart data after deleting the db-->cache folder, it will take a little longer to load than usual. Once all data is loaded, it will load data quickly again.

Start NinjaTrader and check if the issue persists after you reload historical data in a chart.

[bennyforlan]

Hi Jason, thanks for quick reply

My source for historical data is FXCM servers

On the historical data manager,
- for the date of 06.01.2014 the first bar timestamp is 23:01
And should be 21:01, 2hrs of data missing.
- for the date of 06.08.2014 (current trading week) the first bar timestamp is 21:01
And this is correct

Like I said only the previous trading weeks are affected

I've done what you've suggested with cache but it didn't help

Below I've attached one more picture, hope that clarifies the issue

Thanks

[NinjaTrader_Jason]

Can you please switch the historical data source to the NinjaTrader servers and check if you can reproduce. The NinjaTrader servers offer more historical data and is generally loaded quicker than from FXCM's servers.

Once you made the switch, click Next until you finish the connection. Restart NinjaTrader and check if data still disappears when you reload historical data.

[bennyforlan]

Thanks again for quick reply

When i'm using the ninja servers, data is loaded correctly, meaning that every trading week begins at the same time (21:00 my Tzone)
Below I've attached the same chart reloaded after switching H-server to ninja.

But my problem is with the FXCM H-server data !

Data obtained form FXCM H-server is far more reliable and of better quality (no missing ticks etc.) it may be slower and less hitory is avaible but compared to the one served from ninja servers it is bulletproof.

As an example I've attached three charts of GBPJPY, the one with misiing tick was populated using ninja history servers, the other one is correct, just compare the data.
And I could give You dozens of examples like that. The FXCM H-servers offers far better quality.

I just want to be able to use FXCM H-servers (because of the better data quality) without being affected by the issue I've laid out in the first post. Because clearly the data should be there and is not.

Thanks

[NinjaTrader_Jason]

If you now switch back to the FXCM servers and reload historical data in a chart, does the issue still persist?

[bennyforlan]

Unfortunately yes

See attached chart, it is the same chart, the same settings, everything is the same, I've only switched the History servers from ninja to FXCM, reloaded historical data and made the screenshot.

You could easily reproduce the issue on your end, the issue affects every instrument FXCM offers.

Two hours of data is missing.

And if You login to FXCM webstation You'll see that their chart is populated correctly (show weekend data option switched to YES), every trading week starts exatly at 21:00 on sunday (my Tzone).

[NinjaTrader_Jason]

I tested the EURUSD on my end - 15 Minute Ask data - with 'Historical data' set to FXCM's servers. My PC clock is set to US Mountain time zone, so data starts at 3:00 PM on each Sunday. The first bar should be time stamped as 3:15 PM. This is the case on both 6/1 and 6/8 on my end. I reloaded historical data in the chart multiples times, but it does not change.

Data for 6/1 seems to be the same when I compare your screenshot - the same red bar at the start of the session.

Try the following. Shut down NinjaTrader and navigate to the folder below once more.

C:\-->Users-->*Windows user account*-->Documents-->NinjaTrader 7-->db

Transfer the tick, minute, day and cache folders to your desktop. These folders contain all historical data you have present.

Start NinjaTrader and check if the issue remains now that all data is loaded fresh.

[bennyforlan]

From what I see, You are affected by the same issue.

FXCM ticks start flowing from 21:00 on Sunday my Tzone (1:00 PM yours time)

If you first bar is timestamped at 3:15PM as is mine (23:15 my Tzone) then we are missing the first two hours of data on 6/1.

Like I said, check the source, FXCM has it right on their platform (weekend data switched on).


Check the attached chart, it is 6/1 EURUSD ASK 15m, please note the timestamps of
the previous week last bar and the new week first bar, also compare the data, we are missing first two hours of trading on 6/1 and previous sundays when using FXCM h-servers.

[NinjaTrader_Jason]

Forex instruments use the 'Forex' session template which starts at 3:00 PM Mountain time. Forex instruments do not start trading at 1:00 PM Mountain time as far as I know.

Data on 6/8 starts at 3:00 PM as well.

What time zone does your PC clock use?

[bennyforlan]

My Time zone is GMT +01:00

But Jason, check the data, the Session time zone is irrelevant.

I'm using 24/7 template so every piece of data i collect is being depicted on the chart.

Switch history server to ninja servers
Reload the data
Compare the chart with Your previous screenshot on 6/1 (You will see that now you have extra 2hrs of data on your chart)

[NinjaTrader_Jason]

Thank you for clarifying. I see the same when using the 'Default 24/7' session template and NinjaTrader servers. I will send a report to development. Once there is news, I will let you know.

[bennyforlan]

Thanks Jason

It would be great if the Dev could have a look at that issue.

Like I said earlier, compare the data FXCM provide on their webstation/tradestation (weekend data enabled) with the data we have on ninja trader when using FXCM history servers.

Ninja history servers provides that data while FXCM history servers does not.

The issue affects every sunday (first 2 hours) with the exception of the last one (current trading week)

Thanks again

[NinjaTrader_Brandon]

Hello bennyforlan,

Thanks for your reply.

I am unfamiliar with how to enable weekend data for chart within FXCM's Trading Station. So I may look at this further with development can you please provide the steps of how to enable this within FXCM's Trading Station?

[bennyforlan]

Hi Brandon

below is the link to FXCM webstation

https://tsweb.fxcorporate.com/

I've attached screenshot of options window
(press SHIFT + O or go to System then Options then General options and then change the weekend data to SHOW)

I can send you login to demo account but i guess You have Yours

After enabling the weekend, take a look at the data, we are talking about first two hours of trading, every sunday is affected with the exception of the most recent one. You'll notice the difference when You compare the charts.

Big thanks again