processor, back testing

[bcameron]

Im running algo's but while Im doing so I need 120 days of data and my current processor takes 5 to 10 minutes to get data back to me, then another 5 to 10 minutes after reloading ninja script, ideally I need the data to get back to me as quick as possible since I would like to backtest ALL DAY long, but lag is killing me and im assuming its a processor issue since im currently running a amd older processor.....what would the best processor be to get the info to me as quick as possible, fyi my internet speed is not the problem I have a download speed of 50mbs and upload around 12 thank you

[NinjaTrader_RyanL]

Hello bcameron,


Thank you for your forum post.


You can upgrade to any multi-core CPU. For example an Intel Core i3, i5 or i7 Processor.

NinjaTrader 7 has optimized multi-core/multi-processor support. This would allow you to make full usage of a multi-core/ multi-processor hardware on backtesting or optimizing multiple instruments or running multiple iterations of a strategy optimization at the same time.

[bcameron]

that part i was pretty much assuming that it would be the intel core i3-5 or i7, my concern is that if I get the i7 will i get the info that much faster than if i got the i3? and which model should i get, they range from 200 to 1000, I just want the data quick and no lag

[sledge]

What are you using now?

[bcameron]

my current processor is AMD A-8 5600k 3.6ghz
with 8 gb of ram

[NinjaTrader_RyanL]

Hello bcameron,


Thank you for the follow up.


An AMD A-8 5600k is roughly equivalent to an Intel Core i3 3220 CPU.


As for the model, this would be up to you based on the price to performance and what you're willing to spend on the CPU.

[bcameron]

alright for argument sake then, if its close to or equal to an intel I3 then why is it taking me 10 plus minutes to get 120 days of data, tweak it, another 10 plus minutes and so on, always tells me ninjatrader not responding and i take that as taking a long time, but it eventually shows up

[NinjaTrader_RyanL]

Hello bcameron,


Thank you for the follow up.


Back testing can take a long time due to a number of different factors and resources available on your computer.


If you are back testing on tick data and you do not have the data available, NinjaTrader will download the requested period from your data provider which can add time to the back test. If you disconnect from your data provider after downloading the desired period data for your back test, does the speed increase?


Depending on the strategy you are back testing can also take a longer time if the code has a lot of lines or variables used.


For an optimization the time to optimize can be increased exponentially as you begin incrementing as full tests must be run for each possible combination of parameters.


If your database is very large or if the database has any corruption can also cause back test to take a long time.

[bcameron]

thank you for helping me with that,,,,right now is a perfect example ive been working on 1 strategy for the last 2 hours, is there anything i can do to speed up the time? heres some ideas

Processor I7 990x
Server,,,,really dont want to do that though
increase my download upload speed from 50-10mbs to 80-20mbs

my short goal is to be backtesting 2 or 3 strategies at a time and my future goal,,<under a month> is to be running between 40 and 50 algo's simultaneously

it looks to me that data does not affect hard drive space?

[NinjaTrader_RyanL]

Hello bcameron,


Thank you for the follow up.


The data will affect available hard drive space as NinjaTrader downloads data it will be using hard drive space.

With this being said, the historical chart data that is downloaded is usually small but can take up a significant amount of space if downloading large amounts of data.

Historical chart data is saved to the following folder:

• C:\Users\<USERNAME>\Documents\NinjaTrader 7\db
• In the cache, day, minute and tick folders

You might want to manually clear out any data from these folders that you are not using if your are concerned about hard drive space.

[bcameron]

Im sorry to be so bothersome, but I still dont have a definitive answer on what kind of processor would benefit me or if upgrading my download,upload speed would benefit me greatly as well as anything else I can do to increase time management on downloading historial data, 120 days to be exact

[koganam]

Im sorry to be so bothersome, but I still dont have a definitive answer on what kind of processor would benefit me or if upgrading my download,upload speed would benefit me greatly as well as anything else I can do to increase time management on downloading historial data, 120 days to be exact

Your computer is more than adequate to download data. It seems to me that you are downloading the data afresh for each run. You may want to turn off the option that downloads data from the server, once you do your first download and are sure that you have all the current data that you need.

[NinjaTrader_RyanL]

Hello bcameron,


Thank you for the follow up.


Your current CPU is more than equivalent to run NinjaTrader. I do not have any recommendations in regards to the CPU you use as long as it meets the minimum requirements listed on our installation guide:

• http://www.ninjatrader.com/installation-guide.php

It sounds like your internet connection speed is also more than sufficient to download the data quickly. although requesting 120 days of tick data is a very large amount of data and might take a while to download depending on how fast your connection to your data provider is.


If you disconnect from your data provider after downloading the desired period data for your back test, does the speed increase?

Depending on the strategy you are back testing can also take a longer time if the code has a lot of lines or variables used.


For an optimization the time to optimize can be increased exponentially as you begin incrementing as full tests must be run for each possible combination of parameters.


If your database is very large or if the database has any corruption can also cause back test to take a long time.

[sledge]

Im sorry to be so bothersome, but I still dont have a definitive answer on what kind of processor would benefit me or if upgrading my download,upload speed would benefit me greatly as well as anything else I can do to increase time management on downloading historial data, 120 days to be exact

The bottle neck might be on the server side