Hi, I have a strategy that is a resource hog and I'm looking for general performance tuning tips. I'm looking for specific recommendations to improve my strategy, but I'd also really like to eventually see a support thread in the Tips section or a chapter in the documentation on performance tuning and/or memory management.
First, I'll describe my hog. This thing is now 6000+ lines, has almost 300 variables, some of them arrays. It issues long and short entries based on 68 entry criteria (roughly 34 long strategies and 34 short strategies, more or less mirrors of each other). Of these, I consider 16 to be production strategies and the rest to be in development, which means I'm constantly tweaking their entry and exit criteria until they give me the desired results for a while. I send a whole lot of messages to the output window: every time I enter a position, exit a position, exactly 10 minutes after exiting, and at other interesting points in between. A typical day has 1000-3000 lines sent to the output screen, which I manually copy and paste into a spreadsheet about once every 10-15 minutes.
The program uses 4 timeframes: 1-minute, 1-minute on symbol DIA, 1-day, and 3-minute. Most of the logic is in OnMarketData, not OnBarUpdate. Many of the variables are only calculated on a print update or once a minute, and the daily-bar calculations like average daily volume are done just once a day.
I have a 100-symbol Kinetick license and I run 200 instances of this program during the day: 100 for the Sim101 account and the same 100 symbols for my IB account. The program's smart enough to know not to buy into the Sim101 account if it is also buying the symbol in the IB account for the same reason.
In general, this program almost never uses over 1% CPU, but it can spike to 5%. The problem is memory. I have 8GB RAM, and really only about 6.2GB truly usable space for NT before NT slows to a crawl. It'll start off using only 1.2GB and will go up at a slightly increasing rate. It takes about 4-5 hours to hit 6.2GB depending on time of day. It doesn't seem to make much difference if I have 100 instances of the program running or 200 instances running for 100 symbols.
So, here are the questions:
1. All prints to the output window are retained in memory. Will I save memory by printing shorter lines, or are all the lines stored at a fixed length?
2. Does it add to memory usage to play sounds?
3. How much memory space is used up for each trade?
4. How much space would be allocated for a 10-item DateTime array? Is it all allocated at once, or does it take less space if only one element of the array contains data?
5. How much space would be allocated for a 10-item String array?
6. Is there any reason why running through a FOR loop would eat up memory? Specifically, I was looping once a minute from 1 to barsSinceSession (390 iterations at the end of the day) looking through the minute bars to do some calculations. I stored the answers in regular variables, not arrays, so it theoretically shouldn't have taken more memory to loop through 390 minute bars than to loop through 30 minute bars, but I think it did. I changed the logic to accumulate the data each minute instead of looping through all the minute bars each minute, and the memory usage seemed to go down.
7. Does it eat any extra memory to make calls to get account information? I still need the information, but if it saves memory I'll request the information less often.
8. Do the math operations (Math.Min, Math.Max, Math.Abs, etc.) eat up memory that doesn't get released?
9. As mentioned earlier, it seems that having two instances of the program running against the same symbol doesn't cost much more in memory. Why is that?
10. Are there any other tips you might have on memory management? I'm a newbie to C#, so please don't assume I know something that seems obvious to a pro.
First, I'll describe my hog. This thing is now 6000+ lines, has almost 300 variables, some of them arrays. It issues long and short entries based on 68 entry criteria (roughly 34 long strategies and 34 short strategies, more or less mirrors of each other). Of these, I consider 16 to be production strategies and the rest to be in development, which means I'm constantly tweaking their entry and exit criteria until they give me the desired results for a while. I send a whole lot of messages to the output window: every time I enter a position, exit a position, exactly 10 minutes after exiting, and at other interesting points in between. A typical day has 1000-3000 lines sent to the output screen, which I manually copy and paste into a spreadsheet about once every 10-15 minutes.
The program uses 4 timeframes: 1-minute, 1-minute on symbol DIA, 1-day, and 3-minute. Most of the logic is in OnMarketData, not OnBarUpdate. Many of the variables are only calculated on a print update or once a minute, and the daily-bar calculations like average daily volume are done just once a day.
I have a 100-symbol Kinetick license and I run 200 instances of this program during the day: 100 for the Sim101 account and the same 100 symbols for my IB account. The program's smart enough to know not to buy into the Sim101 account if it is also buying the symbol in the IB account for the same reason.
In general, this program almost never uses over 1% CPU, but it can spike to 5%. The problem is memory. I have 8GB RAM, and really only about 6.2GB truly usable space for NT before NT slows to a crawl. It'll start off using only 1.2GB and will go up at a slightly increasing rate. It takes about 4-5 hours to hit 6.2GB depending on time of day. It doesn't seem to make much difference if I have 100 instances of the program running or 200 instances running for 100 symbols.
So, here are the questions:
1. All prints to the output window are retained in memory. Will I save memory by printing shorter lines, or are all the lines stored at a fixed length?
2. Does it add to memory usage to play sounds?
3. How much memory space is used up for each trade?
4. How much space would be allocated for a 10-item DateTime array? Is it all allocated at once, or does it take less space if only one element of the array contains data?
5. How much space would be allocated for a 10-item String array?
6. Is there any reason why running through a FOR loop would eat up memory? Specifically, I was looping once a minute from 1 to barsSinceSession (390 iterations at the end of the day) looking through the minute bars to do some calculations. I stored the answers in regular variables, not arrays, so it theoretically shouldn't have taken more memory to loop through 390 minute bars than to loop through 30 minute bars, but I think it did. I changed the logic to accumulate the data each minute instead of looping through all the minute bars each minute, and the memory usage seemed to go down.
7. Does it eat any extra memory to make calls to get account information? I still need the information, but if it saves memory I'll request the information less often.
8. Do the math operations (Math.Min, Math.Max, Math.Abs, etc.) eat up memory that doesn't get released?
9. As mentioned earlier, it seems that having two instances of the program running against the same symbol doesn't cost much more in memory. Why is that?
10. Are there any other tips you might have on memory management? I'm a newbie to C#, so please don't assume I know something that seems obvious to a pro.
Comment