Change Ninjatrader Folder Location

[carnitron]

But wait, there's more, lol. Actually this is theoretically good news. If you quit OneDrive, it doesn't sync, but keeps the folders local. I.e. while running OneDrive maintains the fiction that all your files are on the cloud, but when it is not running, the local stuff shows up as normal. You can also pause syncing from the Settings menu.

So I will try these tomorrow when live data is coming in. Both disabling syncing and then shutting down OneDrive altogether. If it works, that is actually an acceptable workaround, even if it is a bit annoying.

[carnitron]

I paused OneDrive syncing and turned it off, but I still have my infinite compile problem. I do not have enough information currently to determine whether the infinite compile problem is due to something else, or an issue that was caused by OneDrive while it was syncing (like a corrupted cache of some kind), such that turning it on and off now is irrelevant (since the damage has been done). I will continue to mess with it.

Repair installing and reinstalling NT8 had no effect. I'm basically dead in the water right now in terms of development progress. Next step is to deinstall NT8, purge the bin directory, reinstall NT8 and then reconstruct the bin directory.

NT8 support do have anything to contribute here?

[jeronymite]

carnitron This may or may not be related. Since NT 8.0.26.1, I too have had increasing compile times. I do not use OneDrive or any other networked drive.

I note that compiles proceed as expected, with the compile icon happily ticking over, but then the icon and everything else in the platform freezes for a period of time. The more compiles in a single session, the longer the freeze. I have found that if one waits "long enough", it usually "unfreezes".

I also note that if the duration between compiles is "long enough", the duration of the freeze seems less.

I have not reverted to NT 8.0.26.0 or even NT 8.0.25.0 to test if the problem disappears, but I certainly did not experience it on NT 8.0.25.0.

Restarting the platform seems to help. If I also delete everything in \NinjaTrader 8\cache and \NinjaTrader 8\tmp before starting, it seems to reduce the freeze at least initially.

Hope that is useful additional input.

Thanks.

[NinjaTrader_BrandonH]

Hello carnitron,

Thanks for your notes.

We recommend that OneDrive is disabled when using NinjaTrader.

Are you compiling a lot of code at one time or are you running compiles often when you see the issue with compile times occur?

If you turn OneDrive off and test compiling in a clean environment, is there a difference in the time it takes to compile compared to your original setup?

Creating a clean environment can be done by following the steps below:

1. Close NinjaTrader 8, and rename the "NinjaTrader 8" folder in My Documents to something like: "NinjaTrader 8 OLD"
2. Uninstall NinjaTrader from the Windows Control Panel and reinstall using the installer from http://ninjatrader.com/PlatformDirect
3. Add the required indicators and test the template.

If you ever need to switch back to your original platform, you may do so by closing NinjaTrader and swapping the platform folder names.
For example, Close NinjaTrader and rename the new "NinjaTrader 8" folder to "NinjaTrader 8 NEW" and the "NinjaTrader 8 OLD" folder to "NinjaTrader 8." Then restart the platform. Simply put: NinjaTrader 8 will always load the "NinjaTrader 8" folder in My Documents.

I look forward to assisting further.

[carnitron]

Hi Brandon. Thank you for replying. I am kind of dead in the water here, productivity-wise. On the 2nd recompile of my code, the compile starts but never ends and I have to force quit Ninja Trader 8. I restart and then I get one "good" compile that completes, before I have to force quit and restart again.

So my iteration loop is:

1. Make a code change.
2. Compile.
3. Get one run to collect results.
4. Make another change.
5. The compiler starts but never finishes.
6. Force quit NT8. (I have to, b/c once the infinite compile starts, closing individual NT8 windows causes them to freeze.)
7. Relaunch NT8.
8. Repeat.

I am using 8.0.26.1 of Ninja Trader and Visual Studio 2022. I am compiling a single, simple file that makes up my prototype add on.

I have tried:

1. Deleting the contents of my Custom/bin and Custom/obj directories.
2. Deleting the contents of my cache and tmp directories (as per Jeronymite's suggestion, above).
3. A repair install and a complete reinstall.
4. Deleting my db directory entirely.
5. Turning off OneDrive and suspending it's syncing.
6. Reverting back to 8.0.25.0.
7. Not even launching Visual Studio and just trying to work in the NT8 editor.

Nothing has fixed it so far. I have more things to try, but I appreciate your help here.

My next step is to create an entirely new Ninja Trader 8 directory (as you suggest) and copy only my source files into it.

This used to work. It started happening after I started saving lots of replay data, though I'm not sure if that is simply coincidence.

Thanks in advance for any help you can provide. I hope this isn't some blocking thread/deadlock issue in NT8 itself, since those can be hard to nail. But the fact that various NT8 windows and some UI operations freeze after the infinite compile starts has me wondering if some part of NT8 is getting blocked on some resource that isn't being released quickly, or at all.

[NinjaTrader_BrandonH]

Hello carnitron,

Thanks for your note.

I understand that when you run a second compile on a script after making changes that the compile never completes.

Did you try testing to see if this is reproducible after following the steps in my previous post to create a clean environment?

If you turn OneDrive off and test compiling in a clean environment, do you see the same behavior happen?

Creating a clean environment can be done by following the steps below:

1. Close NinjaTrader 8, and rename the "NinjaTrader 8" folder in My Documents to something like: "NinjaTrader 8 OLD"
2. Uninstall NinjaTrader from the Windows Control Panel and reinstall using the installer from http://ninjatrader.com/PlatformDirect
3. Add the required indicators and test the template.

If you ever need to switch back to your original platform, you may do so by closing NinjaTrader and swapping the platform folder names.
For example, Close NinjaTrader and rename the new "NinjaTrader 8" folder to "NinjaTrader 8 NEW" and the "NinjaTrader 8 OLD" folder to "NinjaTrader 8." Then restart the platform. Simply put: NinjaTrader 8 will always load the "NinjaTrader 8" folder in My Documents.

I look forward to assisting further.

[jeronymite]

There certainly seems to be an issue with how compiles are being performed in NT 8.0.26.1. It may have broader implications, given the symptoms carnitron is experiencing.

I've posted on performance-related issues before (https://ninjatrader.com/support/foru...ormance-issues) that still exist, and although they may be unrelated to the current issue, I made a comment in that specific thread that many performance-related issues can be caused by work being done on the UI thread when it really should be farmed off to its own separate thread. This could apply here, certainly in the symptoms I experience, but it is an important design principle that can be easily missed by developers who design to "do the thing the user clicked on", not realising that unless there is an explicit move to a different thread, all the work is done on the UI thread, and that can easily become catastrophic for UI performance. Just trying to raise awareness of the matter as having a potential contribution, nothing else.

Thanks.

[carnitron]

Brandon, I did as you suggested and it has not fixed the problem. That was a good suggestion, so this is quite unfortunate.

I was able to get multiple compiles off in a row just using the NT8 editor, before starting Visual Studio 2022. But as soon as I started VS2022, I got the infinite compile. So I rebooted, did not start Visual Studio 2022 but got the infinite compile on the 2nd compile anyway. Not sure what that means, other than that it is possible running VS exacerbates the problem.

It's unlikely to help, but is there a way to switch which Visual Studio NT8 talks to? I have both 2019 and 2022 and never had an issue with 2019. This may not be relevant at all, but it seems worth testing.

What else can we try? Something definitely seems seriously broken so I wonder what else we can do to characterize and narrow down the root cause.

Cheers.

[NinjaTrader_Jim]

Hello carnitron,

OneDrive and NinjaTrader compete accessing files in the Documents folder. Changing this affects many other things, including AddOns that may be programmed to look in the Documents folder so for now, we have to resort to user interaction to prevent these programs from competing.

Excluding the Documents\NinjaTrader 8\ folder or disabling OneDrive when you want to use NinjaTrader would be our recommendation for best experience. I generally suggest disabling OneDrive and to enable it again when you are done using NinjaTrader and want to re-active it's syncing behaviors. - This would leave the folders in place when you have a OneDrive setup already.

Changing OneDrive to use another drive may also help. Another idea may be to try other Cloud Syncing software where you would have more control over folders and sync-scheduling.

[carnitron]

I get the infinite compile behavior even when I have OneDrive syncing turned off, so even if live syncing is a cause, it is not the only cause.

I am concerned that maybe the problem is caused by some sort of corruption that might occur when syncing is turned on. The only way to know if this is the case is to do the same directory process above, but with OneDrive turned off the whole time. Like before I install NT8, while I reconstruct the directory and then while I do development. I can try that this weekend.

But another concern is that this has nothing to do with OneDrive. I assumed this was OneDrive initially, b/c of all the sync errors I was getting, and then seeing this thread.

There is now plenty of evidence, however, that there is actually no direct correlation between OneDrive and this issue. It *might* be a contributing factor, but we don't know.

What other things can we try?

[carnitron]

Also, if you guys are concerned that OneDrive may be the cause b/c of issues other people are having, then it sounds like it's a known issue. Is there an ETA for a release that ensures Ninja Trader 8 and OneDrive play nice together? OneDrive is obviously not going away, in fact, it's becoming more integrated and prevalent, so presumably there's no escape from making this work at some point.

What are NinjaTrader's thoughts/plans here (if you can share them)?

[NinjaTrader_Jim]

Hello carnitron,

On my end, I am not running OneDrive, and I do not have any issues recompiling/Rebuilding in Visual Studio 2022 or the NinjaScript Editor. (NinjaTrader 8 Release 26.1 is running with the NinjaScript Editor open at this time.)

Note that we do recommend saving from Visual Studio (instead of compiling) with the NinjaScript Editor open, since making changes to a file with the NinjaScript Editor open triggers a compile from NinjaTrader, and the NinjaScript Editor also adds "NinjaScript Editor Code" wrappers to indicators.

Getting stuck in an area where I am not still raises eyebrows.

Since this is working for me on my end, getting you to a clean slate with that set up would let us rule things out. I.E.

• With OneDrive disabled, and we have a clean installation of NinjaTrader 8 and a clean installation of Visual Studio 2022, do you still see this behavior?

Getting to this level would be the "clean slate" of functionality.

Does it work from here? If you run into the issue, can you isolate steps that cause it to come back up?

If we get to there we could give further insight.

If you cannot get the clean installations working as I am reporting, I may suspect something more environmental.

[carnitron]

I will try clean slate functionality tonight or tomorrow and report back.

One thing actually dawns on me though. I hit the compile button in the NinjaTrader Editor window, b/c it is the only way I know how to get NT8 to reload my code so I can run and/or debug it.

Are there other ways? I can compile the code in Visual Studios 2022 w/out issue. So if there was simply a way to reload/rerun my latest VS build w/out hitting that compile button, I might just be able to sidestep this issue altogether. I prefer using Visual Studio for everything anyway.

Thoughts?

[carnitron]

I finally figured it out.

Running an AddOn that subscribes to FundamentalData causes all subsequent compiles to hang. This typically happens on the 2nd compile, b/c the 1st compile creates the AddOn and "installs" it, such that the "bad" subscription can start and then block whatever follows.

Try this code on your own machine and see what you get. NOTE! That you must uncomment the line below that does the subscription to see the problem:

[FONT=Courier New]//This namespace holds GUI items and is required.
namespace NinjaTrader.Gui.NinjaScript
{[/FONT]
[FONT=Courier New] // NT creates an instance of each class derived from "AddOnBase" and call
  // nWindowCreated/OnWindowDestroyed for every instance and every NTWindow which is created or
  // destroyed...

  public class EvilAddOn : AddOnBase
  {
    private NTMenuItem m_ourMenuItem;
    private NTMenuItem m_existingMenuItem;
    private FundamentalData m_fundamentalData;

    // Same as other NS objects. However there's a difference: this event could be called in any thread
    protected override void OnStateChange()
    {
      if (State == State.SetDefaults)
      {
        Description = "Demonstrates compile hang";
        Name = "Evil AddOn";
      }
      else if (State == State.Configure)
      {
        Instrument instrument = Instrument.GetInstrument("AMD");
        m_fundamentalData = new FundamentalData(instrument);

        // Uncomment this line to see the problem. This needs to be compiled and loaded into
        // NinjaTrader 8 so it shows up in the menu.
        // Once that is accomplished. The NEXT compile will last forever, even if it isn't in this file.
        //
        // m_fundamentalData.Update += OnFundamentalData;
      }
      else if (State == State.Terminated)
      {
        m_fundamentalData.Update -= OnFundamentalData;
      }
    }

    private void OnFundamentalData(object sender, FundamentalDataEventArgs e)
    {
      return;
    }

    // Will be called as a new NTWindow is created. It will be called in the thread of that window
    protected override void OnWindowCreated(Window window)
    {
      // We want to place our AddOn in the Control Center's menus
      ControlCenter cc = window as ControlCenter;
      if (cc == null)
        return;

      /* Determine we want to place our AddOn in the Control Center's "New" menu
         Other menus can be accessed via the control's "Automation ID". For example: toolsMenuItem,
         workspacesMenuItem, connectionsMenuItem, helpMenuItem. */

      m_existingMenuItem = cc.FindFirst("ControlCenterMenuItemNew") as NTMenuItem;
      if (m_existingMenuItem == null)
        return;

      // 'Header' sets the name of our AddOn seen in the menu structure
      m_ourMenuItem = new NTMenuItem
      {
        Header = "Evil AddOn", Style = Application.Current.TryFindResource("MainMenuItem" ) as Style
      };

      // Add our AddOn into the "New" menu
      m_existingMenuItem.Items.Add(m_ourMenuItem);
    }

    // Will be called as a new NTWindow is destroyed. It will be called in the thread of that window
    protected override void OnWindowDestroyed(Window window)
    {
      if (m_ourMenuItem != null && window is ControlCenter)
      {
        if (m_existingMenuItem != null && m_existingMenuItem.Items.Contains(m_ourMenuItem))
          m_existingMenuItem.Items.Remove(m_ourMenuItem);
      }
    }
  }[/FONT]
[FONT=Courier New]}[/FONT]

Given this new info, several questions:

1. Am I subscribing to FundamentalData in the wrong way or at the wrong time? Is there a link that describes how I "should" do it?
2. If this is a bug, when will it be fixed?
3. If it's somehow, not a bug, is there a way to prevent this from happening to other people?

I think we can conclude at this point that this issue has nothing to do with OneDrive, since I can reliably reproduce it regardless of whether OneDrive is on or not.

Regards,
Peter

[jeronymite]

carnitron You are configuring the FundamentalData in State.Configure. According to the documentation for OnStateChange, you probably should be in at least State.DataLoaded before such actions. Try changing on that basis and see what happens.

Thanks.