how to merge database (trade performance) of two computers

[koganam]

Hello Heath,

thanks for the frank words. First of all: The suggestion with the Excel spreadsheet is not an acceptable solution because it lacks all the tools that are available in Ninjatrader8 --> Trade Performance. I would like to see the statistics, the data series, the finely granulated filter options, the data preparation according to weekday, time or other periods, sorting possibilities, right-click->show chart and much more that can be found under "Trade Performance".

I appreciate your warning and really understand this. I wouldn't risk anything on my real and main NT8 installation (desktop PC) and database for the reasons you mentioned. However, I thought of it like this: I install a fresh new VM with Windows10 and a fresh new clean install of NT8. Then I import the backup (trade history) made on my desktop PC and so I will get the same trades on the VM NT8 installation as I have on my desktop PC. When succeeding with the merge, I will apply it onto this new VM. All tests would take place on the VM, so there's no risk of breaking my "real" installation on the desktop PC or having corrupt databases and inconsistencies.

all testing actions will be done on a separate VM with a fresh new NT8 installation, made only for this testing purpose

If the merge works on the VM, I will not execute any trades there. I only want to be able to click on Control Center-->Trade Performance on the VM with the saved data and view the desired statistics in fine granularity. I will do my screenshots, my exports and everything I need from there. When I am finished, I will completely delete the VM and I'm done.

Can you or someone else provide some more information on following questions related to the database structure, please?

1) in my understanding, I need to focus on the tables Executions, Orders and OrderUpdates. Is that correct?
2) what time format is used on NinjaTrader.sdf for example in table Orders and there the column "Time" ? It is obviously not Unix time as I found.
3) can you explain what the difference betwenn table "Orders" and "OrderUpdates" is? what is the purpose of the two different table but similar sounding table names and what is the equivalent in Ninjatrader view?

any further help appreciated, either from you or from the community.
Thanks a lot to everyone!

In that case, you may be best off using an SDF file editor, exporting the tables you need from one database and importing (appending) them into the other. Here is a link to 4 that may be viable: https://www.ilovefreesoftware.com/02...ase-files.html

[patricia70]

koganam thank you so much for your effort and the link. My first try was using CompactView. But when trying to open the file "NinjaTrader.sdf" I got an error that something's missing. Unfortunately the link you posted above contains a dead link to the required service pack. Here's a solution This thread helped. I just downloaded SQLCE 4.0 SP1 as suggested in that thread and after that Compactview was able to open my NinjaTrader.sdf From there on I can view the existing tables "AccountItems, Accounts, Executions, Instrument2InstrumentList, InstrumentLists, Instruments, JournalEntries, Logs, MasterInstruments, Orders, OrderUpdates, Positions, Strategies, Strategy2Account, Strategy2Execution, Strategy2Instrument, Strategy2Order, User2Account, User2MarketDataEntitlement, Users, Versions". The one I need to focus on were Executions, Orders and OrderUpdates. I can run queries and prepare the data I'm looking for.

I will dig into it but I am in good hope to achieve the goal. Thanks so far

and can anyone explain to me, please how the database column "TIME" is constructed? How can I convert this very huge number into a human-reading format to get UTC or LOCAL time ? For example: the date 2021/June/02 09:26:32 (my local time in Germany UTC+1) is shown in NT8 database as value 637582155922190000. How to reverse ?

[patricia70]

Hello again. It is with some anticipation that I report back. I think I made it! first sample tests look good and I'm very optimistic that more detailed tests will pass as well.

Since only the evaluation tools under NT8-->New-->Trade Performance were important to me, I focused exclusively on the "Executions" table and thus did not consider the "Orders" and "OrderUpdates" further for my use case.

I worked exclusively with the SDF database program "CompactView" as already suggested and mentioned here several times. One must consolidate data, several ways lead to Rome. I did it so that I took the database from the PC (NinjaTrader.sdf) and first created a copy called "testing.sdf". I then worked with it in CompactView. Then I created a new table ExecutionsNB (suffix 'NB' = Notebook) with the same schema and PrimaryKey as the original Executions table. Relationships to the other tables were not important, I proceeded according to the KISS principle (Keep it small and simple). Then I imported the previously exported data from the notebook Executions table into the new ExecutionsNB table created on the PC. Here there was a small hurdle, which I could solve with some additional effort. So I had now in the table "Executions" the data from the PC and in the table "ExecutionsNB" the data from the notebook, both in the same database file "testing.sdf". The column "Id" serves as unique identification, I now had to filter out the duplicates. I read everything from the table "ExecutionsNB" and if ExecutionsNB.Id != Exections.Id then the INSERT INTO command was executed. The number of data series before/after I compared of course, also have many screenshots and backups of before/after.

The ready mixed testing.sdf I copied then on my VM, where I had prepared NT8 freshly installed for the tests. The original NinjaTrader.sdf from the test VM/NT8 I had deleted. The finished testing.sdf was renamed to NinjaTrader.sdf. Ninjatrader8 started without problems and so far everything looks great in trade performance. All trades are listed, now everything seems to be consolidated and I can perform my desired evaluations.

So far again BIG THANK YOU to all the helpful tips here. Am pretty happy

[bltdavid]

Congratulations!

That's great news.

[koganam]

Hello again. It is with some anticipation that I report back. I think I made it! first sample tests look good and I'm very optimistic that more detailed tests will pass as well.

Since only the evaluation tools under NT8-->New-->Trade Performance were important to me, I focused exclusively on the "Executions" table and thus did not consider the "Orders" and "OrderUpdates" further for my use case.

I worked exclusively with the SDF database program "CompactView" as already suggested and mentioned here several times. One must consolidate data, several ways lead to Rome. I did it so that I took the database from the PC (NinjaTrader.sdf) and first created a copy called "testing.sdf". I then worked with it in CompactView. Then I created a new table ExecutionsNB (suffix 'NB' = Notebook) with the same schema and PrimaryKey as the original Executions table. Relationships to the other tables were not important, I proceeded according to the KISS principle (Keep it small and simple). Then I imported the previously exported data from the notebook Executions table into the new ExecutionsNB table created on the PC. Here there was a small hurdle, which I could solve with some additional effort. So I had now in the table "Executions" the data from the PC and in the table "ExecutionsNB" the data from the notebook, both in the same database file "testing.sdf". The column "Id" serves as unique identification, I now had to filter out the duplicates. I read everything from the table "ExecutionsNB" and if ExecutionsNB.Id != Exections.Id then the INSERT INTO command was executed. The number of data series before/after I compared of course, also have many screenshots and backups of before/after.

The ready mixed testing.sdf I copied then on my VM, where I had prepared NT8 freshly installed for the tests. The original NinjaTrader.sdf from the test VM/NT8 I had deleted. The finished testing.sdf was renamed to NinjaTrader.sdf. Ninjatrader8 started without problems and so far everything looks great in trade performance. All trades are listed, now everything seems to be consolidated and I can perform my desired evaluations.

So far again BIG THANK YOU to all the helpful tips here. Am pretty happy

I am glad that you figured it out. It has been so long since I ever had to do it myself.