Crashed Database AGAIN!!!

[Ragingbull]

Dear NT,

I don't know if I have to be desperate or very angry with the time I loose trying to use NT.

Problem 1

It has already happened several times that I got my database crashed with NT6.5 and NT7 (see my previous posts).

I can eventually understand that with NT6.5 and Access we could have some limitation in database size (although I was far from pushing the technology to the limits!).

But, with an SQL server database in NT7, I don't get it. Either the system is a real crap or for reason that I ignore you have introduced size limitation on the database!

That the system crashes with NT7 on reaching an inadequate maximum database size is one thing (not very glorious) but on top of that you even don't have a procedure to recuperate the data, the saved optimizations or the saved walk forward analysis....

So basically, I was told last time (previous NT7 version) that everything was lost...

I see that in version 22 this ridiculous database limitation (I am dead again) is still present or the system problem is not solved. So, I just hope that you have now fixed a procedure to recuperate your client’s data... If you don't, I would strongly advice you to start right now to develop a small utility. Your credibility is at stake on this problem. For your info, my database is 1.3G, ridiculous size for a database. You should put the maximum at least to 100G!

Problem 2

But this is not the only problem I have with the database (see my previous post again). When you transfer your database from NT6.5 to NT7, it does not transfer the saved result from optimization and walk forward analysis. OK, I can admit that. Try the following after you have made the transfer, open the Strategy Analyser Window in NT7. Click on the “Save” icon and you will see all the names of your previous NT6.5 saved analysis (although the data are not present anymore). If you try to clean the names using the “delete” button, the system crashes…

Hoping you will bring me back some good news and some pleasure to use your application…

Please don't get me somebody answering "use the database repair utility" since when the database crashes, you cannot even access it.

Best regards,

log.20101011.txt

trace.20101011.txt

[NinjaTrader_JoshP]

Ragingbull,

The only way you can get your database so large in NT7 is if you have huge amounts of trade executions, etc. type of data saved in your NT6.5 database which you migrate over. The way around this is to not migrate NT6.5 to NT7, or clear your executions history from 6.5 before migrating to 7. The issues you are having is simply your DB being too large thus these steps would be necessary.

As far as your saved backtests, a saved backtest does not save the actual trade data. It just saves the results. This is the expected behavior in 7 and is not something to do with migration. For your deletion issue, I just tried on my end and it worked fine in deleting a migrated saved result. The issue on your end likely traces back to the DB size being too large. To proceed would require you to take the steps mentioned earlier prior to migration. Since you have already migrated, you could either start afresh with those steps or just close NT, delete the .sdf database, restart NT to regenerate a new one without any 6.5 information.

[Ragingbull]

Dear Josh,

Thanks for you rapid reply but reading through the lines, I understand that:

• the problem is still not solved in terms of managing the maximum database size. You don't give any clear indication on the database limitation and why a ridiculous 1.3G database is crashing the whole system. I don't know any limitation in SQL server of that type, why is it so using SQL server with NT7.
• the problem is still not solved in terms of having an utility to recuperate the database. You basically have to loose everything not only in terms of stored past data but also in terms of saved backtest, optimization and walk forward analysis. If I have to purge everything in 6.5 to create a new database in 7, what's the point. This is non sense, you should at the minimum be able to recuperate past data.
• Finally, problem N°2 is just a consequence of problem N°1. So be it… but I can tell you that my system crashes when I try to delete transferred 6.5 analysis in NT7.

If NT can guarantee the confidentiality of the information contained in the database, I would like to submit you my last workable 6.5 database for analysis (1.3G). Maybe we will see a little bit more clearly what's going on. Let me know what procedure to follow.

Best regards,

[NinjaTrader_JoshP]

Ragingbull,

I understand this can be frustrating to deal with, but it would not be necessary to send us your 6.5 database. When you migrate your 6.5 database to 7 it brings the whole thing over in full. 6.5 = ~1GB --> 7 = ~1GB. The database size of NT7 however should not be anywhere near 1GB. For instance, mine is 5MBs. The database should not be more than a few hundred MB let alone a GB is the upper limit size of it. This should definitely be possible considering data is not stored in the databsae anymore so any .SDF file of the GB size factor would be problematic.

There is unfortunately no utility to "recover" it. The method development has come up with to address this is to reduce the 6.5 database size prior to migration by clearing out trade executions or not migrating at all. Unfortunately there is no better solution at this point in time.

[Ragingbull]

Josh,

Is there anything I can do with an SQL licence?

[NinjaTrader_JoshP]

Ragingbull,

Unfortunately I am not sure what you mean by an SQL license?

[Ragingbull]

Hi Josh,

Sorry but if you don't know what is a Microsoft SQL Server Licence, I don't understand how you could be qualified to answer questions related to NT database...!

Apparently, NT has switch from an Access (NT6.5) to an SQL database (NT7).

If databases are not your field of competence, please forward this serie of posts to somebody who masters the problem. This will avoid that we both loose our time.

Best regards,

[NinjaTrader_Dierk]

Josh,

Is there anything I can do with an SQL licence?

Josh pointed out below what the cause of your trouble is (NT6.5 DB file size) and what actions to take to overcome the issue. Could you please elaborate why you feel "an SQL licence" would have any impact on that issue? Thanks

[Ragingbull]

Hi Dierk,

If you have a Microsoft SQL Server Licence, I would guess that you could directly access to your NT7 database?

If this is the case, it should be possible to clean up, reduce the database size and make the DB available again for use with NT7.

Let me know what procedure could be executed to restore the DB.

Best regards,

[NinjaTrader_Dierk]

Unfortunately we don't provide support for manually manipulating the DB and even recommend not doing that. Please proceed as Josh suggested and let me know if you had any questions. Thanks

[Ragingbull]

Dierk,

From your answer I conclude that NT has still not make any progress on the management of the DB crashes and the maximum size limitation. I don't understand why NT has introduced such a limit since it certainly does not come from SQL database.

I don't understand why nobody in NT is putting any effort on solving this critical problem.

NT proposed solution resumes to "sorry, no solution, restart all over again and loose all you data...". Great!

Best regards,

[NinjaTrader_Dierk]

As Josh pointed out:
a) you unlikely will experience any such limitation starting with a fresh (!) NT7 DB (not a single report so far)
b) limitations your experience is related to migrating a NT6.5 DB where legacy data which no longer is supported in NT7 needed to be cleared manually before migrating.

Not sure how to make it any clearer. Please let us know if you needed further assistance on either (a) or (b). Thanks