I am considering buying some software, which would do very advanced calculations/simulations for me. If this software provides wrong results, I could go bankrupt.
I specifically did not want to mention what kind of software I am talking about, as I am interested in a general answer.
My question How can I judge the reliability of commercial software?
My best ideas so far are:
What other factors could I use?
EDIT: A couple of you have mentioned testing, which is fine, but in my case it could take months of work to do more than basic testing. I have considered hiring someone do to this, but it seems like I am really doing the job of the software company. In addition, would only be able to do black box testing.
David: I've been in the software industry for over a decade, and to be be blunt, I don't see how any of your ideas could give you an insight into the reliability of the software you are considering.
With one exception: References When the software is used for highly complex or mission-critical tasks, your best bet is to look at who the existing customers are. You seem wary of this because a company could use the software only rarely/have low operational dependence on the software. I would counter this argument by saying that in a high-risk environment companies will not ever use - in any part of their process - software which is not reliable. They can't afford to.
So, look to the case studies, search out discussion boards and blogs (they will be out there), look at the list of users the company has on their website.
And once you make the decision, do a test drive - this is easier/more prevalent with web-based/thin clients than it is with thick clients, but speak to a rep and see if you can do a test drive, and check the results for yourself.
Best of luck.
After you have qualified the software as doing on paper what you need:
I would propose you have to: Test it. Develop a test of standardize calculations that you will be running. Run them on the software. See with what reliability the correct answer is delivered.
If they wont give you a version to run test on -- then it is not the software for you.
A couple of you have mentioned testing, which is fine, but in my case it could take months of work to do more than basic testing. I have considered hiring someone do to this, but it seems like I am really doing the job of the software company, I am would only do black box testing.No. Absolutely no!! The testing will enable you to establish if it's producing the figures/results you expect to see in the format in which you expect to see them. This speaks to business rules, selection of mathematical models, etc.
It has nothing to do with bugs.
The company does not provide a trial version of their software? How about a money back guarantee?