The most viable idea is obviously the one most generating the most profit. Profit is the difference of costs and revenue, so you need to estimate these two factors. In other words, you are basically shopping for information.
Now here's the more interesting part: According to the ‘information paradox’ (Arrow, 1962), there's no economic solution when shopping for information. You'd need to know the value of a good to be able to determine your maximum willingness-to-pay. But to determine the value of information, you would need to know it. Then, there's no need to shop for it, anymore, since you already have it.
Arrow's information paradox affects all kinds of market research questions.
The only way to work around the issue is what I call the waterfall method : You start using the most inexpensive way to find hints (or signals) that indicate whether it makes sense to stop or shop for more information.
In your case, this means to first find the most inexpensive signals for costs and revenue. Let's look at revenue, first, since this is the harder problem, in general. What you are looking for are indicators of relative strength for each product idea i \in (1,...,n).
Here are a few inexpensive ones:
Each of these should be researchable from your desk. This will give you a decision matrix . You may also introduce weights for each indicator if you like. Do the maths and you should get some order of relative strength. Drop the lowest ones, the number is arbitrary.
For the highest ones, try to calculate their relative costs. If costs are approximately the same, make some "real-world" experiments to strengthen your initial evaluation. This will probably costs you some money. Vineet's answer covers the usual ones.
Hope that helps.
Your team will always have lots of interesting ideas, so your goal should really be to figure out a mechanism and culture to evaluate the ideas often (i.e. not just now).
There are many ways of doing this. Here are some of them:
You can do these for multiple ideas and will find some get more traction versus others. One resource that might be helpful, is tapping into the Lean startups community, as they are trying to figure out techniques for evaluating ideas early.
As well as figuring out whether there will be a market or not (which you can get an idea of by asking some of your target audience; by creating a prototype and putting it in front of them; putting together some marketing material and getting feedback etc.), you need to figure out:
The size of the potential market. Are there enough people out there who want this product? Interviews and surveys can help with this.
The cost of creating the product / whether you can actually create it. If one of the ideas is a time travel machine, that's an excellent idea but you're not going to be able to buy it.
How you're going to reach the people in your market and persuade them to buy it, and how much it will cost to persuade each person. Having a technically great product that people want to buy isn't enough if you can't reach those people.
You can test product concepts, designs, logos, domain names by doing an online market research survey. If you have a pool of people to send your surveys use something like survey monkey. If you need a pool of respondents on the cheap, consider askyourtargetmarket.com.
I think the easiest way to initially get feedback on ideas is to have some minimal prototype and then just ask people in some target market.
The key then is how to find some people that are in the target market that are easily approachable. This is where networks come into play.
This is a low budget way just to see if your idea has merit.
You basically ask what would it take for them to be willing to pay for your application.
You can get some immediate feedback if the problem you are trying to solve isn't really much of a problem.
For example, an iphone app to pre-order your food from Wendy's wouldn't really be helpful since the drive-up is as convenient, and cheaper than buying an app.