"Customer comes with a rough software idea" is probably telling you what they think they want and not what they need.
If you want to establish an ongoing relationship with them, you'll want to do some discovery (as Marcus says) to understand the problem. Once you understand the problem better than the client, you are in a position to make a proposal for a solution.
This position is powerful. The client has already invested in you (even if only their time), has built some rapport with you and should be happy that you're listening to them. It also means that you get to understand how much pain the problem causes, convert that to money, and figure out how much you can charge for the product.
Should you get paid for this discovery? That's up to you. For a big client, that we want, we tend to offer to put in fifty percent of the hours as our risk. For example, if the discovery takes 40 hours, we'll bill 20 and eat 20. This shows the client that you're looking to the future, not just to the end of the month.
It's your job to guide your customer through the process of building this software. From your perspective, you should want to be paid for the entire process.
To build this software you need a specification that your customer agrees to and that you can use to build the actual product. If your customer doesn't even have a start on this specification, you need to develop it for him.
Propose that you will develop the specification, charging him at your current hourly rate. Give him a firm estimate for what this part of the project will cost. Make sure to include client meeting time, your own time, and review time for this specification.
You have to determine what needs to be done before you build it, right? You have a few choices:
I do #4 with each project. I feel it's the most honest & ethical, and also gives them an asset they can use even if they do NOT accept my proposal.
Start with some paper sketches. This may prompt some more ideas by getting them to think more specifically about how the app would be designed and functions. Since they are really vague, you may want to start with a prototype a little earlier in the process than normal.
Another approach could be to look at a similar application that is in another field. Or a more generic application that has a specific feature. If they want a discussion board component, you could go over the stack overflow site and look at some of the features.
Try to keep an open mind about radically changing requirements. Seems like you've approached the cost issue. You don't want them to be reluctant to change their mind, but they still need to pay for it.
The best way is using prototypes, the main idea is that you can show to your customer screens that will server as bridge to get all the requirements, and he will know how the software will reacts to their needs.
If you get to a point where he understands how the software is going to work, then you nail it.
The creation of these prototypes will not require too much effort, if you're using some tools you can create some forms very easy.
here's the definition of Software prototyping: http://en.wikipedia.org/wiki/Software_prototyping is very easy, just screens without code, just the transitions between them.
I don't think you will get detailed requirements from the client. I see two options:
In any case, make sure you have change management and risk analysis in place. Avoid a fixed price contract!