I am at a stage of growing a site where it is in open alpha, and there are about 50 people coming to the site per day or so...without any solicitation. They come from stumbleupon, google, and a few scattered places across the web.
What I am trying to do now is increase the conversion rates of pages. My site is a UGC site and it is important that the users register and contribute in the community.
What software out there (preferably free) can I try to help me understand why users are not signing up?
Optimizely (cheap), Visual Website Optimizer (free trial), and Google Website Optimizer (free) will all let you do A/B testing if you want to see how different versions of a particular page element (or even totally different page layouts) affect conversion.
Outside of A/B testing, digging into your analytics can provide many insights as to why people are leaving your site. Three things I'd look at:
Also, try alternative analytics tools like CrazyEgg, which can provide click heatmaps and scroll heatmaps -- are people not seeing the signup prompt? Are people clicking on a logo or image that doesn't actually link to anything?
Finally, learn from others. ABtests.com has many conversion case-studies. As always, your industry / audience has unique considerations, so use these to get ideas and improve what you're initially testing, but make sure you are doing your own testing.
2012 Update: Another tool I've started using is Mixpanel (free up to a certain amount of data per month). They're event tracking analytics; the main way to use this for increasing conversions is through their funnels feature. By defining a series of events (more detailed than page views (e.g. form field clicks, progress through a series of lightbox/modal windows)), you could see exactly which step is where most people are falling out of the funnel.