I and my team members have a handful of outstanding issues we would like to track, ranging from technical to business-oriented. We are looking for a free/low-cost issue tracker, perhaps with CRM.
Data portability (i.e. that I can export all my issues in a relatively standard format) is important to me.
Something integrated with a wiki (again with data portability) is a plus.
Atlassian JIRA is a recognised issue tracker. Atlassian does a $10 license for 10 users, depending on the size of your startup obviously, this could be a good fit. There are also other products that Atlassian sells for $10.
By the way JIRA integrates pretty well with Confluence which is a wiki that most probably would answer your CRM needs. There is a $10 license for Confluence as well.
Disclaimer I work for Atlassian.
If you're familiar with RoR platform — I can recommend setting up Redmine+FatFreeCRM. Both of them are free, but as almost any open-source product they will need some time to set them up according to your specific needs.
I'm also playing with Redmine recently and, for now, it definitely looks great. It seems that community is pretty active and there are quite few plugins available.
It's true that setting it up could be a nightmare, though, but you don't actually need to know much about RoR itself to install it. It doesn't hurt to try to follow instructions - you might succeed in a first attempt. :)
We just launched a hosted help desk software application, it also has lots of contact management features. I don't know if it would be an exact fit for you, but you can try it free for a month. After that, it's $8 a month per user, pretty much the lowest cost available above open source.
There are a lot of free bug tracking & wiki tools out there. For instance, ProjectLocker does bug tracking with the open-source Trac tool. Downloads of your database can be imported into anyone's Trac installation, including yours if you choose to self-host. Pricing starts at free for teams up to 5.
Disclaimer: I work at ProjectLocker.
Do also consider the cost of your time as well as just the raw cost of the product. We are big fans of Fogbugz because it saves us a huge amount to time on organisation and handling customers and forums - time better spent on developing our product.