Generally, it's least expensive to form an LLC in the state where you will be doing business. The reason: If you form the LLC in another state (such as NV or WY) because of its ostensibly low or non-existent taxes, you will pay annual fees to that state, but you also will pay annual fees as a "foreign" LLC to the state where you are doing business. Please see Why (not) form an LLC in Nevada (or Wyoming)?.
Disclaimer: This information does not constitute legal advice and does not establish an attorney-client relationship.
Don't focus on cheap. Focus on what's best for your investors and for you.
Delaware has done an outstanding job of working to become a remarkably "LLC Friendly" state. Good laws on the books and good infrastructure to help you get started.
Here's a link to "Why Choose Delaware?" from the state. Nicely done.
Me and the wife have a small software startup. NO investors (yet). We LLC'ed in Delaware. Works for us.
It costs $50.00 to register your Limited Liability Company, Incorporation, Limited Liability Partnerships, Limit Liability Limited Partnerships, General Partnerships, Professional Corporations and Professional Limited Liability Corporation in Arkansas. As long as you are a 'domestic' company. The yearly franchise tax is .3% of outstanding shares if a Corporation ($150 minimum), $300 for corporations that do not issue shares, and $150 for Limited Liability Companies. The State has become very business friendly and the taxes are relatively low compared to east and west cost states. If you don't mind spending $25 a year for a registered agent/office I would highly recommend Arkansas now.
I have setup a few LLCs in Arizona. Including publishing your LLC in a newspaper the cost is just over $95. The nice part about Arizona is its proximity to California, we actually have a major city and the corporate tax rate is around 7%.
Dana Shultz's post is spot on.
Keep in mind as well that some states have what I think is a horrible requirement- the "publication requirement" for LLCs. New York is one and there are a few others. The requirement is that you must publish an announcement in a local paper when you form the LLC in the state or have it qualified to do business as a foreign LLC. Various service providers can handle it for you, but that costs a few hundred, usually.
Delaware also has laws that protect your private information (such as home address and phone number) from being released, whereas many states that is considered public information.
There's more than taxes that you have to think about though. You need to think about the protections your business, and you individually, receive from the laws.
Delaware has created an entire industry around making it cheap and easy to start an LLC even if you don't live there. You can file all of the initial paperwork and get a registered agent (someone who acts as your representative and lives in the state) for about $200 for the first year and $100 for subsequent years.
That said, there's other legal and tax implications depending on your state, but overall, Delaware is petty darn cheap and business friendly.