It depends on both your endgame and the people you're describing your venture to. Part of being a startup is that you're aiming to grow into something much more substantial down the road. If you're intending to take outside funding, hire on employees down the road, have some sort of liquidity event (IPO, acquisition), then, yeah, I'd say you're a startup. If you aren't ... that is, if you're intending to stay small, developing a small business ... then you aren't really in a "startup" ... but that doesn't mean you can't nestle in with startup founders, contribute to / learn from startup message boards, read books aimed at startups, and so on. The lessons learned and skills developed are almost all the same. And, many would argue, you're freed from the less-important ones, like raising capital.
That being said, although I'm not in a startup (as I've defined it above) — not looking for funding, doubtful that my small business(es) will be acquired, and so on — when I'm describing my work to non-tech-savvy middle-aged people in Virginia, I say "internet startup" because it's a lot easier than boring them with all the details of my business. If I were in Silicon Valley, I'd say "indie web developer".
A one-man shop most definitely counts as a startup in my book.
Yes of course, there are many successful companies that started with one person, some of them are not longer startups because they were bought or became large companies. Some examples:
The point is, all of the above companies were considered startups, even with only one person and they actually turned out to be very successful too.
Examples such as the above are countless, those are only some of my favorite examples since I know the story of those companies very well.