Whether a company is a startup or not is mostly a subjective measure; I think the spirit, mentality and approach it takes is more important than age or size.
As long as the mentality is that everyone does what it takes to succeed irrespective of title or seniority, the organization isn't silo'ed into different bureaucratic entities that don't talk to each other, corporate policies dictate what can and cannot be said about the company in emails, tweets, forums, etc and people go to work with a sense of excitement about all the new things they are going to do - it's a startup.
The startup mentality is likely to die out after an IPO or after the company grows to be bigger than around 150 people, but can still persist in pockets of the company after that - something that definitely should be encouraged.
I don't think it has to be tech related to be a startup, no, although most of the ones I can think of has a strong technology element to them.
I define a startup as a company that:
I do not think they are limited only to technology.
You're a startup when you don't have a dress code. The reason you don't have a dress code is you are too busy trying to "start" the company. And besides, if you wanted to adhere to a set of rules telling you how to dress you wouldn't start your own company. You don't worry about others not following a dress code because you either haven't hired someone else or you don't need to worry about improperly influencing the other people you've hired because there are none of those either. Everyone in the company will do whatever it takes to make the company successful without having to be told what not to wear. To hell with the VC who doesn't appreciate my Iggy Pop T-Shirt with the bleach stain caused by my girl friend's vomit at the Arragon Ball Room concert. Patent infringements, unlawful dismissal threats, taxes, health care & 401k plans can be disregarded since you are not making any money anyway.
Technically speaking, a company is a start-up until they are executing a finely tuned business plan and making profit. As soon as the business is operational and profitable, the start-up "feel" usually goes away because the board & executives get addicted to the profit margin. Process creeps in and all of the things that made it a start-up atmosphere slowly die off.
I have seen companies where the culture sticks when it is profitable, like Fog Creek Software, but it rare.