Unless the startup is relatively far along, everyone should be hands-on in their roles.
CTOs should write code.
VPs of Sales should sell.
I'm not a big believer in pure managerial roles, even within the executive ranks of a startup. My startup is 90 people now and our CTO (me), still writes code.
Yes, they have to be a coder, because unless they're awesome they won't be able to hire other awesome coders.
As "spread thin" as they might be, if they can't hire, motivate, and direct a team of awesome developers, nothing else matters. And no other C_O can do it either.
Assuming that we're talking about a software startup here, not a biotech / hardware / etc.
Early on, a CTO can and often should be a coder. At first, everyone needs to be contributing wherever they are awesome. If your startup isn't purely software-based, then your CTO can be an infrastructure guru or the like. The point is, the CTO has to be able to actively contribute, not just manage. If code is what you're about, then code.
But the CTO can't just be a coder. The CTO has to know how to recruit other technical team members and has to help make the pitch to investors. I know plenty of incredible coders who write great code, but who'd never be at their best in front of VCs. One quality of rockstar coders is not just their great code, but their personalities, their charisma, their ability to make their enthusiasm infect others.
Now remember, too, that when the startup begins to transform into a more organized chaos, the CTO role will shift, too. You'll need to start managing people and budgets and suppliers and schedules. A person who's a great CTO at the beginning may not want to handle those things. You'll need to start thinking about roles like CTO vs VP Engineering, for example.
Absolutely, they should. A startup can't afford to have an "architecture astronaut".
But I wouldn't call running IT and Salesforce.com setup as the CTO/CE's responsibilities. It may be helpful to model the situation as as one person holding multiple positions. Then the question of what the CTO/CE's job is kind of answers itself.
To start off, a single technically-minded person does a sysadmin (Ops), cloud service Mgmt (IT), coding (developer), system architecture design (architect), supervising coders (VP Eng). As the responsibilities grow, you map these tasks to multiple ppl.
Think of it as running multiple virtual servers on a single physical server under light load, and increase the number of physical servers as load increases.
I also believe that "CTO" is a misleading job title in a startup. Something like "Chief Engineer" probably reflects the actual responsibilities better.
On a side note, you should never have a CTO as the only coder. I'm in this position and extremely frustrated. I can't run the business I founded because I'm expected to spend all my time coding.
The CTO needs to be a coder, but they don't necessarily need to code full-time - or, as the product grows, at all. And everyone they hire should be a better coder than they are.
At the very least a CTO should be a competent coder, even though they shouldn't spend all of of their time implementing things. There are multiple advantages to having a CTO who is a hacker:
Don't get hung up on titles. If there's work that needs to be done, someone has to be doing it. If your software startup is three people, your CTO is probably coding. If you are 40 people, there are probably bigger picture jobs for the CTO to be focusing on (you did hire good developers, right?).
Your CTO should have spent considerable time working in the type of environment your company is working on (though not necessarily with a specific technology).
For example, if you're working in Java, your CTO shouldn't be a COBOL guy, but he doesn't necessarily have to have experience using Hibernate.
For a small company (less than 20 people), the CTO should absolutely write code, and his responsibility should be putting together all the disparate parts of the project. For example, if you have someone who's very good at cloud computing and someone who's very good with databases, (s)he should be the one to bridge the gap between them.
They need to know good code when they see it. Talent assessment is key, otherwise they can't delegate.
The last thing they can afford is somebody fobbing them off with technical mumbo jumbo that they don't have the ability to see through and/or hold up to scrutiny.
I'd say, it depends on the kind of product your startup is doing.
If it involves designing and developing software, then I think it is a must. Otherwise, it's just impossible to look ahead.
In addition a lot of programmers now look up to a leader who knows how to code, so it's easy to relate.
CTO should understand and write code, if necessary. Major contributions in the 2 or 3 man startup phase is to make those simple, cost effective decisions that do not come and bite the founder(s)/business in the long run.
CTO should be able to put right requisitions to hire other qualified developers. Totally agree with Raymond G's point on the relevancy of skills/expertise/industry.
In short, CTO should talk the talk and walk the walk!
Gotta agree with Dharmesh - if the CTO doesn't know code, then how will he/she ever know if deadlines are achievable or if estimates coming from engineering are overblown (hint: they are always overblown if the person in charge doesn't know better).
A different perspective:
There are two jobs that need doing in the technology function of any growing company -- building the technology and building the team (hiring, process dev, etc). Without success in both, a startup will falter. Both jobs do not need to be handled by the same person, but the question "Should a CTO code?" pre-supposes that they do. Look for these competencies in your organization and distribute responsibilities accordingly.
As others have said, don't get hung up on titles.
as any other leader, CTO should also lead from the front.
If one is not handson then the decision making is more on conceptual and theoretical basis.
Assuming that the company has several developers, then a CTO may be minimal coding, but he should be able to come in when needed. I have worked for a CTO that was exceptional, but he didn't work in Java, as his languages were LISP and C, so he didn't do any coding, but he had fantastic architects under him that were exceptional mentors and coders, so he worked on the big picture and kept everything going, while getting whatever resources were needed by the developers or anyone else in the tech area.
In a smaller company it may be that the direction goes into a language that the CTO doesn't know, but if he feels comfortable managing, and has a very strong understanding of architecture and is an exceptional hacker, so can recognize excellence when he encounters it, then he may do minimal, if any coding.
For example, you start a solution using .NET and ASP. You then decide that F# may be a better choice, using ASP, then you switch over to Scala and LIFT. It is unlikely that the CTO would have time to learn the new language quickly, with everything on his plate, but over time he should become familiar with it, but he can find people that know the language and can do the mentoring of other developers.
In a very tiny company a CTO should do coding as there may only be 2 or 3 developers and he will be the architect/senior developer/project manager, so a CTO that can't code in the language is almost useless in trying to finish the language, as he isn't earning his pay.
It all depends on the type, size of the company and how you define a CTO's responsibilities. In most small startups where people where a lot of hats a CTO who can't code is a wasted resource. In any of the startups I've been with as the technology team grows the CTO's role has shifted to almost entirely being managing people, high level project management, and communicating vision.
The best CTO I ever worked for and who had tremendous respect from all the developers couldn't code. What he did do great was insulating the developers from all the BS and making it easier for them to do their jobs. But we also had a technology team ranging from 10-20 people there.
It depends on your specific product and the talents available elsewhere on your team. For example, in these days of complex, cloud-based computing and storage platforms, it may be more important in some situations that your CTO grok infrastructure; let somebody else do all the Ruby-on-Rails hacking.
Also, keep in mind that your CTO isn't going to want to turn into the development team leader, reporting 2 levels down from the CTO, once your company grows. All of your C-level employees ought to be comfortable building and operating your business. If you are hiring a CTO purely as a coder, maybe you should reconsider his or her title before making the offer. Downward (or seemingly downward) moves do not inspire loyalty and engagement across your team.