Handling Business Analysis and Managing Scope Creep

As a developer, I produce a high-quality product, I exceed expectations and I deliver on time.

However, as a project manager and business analyst, I could use a lot of work. I am not particularly skilled at managing client expectations, documenting requirements and managing scope creep.

If I were to start my own business, I could try to choose projects that are already spec'd out or perhaps outsource the BA work to an external agency, but both of these options seem like they would severely limit my opportunities.

On the other extreme, I could just jump in and learn by experience, but I can see that leading to a lot of long nights spent effectively working for free, and that doesn't seem to be a path to happiness either.

What options are there for a software engineer to develop his skills as a project manager and business analyst?

So for PM work that should only be an issue if you have more resources then yourself. Developers typically need to provide sizings on work orders and being an ace developer when you have a good set of requirements you can size it well.

For requirement management this can be an issue. Developers typically like to jump to solutions first rather then solicit the objectives. The first thoughts as a customer starts to talk is hey I could solve it this way. This can be create problems when the whole extent of the requirements are documented or just assumed in.

Throwing in a couple links for requirement management.
http://tech.groups.yahoo.com/group/Requirements-Engineering/ http://www.modernanalyst.com/ So I would look at methodologies. One to look at that covers all the different pieces is RUP. It's a bit complex and isn't as sexy as the Agile and Extreme but it covers all the pieces in depth that may need to be documented. Not suggesting that RUP is superiour just has more documentation around it. There are lots of books on this just like development.

Learn as an apprentice to the expert you outsource the work to. Through that partnership you will develop the practical skills and the understanding of the relationship between the requirements and the production when you are client facing.

You will learn in the context of real live income generating experiences rather than within the mind of a consultant that wrote a curriculum or a book. You will be able to ask questions on demand, challenge on a whim, and see the direct impact of specific choices.

Soon you will be doing change orders on your own, then small quick turn-arounds, then bigger projects -- and you will be the master that other wide-eyes apprentices turn to!

You are dealing with three different skill sets in your question: development (D), business analysis (BA), and project management (PM). You correctly note that all three are required if you are to start your own business. You have D nailed. How did you nail it? Did you go to school, read books, follow tutorials, write code? It's all the same for BA and PM. You have to take learning these skills just as seriously as you did D.

I went this same route myself, starting as a programmer. BA was the next logical area to tackle when I wanted to expand. You have to control that D monster in you that wants to start coding right away. You will do that by wearing one hat at a time and embracing that hat while you are wearing it. It is the natural human inclination to fall back to the familiar. Don't do it. BA when you are BAing, PM when you are PMing.

You said, "I am not particularly skilled at managing client expectations, documenting requirements and managing scope creep." Ok, so how do you get skilled? The same way you did at coding: heads down practice. It's work. Do your work and you will become skilled. It's like driving. How does one become an experienced driver? By driving while inexperienced. Yes, you are going to make mistakes. You are going to take longer than a "pro" at the beginning. You are going to look stupid sometimes. Suck it up, it will pass. (Actually, even pros look stupid sometimes. Have you ever spent half a day looking for a missing colon or a misspelled variable name?)

For project management, start by using a project management tool and manage a project with just you as a resource. Yes, it will be extra work but you will learn. Try to pay attention to what it feels like from the resource side when you get direction from your PM side. Once you have the tool down, offer to manage a project for some deserving entity: a charity, something open source, or maybe a user group you like. Then hit the for-profit world.