I'm not sure how to word this but I'll do my best.
I'm in the software business. I know this industry very well and am able to identify problems within this industry. However, there's a ton of competition since the easiest problems to tackle are those that you face every day, in your own industry.
In other words, it's easier to solve your own problems then to put yourself in the feet of someone else.
My question is: how do I put myself in the feet of school teachers, lawyers, doctors, mechanics, artists, taxi drivers, etc. There are some industries that get much less attention from software entrepreneurs because they are unfamiliar.
It really helps if you are the right kind of person for problem solving: You need to have the right kind of personality to be able to observe, empathize (you can put yourself in their shoes), identify problems, and then filter out the real problems, the ones they will not be able to solve by using Excel.
It also helps if you have a lot of experience in general. Many problems are the same problem that looks like a different problem because people call them different names or they solve them a little differently. Once you have a lot of experience, you can look into your toolbox (problems you have seen/solved before) and draw analogies where appropriate.
But in all I have said, the best way is the first thing I mentioned: Do it yourself if possible. Nothing beats doing it yourself for a while.
My experience with this (going from semiconductor manufacturing to healthcare) is only way to figure out someone else's pain is to get to know their business. What is interesting about healthcare (coming from semiconductors) is that they have similar problems.
For example, quality is a big deal. Such a big deal, in fact, that they have 6 Sigma black belts (just like semiconductors) working on improving their processes. It's in the early stages, but the trend is there that quality systems are an integral part of patent safety and quality healthcare. They are doing this by apply software systems to manage patent records, laboratory testing and operating room equipment. All essentially ERP systems.
You might want to look at some macro trends first and how you solved them in software. It took me a couple of years to really understanding healthcare but once I figured out the general problems, I found that relating them to how I solved semiconductor manufacturing issues worked most of the time.
You say there's not a lot of competition in other industries, but that's probably because you're not familiar with them.
You list "doctors" among the people who are ill-served by software, but there's a TON of software for doctors -- practice management, hospitals, and everything in between.
Also many folks don't want software. A taxi driver who don't own a smartphone and gets instructions from home base might not have any pain you can solve with a webapp, for example.
Bottom line is: Like writing, you can't do what you don't know. "Getting in front of" a few folks isn't enough either -- how will you get in front of 100s of 1000s of them with your marketing? How will you speak their language when you make sales?
In my experience, companies like this are successful when a technical founder is paired with a sales-y founder who understands the industry inside-out. Then you (the technical one) don't have any of the problems above, and you can become a domain expert over time.
Get out of the office and talk to potential clients.
You can also find someone who is in the industry to help you. I worked for a small software development shop for a few years and each time we got a new project it was in a new industry (health care, banking, oil and gas). When I started the projects I didn't know much about the industry but after working with end users on a daily basis for a few weeks I would start to pick up the lingo and after a few months I knew quite a bit. My current product I am working on is with animal rescue groups, I started by finding two strong people that have been doing it for years and agreed to give them free service for a year in return for helping me build the product.
I think the best way to identify problems in an industry you know nothing about is to ask people in that industry. Pick an industry you are interested in. Research the major companies and people in that industry and ask them. If you are upfront telling them you are in the software business and do not have a lot of experience in that industry but are looking for problems to solve, people may be open to giving you information. I am always surprised that when I ask a lot of questions and listen well, people say a lot. AND they like talking about what is not working. Most people also like helping others out.
That is how I learned about the book industry when I started. It is also the same way, I would learn about problems (errors, fraud, etc) in companies when I did audits.