I've probably said it here a dozen times, but I don't think you can make a judgement like this without considering the whole picture.
I've applied at jobs where they've kept salary information a secret (for example, the posting says nothing about salary) and while the company seemed interesting, salary was my biggest concern. While I don't think I'd ask about salary first, there's definitely a strategy to getting to the point quickly. If he knows he won't take anything less than 100K, but you can't pay more than 50K, then you can both move on to other possibilities without going through the motions of an interview that is destined to not work out.
I wouldn't call it a red flag though. Maybe a yellow flag. I'd keep the interview going, and try to get a broader understanding of the candidate. Make the decision holistically, not based on any one specific question.