Do you have a "data-mining" guy?


Right now I'm in the process of bootstrapping a start-up, so I and my co-founder are working pretty long hours. My only source of new information seems to be Hacker News at the moment, but honestly, besides the thirty minutes I have when going to bed and before falling asleep, I don't wanna spend unnecessary time browsing the web when I could be working. 30 minutes aren't nearly enough to process a day's
worth of new information anyway (especially when I have over ~15-20 subscribed blogs in Google Reader I consider ESSENTIAL, that is, after cutting the number down from 40-50 or so).

So recently I was day-dreaming about how when we get rich and famous :), we would hire a dedicated data-mining guy - they would do everything from reading all those blogs, listening to podcasts, watching YouTube presentations, keeping an eye on financial & business outlets, looking for cool new inventions, exploring new markets opportunities etc. Then at the end of the day, they would do a short presentation about the hot topics of the day, focusing on what might benefit the company in the short and long runs, and send out a mail containing a short write-up, their thoughts and 2 to 3 links AT MOST that they consider important enough to be read / watched in their entirety (obviously not a job for everyone - they should be intimately familiar with the company's technology and owners' philosophy in order to be able to filter out the relevant topics). Coming to think of it, we'd need 2 or 3 of those bad guys :)

Has anyone actually hired someone with a similar task set? Would you if your budget allowed for it?

Ideas Data Budget

asked Jul 30 '11 at 18:54
Philip K
113 points

2 Answers


It can be exhausting to constantly search for new information, so you're thinking about a position in the company to mine information, process what is important, and share it with the company.

You are doing well to think about this, because understanding the market can lead to adjustments in strategy and new opportunities. You can improve the depth at which you think about it, because grafting on new positions should be a later step to solving this. Here are some things to think about:

(1) What if you build an online presence (even small)? People share ideas and market trends with those they follow. Here's a great blog on how to do this by Fred Wilson School of Blogging (2) What if once a quarter you have a strategy meeting with your partner and maybe some external customers or entrepreneurs in the same sector to discuss the trends they've seen over the last 3-4 months. Everyone could be responsible for their research beforehand.

(3) What if you and your partner decide on 1 content agg website and 2 blogs to take responsibility for (Techmeme, HN, M Suster, B Feld, etc.)? As long as you took responsibility, discussion around ideas would come up naturally in interactions.

answered Jul 31 '11 at 00:25
314 points


Different people absorb different part of information. I doubt that it is impossible to hire someone to "concentrate" information that is useful for everyone. My second concern is that it is very important to have "information diversity". If someone is filtering information this will lead to diversity reduction.

answered Jul 30 '11 at 23:08
2,288 points

Your Answer

  • Bold
  • Italic
  • • Bullets
  • 1. Numbers
  • Quote
Not the answer you're looking for? Ask your own question or browse other questions in these topics:

Ideas Data Budget