Copyright Issues With Creating "Social Search Engine"


I want to create a "social search engine" website. The idea is for people to search for jobs (or articles, or recipes -- I am necessarily targeting jobs), and be able to view them on my site, comment on them, vote on them, share them, etc. An example of this kind of idea is which targets recipes.

My problem is with the copyright. Do I need to get permission from every site I want to crawl to display their content on my own site -- even if I am linking back to them as the source? If so, how would I go about this?

Also, are there any good ways I can have the social features while still being treated as merely a search engine? (As opposed to "stealing" content.)

Website Copyright Best Practices Search Engines

asked Dec 14 '10 at 07:11
116 points
Top digital marketing agency for SEO, content marketing, and PR: Demand Roll

2 Answers


Most sites have a file called Robots.txt, placed in the site root, that tells which parts of the site you may, or may not, access with non-human behaviour. Also make sure you follow fair use when you publish content (publish an abstract, always link to the source site etc.). The Electronic Frontier Foundation has some good info on legals.

In some cases you may collect the data using the site's API. Just make sure the Terms of Service, the EULA or whatever agreement covers the API allows for your usage.

answered Dec 14 '10 at 07:34
John Sjölander
2,082 points
  • after reading this i tried to downvote my own answer. – Frank 13 years ago


If you are doing as google does, just putting a title in the search results and description you are fine. Just give your site owners the option to verify that they own a domain name and opt out.

answered Dec 14 '10 at 07:20
2,079 points
  • In my question I mentioned that that isn't exactly what I want to be doing, so this isn't so helpful. Thanks for the time, though. – Moteutsch 13 years ago

Your Answer

  • Bold
  • Italic
  • • Bullets
  • 1. Numbers
  • Quote
Not the answer you're looking for? Ask your own question or browse other questions in these topics:

Website Copyright Best Practices Search Engines