#827 Allow project to opt into having the code indexed

[Yura]

currently many other code hosting sites allow code to be indexed by search engines, and this is very useful as coding problems that have already been solved and are well documented in the code can easily be found by new engineers and people learning about code.

I have had a look at the https://sourceforge.net/robots.txt
it says it should: Allow: /p/*/code/HEAD/tree/$
but then it says: Disallow: /p/*/code/*

and this blocks all the source files from being indexed, none of the code in: https://sourceforge.net/p/domination/code/HEAD/tree/ or any other project is being indexed by any search engine.

I know it may be a lot of code there, but i am sure search engines have solved this problem and can manage, also if you make it opt-in then only some projects that really want this will use it.

I really need my projects to have the code indexed, this could either be done by allowing the indexing or by making it easy to mirror the code to another open source code hosting provider like github or gitlab.

currently it is very complicated to set up the auto mirroring to another site, so this may put projects off, and instead push them to just use another website for hosting their code and project.

[Logan Abbott]

Hi, we are considering doing this in the near future. Will update this ticket when we do.