Re: [E-devel] http://git.enlightenment.org down for now

[Jonathan A. <jaq...@ea...>]

Hi Carsten, 

What advantage does http for git have? Isn't git better to use to pull
and push stuff to the repositories or is cgit more a web based ui to
view the repository history etc?

---
Regards, 

Jonathan Aquilina 

On 2017-04-26 04:18, Carsten Haitzler wrote:

> I've had to disable the whole http support for now for git.enlightenment.org
> because several bots are crawling it causing our VM to basically be loaded with
> 10-20 cgit cgi's running git queries for history etc. continually. I/O and
> system load is going through the roof as a result and causing other stuff like
> phab to crawl and begin timing out.
> 
> So anyone using HTTP for doing cmdline git stuff is, at this moment, going to
> find things not working. SSH and GIT protocol should still work. I'll keep this
> shut down for a few hours hoping the bots give up.
> 
> I added a robots.txt and edited the cigtrc to deny all bots from indexing
> git.enlightenment.org - but the bots seem to be ignoring that now that they
> have decided to start indexing.
> 
> I am wondering if this has been the cause of our issues - being overloaded by
> indexer bots. FYI I counted 3 different bots indexing cgit at the same time:
> Baiduspider, Semrushbot, Ahrefsbot.
> 
> I hope later they will start listening to robots.txt, but for now I need to
> keep things off until the bots give up.