Re: [Roundup-users] Spam attack, observations, how to repair

[Kay H. <kay...@gm...>]

Hello Ralf,

2014-07-05 11:15 GMT+02:00 Ralf Schlatterbeck <rs...@ru...>:

> On Sat, Jul 05, 2014 at 10:27:31AM +0200, Kay Hayen wrote:
> > So, should I create a trac ticket for this. I certainly would like to
> alert
> > Debian to this issue. I consider it security relevant, not sure if that
> is
> > by the definition though. For that an upstream issue will be best.
>
> Where do you want to create a trac ticket?
> Do you mean a roundup ticket? Yes, would be fine although it's already
> fixed and would be closed (and therefore probably missed by debian), if
> you want to alert debian, you should report it directly. I think the
> debian maintainer for roundup is following discussions on the list
> though.
>

Embarrassed about that "trac" mention. Involved in too many communities
error. ;-)

Ok, but that is why I was asking. Don't want to create unnecessary work.

As an aside: I also noticed that "pip install roundup" nor "apt-get install
roundup" didn't give me your "spam-removal" script, which is very handy and
served me well to solve the breakage. I might be wrong about pip, as I am
not so sure, if I need to look outside "/usr/local" for pip and I checked
"/usr/share/doc/roundup/examples" only, so I cloned the repository to get
it.

Your proposed checks that forbid content-type changes could be
> implemented as an auditor for the file class, e.g.  (completely
> untested):

[...]


> Extending this to allow content-type changes by certain roles or
> disallow attachments of certain content-type when creating a file is
> left as an exercise to the reader :-)
>

Yup, I will try and provide something to the wiki that limits it to admins
if I can.


> Maybe we should add a (configurable) whitelist of allowed mime types
> that are *not* rewritten to application/octet-stream when served via the
> web interface.
>
> About your concerns of search engines following links: Do these also
> interpret content that is served as application/octet-stream? My
> understanding so far was that they don't ...
>

No, "application/octet-stream" would be safe, I am pretty sure. I only meant
to emphasize that need to be defensive. I would easily sacrifice the display
of attachments in browser.

For the links, "rel=nofollow" already is a good think, and robots.txt ought
to
be efficient measure. I was thinking of ways how to discourage spammers
even further.

And I found
https://developers.google.com/webmasters/control-crawl-index/docs/robots_meta_tag?hl=de

First: HTML attachment files if allowed and to be displayed could have
<meta name="robots" context="noindex"> as that is probably very likely to
prevent any benefit from making such attachments.

Second: As a server, roundup should provide "X-Robots-Tag: noindex" to any
/filexxx and /userxxx URLs at least.

That way, attaching files is pretty much automatically not relevant to
search engines anymore, and it would be merely an issue of hosting foreign
content.

Oh, and another aside, while at it, Python files attached to the database,
are a text/plain, I would rather see mimetypes module employed:

>>> import mimetypes
>>> mimetypes.guess_type("test.py")
('text/x-python', None)

That could enable syntax highlighting in clients.

Yours,
Kay