Re: Spam attack, observations, how to repair

On Sat, Jul 05, 2014 at 10:27:31AM +0200, Kay Hayen wrote:
> So, should I create a trac ticket for this. I certainly would like to alert
> Debian to this issue. I consider it security relevant, not sure if that is
> by the definition though. For that an upstream issue will be best.

Where do you want to create a trac ticket?
Do you mean a roundup ticket? Yes, would be fine although it's already
fixed and would be closed (and therefore probably missed by debian), if
you want to alert debian, you should report it directly. I think the
debian maintainer for roundup is following discussions on the list
though.

I've clearly marked the change in the release notes as security
relevant. But see below, maybe we need to do more.

> It's also relevant that search engines believe it is HTML and that it is
> indexing links. Even browsers were not to render, that would still be
> sufficient to steal our site rankings for spam links.
> 
> The trailing slash definitely made a difference to the browser. I have
> seen the other discussion of why it is potentially useful, but I would
> want to see ambiguity removed. The browser seeing "/something"
> or "/something/" makes apparently more likely for the later to apply
> guesswork.

See recent discussions on the topic.
I think it's a good idea to not allow trailing slash for other
resources and maybe forbid additional slashes in file resources.

> Also please let me know once it is pip installable. For now I am going
> to apply the script of yours to make the cleanups, but I am going to
> be vulnerable still.

No, I don't think it's currently installable via pip.

> BTW: Content type changes is probably something that I would like to
> not allow I guess. And certainly not to random strings:

I've used content-type changes in the past to fix attached files coming
in via email for which the content type wasn't set correctly by the
email client of the sender. This could be limited to certain admin
roles, though.
But I don't think you'll get much added security because the
content-type could be set when the file is added in the first place (not
needing to change it later). What you'd really need is a whitelist of
allowed content-types. But I wouldn't prevent the setting of these
content-type but instead rewrite the content type to something
considered safe when rendering, see below.

Your proposed checks that forbid content-type changes could be
implemented as an auditor for the file class, e.g.  (completely
untested):

from roundup.exceptions import Reject

def check_content_type (db, cl, nodeid, new_values):
    if 'type' in new_values:
        raise Reject ("Content-Type changes are not allowed")

...

def init (db):
    db.file.audit ("set", check_content_type)

Extending this to allow content-type changes by certain roles or
disallow attachments of certain content-type when creating a file is
left as an exercise to the reader :-)

> kayhayensettype: html -> html2
> This is also something that enabled the attack in the first place. With
> the users being allowed to set non-Mime types, we allow an attacker
> to force the wholeBrowser / Search engine guessing.

As already said above: This would require a whitelist of allowed content
types (and would probably be a good idea). A minimal syntax change of
the content type would also be nice.

> And while there, I think, with "allow_html_file = no", the following
> ought to be prevented:
> 
> kayhayensettype: html2 -> text/html

I don't think so: By default we are already creating mime types with
text/html (e.g. when something comes in via email), but are rewriting
this when the page is rendered. The problem at hand was that this check
failed because browsers interpret all sorts of mime type errors as html.
So not allowing to set a content-type for which our mechanism correctly
rewrites the content type to something safe won't get us better
security.
What worries me is the guesswork of browsers what they chose to
interpret as html. Looks like a moving target to me.

Maybe we should add a (configurable) whitelist of allowed mime types
that are *not* rewritten to application/octet-stream when served via the
web interface.

About your concerns of search engines following links: Do these also
interpret content that is served as application/octet-stream? My
understanding so far was that they don't ...

Ralf
-- 
Dr. Ralf Schlatterbeck                  Tel:   +43/2243/26465-16
Open Source Consulting                  www:   http://www.runtux.com
Reichergasse 131, A-3411 Weidling       email: [email protected]
allmenda.com member                     email: [email protected]

------------------------------------------------------------------------------
Open source business process management suite built on Java and Eclipse
Turn processes into business applications with Bonita BPM Community Edition
Quickly connect people, data, and systems into organized workflows
Winner of BOSSIE, CODIE, OW2 and Gartner awards
http://p.sf.net/sfu/Bonitasoft