Re: repairing the From: headers of sed-users ...

On 01Jan2015 09:27, Cameron Simpson <[email protected]> wrote:
>As I'm sure you're all aware, Yahoo Groups began mangling our From: 
>headers last year to be from the list, not the author, because Yahoo
>moved to DKIM validation. [...]

Eliana <[email protected]> sent me some sample headers from some other DKIM 
"From:" mangling lists, which were not dealt with by the original script.

Therefore I have done the following:

 - made "fix-dkim-from" a little more flexible to cope with some
   of the examples from other non-yahoo lists

 - made a different script named "fix-dkim-from-swap" which swaps out the
   "From:" header for the "X-Original-From:" header to handle the
   other examples; this script will optionally accept an argument
   naming the other header if it is not "X-Original-From:".

Therefore, the example procmailrc recipes now look like this:

 # DKIM messages which can be drectly unmangled
 :0whf
 * ^from:.*<.*@yahoogroups.com>
 | fix-dkim-from

 # DKIM messages which provide the old From: header in another header
 :0whf
 * ^from:.*<[email protected]>
 | fix-dkim-from-swap

I've attached the updated and new scripts to this message, but in case they get 
stripped they can be fetched here:

 https://bitbucket.org/cameron_simpson/css/src/tip/bin/fix-dkim-from
 https://bitbucket.org/cameron_simpson/css/src/tip/bin/fix-dkim-from-swap

The "Raw" link towards the top right should get you the raw source. Note that 
there are some bare TAB characters in the sed script regexps.

Cheers,
Cameron Simpson <[email protected]>

I couldn't think of anything else to do with it, so I put it on the web.

  ----------

#!/usr/bin/sed -nf
#
# Fix up the From: lines of messages which have passed through a
# DKIM verifying mailing list such as (gah!) all of Yahoo's ones.
#
# They unilaterally decided to introduce this and as fallout they
# could not leave the From: lines alone like any sane mailing list.
# Fortunately, the mangled lines look like this:
#
#  From: "full name local@domain [sed-users]" <[email protected]>
# or:
#  From: "local@domain [sed-users]" <[email protected]>
#
# This can readily be undone and converted into:
#
#  From: "full name" <local@domain>
# or:
#  From: <local@domain>
#
# leaving the message ready for use.
#
# In my mailfiler tool I have these rules:
#
#   from:s/"(?P<fullname>.*\S)\s+(?P<coreaddr>\S+@\S+)\s+\[sed-users\]".*<[email protected]>/"$fullname" <$coreaddr>/
#                           sender:[email protected]
#                           from:/<[email protected]>
#   from:s/"(?P<coreaddr>\S+@\S+)\s+\[sed-users\]".*<[email protected]>/<$coreaddr>/
#                           sender:[email protected]
#                           from:/<[email protected]>
#   sh      SedUsers        to,cc,sender,x-apparently-to:[email protected]
#
# Gory, but it rewrites the From: lines on the sed-users list and
# then files the message in my "sh" folder. There are two s/this/that/
# rules there, one for "full name local@domain" and one for just
# "local@domain".
#
# The sed script below does the same as an external tool for use
# with systems like procmail. Note that you need know that the
# message is suitable to be unmangled, so you will need to phrase the
# procmail rule along these lines:
#
#  :0whf
#  * from:.*<[email protected]>
#  | fix-dkim-from
#
# Hoping this helps,
#   - Cameron Simpson <[email protected]> 31dec2014
#

:top

# save the first line to the hold space
1{
  h
  d
}

# append continuation lines to the hold space
/^[ 	]/{
  H
  d
}

# save current line to hold space, pull back saved line
x

# rewrite the From: line to undo the from-mangling
/^[Ff][Rr][Oo][Mm]:/{

  y/	/ /
  y/\n/ /

  # trim list name from @yahoogroups.com messages
  s/  *\[[^\]*]*\]" \(<[^@>]*@yahoogroups\.com>\)/" \1/

  # replace "full name local@domain" <listname@blah>
  s/"\(.*[^ ]\)  *\([^ ][^ ]*@[^ ][^ ]*\)" *<[^@>]*@[^@>]*>/"\1" <\2>/
  t done
  # replace "local@domain" <listname@blah>
  s/"\([^ ][^ ]*@[^ ][^ ]*\)" *<[^@>]*@[^@>]*>/<\1>/
  :done
}

p

# last line; retrieve saved line and print it
${
  x
  p
}

  ----------

#!/bin/sh
#
# A different DKIM fixup from fix-dkim-from.
#
# fix-dkim-from works for lists where the original From: information
# is preserved in a quotes string in the mangled From: header.
#
# fix-dkim-from-swap is for lists where (arguably more sensibly)
# the original from header has been renamed to (by default)
# X-Original-From:. We just swap them around.
#   - Cameron Simpson <[email protected]> 02jan2015
#

set -ue

# a basic regular expression matching the former From: header name
otherfrom_re=X-Original-From

cmd=$(basename "$0")
usage="Usage: $cmd [orig-hdr-name-re] <dkim-headers >fixed-headers
  Default orig-hdr-name-re: $otherfrom_re"

badopts=

while [ $# -gt 0 ]
do
  case "$1" in
    -*) echo "$cmd: unrecognised option: $1" >&2
        badopts=1
        ;;
    *)  break ;;
  esac
  shift
done

if [ $# -gt 0 ]
then
  otherfrom_re=$1
  shift
  case "$otherfrom_re" in
    */*)
      echo "$cmd: slash (/) characters forbidden in orig-hdr-name-re: $otherfrom_re" >&2
      badopts=1
      ;;
  esac
  if [ $# -gt 0 ]
  then
    echo "$cmd: extra arguments after orig-hdr-name-re: $*" >&2
    badopts=1
  fi
fi

[ $badopts ] && { echo "$usage" >&2; exit 2; }

sed -e "s/^[Ff][Rr][Oo][Mm]:/X-Old-&/
        t done
        s/^$otherfrom_re:/From:/
        :done"



[Non-text portions of this message have been removed]