+ Post New Thread
Results 1 to 7 of 7
*nix Thread, zimbra / postfix content filtering for Bad Language in Technical; I just got this working, any improvements gratefully received. (this is internal documentation but I thought I'd share This howto ...
  1. #1


    Join Date
    Jan 2006
    Posts
    8,202
    Thank Post
    442
    Thanked 1,032 Times in 812 Posts
    Rep Power
    338

    zimbra / postfix content filtering for Bad Language

    I just got this working, any improvements gratefully received.
    (this is internal documentation but I thought I'd share

    This howto was specifically written for a Zimbra installation but should work on any postfix server.

    It is necessary for us to filter bad words from any email that has been sent internally or externally.
    The plan is to use postfix after queue content filter to run a script that extracts email from the queue, process the email then injects it back into the queue for delivery. We will be using grep to check the email against a wordlist of regular expressions. We will avoid email loops by using putmail.py to re inject the email into the mail queue.

    The following software was used:
    Centos 4.3 http://centos.org
    Zimbra 3.1.3 http://www.zimbra.com
    putmail.py 1.3 http://putmail.sourceforge.net/home.html
    python
    grep
    A wordlist of regular expression to Ban
    filter.sh €“ described below

    Zimbra was installed into a default installation of Centos 4.3

    A directory must be created to spool the mail to whilst it is being processed.
    here we will make it in the zimbra directory, /var/spool/filter would be another sensible option.
    We will make zimbra the owner of this directory and make sure it has read write execute access.
    On a non zimbra install another non-root user should be created to run the script.

    Code:
    mkdir /opt/zimbra/postfix/spool/filter
    chown zimbra:zimbra /opt/zimbra/postfix/spool/filter
    chmod 700 /opt/zimbra/postfix/spool/filter
    We will copy the putmail.py program to /usr/local/sbin and make sure the zimbra user can access it:

    Code:
    cp putmail.py /usr/local/sbin
    chown zimbra /usr/local/sbin/putmail.py
    chmod 500 /usr/local/sbin/putmail.py
    create the putmail configuration home directory:
    Code:
    mkdir /opt/zimbra/.putmail
    chown zimbra:zimbra /opt/zimbra/.putmail
    add the configuration file to this directory:
    Code:
    jmacs /opt/zimbra/.putmail/putmailrc
    and make sure it contains the port number specified in the postfix master.cf file:

    Code:
    [config]
    server = localhost
    email = amin@your.org
    port = 10026
    We need to configure putmail, and edit the putmail program to tell it where its configuration is located:

    Code:
    emacs /usr/local/sbin/putmail.py
    make sure this line points to the configuration home directory:
    Code:
     
    HOME_DIR = '/opt/zimbra/'
    You now need to make a word list file containing the bad language, this is the most fun part of the entire exercise, I recommend doing this over a few beers.
    Your wordlist should contain the prohibited words in the following format:

    Code:
    \<badword\>
    \<badword1\>
    \<badword1 
    \<badword2\>
    \<badword2
    Call this file 'words' and place it into a configuration directory

    Code:
    cp words /etc/
    We now need to create and edit the script that ties it all together, probably you can copy it from here .This script was modified from the postfix website.
    Check that INSPECT_DIR points to your filter directory, SENDMAIL points to your putmail.py program, BADWORDS points to the location of your words file and that MAILADMIN is set to the email address of the administrator.

    Code:
    !/bin/sh
    
      # Simple shell-based filter. It is meant to be invoked as follows:
      #       /path/to/script -f sender recipients...
      # Localize these. The -G option does nothing before Postfix 2.3.
    
    INSPECT_DIR=/opt/zimbra/postfix/spool/filter
    SENDMAIL=/usr/local/sbin/putmail.py 
    MAILADMIN=email@address.of.your.org
    BADWORDS=/etc/words
    
      # Exit codes from <sysexits.h>
    EX_TEMPFAIL=75
    EX_UNAVAILABLE=69
    
    # Clean up when done or when aborting.
    trap "rm -f in.$$" 0 1 2 3 15
    
    #Start processing.
    cd $INSPECT_DIR || {
    echo $INSPECT_DIR does not exist; exit $EX_TEMPFAIL; }
    cat >in.$$ || {
    echo Cannot save mail to file; exit $EX_TEMPFAIL; }
    # Specify your content filter here.
    cat in.$$ | while read line; do echo $line|grep -v 'Content-Disposition: attachment;' || break; done | /bin/grep -if /etc/words  && {
    echo ": The message content has been rejected, your actions have been reported" ; $SENDMAIL $MAILADMIN <in.$$ ; exit $EX_UNAVAILABLE; }
    
    
    $SENDMAIL "$@" <in.$$
    exit $?

    Now we need to adapt the postfix master.cf file to use the ammendments that we have just made.
    On a default zimbra install it lives in /opt/zimbra/postfix/conf/master.cf




    Backup the original master.cf file then edit it.
    Code:
    cd /opt/zimbra/postfix/conf
    cp master.cf master.cf.orig
    emacs master.cf
    add this line (to the bit after maildrop, although I'm not sure if its position in the config makes any difference)
    Code:
    filter    unix  -       n       n       -       -      pipe
            flags=Rq user=zimbra argv=/usr/local/sbin/filter -f ${sender} -- ${recipient}
    zimbra

    Navigate to this line and add the content filter named filter to the option -o content_filter=

    Code:
    127.0.0.1:10025 inet n  -       n       -       -  smtpd
                      -o content_filter=filter:dummy
    Then add the following to the end of the file:

    Code:
    127.0.0.1:10026 inet n  -       n       -       -  smtpd
            -o content_filter=
            -o smtpd_authorized_xforward_hosts=127.0.0.0/8
            -o local_recipient_maps=
            -o virtual_mailbox_maps=
            -o virtual_alias_maps=
            -o relay_recipient_maps=
            -o smtpd_restriction_classes=
            -o smtpd_delay_reject=no
            -o smtpd_client_restrictions=permit_mynetworks,reject
            -o smtpd_helo_restrictions=
            -o smtpd_sender_restrictions=
            -o smtpd_recipient_restrictions=permit_mynetworks,reject
            -o mynetworks_style=host
            -o mynetworks=127.0.0.0/8
            -o strict_rfc821_envelopes=yes
            -o smtpd_error_sleep_time=0
            -o smtpd_soft_error_limit=1001
            -o smtpd_hard_error_limit=1000
            -o smtpd_client_connection_count_limit=0
            -o smtpd_client_connection_rate_limit=0
            -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks,no_address_mappings
    Restart Zimbra (or postfix)

    Code:
    su €“ zimbra
    zmcontrol stop
    zmcontrol start
    job done.

    Further reading (adding disclaimers etc):

    http://www.postfix.org/FILTER_README.html
    http://www.zimbra.com/forums/showthr...2&page=2&pp=10
    http://putmail.sourceforge.net/home.html
    Last edited by CyberNerd; 15th February 2008 at 09:37 AM.

  2. 3 Thanks to CyberNerd:

    Ric_ (15th February 2008), Sylv3r (16th February 2008), webman (15th February 2008)

  3. #2

    Join Date
    May 2006
    Posts
    66
    Thank Post
    0
    Thanked 0 Times in 0 Posts
    Rep Power
    0

    Re: zimbra / postfix content filtering for Bad Language

    Cool, maybe this should go in a wiki somewhere, I think there is one on this site, or maybe send it to the zimbra project or tldp.org.

  4. #3

    webman's Avatar
    Join Date
    Nov 2005
    Location
    North East England
    Posts
    8,372
    Thank Post
    624
    Thanked 949 Times in 651 Posts
    Blog Entries
    2
    Rep Power
    317
    I know this is an ancient thread; but I've tried this on our Zimbra test server and come across a little problem. On messages with attachments, the filter regularly picks up bad words in the Base64-encoded attachment section of the email.

    Is there any way to prevent this that you are aware of?

  5. #4


    Join Date
    Jan 2006
    Posts
    8,202
    Thank Post
    442
    Thanked 1,032 Times in 812 Posts
    Rep Power
    338
    @ webman, yes one of our yr 11's came up with this:

    Code:
    cat in.$$ | while read line; do echo $line|grep -v 'Content-Disposition: attachment;' || break; done | /bin/grep -if /etc/words  && {
    echo ": The message content has been rejected, your actions have been reported" ; $SENDMAIL $MAILADMIN <in.$$ ; exit $EX_UNAVAILABLE; }
    It reads after the 'content-disposition' line that zimbra puts for an attachment.

    I'll put this on the wiki now

  6. Thanks to CyberNerd from:

    webman (15th February 2008)

  7. #5


    Join Date
    Jan 2006
    Posts
    8,202
    Thank Post
    442
    Thanked 1,032 Times in 812 Posts
    Rep Power
    338
    I put this on the wiki, so if anyone wants to improve this method go ahead..
    http://www.edugeek.net/wiki/index.ph...content_filter

  8. #6

    webman's Avatar
    Join Date
    Nov 2005
    Location
    North East England
    Posts
    8,372
    Thank Post
    624
    Thanked 949 Times in 651 Posts
    Blog Entries
    2
    Rep Power
    317
    Ah, excellent! Thanks CyberNerd and the Year11... will have another go

  9. #7

    Sylv3r's Avatar
    Join Date
    Jul 2005
    Location
    Co. Durham
    Posts
    3,151
    Thank Post
    369
    Thanked 365 Times in 323 Posts
    Rep Power
    145
    Excellent Doc Cybernerd.

    Once I get my admin server moved across to the new box I will be installing Zimbra for students use onto the old admin server.

    From what I have seen it looks a good product.

SHARE:
+ Post New Thread

Similar Threads

  1. Web Content Filtering
    By richard.thomas in forum Network and Classroom Management
    Replies: 5
    Last Post: 14th November 2007, 09:50 AM
  2. Proxy with content filtering
    By gwendes in forum *nix
    Replies: 21
    Last Post: 8th March 2007, 12:11 AM
  3. Content filtering
    By mitcheln in forum School ICT Policies
    Replies: 11
    Last Post: 20th December 2006, 08:38 PM
  4. Rm SmartCache Content Filtering
    By Mintsoft in forum Windows
    Replies: 9
    Last Post: 2nd March 2006, 09:51 AM

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •