Koozali.org: home of the SME Server

Save email attachments to iBay

Offline wires12

  • ***
  • 56
  • +0/-0
Save email attachments to iBay
« on: November 13, 2009, 08:32:56 PM »
I need a workaround.

Being a bit older the KM-4035 copy/fax/scan machine apparently will not scan directly to a file share. It will scan to email, scan to a winPC with client software, or to a database connection that appears to be winPC only. Since it will scan to email I have been researching using a script to process an incoming email and save the attachment to a directory. The most promising thing I have found is on http://tek-tips.com/faqs.cfm?fid=4138:

Quote
Perl script to save an email attachment to disk
faq80-4138
Posted: 9 Sep 03

From unknown source, bless his perl skills. Change the TMPDIR path to where you want to save the attachments. I called it unmime, you are free to call it anything you want.

Things to watch out for:

Path to perl RTE.
Executable rights for script.

Define an alias in sendmail's /etc/aliases to call the script, e.g:
detach: |"/your/path/to/script/unmime"

Mailing to the alias will detach the file to your preferred directory. Remember to run "newaliases" after adding the alias.
Code: [Select]
#!/usr/bin/perl
#
# Un-MIME regular message from stdin.
# Non-text version saved in directory ~/mail/MIME and proper indication is
# left in the dumped message. Text is otherwise dumped and deleted from there.
#
# Intended to be used with mailagent thanks to the following incantation rule:
#
# Mime-Version: /^\d/    { SAVE +mime; FEED ~/mail/unmime; RESYNC; REJECT };
#
# Options:
#   -e: pass the quoted-printable decoder over the message and that's it.
#    -x: translate chars not understood by some iso8859-1 fonts.
#    -X: translate all accents to non-accentuated letters (plain ASCII).

($me = $0) =~ s|.*/(.*)|$1|;

require "getopts.pl";
Getopts('exX');

$opt_x++ if $opt_X;        # -X implies -x

$TMPDIR = "/var/spool/mail/MIME";

use MIME::Parser;

#------------------------------------------------------------
# dump_entity - idempotent routine for dumping an entity
#------------------------------------------------------------

sub dump_entity {
  my ($entity) = @_;
  my $IO;
  my $not_first_part = 0;
 
  # Print the header, converting accents if any
  my $head = $entity->head->original_text;
  $head =~ s/^(Subject:.*)/no_iso_markup($1)/me
    if $head =~ /^Subject:.*=\?iso-8859-1\?Q\?/mi;
  print $head, "\n";
 
  # Output the body:
  my @parts = $entity->parts;
  if (@parts) {            # multipart...
    my $i;
    foreach $i (0 .. $#parts) { # dump each part...
      dump_entity($parts[$i]);
    }
  } else {            # single part...
    # Get MIME type, and display accordingly...
    my ($type, $subtype) = split('/', $entity->head->mime_type);
    #print STDERR "type - $type\n";
   
    my $body = $entity->bodyhandle;
    my $path = $body->path;
    if ($type =~ /^(text|message)$/ || -T $path) {     # text: display it...
      if ($IO = $body->open("r")) {
    print "\n" if $not_first_part++;
    print to_ascii($_) while (defined($_ = $IO->getline));
    $IO->close;
   
    # If message is text/message, chances that we did the right
    # thing are extremely high. So unlink the message if lying on
    # the disk... -- RAM, 19/11/96

    #unlink($path) or warn "$me: can't unlink $path: $!\n"
    #  if defined $path && -f $path;
   
      } else {            # d'oh!
    die "$me: couldn't find/open '$file': $!";
      }
    } else {            # binary: just summarize it...
      my $size = ($path ? (-s $path) : '???');
      print ">>> This is a non-text message, $size bytes long.\n";
      print ">>> It is stored in ", ($path ? "'$path'" : 'core'),".\n\n";
    }
  }
  print "\n";
 
  1;
}

#------------------------------------------------------------
# smart_pack
#------------------------------------------------------------
sub smart_pack {
  my ($hexa) = @_;
  my $val = hex($hexa);
  return "=$hexa" if $val >= 128; # We're smart right there!
  return pack('C', $val);
}

#------------------------------------------------------------
# no_accent
#------------------------------------------------------------
sub no_accent {
  local ($_) = @_;
  tr/\xab\xbb\xe0\xe2\xe7\xe8\xe9\xea\xee\xef\xf4\xf9\xfb/""aaceeeiiouu/;
  return $_;
}

#------------------------------------------------------------
# to_ascii
#------------------------------------------------------------
sub to_ascii {
  my ($l) = @_;
  return $l unless $opt_x;    # Don't loose info unless -x or -X
  $l =~ tr/\x92/'/ if $opt_x;    # ';
  $l = no_accent($l) if $opt_X;
  return $l;
}

#------------------------------------------------------------
# to_txt -- combines =xx packing with no_accent()
#------------------------------------------------------------
sub to_txt {
  my ($l) = @_;
  $l =~ s/=([\da-fA-F]{2})/pack('C', hex($1))/ge;
  return no_accent($l);
}

#------------------------------------------------------------
# no_iso_markup -- removes ugly ?iso-8859-1?Q escapes
#------------------------------------------------------------
sub no_iso_markup {
  local ($_) = @_;
  s/^(.*?)=\?iso-8859-1\?Q\?(.*)\?=/$1 . to_txt($2)/ie;
  s/_/ /g;
  return $_;
}

#------------------------------------------------------------
# unquote_stdin
#------------------------------------------------------------
sub unquote_stdin {
  local $_;
  my $encoded = 0;
  my $in_header = 1;
  while (<STDIN>) {
    $in_header = 0 if /^\s*$/;
   
    # All Subject: line with accents to be "un-mimed" as well.
    s/^(Subject:.*)/no_iso_markup($1)/e
      if $in_header && /^Subject:.*=\?iso-8859-1\?Q\?/i;
   
    # Avoid decoding inlined uuencoded/btoa stuff... since they might
    # accidentally bear valid =xx escapes... The leading \w character
    # is there in case the thing is shar'ed...
    # Likewise, all the lines longer than 60 chars and with no space
    # in them are treated as being encoded iff they begin with M.

    $encoded = 1 if /^\w?begin\s+\d+\s+\S+\s*$/ || /^\w?xbtoa Begin\s*$/;
    $encoded = 0 if /^\w?end\s*$/ || /^\w?xbtoa End/;
   
    if ($encoded || (length > 60 && !/ / && /^M/)) {
      print $_;
    } else {
      # Can't use decode_qp from MIME::QuotedPrint because we might not
      # face a real quoted-printable message...
      # Inline an alternate  version.
     
      s/\s+(\r?\n)/$1/g;    # Trailing white spaces
      s/^=\r?\n//;        # Soft line breaks
      s/([^=])=\r?\n/$1/;    # Soft line breaks, but not for trailing ==
      s/=([\da-fA-F]{2})/smart_pack($1)/ge;    # Hehe
      print to_ascii($_);
    }
  }
  return 1;    # OK
}

#------------------------------------------------------------
# main
#------------------------------------------------------------

sub main {
  return &unquote_stdin if $opt_e;
 
  # Create a new MIME parser:
  my $parser = new MIME::Parser;
 
  # Create and set the output directory:
  $parser->output_dir($TMPDIR);
 
  # Read the MIME message:
  $entity = $parser->read(\*STDIN) or
    die "$me: couldn't parse MIME stream";
 
  # Dump it out:
  dump_entity($entity);
  unlink<$TMPDIR/msg-*.txt> or warn "can't unlink: $!\n";
}

exit(&main ? 0 : -1);
#------------------------------------------------------------
1;
#
# This bit below saves the message body to file, uncomment if wanted
#
#unlink</var/spool/mail/MIME/msg-*.txt> or warn "can't unlink: $!\n";

Is this likely to work on SME 7.4?

Do I create the alias in the web GUI?

How do I "Define an alias in sendmail's /etc/aliases to call the script, e.g:
detach: |"/your/path/to/script/unmime" ?

If I create the alias in the web GUI doe it run "newaliases"?

Is this sort of thing going to break something else?


TIA!


P.S. Other options I have found are these:
http://wiki.mutt.org/?MuttFaq/Attachment
http://www.experts-exchange.com/Programming/Languages/Scripting/Perl/Q_22835085.html
http://ask.metafilter.com/31730/Strip-email-attachments-based-on-the-subject-line



Offline versa

  • ****
  • 109
  • +0/-0
Re: Save email attachments to iBay
« Reply #1 on: November 13, 2009, 09:02:29 PM »
I cannot remember for sure but I thaught it could scan to ftp?
So set it to scan to the iBay path with the admin username and password.
......

Offline wires12

  • ***
  • 56
  • +0/-0
Re: Save email attachments to iBay
« Reply #2 on: November 13, 2009, 09:10:14 PM »
I cannot remember for sure but I thaught it could scan to ftp?
So set it to scan to the iBay path with the admin username and password.

Nope. Tech support just returned my call from yesterday. They say machine is too old to do anything except the email and Windows only options mentioned before.

Offline versa

  • ****
  • 109
  • +0/-0
Re: Save email attachments to iBay
« Reply #3 on: November 13, 2009, 10:21:06 PM »
Have a look at point 5 on the second page.

http://tuamwan.org/kyocera/SB--7163.pdf

See if the service guys can upgrade your firmware.
......

Offline CharlieBrady

  • *
  • 6,918
  • +3/-0
Re: Save email attachments to iBay
« Reply #4 on: November 13, 2009, 10:32:52 PM »
SME server does not use sendmail/newaliases, so your recipe will not work without substantial modification.

Offline wires12

  • ***
  • 56
  • +0/-0
Re: Save email attachments to iBay
« Reply #5 on: November 13, 2009, 11:13:57 PM »
versa - Thanks! I will give them a call on Monday to see about a FTP upgrade.

CharlieBrady - Good point! I thought I was missing something since aliases didn't show up in /etc/aliases. I think I found them in /var/qmail/users/assign. I don't think sendmail would have been the correct place for this anyway since I want this to work on incoming mail.

Would /home/e-smith/files/users/username/.qmail-scriptname be the correct location to fire a script?

Offline CharlieBrady

  • *
  • 6,918
  • +3/-0
Re: Save email attachments to iBay
« Reply #6 on: November 14, 2009, 05:26:18 AM »
Would /home/e-smith/files/users/username/.qmail-scriptname be the correct location to fire a script?

It would be if the addressee of the email was username-scriptname@domain.name.

Offline wires12

  • ***
  • 56
  • +0/-0
Re: Save email attachments to iBay
« Reply #7 on: November 14, 2009, 08:02:30 AM »
It would be if the addressee of the email was username-scriptname@domain.name.

Yep - I just learned about that from reading www.evolt.org/incoming_mail_and_php. Quite handy.

...your recipe will not work without substantial modification.

Actually very little modification is required to get it running. Simply adding a pipe and location of the script at the end of .qmail file will fire the script and changing the $TMPDIR = "/home/e-smith/files/users/scanner/home"; line works. What is puzzling me now is why I can't set the $TMPDIR = to an i-bay where the user has RW privileges and have it work. It works fine in the user's home directory with the $TMPDIR = change. I guess I could learn a bit of perl and do file copies and deletes at the end of unmime.sh

I also have not figured out the best way to handle the .qmail template since implementation depends on how the i-bay issue resolves. If user scanner can save to an i-bay then that user could receive all the scans. If not then each user could get a .qmail-scan alias so email to username-scan@domain.name would go to their home directory.

working .qmail file:
Code: [Select]
#------------------------------------------------------------
#              !!DO NOT MODIFY THIS FILE!!
#
# Manual changes will be lost when this file is regenerated.
#
# Please read the developer's guide, which is available
# at http://www.contribs.org/development/
#
# Copyright (C) 1999-2006 Mitel Networks Corporation
#------------------------------------------------------------

| condredirect scanner-junkmail headermatch 'X-Spam-Status: Yes'
# Forward not set
./Maildir/
|/home/e-smith/files/users/scanner/unmime.sh

working unmime.sh file:
Code: [Select]
#!/usr/bin/perl
#
# Un-MIME regular message from stdin.
# Non-text version saved in directory ~/mail/MIME and proper indication is
# left in the dumped message. Text is otherwise dumped and deleted from there.
#
# Intended to be used with mailagent thanks to the following incantation rule:
#
# Mime-Version: /^\d/    { SAVE +mime; FEED ~/mail/unmime; RESYNC; REJECT };
#
# Options:
#   -e: pass the quoted-printable decoder over the message and that's it.
#    -x: translate chars not understood by some iso8859-1 fonts.
#    -X: translate all accents to non-accentuated letters (plain ASCII).

($me = $0) =~ s|.*/(.*)|$1|;

require "getopts.pl";
Getopts('exX');

$opt_x++ if $opt_X;        # -X implies -x

#       $TMPDIR = "/var/spool/mail/MIME";
#$TMPDIR = "/home/e-smith/files/users/scanner/Maildir/tmp";
$TMPDIR = "/home/e-smith/files/users/scanner/home";
#$TMPDIR = "/home/e-smith/files/ibays/public1/files";

use MIME::Parser;

#------------------------------------------------------------
# dump_entity - idempotent routine for dumping an entity
#------------------------------------------------------------

sub dump_entity {
  my ($entity) = @_;
  my $IO;
  my $not_first_part = 0;

  # Print the header, converting accents if any
  my $head = $entity->head->original_text;
  $head =~ s/^(Subject:.*)/no_iso_markup($1)/me
    if $head =~ /^Subject:.*=\?iso-8859-1\?Q\?/mi;
  print $head, "\n";

  # Output the body:
  my @parts = $entity->parts;
  if (@parts) {            # multipart...
    my $i;
    foreach $i (0 .. $#parts) { # dump each part...
      dump_entity($parts[$i]);
    }
  } else {            # single part...
    # Get MIME type, and display accordingly...
    my ($type, $subtype) = split('/', $entity->head->mime_type);
    #print STDERR "type - $type\n";

    my $body = $entity->bodyhandle;
    my $path = $body->path;
    if ($type =~ /^(text|message)$/ || -T $path) {     # text: display it...
      if ($IO = $body->open("r")) {
    print "\n" if $not_first_part++;
    print to_ascii($_) while (defined($_ = $IO->getline));
    $IO->close;

    # If message is text/message, chances that we did the right
    # thing are extremely high. So unlink the message if lying on
    # the disk... -- RAM, 19/11/96

    #unlink($path) or warn "$me: can't unlink $path: $!\n"
    #  if defined $path && -f $path;

      } else {            # d'oh!
    die "$me: couldn't find/open '$file': $!";
      }
    } else {            # binary: just summarize it...
      my $size = ($path ? (-s $path) : '???');
      print ">>> This is a non-text message, $size bytes long.\n";
      print ">>> It is stored in ", ($path ? "'$path'" : 'core'),".\n\n";
    }
  }
  print "\n";

  1;
}

#------------------------------------------------------------
# smart_pack
#------------------------------------------------------------
sub smart_pack {
  my ($hexa) = @_;
  my $val = hex($hexa);
  return "=$hexa" if $val >= 128; # We're smart right there!
  return pack('C', $val);
}

#------------------------------------------------------------
# no_accent
#------------------------------------------------------------
sub no_accent {
  local ($_) = @_;
  tr/\xab\xbb\xe0\xe2\xe7\xe8\xe9\xea\xee\xef\xf4\xf9\xfb/""aaceeeiiouu/;
  return $_;
}

#------------------------------------------------------------
# to_ascii
#------------------------------------------------------------
sub to_ascii {
  my ($l) = @_;
  return $l unless $opt_x;    # Don't loose info unless -x or -X
  $l =~ tr/\x92/'/ if $opt_x;    # ';
  $l = no_accent($l) if $opt_X;
  return $l;
}

#------------------------------------------------------------
# to_txt -- combines =xx packing with no_accent()
#------------------------------------------------------------
sub to_txt {
  my ($l) = @_;
  $l =~ s/=([\da-fA-F]{2})/pack('C', hex($1))/ge;
  return no_accent($l);
}

#------------------------------------------------------------
# no_iso_markup -- removes ugly ?iso-8859-1?Q escapes
#------------------------------------------------------------
sub no_iso_markup {
  local ($_) = @_;
  s/^(.*?)=\?iso-8859-1\?Q\?(.*)\?=/$1 . to_txt($2)/ie;
  s/_/ /g;
  return $_;
}

#------------------------------------------------------------
# unquote_stdin
#------------------------------------------------------------
sub unquote_stdin {
  local $_;
  my $encoded = 0;
  my $in_header = 1;
  while (<STDIN>) {
    $in_header = 0 if /^\s*$/;

    # All Subject: line with accents to be "un-mimed" as well.
    s/^(Subject:.*)/no_iso_markup($1)/e
      if $in_header && /^Subject:.*=\?iso-8859-1\?Q\?/i;

    # Avoid decoding inlined uuencoded/btoa stuff... since they might
    # accidentally bear valid =xx escapes... The leading \w character
    # is there in case the thing is shar'ed...
    # Likewise, all the lines longer than 60 chars and with no space
    # in them are treated as being encoded iff they begin with M.

    $encoded = 1 if /^\w?begin\s+\d+\s+\S+\s*$/ || /^\w?xbtoa Begin\s*$/;
    $encoded = 0 if /^\w?end\s*$/ || /^\w?xbtoa End/;

    if ($encoded || (length > 60 && !/ / && /^M/)) {
      print $_;
    } else {
      # Can't use decode_qp from MIME::QuotedPrint because we might not
      # face a real quoted-printable message...
      # Inline an alternate  version.

      s/\s+(\r?\n)/$1/g;    # Trailing white spaces
      s/^=\r?\n//;        # Soft line breaks
      s/([^=])=\r?\n/$1/;    # Soft line breaks, but not for trailing ==
      s/=([\da-fA-F]{2})/smart_pack($1)/ge;    # Hehe
      print to_ascii($_);
    }
  }
  return 1;    # OK
}

#------------------------------------------------------------
# main
#------------------------------------------------------------

sub main {
  return &unquote_stdin if $opt_e;

  # Create a new MIME parser:
  my $parser = new MIME::Parser;

  # Create and set the output directory:
  $parser->output_dir($TMPDIR);

  # Read the MIME message:
  $entity = $parser->read(\*STDIN) or
    die "$me: couldn't parse MIME stream";

  # Dump it out:
  dump_entity($entity);
  unlink<$TMPDIR/msg-*.txt> or warn "can't unlink: $!\n";
}

exit(&main ? 0 : -1);
#------------------------------------------------------------
1;
#
# This bit below saves the message body to file, uncomment if wanted
#
#unlink</var/spool/mail/MIME/msg-*.txt> or warn "can't unlink: $!\n";

Questions:

Why does $TMPDIR = "/home/e-smith/files/ibays/public1/files"; not work?

When are the .qmail files regenerated from templates?

If I make a .qmail-scan file in the user's home directory will it be effected by a regeneration from templates?

Offline CharlieBrady

  • *
  • 6,918
  • +3/-0
Re: Save email attachments to iBay
« Reply #8 on: November 14, 2009, 04:42:50 PM »
What is puzzling me now is why I can't set the $TMPDIR = to an i-bay where the user has RW privileges and have it work. It works fine in the user's home directory with the $TMPDIR = change.

My guess is that qmail-local does not set supplementary groups when changing userid for delivery. Only a user's primary group would be set, and that group does not have write privilege to the i-bay.

Quote
I guess I could learn a bit of perl and do file copies and deletes at the end of unmime.sh

If my guess is correct then that won't help.

Quote
When are the .qmail files regenerated from templates?

When a user is created or modified.

Quote
If I make a .qmail-scan file in the user's home directory will it be effected by a regeneration from templates?

No.

Offline wires12

  • ***
  • 56
  • +0/-0
Re: Save email attachments to iBay
« Reply #9 on: November 14, 2009, 07:20:52 PM »
Charlie, I suspect you are correct about the write privilege to the i-bay. If the FTP thing does not work out I will pursue each user getting their own alias. Thanks!