Koozali.org: home of the SME Server

Adding custom headers to all outgoing emails

Offline biogenic

  • 12
  • +0/-0
Adding custom headers to all outgoing emails
« on: July 21, 2021, 08:26:59 AM »
Hi,
would like to know if there's a plugin or a native way
to add a custom header to all outgoing emails.

The custom header will be the same for all outbound emails, like this:

X-ETP=DOMAIN-AUTH-TOKEN <Id Token>

Thanks in advance!


Online Jean-Philippe Pialasse

  • *
  • 2,746
  • +11/-0
  • aka Unnilennium
    • http://smeserver.pialasse.com
Re: Adding custom headers to all outgoing emails
« Reply #1 on: July 21, 2021, 01:04:27 PM »
write a custom plugin for qpsmtpd and enable it.

only limit, by default deamons on server will send directly thru qmail so those mail will not have the header. most of it is to internal account except probably php generated emails.

Offline biogenic

  • 12
  • +0/-0
Re: Adding custom headers to all outgoing emails
« Reply #2 on: July 22, 2021, 05:44:32 AM »
Thanks for the reply.

Writing plugins - I have zero knowledge of this,
but would like to give it a shot.

The best I was able to come up with from searching was this excerpt from
https://wiki.qpsmtpd.org/doku.php?id=plugins

It is stated where we can add, modify headers:

hook_data_post_headers
hook_data_post
hook_queue_pre


The nearest similar plugin I found was:
https://steve.fi/software/qpsmtpd/log_details/

which adds headers to each incoming message.

from the original:
Code: [Select]
#!/usr/bin/perl
#
#  Alter the message before it is delivered to add headers to it.
#
# Steve
# --

sub hook_data_post
{
    my ( $self, $transaction ) = (@_);

    #
    # get the sender + ip + helo name for logging.
    #
    my $remote_ip   = $self->qp->connection->remote_ip;
    my $remote_host = $self->qp->connection->remote_host;
    my $helo        = $self->qp->connection->hello_host;

    #
    # Store hosts.
    #
    $transaction->header->add( 'X-HELO',        $helo );
    $transaction->header->add( 'X-REMOTE-IP',   $remote_ip );
    $transaction->header->add( 'X-REMOTE-HOST', $remote_host );

    #
    # Store sender & recipient
    #
    my $sender = $transaction->sender->address || undef;
    $transaction->header->add( 'X-MAIL-FROM', $sender ) if ( $sender );

    my @to = transaction->recipients || undef;
    $transaction->header->add( 'X-RCPT-TO', join( " ", @to ) ) if ( @to && scalar( @to ) );

    #
    #  All done
    #
    return DECLINED;
}


I guess we can modify this to something like this?

Should we always return DECLINED, though?

Total newbie question - how would we check if the email is outbound?
Is checking for $sender domain a good idea?

Code: [Select]
sub hook_data_post
{
    my ( $self, $transaction ) = (@_);
    $token = 'XXXXXXX';
    // if outbound
    // if $sender is from internal domain??, then:
         $transaction->header->add( 'X-ETP=DOMAIN-AUTH-TOKEN',        $token);
    //
    return DECLINED;
}


Next question:
Where do we put this, how do we test and get it to run in our SME Server?
Any recommended resources to follow along?

Thanks for the guidance!
« Last Edit: July 22, 2021, 05:57:02 AM by biogenic »

Offline biogenic

  • 12
  • +0/-0
Hi! :)
Trying to write and register a plugin, 
hope someone could help, totally new to this..

Here are my questions:

1) Is the code for the new plugin correct for its intended effect?
2) How to register a new qpsmtpd plugin correctly?
3) How to debug?

Here's the background:

I've tried adding a new qpsmtpd plugin into SME Server,
for adding a custom header to all emails.
However, the plugin seems not to work as the custom header is not being added to emails.

What I know:
1) I'm sure the new plugin code is getting read / executed because early on there was a syntax error in my code and the line number and error showed up in the qpsmtpd logs.  After I fixed it, no more errors showed up in the logs.

Here's what I did:

1) I added a new file in /usr/share/qpsmtpd/plugins named "add_custom_header_token"
    and put the ff custom code:
Code: [Select]
#!perl -w
=head1 NAME
add_custom_header_token
=head1 SYNOPSIS
add a custom header_token
=head1 DESCRIPTION
add custom header token to all (outgoing (todo)) emails
=cut

sub hook_data_post_headers {
    my ($self, $transaction) = (@_);
    $transaction->header->add( 'X-ETP-DOMAIN-AUTH-TOKEN', ' TEST123AAAAA');
    return DECLINED;
}

2) Next, this is probably all wrong but here's exactly what I did (sorry, I really don't know what I don't know):

I added an empty file named
   01add_custom_header_token
in
   /etc/e-smith/templates/var/service/qpsmtpd/config
Inside the empty file I then added the line "add_custom_header_token" without quotes.

Then I did the same in the directory
   /etc/e-smith/templates/var/service/qpsmtpd/peers/0

3) Then, the commands:
   
Code: [Select]
    sv t qpsmtpd
    signal-event email-update
     

4) For testing, sent an email to self, checked the headers in the message, the header X-ETP-DOMAIN-AUTH-TOKEN was still missing..  :-(

Thanks in advance for the assistance!



« Last Edit: July 25, 2021, 01:41:39 PM by biogenic »

Online Jean-Philippe Pialasse

  • *
  • 2,746
  • +11/-0
  • aka Unnilennium
    • http://smeserver.pialasse.com
Re: Adding custom headers to all outgoing emails
« Reply #4 on: July 25, 2021, 02:06:33 PM »
for qpsmtpd dev relating question you might have better chance actually asking to the qpsmtpd community. 
Most people here are using it without knowing they do.
Few people are aware that they use it and are able to play a little with plugin config.
Less than a handfull are able to fix small issues in the code of plugins. And less than that have actually coded a plugin. Not that it is a hard task, just you have to read the doc to do it ;).


What you have to know is that it is not because qpsmpd check the plugins that it will actually use them, it depends on the config specific to the network.

Here is where i can help. 

first you shall not play in the /etc/e-smith/templates.
this is for rpms. you should use /etc/e-smith/templates-custom.

then you have to ask: do i want only horde /zimbra users to use the plugin or all users ?
if all users you need to configure both for sqpsmtpd and qpsmtpd.
the. do you want only user of the lan or users whatever the network to use it ?

the intended config are
/var/service/qpsmtpd/config/peers/0
/var/service/qpsmtpd/config/peers/local
/var/service/sqpsmtpd/config/peers/0
/var/service/sqpsmtpd/config/peers/local

their templates are in

/etc/e-smith/templates/var/service/sqpsmtpd/config/peers/local/
and
/etc/e-smith/templates/var/service/sqpsmtpd/config/peers/0/

so you have to do
mkdir -p /etc/e-smith/templates-custom/var/service/sqpsmtpd/config/peers/0/ /etc/e-smith/templates-custom/var/service/sqpsmtpd/config/peers/local /etc/e-smith/templates-custom/var/service/sqpsmtpd/config/plugins

and then either put it in 0 or local dir or in the plugins one with link to it in both local and 0.

then
signal-event email-update should do


Offline biogenic

  • 12
  • +0/-0
Re: Adding custom headers to all outgoing emails
« Reply #5 on: July 25, 2021, 03:23:43 PM »
Thanks for your response! It worked! (edited- my first response said it didn't work, but I checked again, it did)

I'm still unsure about #3 though - what does it mean?

I followed along your instructions.

1) I removed the files I created inside /templates - based on this:

first you shall not play in the /etc/e-smith/templates.
this is for rpms. you should use /etc/e-smith/templates-custom.

-------------------

2) We do want for all users (horde and desktop clients), and for local only (since outgoing emails only?), hence I updated the /local 

then you have to ask: do i want only horde /zimbra users to use the plugin or all users ?
if all users you need to configure both for sqpsmtpd and qpsmtpd.
the. do you want only user of the lan or users whatever the network to use it ?

--------------

3) So I did the mkdir command, then

I edited only the  /local folder, /etc/e-smith/templates-custom/var/service/sqpsmtpd/config/peers/local
patterning after the template, adding the file 10add_custom_header_token
with content: add_custom_header_token

but am confused by the ff. Should I use put the file in both /local and /plugins?
What does "in the plugins one with link to it" mean? What should I put in /plugins?

so you have to do
mkdir -p /etc/e-smith/templates-custom/var/service/sqpsmtpd/config/peers/0/ /etc/e-smith/templates-custom/var/service/sqpsmtpd/config/peers/local /etc/e-smith/templates-custom/var/service/sqpsmtpd/config/plugins

and then either put it in 0 or local dir or in the plugins one with link to it in both local and 0.

---------------
4) Tested - worked!

Thanks a lot! It's really great!
« Last Edit: July 25, 2021, 03:30:13 PM by biogenic »

Online Jean-Philippe Pialasse

  • *
  • 2,746
  • +11/-0
  • aka Unnilennium
    • http://smeserver.pialasse.com
Re: Adding custom headers to all outgoing emails
« Reply #6 on: July 25, 2021, 07:05:51 PM »
check the content of the original templates.  you will see that with a ll command

somme files are links pointing to the plugins directory where the actual fragment lies. this is when the same file is shared between the local and 0 config file.
If you need this one also tagged then you need to add the plugin to 0.   


local means client is on localhost or LAN
0 means the client is road warrior anywhere on the internet and is using its laptop mail client (eg thunderbird) to send mail using sqpsmtpd. 

Online Jean-Philippe Pialasse

  • *
  • 2,746
  • +11/-0
  • aka Unnilennium
    • http://smeserver.pialasse.com
Re: Adding custom headers to all outgoing emails
« Reply #7 on: July 25, 2021, 07:07:51 PM »
try to make few test also to see if it does not tag also incoming emails ;)


Offline biogenic

  • 12
  • +0/-0
Re: Adding custom headers to all outgoing emails
« Reply #8 on: July 26, 2021, 01:30:42 AM »
try to make few test also to see if it does not tag also incoming emails ;)
Oh, it does tag everything, including incoming emails.. (although not a big problem).

I suppose should add some condition to the plugin code.

What could be a good thing to check?
Maybe check the From? or check the connection where it's coming from?
I'll try to see..
« Last Edit: July 26, 2021, 01:46:25 AM by biogenic »

Offline biogenic

  • 12
  • +0/-0
Re: Adding custom headers to all outgoing emails
« Reply #9 on: July 26, 2021, 01:45:48 AM »
check the content of the original templates.  you will see that with a ll command

somme files are links pointing to the plugins directory where the actual fragment lies. this is when the same file is shared between the local and 0 config file.
If you need this one also tagged then you need to add the plugin to 0.   


local means client is on localhost or LAN
0 means the client is road warrior anywhere on the internet and is using its laptop mail client (eg thunderbird) to send mail using sqpsmtpd.

That clears things up -- 
I was not aware of the "link" / ln type of file, now makes sense.  :?

Thanks a lot for the big help!  :grin:

Online Jean-Philippe Pialasse

  • *
  • 2,746
  • +11/-0
  • aka Unnilennium
    • http://smeserver.pialasse.com
Re: Adding custom headers to all outgoing emails
« Reply #10 on: July 26, 2021, 01:50:15 AM »
you will need to check connection and from field.
one easy way could be tag only if a user is authentified. 

this way you should not tag authorized relay and not tag incoming, without the hassle of testing if from is in the local domain list that you will have to generate (probably by a templated config file with the list of domains as the plugin will not access the domain db.)