#62 Change request on Multiple Processes

[Leona]

Hello Graeme,
Every time I have a new destination server, which needs to authenticate to forward the message it's been relayed, I need to create a new configuration on bat/cfg file to specify the "forward-to" address of the new server.
Doing this also obligates me to specify a new listening "port" for the process, if port 25 is already in use by another Emailrelay process.

My request is, is it possible to separate the listening port, keeping it always the same, from the processes running to forward the messages?

This is because I need to listen on port 25 of the Emailrelay server, while been able to forward messages to different mail servers.

Thanks in advance.

[Graeme Walker]

Hi Leona, yes it is a basic feature of emailrelay that you can do the 'storing' separately from the 'forwarding', sharing the same spool directory. The "--as-server" option can be used to run the storing part and "--as-client" to do one pass of forwarding: "--as-server" is equivalent to "--log --close-stderr", and "--as-client" is equivalent to "--log --no-syslog --no-daemon --dont-serve --forward --forward-to".

In your case it sounds like you need to do the forwarding continuously, rather than in a single pass. To do that you replace "--dont-serve" with "--no-smtp" and also add "--poll".

(For low latency you can use a filter on the storing server to poke the forwarding client through its "--admin" interface.)

The complication on windows is that you will probably need to have two emailrelay services instead of one. There is a "running as a service" section in the "windows.txt" document that might be useful. Start by creating the two configuration files and running the two emailrelay processes just from their startup batch files. Only create the services once you are confidant that the two configurations are doing what you want.

You could also consider using client account selectors as an alternative approach -- a filter can be used to set a forwarding address and account selector in each message's envelope file, with the secrets file containing multiple downstream accounts.

[Leona]

Hi Graeme, I am struggling to make it work.
Previously my single bat/service configuration was:

start emailrelay --server-auth=@app/EmailRelay.auth --port=25 --remote-clients --client-auth=@app/EmailRelay.auth --client-tls --forward --forward-on-disconnect --forward-to-some --forward-to=smtp.office365.com:25 --close-stderr --spool-dir=@app/spool --log --log-address --log-file=@app/EmailRelay-log-%%d.txt --log-time --pid-file=@app/EmailRelay.pid --client-filter=msgid: --verbose --X-remote-clients --X-spool-dir=@app/spool\587 --X-client-auth=@app/EmailRelay.auth --X-server-auth=@app/EmailRelay.auth --X-port=587 --X-forward-to=127.0.0.1:25 --X-forward --X-forward-on-disconnect --X-forward-to-some --X-client-tls --X-filter=@app/emailrelay-selector.js --X-filter=split: --X-client-filter=mx:1.1.1.1:53 --X-client-filter=msgid:

This make possible to run two instances receiving messages on separated ports, one forwarding to Office365 using oauth token, and another as mta, forwarding based on mx records.

Now, I need to add new destination servers, without public mx records. And possibly receive messages for all destinations through port 25.

I tried to create the two configuration files you indicated, as follow:

"start emailrelay --server-auth=@app/EmailRelay.auth --port=25 --remote-clients --close-stderr --log --log-format=time,address,port --log-file=@app/EmailRelay-log-%%d.txt --pid-file=@app/EmailRelay.pid --spool-dir=@app/Spool --filter @app/emailrelay-impersonation.js --filter=split: --filter=@app/emailrelay-selector.js --verbose"

"start emailrelay --log --no-syslog --syslog --log-format=time,address,port --log-file=@app/EmailRelay-log-%%d.txt --no-daemon --no-smtp --client-auth=@app/EmailRelay.auth --forward-to=127.0.0.1:25 --forward --poll=30 --forward-to-some --client-tls --client-filter=mx:1.1.1.1:53 --client-filter=msgid: --verbose"

With this I guess I will need to create new configuration files --as-client, to be able to add Office365 and other destination servers!??
If so, will it run parallelly with another --as-client process ?

May thanks.

[Graeme Walker]

You are already using a two-in-one configuration, so two batch files and two service instances is not a good idea.

FWIW, when using multiple configurations in one process I call these separate "units".

To summarise your current configuration you have the default unit listening on port 25 forwarding to office365 as a smarthost, and the "x" unit listening on port 587 doing its own MX routing and using the default unit as a fallback. You have a "emailrelay-selector.js" filter that (from the name) is presumably setting the ClientAccountSelector, before the "split:" filter sets the ForwardTo address and the "mx:" client filter looks up the ForwardToAddress.

But it looks like the "x" unit will never actually forward to the default unit because the cloudflare DNS server will never return the special 0.0.0.0 address. If there is no MX record then message just fails. Is that what you see?

You say "I need to add new destination servers, without public mx record". So would it solve your problem if a failed MX lookup used the default smarthost unit as a fallback? Or do you need a failed MX lookup to result in emailrelay connecting directly using the A/AAAA address (with or without client authentication)?

In either case you need a new "--client-filter" that runs before the built-in "mx:" filter. (Alternatively I suppose you could use a local DNS server rather than CloudFlare.) The new filter would clear the ForwardTo field to use the default unit, or move the ForwardTo value into the ForwardToAddress field to make a A/AAAA connection.

Let me know if I am misunderstanding your problem.

[Leona]

You nailed it! The problem is well described.

Actually, the "split" is running before the ClientAccountSelector. You say it's better to be after it?
I had to set the "forward-to" because it is required by the "forward" parameter. Otherwise I get an error on starting the service.

As for your question, I guess would be better if a failed MX lookup result in connecting through AAAA address. And I prefer simplify the management, avoiding using local DNS.

Adding a client filter to set the two fields in the envelope solve the issue. Now, my current configuration is this:

"start emailrelay --server-auth=$env:programdata\e-mailrelay\$svcname.auth --port=25 --remote-clients --close-stderr --log --log-format=time,unit,address --log-file=$env:programdata\e-mailrelay\$svcname-log-%%d.txt --pid-file=$env:programdata\e-mailrelay\$svcname.pid --spool-dir=$spool --filter=split: --filter @app/emailrelay-select.js --verbose --X-log --X-no-syslog --X-syslog --X-no-smtp --X-client-auth=$env:programdata\e-mailrelay\$svcname.auth --X-forward --X-forward-to=fallback:25 --X-poll=10 --X-forward-to-some --X-spool-dir=$spool --X-client-tls --X-client-filter=@app/client.ps1 --X-client-filter=mx:1.1.1.1:53 --X-client-filter=msgid:"

Many thanks.

[Graeme Walker]

I'm glad you've made progress.

If you don't want to forward from the "x" unit to the default unit at all then it might be clearer/safer to use a bogus port number (such as zero or nine) for the "--forward-to" option, rather than 25.

The "split:" filter should be the last in the chain because it potentially creates more messages and any filters after "split:" only run against the first message. If you need further processing it can go into a client-side filter, in which case its operation should be made idempotent to facilitate retries.

If you are doing client account selection before the split then the same account will be used for all possible recipient-address domains. I would normally expect client account selection and routing to be dependent on each other, but YMMV.

I note that falling back to A/AAAA address lookup after the MX lookup has come back with nothing is expected behaviour, so it should probably be at least a configuration option for the built-in "mx:" filter. Please open a feature request if that would be useful to you.

[Leona]

On doing some tests with the "split" at the end, or any position, I've noticed a bug when the client process the second message.
Looks like the ClientAccountSelector is not used when connecting to the remote server if the content's "To" field has more than one recipient. Even if the envelope is all correct.
Am I missing something?