CiviCRM Community Forums (archive)

*

News:

Have a question about CiviCRM?
Get it answered quickly at the new
CiviCRM Stack Exchange Q+A site

This forum was archived on 25 November 2017. Learn more.
How to get involved.
What to do if you think you've found a bug.



  • CiviCRM Community Forums (archive) »
  • Old sections (read-only, deprecated) »
  • Support »
  • Using CiviCRM »
  • Using CiviMail (Moderator: Piotr Szotkowski) »
  • Return path and reply to appear to be switched.
Pages: [1]

Author Topic: Return path and reply to appear to be switched.  (Read 2633 times)

dschafer

  • I post occasionally
  • **
  • Posts: 93
  • Karma: 3
    • Backoffice Thinking
  • CiviCRM version: 3.2.3 - 4.4.4
  • CMS version: Drupal 6.x, 7.x, Wordpress
  • MySQL version: 5.1.x - 5.5.x
  • PHP version: 5.2.x - 5.4.x
Return path and reply to appear to be switched.
April 05, 2010, 01:30:49 pm
Trying to get bound processing working on 3.0.3.

It appears that compared to the demo site for 3.1 my 3.0.3 config doesn't produce the same mail header. But I can't find any place to make an adjustment.

I get the following header from my install. (return path is set to the from email address) VERP override is off.
Return-Path: <admin@wocumc.org>
Received: from westohioumc.virtual.vps-host.net (westohioumc.virtual.vps-host.net [69.89.10.186]) by maila61.webcontrolcenter.com with SMTP;
   Mon, 5 Apr 2010 13:00:03 -0700
Received-SPF: pass (westohioumc.virtual.vps-host.net: domain of admin@wocumc.org designates 127.0.0.1 as permitted sender) receiver=westohioumc.virtual.vps-host.net; client-ip=127.0.0.1; helo=westohioumc.virtual.vps-host.net; envelope-from=admin@wocumc.org; x-software=spfmilter 0.97 http://www.acme.com/software/spfmilter/ with libspf2-1.0.0;
Received: from westohioumc.virtual.vps-host.net (localhost.localdomain [127.0.0.1])
   by westohioumc.virtual.vps-host.net (8.14.3/8.14.3) with ESMTP id o35K02fx017477
   for <dschafer@backofficethinking.com>; Mon, 5 Apr 2010 16:00:02 -0400
Received: (from apache@localhost)
   by westohioumc.virtual.vps-host.net (8.14.3/8.14.3/Submit) id o35K02Ie017476
   for dschafer@backofficethinking.com; Mon, 5 Apr 2010 16:00:02 -0400
Message-Id: <201004052000.o35K02Ie017476@westohioumc.virtual.vps-host.net>
X-Authentication-Warning: westohioumc.virtual.vps-host.net: apache set sender to admin@wocumc.org using -f
MIME-Version: 1.0
Reply-To: r.22.2901.617061e1959ffe40@westohiouumc.virtual.vps-host.net
From: "\"admin\"" <admin@wocumc.org>
Subject: Message bounce test
List-Unsubscribe: "" <mailto:u.22.2901.617061e1959ffe40@westohiouumc.virtual.vps-host.net>
To: "\"Role Tester\"" <dschafer@backofficethinking.com>
Content-Type: multipart/alternative;
   boundary="=_aa2eb4ec755fe40e446af014a30a317e"
Date: Mon, 05 Apr 2010 16:00:02 -0400
X-SmarterMail-Spam: SpamAssassin 0 [raw: 0], SPF_None, DK_None, DKIM_None
X-SmarterMail-TotalSpamWeight: 0

I get the following header from the demo site.
Return-Path: <b.8.8.01b5a1bcc7d5b2ac@fixme.org>
Received: from sushi.osuosl.org (sushi.osuosl.org [140.211.166.55]) by maila61.webcontrolcenter.com with SMTP;
   Mon, 5 Apr 2010 12:30:15 -0700
Received: by sushi.osuosl.org (Postfix, from userid 33)
   id 7B47D85EA; Mon,  5 Apr 2010 12:30:14 -0700 (PDT)
MIME-Version: 1.0
Reply-To: r.8.8.01b5a1bcc7d5b2ac@FIXME.ORG
From: "=?utf-8?Q?\"CiviCRM=20Test=20Mailer=20(=E6=B8=AC=E8=A9=A6again)\"?=" <info@test.com>
Subject: Test Mailing: Return path test
List-Unsubscribe: "" <mailto:u.8.8.01b5a1bcc7d5b2ac@FIXME.ORG>
To: "Charles Darwin" <dschafer@backofficethinking.com>
Precedence: bulk
X-CiviMail-Bounce: b.8.8.01b5a1bcc7d5b2ac@FIXME.ORG
Content-Type: multipart/alternative;
   boundary="=_7806335b843d5b53d9109a31126d25a7"
Date: Mon, 05 Apr 2010 12:30:14 -0700
Message-Id: <20100405193014.7B47D85EA@sushi.osuosl.org>
X-SmarterMail-Spam: SpamAssassin 0 [raw: 0], SPF_None, DK_None, DKIM_None
X-SmarterMail-TotalSpamWeight: 0

Return path is correct.   

Can anyone clear this up for me?

Also in the docs you should indicate that the default mail account is the one that is used for the VERP address.

Thanks

xavier

  • Forum Godess / God
  • I’m (like) Lobo ;)
  • *****
  • Posts: 4453
  • Karma: 161
    • Tech To The People
  • CiviCRM version: yes probably
  • CMS version: drupal
Re: Return path and reply to appear to be switched.
April 05, 2010, 02:24:00 pm
what mail sending process ? smtp or sendmail ?

what is the default mail account ?
-Hackathon and data journalism about the European parliament 24-26 jan. Watch out the result

dschafer

  • I post occasionally
  • **
  • Posts: 93
  • Karma: 3
    • Backoffice Thinking
  • CiviCRM version: 3.2.3 - 4.4.4
  • CMS version: Drupal 6.x, 7.x, Wordpress
  • MySQL version: 5.1.x - 5.5.x
  • PHP version: 5.2.x - 5.4.x
Re: Return path and reply to appear to be switched.
April 06, 2010, 05:49:38 am
We're using sendmail on the local host.

The default mail account is the catch all set at bouncehandler@westohioumc.virtual.vps-host.net


Piotr Szotkowski

  • Moderator
  • I live on this forum
  • *****
  • Posts: 1497
  • Karma: 57
Re: Return path and reply to appear to be switched.
April 06, 2010, 07:40:35 am
Quote from: dschafer on April 05, 2010, 01:30:49 pm
I get the following header from my install. (return path is set to the from email address) VERP override is off.

Code: [Select]
Return-Path: <admin@wocumc.org>
…
Reply-To: r.22.2901.617061e1959ffe40@westohiouumc.virtual.vps-host.net

I get the following header from the demo site.

Code: [Select]
Return-Path: <b.8.8.01b5a1bcc7d5b2ac@fixme.org>
…
Reply-To: r.8.8.01b5a1bcc7d5b2ac@FIXME.ORG

Hm, both seem exactly as I’d expect them to; Reply-To: header has the VERP with the r action and Return-Path: has the bounce (b) VERP unless overridden…

Quote
Also in the docs you should indicate that the default mail account is the one that is used for the VERP address.

The docs are a wiki, please do feel free to fix them as appropriate.
If you found the above helpful, please consider helping us in return – you can even steer CiviCRM’s future and help us extend CiviCRM in ways useful to you.

xavier

  • Forum Godess / God
  • I’m (like) Lobo ;)
  • *****
  • Posts: 4453
  • Karma: 161
    • Tech To The People
  • CiviCRM version: yes probably
  • CMS version: drupal
Re: Return path and reply to appear to be switched.
April 06, 2010, 07:43:50 am
Hi,

I'm confused, I thought the override VERP was only applying to replies, not to bounce ?

X+
-Hackathon and data journalism about the European parliament 24-26 jan. Watch out the result

Piotr Szotkowski

  • Moderator
  • I live on this forum
  • *****
  • Posts: 1497
  • Karma: 57
Re: Return path and reply to appear to be switched.
April 07, 2010, 03:02:07 am
Yes, sorry, I’m an idiot.

VERP override was off, which explains why both Reply-To:s were VERP-ed.

As for the Return-Path:, my gut feeling is that either the install is patched to not set the Return-Path: or (IMHO more likely) the SMTP server overrides any user-set Return-Path: with the SMTP envelope’s FROM.

Does this make sense?
If you found the above helpful, please consider helping us in return – you can even steer CiviCRM’s future and help us extend CiviCRM in ways useful to you.

xavier

  • Forum Godess / God
  • I’m (like) Lobo ;)
  • *****
  • Posts: 4453
  • Karma: 161
    • Tech To The People
  • CiviCRM version: yes probably
  • CMS version: drupal
Re: Return path and reply to appear to be switched.
April 07, 2010, 05:26:04 am
Quote from: Piotr Szotkowski on April 07, 2010, 03:02:07 am
As for the Return-Path:, my gut feeling is that either the install is patched to not set the Return-Path: or (IMHO more likely) the SMTP server overrides any user-set Return-Path: with the SMTP envelope’s FROM.



You are right, and I did patch it indeed
http://issues.civicrm.org/jira/browse/CRM-5946

dschafer, you will need to upgrade to the latest of 3.1 or backport it (nothing very complicated).
-Hackathon and data journalism about the European parliament 24-26 jan. Watch out the result

lcdweb

  • Forum Godess / God
  • I live on this forum
  • *****
  • Posts: 1620
  • Karma: 116
    • www.lcdservices.biz
  • CiviCRM version: many versions...
  • CMS version: Joomla/Drupal
  • MySQL version: 5.1+
  • PHP version: 5.2+
Re: Return path and reply to appear to be switched.
May 26, 2010, 09:23:49 am
I question whether this patch is good, or maybe needs some additional work --

It assumes that all sites will be using the return mail processing. In previous versions, the return-path was defaulted to whatever the from address was. But now -- I believe as a result of this mod -- the return path is the processed value by default unless the override VERP option is selected.

I think we should assume mail processing is not in use, as that's the out of the box behavior -- and many orgs will use civimail that way. Only if the mail processor is implemented should the dynamic return-path kick in and verp override be required if you don't want to use it. Not sure if that means we have some master switch to turn mail processing on via the interface or what. But the current behavior is problematic and easy to miss (someone could setup outgoing civimail, do lots of testing, and never actually notice that the reply-to email is not the from email).
support CiviCRM through 'make it happen' initiatives!
http://civicrm.org/mih

xavier

  • Forum Godess / God
  • I’m (like) Lobo ;)
  • *****
  • Posts: 4453
  • Karma: 161
    • Tech To The People
  • CiviCRM version: yes probably
  • CMS version: drupal
Re: Return path and reply to appear to be switched.
May 26, 2010, 09:49:34 am
Hi,

Not sure it was the case to default to from (or that it was done on purpose ;). You might have chosen the right outbound option that did default it to the from, no matter if you did set the email processor or not.

I'm pretty sure there isn't any test to check if the mail processor is set or not. This being said, it should be complicated to implement and test that if the default email is FIXME or whatever is the default, then use the from.

Wanna get a go and try to add it ? Piotr, what do you think (vaguely remember having discussed it with you)?

X+
-Hackathon and data journalism about the European parliament 24-26 jan. Watch out the result

Donald Lobo

  • Administrator
  • I’m (like) Lobo ;)
  • *****
  • Posts: 15963
  • Karma: 470
    • CiviCRM site
  • CiviCRM version: 4.2+
  • CMS version: Drupal 7, Joomla 2.5+
  • MySQL version: 5.5.x
  • PHP version: 5.4.x
Re: Return path and reply to appear to be switched.
May 26, 2010, 10:02:52 am

My 2 cents:

1. In general we should encourage / coerce folks to set up the return processing channel. Handling it manually does not scale. If they dont have the skills/infrastructure to set it up, they should outsource to someone like SMTP. Hence i'd strongly advocate that the return-path is VERP based by default

2. Not setting this up automatically, allows orgs to ignore/not-process unsub / opt-out requests from users. as a user this is sucky, IMHO

3. The default install of CiviMail should facilitate best practices

lobo


A new CiviCRM Q&A resource needs YOUR help to get started. Visit our StackExchange proposed site, sign up and vote on 5 questions

lcdweb

  • Forum Godess / God
  • I live on this forum
  • *****
  • Posts: 1620
  • Karma: 116
    • www.lcdservices.biz
  • CiviCRM version: many versions...
  • CMS version: Joomla/Drupal
  • MySQL version: 5.1+
  • PHP version: 5.2+
Re: Return path and reply to appear to be switched.
May 26, 2010, 10:24:24 am
I respectfully disagree. You're going to have a lot of small orgs that are only using CiviMail for outbound email, or who's hosting environment won't accommodate the catchall requirements, or who have email server structures that don't easily accommodate the needs (Exchange, GApps).

While the php processing solution definitely makes full mail processing more attainable, I'm pretty sure there's still a healthy audience not able or interested in implementing it.

As it stands, mail processing requires non-interface involvement (cron job). So I don't think it's a stretch to say that you would need to flip a switch internally, in addition to setting up the cron job, to get that functionality working. Or at least implement X's proposal of reverting to the from email if no return account is configured. On that note -- why don't we just not ship with a return mail account dummy record? It has to be modified to work anyway. Ship with no record, ensure that the absence of a record means it uses the FROM address, and you won't need a mail processor "switch" -- if a mail processor record exists, then the switch is on, else off.

No doubt that the absence of mail processing won't scale. But for smaller orgs it's not necessarily an issue. It may even be a preferred process, as they'll be notified via email when a bad address is bounced and can address as it comes up, rather than running 'on hold' reports from the system.

My biggest concern is that it would be very easy to overlook the fact that a different reply-to value is inserted by default, if you're only focused on setting up the outbound piece.
support CiviCRM through 'make it happen' initiatives!
http://civicrm.org/mih

xavier

  • Forum Godess / God
  • I’m (like) Lobo ;)
  • *****
  • Posts: 4453
  • Karma: 161
    • Tech To The People
  • CiviCRM version: yes probably
  • CMS version: drupal
Re: Return path and reply to appear to be switched.
May 26, 2010, 11:01:19 am
Hi,

Read the book, we tried to describe a solution that works with Gapps, without having to use catch all.

Lobo, we can process unsubscribe and co via urls, so it mitigates part of the problem, doesn't it ?
However, the major issue still is that if you don't process the bounces and keep hitting the same invalid emails, you will end up being blacklisted (hey, close to say they deserve it, but still, we shouldn't provide the rope to the guys that choose to hand themselves).

This being said, the actually solution (sending even if they don't set the return channel, using the default FIXME address) is even more crappy than not having a return channel but sending the bounces to the from.

I think either we shouldn't allow to create new mailings if the mailbox/return channel isn't changed from FIXME (ie. we enforce the best practice), or default to the from (ie. we go the small orgs friendly but inferior option).

Don't mind about which option is chosen. Either way is fine IMO.

X+
-Hackathon and data journalism about the European parliament 24-26 jan. Watch out the result

yung

  • Guest
Re: Return path and reply to appear to be switched.
July 18, 2010, 02:11:25 pm
Lots of discussions, no answers.

Pages: [1]
  • CiviCRM Community Forums (archive) »
  • Old sections (read-only, deprecated) »
  • Support »
  • Using CiviCRM »
  • Using CiviMail (Moderator: Piotr Szotkowski) »
  • Return path and reply to appear to be switched.

This forum was archived on 2017-11-26.