Queue sending order emails to speed order confirmation?

[Chandra]

Why not separate out email sending process to some kind of queue mechanism so it is not on the critical path for order confirmation?

 

Having this feature allows using SMTP authenticated email that is delivered more reliably, and does not delay order confirmation by many seconds either.

[datakick]

This is definitely a good idea. The implementation is tricky, though. The best way would be to have dedicated task queue running in separate process. But that would require your own server, and it would not be trivial to configure. Alternative is cron-based task queue. There are always some problems with crons, timeouts, etc.

But it's definitely a good thing to think about. Especially if we combine this with task scheduler to run maintenance tasks

[Chandra]

Thanks @datakick

Mautic and Phplist have a queuing mechanism that I thought should work well for the tb scenario.

I see mautic creates emails and dumps them in a folder within the mautic installation. A cron runs periodically to pick and send out a configurable number of those emails through any standard mail server. We have cron functionality in tb for a variety of tasks anyway.

Thinking about it, even Amazon does not send order emails immediately these days.

Edited February 4, 2019 by Chandra

[Chandra]

Oh oh, feature request submissions are closed! 😞

[30knees]

On 2/4/2019 at 7:50 AM, Chandra said:

Thanks @datakick

Mautic and Phplist have a queuing mechanism that I thought should work well for the tb scenario.

I see mautic creates emails and dumps them in a folder within the mautic installation. A cron runs periodically to pick and send out a configurable number of those emails through any standard mail server. We have cron functionality in tb for a variety of tasks anyway.

Thinking about it, even Amazon does not send order emails immediately these days.

@ChandraDo you use phplist and if you do, how do you sync tb and phplist? I couldn't find a module for that. Thank you 🙂

[Chandra]

@30knees

No, I don't use phplist with tb. I doubt phplist will be a good tool for transactional emails. However I believe you could export and import users and newsletter preferences. I remember there was also a php script you could call to add members to phplist, that I had used elsewhere many years ago.

 

For my original point of queuing emails for orders from tb however, I have now added delay to the configuration in postfix, since I run my own mail server. I believe this should allow creation of the email quickly but deliver it leisurely, not holding up the order completion. However, I have hit a snag in that presta/tb don't seem to work well with SMTP email, so I am sticking to phpmail until I can sort this out.

[Kashir2000]

Just to note, with good mail servers/services mail send takes few miliseconds.

But implementing it has another tricky part would be requiring of cron or services as @datakick said. Which would increase requirements of knowledge from one setting up a shop - some don't even know what CRON is and setup their own shops. Do you really want to increase complexity of TB setup? So definitely that would have to be an option I think.

[datakick]

19 hours ago, Kashir2000 said:

Just to note, with good mail servers/services mail send takes few miliseconds.

But implementing it has another tricky part would be requiring of cron or services as @datakick said. Which would increase requirements of knowledge from one setting up a shop - some don't even know what CRON is and setup their own shops. Do you really want to increase complexity of TB setup? So definitely that would have to be an option I think.

This could be implemented transparently. For example, there could be some ImmediateTaskExecutor that would execute tasks arriving into the queue synchronously, in the same thread / process. This way everything would continue to work *in the same manner* as it does now. No cron / external services would be required, and emails would still block / slow down the process execution. This would still work for most of the merchants.

But there would be an option to set up external task executor and push the these expensive tasks to background, for those merchants that need that.

[Kashir2000]

@datakick Acually I think ther could be a Hook actionSendMail which could be able to "break" mail sending and do its own magic. That way TB could implement mail sending in many ways. One could use PHPList integration, other one mailchimp or any kind of other service. And those who want could stay as it is by default!

[Chandra]

On 12/23/2019 at 9:42 AM, Chandra said:

For my original point of queuing emails for orders from tb however, I have now added delay to the configuration in postfix, since I run my own mail server. I believe this should allow creation of the email quickly but deliver it leisurely, not holding up the order completion. However, I have hit a snag in that presta/tb don't seem to work well with SMTP email, so I am sticking to phpmail until I can sort this out.

I wanted to provide an update that this approach seems to be working well based on my initial tests. I figured the TB SMTP bit and use Postfix queue to create emails quickly. By having a small delay to send, I believe the email is created right away and placed in a queue instead of holding up the order confirmation until negotiating with the receiving server to send the email is complete. 

I have to test a little more but believe this addresses the order confirmation delay, without reinventing the wheel for a new queue mechanism.