Long delays after clicking "I confirm my order" (bank wire payment)

[dynambee]

I'm currently generating some test order data to use with the API development I'm doing.

I have noticed that after I click "I confirm my order" on the last page of the bank wire checkout/payment process that it takes anywhere from 5 to 15 seconds to process the order and take me to the bank wire payment details page. Everything else on the site is super fast, both in the front and back office.

These are small orders with 1 to 3 items per order.

As I am doing development work I have all caching turned off. Is this the likely cause? Will it get faster with caching enabled?

It's a Cloudways server located about 500km from me. Server is barely used right now, a few very low traffic sites and a few dev & test sites. Using 30bz v1.0.3.

Everything on the server feels very fast except this one thing.

Ideas?

[Traumflug]

Will it get faster with caching enabled?

Easiest way to find this out would be to turn caching on. Obviously.

[dynambee]

@traumflug said in Long delays after clicking "I confirm my order" (bank wire payment):

Easiest way to find this out would be to turn caching on. Obviously.

I was hoping that perhaps it was a known issue that could be answered immediately by someone else who had already come across it.

Right now it looks like the delay is caused by the order confirmation email being sent. Working to confirm this and to see if trying different SMTP servers resolves the issue or not.

[lesley]

Are you using Google for mail? If so, that will cause it.

[dynambee]

@lesley said in Long delays after clicking "I confirm my order" (bank wire payment):

Are you using Google for mail? If so, that will cause it.

Nope, it's postfix which is running on a separate VPS in the same data center as the 30bz instance is.

[dynambee]

I just set email to, "Never send emails (may be useful for testing purposes)" and now the order processing is instant. Filled the cart up with 19 different items and the bank wire details page loaded fast enough that I couldn't get an accurate stopwatch counter on the time taken.

I'm going to test the performance of the postfix server and see if that is the bottleneck or if the bottleneck is on the Cloudways server/30bz/swiftmail side of things.

[lesley]

I think it is the external server. I have seen this happen before. Also to note, this slowdown is also happening with registrations as well, because of the emails triggered there.

[dynambee]

I don't notice any slowdown with the registration system, it seems to be pretty much immediate. However I just created another test order with emails enabled that took 19.8 seconds to process. :(

I'm going to use Apache JMeter to test the SMTP server and see how well (or badly) it's performing. I'll also try using a Google Apps account with SMTP to see how that compares and may try signing up for SendGrid's free trial and see how that goes too.

[lesley]

Google might be the same way, that is why I mentioned it. We have had several clients complain over it taking a long time. Google is such a massive service that sometimes the mail servers are under load and it takes a long time, others its quick as it should be.

[dynambee]

Just trying to narrow down the cause of this very odd delay.

My postfix server is under zero load as I am not using it on any production websites yet. So it really shouldn't be the source of the bottleneck, and it's part of a mailcow installation so there shouldn't be any weird configuration issues, and it's running on an overspec'd VULTR instance which also shouldn't be causing any problems...

Of course "shouldn't be" and reality can be quite different so I want to try a few things to see which "shouldn't be" is actually causing the issue.

[dynambee]

Have to say I am very impressed with Apache JMeter as a testing tool. It was easy to set up and I was able to get helpful data back quickly. MX Toolbox was also helpful.

It does seem to be an issue with the postfix server being slow to accept incoming connections. I seem to remember reading that this is a security measure of sorts but I can't find information on it right now.

Rather than monkey around trying to fix this issue I think I am going to use Postmark to send out transactional emails such as welcome messages, order receipts, and bank wire payment details. Postmark has a very fair pricing system where you buy credits for emails ($1.50 per 1000 for small users) and the credits do not expire. No monthly charges, just pay for what you use. First 25,000 emails are free.

However, this does leave me wondering why the new user signup isn't delayed like the bank wire purchases are. The only answer I can come up with is that perhaps the welcome mail is being sent asynchronously while the order details mails are synchronous and thus causing the delay?

@mdekker, any thoughts on this? Possible to send all mails asynchronously to avoid any delays?

[dynambee]

Well, I've sure learned a lot about email in the past couple of days.

The issue with my SMTP server is known as "tarpitting". The server intentionally responds slowly to incoming connections to make it less likely that spammers will connect to the server. Apparently these days spammers connect directly to SMTP servers and try "delivering" email directly to users that may or may not exist on those servers.

Rather that mess around with my SMTP server I decided to sign up with Postmark as I mentioned in my previous message. Using a service like Postmark for transactional emails (welcome messages, order confirmations, payment details, shipping confirmations, etc) greatly increases the chance that the messages will actually get delivered. Postmark is very reasonably priced and they only allow transactional emails -- no newsletters or other marketing type emails.

Anyway, changing to Postmark's SMTP servers reduced the post-"I confirm my order" delay from ~10 seconds to ~5.4 seconds. Postmark has SMTP servers in multiple locations and will automatically route connections to the closest location. However as my development server is in Japan there is no server that is particularly close to me. Add in that SMTP was never designed to be a fast protocol and some delay is to be expected.

In the hopes of reducing the delay further I changed to using Postmark's API to send emails. They have an open source PHP library available that made integration relatively straightforward. I'm nowhere near proficient with PHP so it took me longer than it would take anyone with a little PHP experience but still, it wasn't a huge amount of work. Using their API the post-"I confirm my order" delay is now ~3 seconds.

Compared to how fast the rest of the 30bz platform is that 3 second delay still feels long. I think the only way to get it faster would be to send emails using a background process. This would require work to set up that is unfortunately beyond my meager abilities with PHP. :(

I'm happy to share my hacked together PHP code with anyone who would like to use Postmark with 30bz. However you will also need to install the Postmark PHP library using composer. It's just a single command (composer require wildbit/postmark-php) but I do not know what, if any, risk it may have for an already developed site. So far I do not see any ill effects from having installed the library but hearing from @mdekker about what issues this may cause seems like a good idea.