Impossible to make the migration - download keeps failing

[j.kaspar]

Hi everyone, I finaly decided to give up on prestashop, and move to TB. However, after several hours of trials, repeated errors and some changes in PHP and web server configuration, I am stuck at the very beginning - in the migration module, I keep getting an error "Download complete but md5 the sum of the core package does not match" over and over again, no matter what I do. I found out, that the main archive is never completely downloaded, thus, the MD5 is really incorrect. The final size of the file is much smaller, than it should be. The second archive, that is being downloaded after the main one (thirtybees-extra...), is downloaded without any problem. If I download the main archive manually, using for example "wget https://github.com/thirtybees/thirtybees/releases/download/1.0.1/thirtybees-v1.0.1.zip" the download succeeds (it is considerably slow, but it has nothing to do with PS or TB). I have no other problems with prestashop or any other application on the server, nor the internet connection.

My current suspition is, that the migration module, combined with some other factors, is to blame. I cannot imagine what could interrupt a download in the middle of the process, while not interrupting the script completely. I appreciate any hints and ideas. Thank you!

My current configuration is following:

Prestashop 1.6.1.18 Debian 9 nginx/1.10 PHP 7.0.27 (php-fpm) memorylimit = 512M maxexecutiontime = 3000 defaultsockettimeout = 3000 maxinputtime = 3000 uploadmaxfilesize = 60M postmax_size = 60M server is in a datacenter, on a reliable internet connection > 500/200 MBit/s

[toplakd]

Try with php 5.5 or 5.6. I think that worked for me when I was testing the upgrade on my test vps a day ago.

[lesley]

Do you have command line access on your site? Are you able to curl the current package and download it fully? I am betting either you have a non standard curl implementation or you are reverse proxying apache and the timeout is really low and catching the file in the middle.

[j.kaspar]

@lesley Yes, I do have a full root access to the server. I tried "curl -L --remote-name https://github.com/thirtybees/thirtybees/releases/download/1.0.1/thirtybees-v1.0.1.zip" and the file was downloaded flawlessly, same as with the wget. Nginx is my primary web server (not a reverse proxy) and I don't have apache installed at all, so this side should also be OK. The problem still persist. Most of the time, the final size of the file is around 5MB, which tells me, that the download keeps failing at about the same time. But what that means, I don't know... I am thinking - is there a way, to download the file manually and let the module to use it, instead of deleting it and trying to download it again? I would love to solve the problem, but this could also be a way out. At least temporarily.

@toplakd as I currently don't have older versions of php installed, I would rather not go this way, but if everything else fails, I am going to give it a try. Thanks

[Traumflug]

Most of the time, the final size of the file is around 5MB

Sounds like a file upload/download size limit kicks in. For example this one: https://nginx.org/en/docs/http/ngxhttpcoremodule.html#clientmaxbodysize or these two: https://stackoverflow.com/questions/2184513/change-the-maximum-upload-file-size

[j.kaspar]

@Traumflug I set clientmaxbodysize to 60M The uploadmaxfilesize and postmax_size already was on 60M.

Unfortunatelly, the behavior is still the same. It looks like some timeout is the problem, rather than the size. So I even set fastcgireadtimeout to 3600. Still the same...

[DaoKakao]

I have those params set to 256m each

[j.kaspar]

It is worth mentioning, that recent 1 click upgrade from PS 1.6.1.13, to 1.6.1.18, worked like a charm

[dosbiner]

@toplakd I'm using this workaround for same issue, don't know why but it works

[j.kaspar]

So is it a bug/incompatibility of the module?

[j.kaspar]

Shall create a bug report, or is using php 5.6 the oficial "solution" ?

[lesley]

Did it work using 5.6?

[j.kaspar]

I would like to avoid unnecessary installation of old versions of php, so I haven't tested it yet. But as I mentioned before, 1 click upgrade of PS worked fine (few days ago) and there was no change on the server since then. I don't use anything non-standard and ultimately - two users here apparently had the same issue, and php 5.6 worked for them.

[lesley]

It is likely a bug in our migration module then. You can open a bug report on Github for it.

[j.kaspar]

Thank you

[lesley]

Check your error logs on the server and let us know of any error message that relates to it as well.

[j.kaspar]

Sorry I entirely forgot to post the log. So, this message appears immediately after the migration process is started:

2018/05/07 07:24:54 [error] 3581#3581: *8323 FastCGI sent in stderr: "PHP message: PHP Notice: Trying to get property of non-object in /var/www/clients/client1/web10/web/modules/psonesixmigrator/classes/ObjectModel.php on line 170 PHP message: PHP Notice: Trying to get property of non-object in /var/www/clients/client1/web10/web/modules/psonesixmigrator/classes/ObjectModel.php on line 170" while reading response header from upstream, client: XX.XX.XX.XX, server: XXXX.eu, request: "POST /XXXX/autoupgrade/ajax-upgradetab.php HTTP/2.0", upstream: "fastcgi://unix:/var/lib/php7.0-fpm/web10.sock:", host: "www.XXXXX.eu", referrer: "https://www.XXXXX.eu/XXXX/index.php?controller=AdminThirtyBeesMigrate&token=3ac0cdd5c5e19b17b026dd0e14c6d69f"

And this message appears, when the module reports the failure:

2018/05/07 07:25:59 [error] 3581#3581: *8323 FastCGI sent in stderr: "PHP message: PHP Warning: sprintf(): Too few arguments in /var/www/clients/client1/web10/web/modules/psonesixmigrator/classes/AjaxProcessor.php on line 307" while reading response header from upstream, client: XX.XX.XX.XX, server: XXXXX.eu, request: "POST /XXXX/autoupgrade/ajax-upgradetab.php HTTP/2.0", upstream: "fastcgi://unix:/var/lib/php7.0-fpm/web10.sock:", host: "www.XXXXX.eu", referrer: "https://www.XXXXX.eu/XXXX/index.php?controller=AdminThirtyBeesMigrate&token=3ac0cdd5c5e19b17b026dd0e14c6d69f"

[Traumflug]

Filed this here: https://github.com/thirtybees/psonesixmigrator/issues/12

[j.kaspar]

@Traumflug great, thank you

[j.kaspar]

Do you have any idea roughly when this could be fixed? I am still on PS 1.6 and pretty much happy, but I would still like to give TB a try... Thanks

[dosbiner]

@j-kaspar this issue has been fixed in new version, make sure use 1.0.2 version, see here: https://github.com/thirtybees/psonesixmigrator/releases Kudos to @Traumflug ?

I see a lot of new member has been migrate from ps 1.6 to thirtybees, is there some statistics how much download, or websites that use thirty bees right now? cc @lesley

[lesley]

We are hovering around 20k live sites now. Its hard to keep a good track, because there are a lot of dev installs, some go offline, ect. We try to track as loosely as possible, so we aren't collecting everyones data like other platforms do.