dynambee

CentOS Web Panel (CWP) user here. Relatively new to it but so far I like the features & flexibility. I've used cPanel and Plesk in the past but never really liked either. I tried a selection of the top free panels like VestaCP (and a couple of forks), ISPConfig, and Webmin as well as doing a lot of reading on other panels. In the end I decided that CWP was the best balance between ease of use, overall power, regular updates, and flexibility. So far it has not disappointed and I will be migrating all my dev & test sites off Cloudways and onto the new VULTR server running CWP. Next up will be setting up an Elastic server to test that out. Wish me luck. 🙂

I didn't have problems accessing the webservice, it managed GET requests perfectly, so the webservice itself was fine. The issue was that Apache was configured in such a way as to block PUT requests, or at least the type of PUT request needed for the webservice. This isn't necessarily a bad thing as the vast majority of people don't use APIs and don't need this sort of access. In any case this meant I could retrieve data from the webservice but not make any changes. I couldn't enable/disable carriers/countries/zones/etc or send any delete requests. After making the changes I listed to the vhost file for my domain I can fully work with the webservice, exactly as I could on Cloudways. I'm still waiting for confirmation from the CWP support guys but for my particular situation I don't think I my changes have affected server security in any meaningful way. (It could make a cross-site attack easier in some situations but since all the sites on the server will be mine I am not concerned about this.)

Thanks @SLiCK_303 and @lesley, I will test out the fixes later today my local time. I have found a couple of other issues with the carriers page in the checkout as well, and have opened a git issue about them. In short though, the "delivery time" is not displayed correctly for each carrier. The delivery time for the first carrier is displayed for all carriers. The second issue is that the price column doesn't align properly. Here are the same images that I posted in the git issue: A big thank you to the dev team for all the hard work that has gone into this release. It's looking really good!

There is certainly no denying CWP's steep learning curve compared to cPanel or Plesk. However I think CWP is easier to work with than something like ISPConfig which I couldn't even get to install properly. I tried a bunch of other well-known control panels as well but for one reason or another I found CWP to be the best fit for what I want to do. It's powerful, flexible, and not insanely over-complicated. It probably helps that it doesn't try to work with a dozen different distros and just focuses on CentOS.

It's been quite an adventure moving away from managed hosting. Slowly getting a handle on it, haven't had to do anything like this for a while. Good for my brain. 🙂

Installed onto my new CentOS7 server and the system seems to work well. Webservice working as with earlier versions. My custom modules also seem to work without any issues. Found a couple of minor design issues: Front office create new account, "Mr. & Mrs." options are not aligned On page 4 of checkout can't click on the small circles to select the carrier. Must click on the carrier description. Should be able to click on both, and ideally those little circles should be a bit bigger as many customers will try to click on them (as I did, and thought I couldn't choose the carrier at all.) I'll keep testing things out over the coming days and see what comes up.

After spending a bunch more time and doing a lot more reading I have managed to figure out something that works, but I don't know if it is a good solution or not from a security standpoint. I modified the Apache vhost configuration file as follows: <Directory "/home/[sitedir]/public_html"> # Commented out this line: #Options -Indexes -FollowSymLinks +SymLinksIfOwnerMatch # Added this line: Options FollowSymLinks Indexes # No change to this line: AllowOverride All Options=ExecCGI,Includes,IncludesNOEXEC,Indexes,MultiViews,SymLinksIfOwnerMatch # Added these 3 lines: Order allow,deny Allow from all Require all granted </Directory> I have seen many people post about getting 403 Forbidden errors when trying to use PUT with the Prestashop 1.6 webservice. Some manage solutions through the .htaccess file but others haven't been able to fix the problem and just end up changing hosts. It would seem somewhat likely that the problem is that the host has locked everything down extra-tight thus causing this problem. As I mentioned above though, I am not sure how much the changes I made impact the security of the site. I am not concerned about other users on the server as the server is mine, there won't be any other users. I am however concerned that it might weaken external security. Does anyone have any thoughts or feedback about these changes from a security standpoint??

Just for certainty I rechecked the error logs with this command: grep [my office ip removed] *.error.log|grep ModSecurity There were no results. I have a fixed IP here so there is no chance that errors would appear under a previous dynamic IP or anything like that. However to be extra safe I did this as well: grep ModSecurity *.error.log There were still no errors. However when I run this: grep proxy_fcgi *.error.log I get a lot of general errors that don't seem to be related to the API. I haven't had to check this sort of thing before (I don't think I even had access to these logs on Cloudways) so I don't know if this is normal or not. This error seems to repeat many times, I guess from the recent back office product edit for that photo upload: [domain removed].error.log:[Tue Jun 25 07:13:29.934861 2019] [proxy_fcgi:error] [pid 1945:tid 139881086928640] [client [ip removed]:60690] AH01071: Got error 'PHP message: PHP Warning: Declaration of Cart::getDeliveryOptionList() should be compatible with CartCore::getDeliveryOptionList(?Country $defaultCountry = NULL, $flush = false) in /home/[removed]/public_html/override/classes/Cart.php on line 0\n', referer: https://[domain removed]/admin/index.php?controller=AdminProducts&id_product=1&updateproduct&conf=4&key_tab=Informations&token=[token removed] Edit: Credit where it's due, above grep commands and log locations learned from here.

(My apologies for the image misunderstanding. Since I do everything through the webservice I'm actually not all that familiar with how the products section of the back office works.)

I uploaded the image in the back office product catalog. However I just tried adding an image to the description and uploading it that way and that works too:

mod_sec is definitely turned off. I actually had this problem before mod_security was installed at all. I was starting to get desperate so I installed mod_security it in the hopes that either it would make the problem go away (lol) or would give me some sort of insight into what the problem was. Neither seemed to happen so I disabled it for the domain with tb installed. I have also checked the logs and there are no ModSecurity errors in them at all. Having testing (and had the same results) with the firewall off, with mod_security off, and having reset all the file permissions I figured this might be some sort of apache configuration issue. I'm good a troubleshooting (trial and error, narrow things down carefully) but not skilled with webserver configs, unfortunately.

No problem, willing to try anything at this point! The image uploaded fine and appears correctly on the product in both the back and front offices.

After commenting out those lines the GET request failed with a "![CDATA[Authentication key is empty]" error. The PUT request failed with the same 403 forbidden error as before. Uncommenting those lines returned things to as they were previously (GET=ok, PUT=403).

I'd be more than happy to give you access to the server if you would be willing to have a look around.

Thanks for the quick reply @lesley. CWP takes care of the overall configuration settings so I haven't had to set things up (heh, screw things up...) manually. However it is using PHP-FPM which as far as I understand it (and that probably isn't very far...) is a process manager to manage FastCGI. So yes, I believe the server is using fcgi. I have tested the webservice with "Enable CGI mode for PHP" both on and off. I also just used a Firefox plugin to test the webservice from Firefox to see if it was something about how the request was being formed or if there was a different problem with my automation but it returned the same 403 forbidden error. I sent the following XML via Firefox: <?xml version="1.0" encoding="utf-8"?> <thirtybees> <country> <id xmlns="">1</id> <id_zone xmlns="">0</id_zone> <id_currency xmlns="">0</id_currency> <call_prefix xmlns="">49</call_prefix> <iso_code xmlns="">DE</iso_code> <active xmlns="">0</active> <contains_states xmlns="">0</contains_states> <need_identification_number xmlns="">0</need_identification_number> <need_zip_code xmlns="">1</need_zip_code> <zip_code_format xmlns="">NNNNN</zip_code_format> <display_tax_label xmlns="">0</display_tax_label> <name xmlns=""> <language id="1">Germany</language> </name> </country> </thirtybees> The exact response via Firefox was as follows: Server: nginx/1.16.0 Date: Tue, 25 Jun 2019 06:16:27 GMT Content-Type: text/html; charset=iso-8859-1 Content-Length: 222 Connection: keep-alive Keep-Alive: timeout=60 <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN"> <html><head> <title>403 Forbidden</title> </head><body> <h1>Forbidden</h1> <p>You don't have permission to access /api/countries on this server.<br /> </p> </body></html> I have also done tests with all the server firewalls temporarily disabled to make sure nothing was being unexpectedly blocked. I think (but could easily be wrong!) that I have narrowed it down to a webserver configuration issue. GET works fine but PUT fails.

A few things I forgot to include: * I'm running TB "1.0.8-1.0.x bleeding edge" with all updates as of yesterday * I have confirmed that the TB webservice (API) key that I am using has full permissions and is not blocked from PUT or POST requests * I can manually make changes through the back office web interface and they are saved properly. (Tested with the same changes I attempted by API.) A couple of lines from the access_log file: "GET /api/countries?display=full&date=1&ws_key=[key removed] HTTP/1.0" 200 163545 "PUT /api/countries?ws_key=[key removed] HTTP/1.0" 403 222

After a particularly terrible experience with Cloudways support I'm trying to migrate away from Cloudways and towards a self-managed VULTR server running Centos7 + Centos Web Panel (CWP). (It took Cloudways support 18 hours to resolve an SSH key issue that they somehow created. I couldn't access the server with public/private keys during this time. I have no idea how long it would take them to fix an actually serious problem and I do not wish to find out...) After some initial struggles things are now working quite well and I am learning how to get things done within CWP. I can get sites installed and running quite easily now and am overall very happy with the way CWP works. I do however have one problem, and that is the webservice (aka api) is not operating properly. I can send GET requests and get data from the webservice. These show up in the apache logs with status code 200 and the data is displayed correctly. However when I try to use a PUT request to make changes to server data it fails with a 403 Forbidden error. This same code works just fine on cloudways and has worked with other hosts so I think there is some sort of permissions error going on. For the life of me I cannot figure this out. I have spent a lot of time over the past couple of days banging my head against the wall which has been great for learning but I'm out of ideas. There are a lot of suggestions on Stack Exchange about how to rectify common 403 errors with Apache but none have worked for me. I have been very careful about not making any permanent changes and always undoing any changes I make along the way. A few of the things I have tried: * Making sure perms for all folders are set to 755 and for files to 644. These are the recommended perms by CWP. * Making sure perms for all folders are set to 775 and for files to 664. These more permissive perms are what Cloudways uses. * Adding "Options FollowSymLinks" to the .htaccess file (this changed the error code from 403 to 500 and made the entire website inaccessible) * Adding "Options +FollowSymLinks" to the .htaccess file (this changed the error code from 403 to 500 and made the entire website inaccessible) * Making sure mod_security is OFF for the domain I am working with (can be turned on and off on a per-domain basis with CWP) * A LOT of time trying to interpret the config files for Apache which are really above my ability to understand. Despite doing a LOT of reading about these files in the end I didn't feel comfortable monkeying with them lest I make things much worse. Some info on the server: CentOS 7 with all updates CWP 0.9.8.836 Apache 2.4.39 Nginx 1.16.0 PHP-FPM v7.1.30 MariaDB 10.1.40 VULTR's standard firewall is disabled, the firewall included with CWP is enabled (CSF + LFD) Varnish is also installed but is disabled for this domain. So this domain is running using a similar set of services to what Cloudways uses, Nginx -> Apache -> PHP-FPM. This overall server setup was created by CWP. In other words, I didn't personally install and configure each service, the server is an automated install using CWP that configures things in a standard way. So, does anyone have any ideas on what I need to change to make this work?

Well I was going to go with servercow's hosted mailcow but the migration didn't end up happening. Last weekend (planned time) they had some issues connecting to the server that were my fault. I rectified that so the migration could happen either last Sunday or this weekend but despite sending a few emails asking for an update I have heard nothing at all back from them. Considering that much of the reason for not hosting my own email is to have support from someone else in case things go wrong the lack of response from servercow has been very concerning. I've therefore abandoned the idea of going with servercow and will instead be migrating to OpenSRS. There is no push email but I can live with that if it means having stability & support at a reasonable price. I wanted to support servercow with my business (open source etc) but I can't risk the lack of support. I just signed up with OpenSRS and made the $95 deposit (it gets added as a credit to your account) and am now starting the migration. I should have most of my domains migrated over in the next few days and can then nuke the VULTR instance that contains my current mailcow server. Will be sorry to see it go.

The general way that ES is secured is by only allowing connections from localhost and not opening the firewall port to the outside world. In larger multi-server installations the ES server(s) resides on the local network and is not exposed to the outside world at all, all access comes from the web server. It is in the documentation, and I have been telling you from the beginning that the module is only known to be compatible with 5.4 and 6.0. Why you have ignored every one of my comment replies I do not know. @Factor even posted a link to the documentation in the thread you started about ES. The documentation says, "Therefore, at the moment of writing, you will need a version of Elasticsearch that is: Version 5.4 or higher, including 6.0". At the time that the module (and documentation) was written 6.0 was the latest version of ES.

That is very good to hear. Thank you. I don't have any problem with that. ES 5.4 is absolutely fine for index & search of any website up to 100s of thousands of products, and probably well beyond that. ES is capable of indexing & quickly searching vast amounts of data. I'm making changes to my automation systems right now but once that is done (should be this week) I will be able to set up a decent sized test system to better check the performance and issues of the current module on 1.0.8 bleeding edge. It is difficult to properly test something like ES with only a small handful of products.

No, one of the good things about the ES module was that it supported proper faceted navigation, and in a way that worked extremely quickly. I should have a better test site available for the ES module soon, one with a few thousand products on it. As far as I know the ES module worked fine with early versions of TB, there was much testing done. Later versions of TB have made changes that have caused incompatibilities with the module. Part of the agreement when the module was funded & created was that it would be supported by the TB team along with other TB-created modules. It seems very unreasonable to renege on this the very first time problems are reported.

When mdekker created the ES module a lot of time was spent to make it better than the modules that were already out there for PS. Each of the PS modules that existed at that time did some things well and other things poorly. The goal was to make the TB ES module the best of all worlds, one that did each aspect of ES functionality in the best, fastest way possible. It took a lot of time, effort, and money to get the project to the point that it is at now. It would be a real shame to abandon such an advanced, high performance module at this point. As far as the users who have been having trouble with it, there seem to be two. One is me, who has been testing and troubleshooting various aspects of the module, and the other doesn't seem to be able to get his ES server installs to work. I suspect he is using incompatible versions of the ES server or that something is misconfigured with his ES installation or the server that ES is running on. He doesn't really reply to my messages asking him to use compatible versions of Elasticsearch Server so it is quite difficult to help him. As far as if the module is worth the trouble or not, I would say that it absolutely is. Once it is operating correctly it would just be a case of installing the module and turning it on. For users of something like Cloudways they don't even need to install Elasticsearch itself as it already exists on Cloudways servers. The module is very easy to install and configure and the resulting search performance is staggering.

Have you tried using a version of Elasticsearch that is *actually* supported by the module? That would be 5.4 or 6.0. I have various problems with the module but it does connect and index with ES 5.4 without trouble. I cannot currently test it with 6.0 but it was tested against both versions over the development of the module so I suspect it will work. The thing with ES is that the way ES clients & servers communicate with each other changes a bit with each version. This is normal, expected behavior. As the server versions increase new features are added and the communication protocols change. Using unsupported server versions (ie not 5.4 or 6.0) is much more likely to lead to problems such as not being able to index properly. There is a certain amount of backwards compatibility, of course, but it is always best to start with something that is known to have worked in the past. If you have used 5.4 and 6.0 and still cannot get the client to connect to the server then it is likely a server configuration issue. Firewall problems, port problems, something like that. I say this because I, personally, have tested ES 5.4 with TB 1.0.8 and 1.0.8 "bleeding edge" and it does connect and index.

Regarding turning off the state/province requirement and disabling the postal code validation, it is important to know that doing this will almost certainly result in you getting more invalid addresses from customers. People are not very careful when entering their addresses and the verification settings help avoid a lot of common errors.