[Free Module] Consistency checks

[datakick]

Hi everyone, 

I'm presenting you today a new module I always wanted to create: Consistency checks.

Some of you probably know that I offer paid support services for thirtybees (and prestashop as well). In my line of work I have regularly encountered issues that were caused by inconsistencies between expected and actual environment. Thirtybees core, and modules as well, often expects that some preconditions are met. And if they are not, things can go awry pretty fast. Let me show some examples or issues I've seen:

Extra file in /classes directory

My customer accidentally copied php file /classes/Tools.php to /classes/db/Tools.php. That's not a big deal, right? Some extra file shouldn't have any affect on the system, right? Well, it does. When thirtybees generates class_index.php (index file for class autoloader) it includes all files from the /classes/* directory into it. In this case, the result was that thirtybees core used Tools class declared in file /classes/db/Tools.php instead of /classes/Tools.php.

At the time this accidental copy happened, it wasn't a problem yet. Because both files were the same. But when my customer decided to upgrade to new version, it started to matter. And it bring down the entire server. That's because new version of thirtybees was used with old version of Tools class...

It took me 4 hours (and my customer 200 EUR) to figure out the root cause of this problem. The fix was simple -- just delete the extra file. 

Module were deleted from filesystem without being properly uninstalled

We all are guilty of doing this. Sometimes you install some module, and it crashes your shop. HTTP 500 error code. So you login via your ftp and delete the module. And the shop works ok again...

But, there's a problem. There are remnants of the module in the system. Entry in modules table, registered hooks, possible overrides,... these all can cause weird stuff and bugs.

For example, core updater will not allow you to perform update if it detects that some incompatible module is installed -- but what if you already deleted this module from /modules directory? Sorry, you can't upgrade. Not without going to your database and delete the module information from there...

Also, these deleted modules slow down your system a bit. If there are hooks registered by delete module, thirtybees core have to evaluate this situation over and over again, every time the registered hook is executed. That's waste of resources.

Images aren't displayed, because the image file is not on filesystem

This is classic problem associated with migration. You copied your /img/p directory from old server, and then you import your data into database. It's very easy to make a mistake, and the data in tb_image tables can have different IDs. If that's a case, your images won't show... because  the image ID does not correspond with id used to store file on filesystem...

Consistency checks module

All these issues can't be reproduced on vanilla installation. That's because the inconsistencies does not exists (yet). There are no extra files in classes directory, no modules were force-deleted, and all images are present,... But these inconsistencies will appear eventually, simply by using the system.

So I decided to create a module that runs a bunch of tests against your system in order to find these problems. And, if possible, offers a fix.

By using this module, you can get your system into the state that matches the vanilla installation as much as possible. And that can help you reduce the chance of bad things happening.

At the moment, the module does not really contain a lot of tests, there are only 6 of them. But I plan to implement new along the way -- every time I will work on some problem that is caused by some inconsistency. So you can expect a lot of updates in the future. 

Also, if you have idea for some checks, let me know. 

Bug detection

This module can also help detect bugs in the core or modules. If you fix some inconsistency, and it re-appears, it's likely there is some bug that wants to be fixed. Don't be shy and report this. Help thirtybees be better and more stable. 

Download

I decided to release this module for free in order to help this project to grow. You can download the latest version of the module here: https://store.getdatakick.com/en/modules/consistency

[datakick]

New version 1.1.0

Added new test: Finds records that are not associated with shops

Inspired by this thread.

This new test goes through (most) multistore enabled records in the system and verifies that for every record in primary table there exists at least one record in primary_shop table. If no such record exists, it is most likely a bug. You should either associate the record with some store, or delete it completely.

[musicmaster]

Nice initiative.

With the Integrity Checks page in Prestools I provide some consistency checks too. Feel free to copy.

[datakick]

1 hour ago, musicmaster said:

Nice initiative.

With the Integrity Checks page in Prestools I provide some consistency checks too. Feel free to copy.

Thanks, I'll look at your module sometimes. Maybe I'll borrow some of the code from there

[musicmaster]

One thing I have considered - but I believe should be done with direct access to the backoffice - is price calculation. You should enter a cart number or an order number and see a complete explanation of how the pricing is processed and what system settings play a role. The script should both make its own calculations and retrieve the values that the TB functions have calculated. 

Rounding questions regularly come back on the forums and with such a functionality people could provide a screendump with all the relevant information.

[movieseals]

I absolutely love this module.  It is a great initiative.  It fixed problems I did not even know I had and it helped me fix others I knew I had but had no idea how to fix them.  Great tool!  Thank you so much!

[movieseals]

Hi again!

I ran the module today and everything checked out except for one thing.  It said that there was one image missing.  It gave me the product ID.  I regenerated this image and re-ran the tests.

Still got the error.

So I click on the fix button in the module and ran the tests again - this time everything is A-OK.

Just for my understanding, what does the module do that the regeneration of the product images did not?

[the.rampage.rado]

@datakick today I decided to check this module and it was showing this:
(note: all images are there when I check them and all of them were uploaded after the image rewrite update). Can you replicate this or I messed something up again on my local setup? 🙂

[datakick]

1 hour ago, the.rampage.rado said:

@datakick today I decided to check this module and it was showing this:
(note: all images are there when I check them and all of them were uploaded after the image rewrite update). Can you replicate this or I messed something up again on my local setup? 🙂

 

Thanks for letting me know. This module checks only existence of .jpg source file. With the image rewrite, we can now upload and store source files in different formats as well (webp, avif, png,...)

I will have to update the module to check for those as well