New collectlogs module

[datakick]

Hi all,

we have just released first version of collectlogs module. You should already see it in your back office module page.

This module requires 1.4.0, so this is relevant to you only if you already updated your store.

This is a very simple module, but it very quickly become one of my favourite.  

It's main task is to collects php errors, warnings, deprecations, etc, and displays them nicely in back office overview. You can also set up cron job and you will be informed by email every time new error is detected. This is great way to catch new errors early, for example when you install new module.

In the module back office overview you can see list of errors, see how many times they occurred, when was the last time it happened, etc. If you click on error, you will see additional details and context. Note that these details are collected the first time error happened.

All these information make it quite easy for a developer to figure out where the problem is, and reproduce it. For you, it makes it super easy to track issues with your codebase and slowly fix them. 

If you are, for example, planning updating to PHP8, I suggest you run this module in PHP7.4 for a while, and make your developer fix all DEPRECATION warnings. With those fixed, the update to PHP8 should be smooth. 

As a side note -- we at thirty bees offer support services, we can fix these issues with overrides and third party modules for you. Don't hesitate to reach out.

 

[datakick]

If you detect any error that is clearly caused by core, or native module, don't hesitate to file issue on github: https://github.com/thirtybees/thirtybees/issues

Copy as much details as possible (omit sensitive data, of course)

Sometimes it's hard to determine the culprit. The good rule of thumb is to look at the stacktrace, if there is any module mentioned. If there is, it's may be problem with that module. But still it can be issue with the core. If unsure, file github issue. The worst that can happen is that we close it as not relevant 🙂

[wakabayashi]

Please all install this module and report the warnings you got. It will help, to reduce all kind of bugs. For a dev its very helpful to see, if multiple merchants report the same stuff.

While it sucks to see these warnings, we also should see new PHP versions as chance to make the system more clean. 🤓

[wakabayashi]

@datakick What do you think about a "clearAllLogs" button? When you have fixed a lot/done a shop upgrade, you might want to start with a clean log again. 

[datakick]

12 minutes ago, wakabayashi said:

@datakick What do you think about a "clearAllLogs" button? When you have fixed a lot/done a shop upgrade, you might want to start with a clean log again. 

There is already bulk delete implemented (will be in the upcoming version), so that would be a workaround for this.

My idea is to have some sort of 'resolved' status, where you could mark issue as resolved. Resolved issues would be hidden from the list, but could be returned if the issue re-appeared. 

There could be different resolve statuses, like 'Resolved in the upcoming releases', or 'Resolved in the next version of module', 'Resolved' etc

[wakabayashi]

3 minutes ago, datakick said:

My idea is to have some sort of 'resolved' status, where you could mark issue as resolved. Resolved issues would be hidden from the list, but could be returned if the issue re-appeared. 

There could be different resolve statuses, like 'Resolved in the upcoming releases', or 'Resolved in the next version of module', 'Resolved' etc

Sounds even better 😎 But this is only locally right? I mean I can't give a hint to merchant, that he uses an outdated version, right?

[musicmaster]

I have in Prestools a function with which you can reduce the server error log so that each error message only is listed once (the most recent version). Maybe such a thing is an idea for Thirty Bees too?

[datakick]

12 hours ago, musicmaster said:

I have in Prestools a function with which you can reduce the server error log so that each error message only is listed once (the most recent version). Maybe such a thing is an idea for Thirty Bees too?

collectlogs module does the similar thing already -- it groups the same errors together. The difference is that it does it in runtime, and not by parsing server error files. There are few reasons why we used runtime method instead of log file parsing. The most important one is that we can collect additional information like context (server variables, application state,...) and stacktrace. Other reason is practical -- server error logs are usually stored in a directory that is not accessible by web application (ie /usr/local/var/log/php-fpm/error.log). If we tried to open this file from PHP app, we would be blocked by php security settings.

[wakabayashi]

Houston we (or I?) do have a problem:

    vendor/smarty/smarty/libs/sysplugins/smarty_internal_runtime_cacheresourcefile.php:76 is bloating my DB 😳

[datakick]

@wakabayashi this is an indicator that there is some issue within smarty lib regarding cache directory cleanup. That's good to know 🙂

Bad thing is that collectlogs consider these as a separate issues (because of different error message). To fix this, you can add new line into database table <PREFIX>collectlogs_convert_message

This line should help:

INSERT INTO tb_collectlogs_convert_message(`search`, `replace`)
VALUES("/^rmdir.*Directory not empty.*/", "rmdir(): Directory not empty");

Collectlogs will then track all these errors under single issue.

You will probably want to clean up your tb_collectlogs_logs table afterwards 🙂

[wakabayashi]

@datakick Thanks a lot! I did a full reset of the module and INSERTED it afterwards 🙂 This time I was even smart enough to change the PREFIX 🤣

Will this entry be added automatically on upcoming version? I assume that this is a general problem with smarty, as all kind of module folders throw this warnings.

[datakick]

8 minutes ago, wakabayashi said:

@datakick Thanks a lot! I did a full reset of the module and INSERTED it afterwards 🙂 This time I was even smart enough to change the PREFIX 🤣

Will this entry be added automatically on upcoming version? I assume that this is a general problem with smarty, as all kind of module folders throw this warnings.

Yes, it will be part of next version of collectlogs module.