Jump to content
thirty bees forum

Mark

Members
  • Posts

    277
  • Joined

  • Last visited

  • Days Won

    5

Posts posted by Mark

  1. 16 hours ago, datakick said:

    Sorry about the delay, I totally forgot about this one 🙂

    This new tracking feature in 1.3.0 is basically a mechanism to send anonymous usage data to thirty bees api server. We made very sure that no personal or even domain information are being shared with using this mechanism, so it's aligned with GDPR requirements.

    At the moment we track only the very basic info about the installation: php version, installed php extensions, database version, and basic server settings (max exec time, memory limit, etc). In the future, this mechanism will be extended to track other informations like feature usage. 

    Why do we need this information? There are couple of reasons, of course.

    First of all, we are investing a lot of effort and money into this thirty bees enterprise. And we need to have some way to actually measure the adoption of the software. When we were considering purchasing thirty bees we discovered that nobody actually knew how many stores are using this software in the wild. That's not good really. Without this information, it's quite hard to prepare business plan, monetization strategy, or whatnot. Merchants are using google analytics, matomo, or other tracking software to gather this information about their visitors. We are using this new feature to do the same.

    This first reason behind this feature is not something that merchants benefits from, thats for us. But there are other reasons that have (or will have) direct impact on merchants. With the collected statistics we can much better plan the development.

    For example, we are considering dropping support for some php versions - one of the reasons being that it makes it very hard to update dependencies / third party libraries might not support older version of php. But we don't want to just drop the support out of the blue, and leave merchants in the bind. Before we do such drastic thing, we want to be sure that the impact will be minimal. 

    Another example: feature usage, that we currently do not track, but we want to - if we knew how many merchants are actually using some feature (ASM, multistore, theme, module), we can much better plan our efforts. If we knew that only 5% of merchants are using ASM, then probably it's not a good idea to spend a lot of effort in fixing the bugs, but maybe it's time to consider dropping this feature from the core (moving to module) instead.

    Php extensions are another interesting info to have, as it can answer the questions about potential features that might depend on some extension. 

    Regarding consents. As I already wrote, the collected data are anonymised, there's no law or regulation that requires for us to obtain a consent. Regardless, we want to have the merchants the ability to decide if they want to help us collect this info or not. Unfortunately there wasn't enough time to implement all aspects of this consent mechanism. Currently, there is only a database table that governs consents. If you do not want to allow sending the info to api server, you can update table `tb_tracking_consent` and nothing will be sent to us. Of course, changing the consent in DB is not very user friendly 🙂 In the next major release there will be UI for this. There were other, more imporant tasks, to implement for 1.3.0. These consents will gain more importance once there are more 'personal' type of data to be collected.

    Work queue is a mechanism to run tasks outside of the main execution thread 🙂 

    Yeah, I know. That didn't help much. 

    During execution of php script there might be some tasks that do not need to be executed immediately. For example, when you change order status, an email might be send. Sending email can actually take a lot of time, and that impacts the page response time. Employees have to wait couple of seconds just to see the result of this simple operation. Also, what happens if the smtp server is currently down? Yeah, the process will fail, back office page will display error message, and employee do not know what happened.

    With work queue we do not send emails. Instead, we just describe the 'Send email' operation as a task, and submit it to work queue executor, and immediately continue. Work queue executor will ensure that the task is executed as soon as possible. The task description can be serialized and stored somewhere. This allows us to actually run work queue executors on different servers, or in different process on the same server. It also allows us to try to recover from errors -- for example, if sending email fails, we can retry it later after some delay. 

    There are a lot of operations that can be converted to work queue tasks. Clearing cache, generating images, webservice calls, cleanup tasks, and more. 

    By converting these operations to work queue tasks, we can also trigger them differently. We can have a cli tool to actually run these tasks -- so you can ssh to your server and run command to clear cache, or regenerate the images. These tasks can also be scheduled using build-in scheduler. 

    In 1.3.0, we only introduced the basic mechanisms for work queue, mostly because of scheduler that we needed. There is still a lot of work on this functionality. For example, we need to actually implement different types of work queue executors. Currently, we only have 'Immediate executor', that runs submitted tasks immediately in the context of the current thread. But in the next release, we will introduce new type of executors. There will be

    • immediate executor -- currently available, executed tasks immediately in the context of the thread. Does not support retrying. There is not really a benefit, apart from error handling.
    • deferred executor -- the tasks will be executed in the context of the php thread, but only **after** the response is send to the user (not all servers supports this feature, though). This will increase page response time.
    • standalone executor -- standalone php application that can run on the same server or on other server as well. Will receive the work queue tasks via some queue (redis, kafka, memcache...). This is solution for serious merchants that want to run high performant stores

    I'm happy to answer any questions.

    Great answer thanks @datakick and all important groundwork for the future of TB. Sounds like, for non anonymised and specific data, there will be a consent process put in place, which is where I was coming from. I did mention and suggest that TB did this myself probably a year or so ago, so its good to see it begin to happen...  !  It would be great for us to see exactly what data was actually sent to TB at some point.

     

  2. I got this ajax error when updating database ....

     

    There were hundreds of files that needed updating.... A bit daunting to do that... Maybe a "don't worry" about it comment?

  3. Something that tracks a referrer code on any page like this soon-to-be-doomed core feature claims to and where someone can login and see what the clicks and resulting sales and user actions were (of those clicks) without logging into admin. 

    Tracking clicks to sales might not be straightforward and neither might a login page.

    This core feature does have that stuff but it's pretty complex like Datakick says... And pretty sketchy at best.

  4. 22 minutes ago, wakabayashi said:

    @Mark The answer is almost the same in my case. It's: genzo_affiliate. Which means it's my own development.

    While most of my mobiles are out of question, to be maked public. I could consider to release this one day. It's very small without a lot of features. But on the other it's very clean IMO. BTW it's really only affiliate and not referal.

    Btw: How do you know I am using affiliate? Did you check my sites source code?

     

    No lol I don't even know what your site is. You mentioned about affiliates being one of being the only modules you really use outside core a few days ago.  So I thought I'd see what module you used. Looks like Datakick is going to seriously consider dropping this from Core (understandably) and Im after an Affiliate module

  5. 1 hour ago, datakick said:

    It works properly, or at least it works as it was designed. 

    There are couple of gotchas.

    • make sure you 'Save direct traffic' enabled, otherwise your testing will not work. When you test by entering url into browser, it's considered direct traffic -- there's no referral http header.
    • When you define request uri include key, you need to surround it with '%', as it is basically a sql subexpression. For example, if you want to attribute links with &referrer=123, you should use '%referrer=123%'. You can't simply write 'referrer=123', because it would not match -- the resulting sql looks something like this 'AND url LIKE "referrer=123"'
    • Every time you change the referrer, you should rebuild the index. 
    • And you need to refresh the cache regularly, or the list will not display up to date informations.

    I must say, this is one of the most f*cked up feature in the core. It's way too complicated, depends on caches and other transient data, it's hard to set up and debug,... and nobody is using it. This it the prime candidate for removal, or at least to transfer to 

    I definitely enjoyed that response cheers @datakick 🙂. The bit I was initially referring to is a login portal  where Affiliates go to see their clicks and referrals and so on, it doesnt work at all. As far as recording the data, Ive tried what you suggest but I couldn't get it going particularly easily.

    I agree this should not be Core, it should be a module that people can pay for and it should be much better, more bullet proof and reliable than this. I see you use Affiliate tracking @wakabayashi how do you do it?, Which module?

  6. 1386658006_Screenshotfrom2021-09-2914-54-30.thumb.png.a11e11f31920a1b782057b886d9a4c2a.png

    On 7/1/2021 at 2:53 PM, Mark said:

    @datakick when will modules/trackingfront/stats.php be fixed and this affiliate tracking finally fixed?

    Been waiting about 1.25 years for this to be fixed @datakick. Has it been fixed in the 1.3 upgrade at all? Looks like the Stats module is the problem?

    Referral agents also appear to not be working.... if I create jjj as a Request_URI key ... then access url?jjj then nothing gets recorded

  7. 23 hours ago, wakabayashi said:

    Btw: I worked like 10 hours at the weekend to push my theme project a bit forward. I am very happy with the progress. 😎

    IMO I already found a very stupid behaviour that each theme copies from the other themes (root is probably 1.6 PS theme), but was probably never intended to do so. If you have ever looked into the tpl files, you have maybe noticed that some tags are opened in header.tpl and closed in footer.tpl. Very unnecessary... Just use layout.tpl for it. Such stuff is the reason, why I prefer to do it from an empty folder.

    Hopefully you'll see a way to take this that might not be as bad as it first looks (or at least you'll be able to define exactly whats involved) and maybe others will help you in that if you provide the framework by which to do it.

  8. On 9/24/2021 at 8:51 PM, haylau said:

    Or

    Rather than re-inventing the wheel, could the 'owners' of thirtybees have conversations with some of those theme developers (only the best / most configurable themes) and offer a one off payment to adopt that theme as the new default theme and simply port it across and adopt it as the standard theme when TB is installed making it effectively open source. Perhaps the theme (Panda? @Jonny?) could actually supply a slightly cut down version with big advertising within their theme for the extra modules so they gain even more?

    I am sure some of us would financially support that one off payment - I am sure it would not be too much as they won't really be getting many sales from ThirtyBees customers anyway.

    Many of these themes are highly configurable and work with templates which can be imported and exported, then:

    @jollyfrog could dessign some standard templates with help from users such as @Mark and @led24ee and @cienislaw who have perhaps some specific design features in mind

    And them some development of the theme would be down to devs like @datakick, as part of the thirtybees environment, but other great devs like @wakabayashi, @yaniv14 are more likely to submit improvements and bug fixes on Github?

    Win/Win??

    I asked this earlier in this topic of @Smile and got no response unfortunately @haylau

    I have spoken to the Panda makers once who said they'd like to help but

    a) its a big piece of work

    b) TB base is too small

    Doing this kind of work is, in all fairness, beyond the realm of volunteer work and its not surprising the guys here reluctant to help out.

    What I could see working possibly is this:

    The theme could be built as a collaboration between Panda (for example) and the guys here. We do very much need this. What we have is all we have, and it isnt enough now, let alone the future.

    Panda have done this before, they will probably know the best path under collaboration from people here.

    Smile needs to talk to Panda, he's the boss, its his baby.

     

    BOTTOM LINE:

    We members are going to have to pay for it, and we should have expectations around what we get. The money we pay should go to thirtybees, and Smile the owner of thirtybees, should pay Panda to get the work done with help from the guys here to keep costs down.

     

    Without this, thirtybees gets closer to to its own untimely demise.

  9. 3 hours ago, datakick said:

    I don't think there is any such mechanism in the core. You might have some module installed (blackhole for bots, for example). Maybe some security apache module. Or the crawler could be flagged by couldflare or similar proxy you might be using

    Following a rather disconcerting hack even through Cloudflare I do use Cloudflare to block just about every country I dont intend trading in to some degree now (except search engines). I also use your blackhole for bots, however they arent crawling, just accessing image url's I give them, so its probably not that. Right?

    I did whitelist their ip, but its probably Cloudflare, I'll raise it in that community

    Thanks.

  10. Leaving out most 3rd party modules is a totally understandable thing from a build point of view. I agree minimal is good, however it's obviously a pretty terminal thing for those modules people have or want if they aren't compatible so there needs be a clear process for assessing and handling what modules will no longer work properly/at all with the theme and what options the site owner has ... and what module developers need to do to get the module compatible.

  11. Certainly sounds like a pretty dire need to get a quality theme going.... But from scratch @wakabayashi?? That sounds like an incredibly long hard road? Wouldnt it be easier to take an existing great theme like say Panda (that doesnt work properly with TB unfortunately), and speak to Sunnytoo (the maker of Panda). I spoke to him once before and it was just too much work for small sales to get it compatible with TB.

    Maybe if the talents here that have already volunteered to help could make Sunnytoo's life easier getting it compatible then it we could have something high class for TB?

    His code is obviously not open source (a far as I know).

    I am sure many here including me would pay for a decent theme (or at least components of a decent theme), it is  a pretty large achilles heel with TB.

     

  12. Does anyone have a good understanding of what themes are currently compatible and will be compatible with the next upgrade of TB?

    I use Niara, Im of the belief that its about the only thing thats compatible with TB and modernish - given that sunnytoo has moved on from TB.

    I dont particularly like Niara, its pretty dated especially the checkout page. But I dont believe there's anything better, can someone prove me wrong?

  13. 56 minutes ago, datakick said:

    This is not usually what happens. Spammers use email addresses of real people, not their own. They are abusing your server to act as a sender of spam mails. Of course, they occasionally use their own email address to check that the emails are sent, but in most cases they use real email addresses.

     The most efficient way to mitigate this is to block outgoing contact_form emails. Spammers will have no reason to abuse your website, if it does not send emails to end users

    You are bound to be right @datakick, but in my sendinblue account, a number of spam accounts have hard bounced... I dont know why...I just know Ive been ratelimited as a result... Is there a UI way to block contact_form emails? or will I need to make code adjustments...?

    By the way thanks @AndyC I did have that running before my old site was hacked, I hadnt resurrected it on the rebuilt site so thanks for the reminder.

  14. What are people using to control spam?

    There's two problems

    1/ The incoming spam itself

    2/ The mail that is sent to spammers confirming their message has been processed.

    In the second case, sending the spammers an email confirmation, usually to fake emails, causes hard bounces and therefore degrades credibility with 3rd party mail providers (I use sendinblue). This loss of credibility means real users mail arrives more slowly.

  15. I have version 1.2 ... I went to Core Updater and noticed many files ready to update presumably for 1.3. By good fortune I read this first. However I went to Modules and checked for updates, particularly for Core Updater module but it said Core Updater was up to date, which seemed odd given your comments above @datakick
     

    I'm not sure now whether to just go ahead with Core Updater, given that the module hasn't been updated... I don't want to stuff the db trying to update

×
×
  • Create New...