Let's talk about Search!



  • The standard search functions of most shopping carts are pretty disappointing. I haven’t looked into this in fine detail with TB but my expectation & understanding is that TB 1.0.x search is the same as PS 1.6 search, and as such it’s slow and overall just not very good. No “instant search” with results as users type. No “fuzzy search” that gives results even if the user makes spelling error(s). No ability to filter searches by price / category / brand / supplier / attributes / etc.

    Worst of all is that the results themselves don’t seem to be particularly good and as you add more products to a store it takes a lot longer for searches to return results. If the site is at all busy and has a higher number of SKUs then standard search will grind to a halt completely.

    Understandably all of this is very frustrating for visitors and frustrated visitors are less likely to convert into customers.

    So, what to do?

    There are some incredibly powerful yet rather pricey options out there such as Algolia hosted cloud search. @lesley has a very slick Algolia module available and this is, IMO, the best PS/TB solution for those who have the budget.

    Unfortunately I fall into the group of people with no monthly budget for search services so I’ve been looking into free high performance search options that are compatible with TB. A while back I came across a potential solution that I think might be of interest to other TB users: Elasticsearch + Brad.

    Elasticsearch is a free high performance full-text search engine that is used by some of the largest & busiest sites online. Facebook, Wikipedia, Netflix, GitHub, eBay, and the New York Times all use Elasticsearch to power their indexing & searching.

    Brad is a free module for PS 1.6 that enables searching through Elasticsearch on a PS 1.6 website. Brad has been around for a while and is currently at version 2.1. I installed Brad on a TB development server (a Cloudways VULTR server) that has Elasticsearch 2.4.4 running and Brad seems to work with TB – mostly. There are some things I haven’t figured out yet and I’m not sure if these are caused by differences between PS and TB or if I just don’t have things working properly yet.

    What works out-of-the-box with Brad on TB:

    1. The module installs with no errors as long as you follow the directions on the GitHub download page.

    2. The back office seems to work fine. The Re-index buttons work and all the settings I tried can be adjusted and saved.

    3. In the left hand column of the back office there are various menu choices for Brad when looking at the module configuration. Settings, Advanced Settings, Filters, Filter templates, Info. These all seem to work as well and changes made are saved correctly.

    4. Searching for items. Once Brad is turned on searching is an entirely different experience. Results start to appear as the visitor types their search terms (this can be turned off in the BO if desired.) Type “sumer” instead of “summer” and you still get good results. Other spelling errors also usually still produce quite good results. Search results are fast, but it’s difficult to gauge speeds on a dev site with few items.

    What seems a little wonky:

    1. My dev site is using the standard TB theme. The standard TB theme search box at the top, to the right of the logo, doesn’t change to a Bradsearch box. Instead a new search box appears just below the logo. This is likely due to a difference in hooks or ID, something like that. I’ll look into it but I’m not a web developer so my abilities are limited.

    2. Brad supports search filters out of the box but I can’t get them to appear in TB. I’m guessing this is similar to the above issue and is caused by template differences but it could equally be something I’m doing wrong.

    While this system isn’t completely plug-and-play at this point it improves the standard search even without the filters working properly. I will continue to monkey around with Brad in the hopes that I can figure out what is going on with the extra search box and the missing filters. I would be most appreciative if any other interested parties could do likewise. Perhaps working together we can figure things out and have a fantastic free search solution for TB!


  • administrators

    Very interesting!

    We are currently working on the Algolia module to make it suitable for thirty bees and make it a native module. It will likely not be plug and play for a while because of the problems you’ve mentioned.

    Both Brad and Algolia search modules have a hard time integrating in themes, because theme integration was never a priority for PrestaShop (just look at mess we are seeing with 1.7) and the lack of standardization.

    We do expect that for a long time in thirty bees you will have to customize the search templates yourself, but for that we have introduced the ACE editor in thirty bees. So whenever a module needs to you to customize a template, you can directly edit it from the Back Office and keep tweaking the HTML until it looks great. I think that, for now, that would be the best solution to fully optimize your search with ease.

    For Brad (ElasticSearch) I am not sure if we can add it. We will likely have to contact Invertus to see if we can fork the module as they haven’t added a standard open source license, it seems. Maybe we can work together to get thirty bees support into their module, who knows?

    The (renewed) Algolia module is currently being tested in production and will likely need a few more optimizations to make it “modular” enough to be presented as a native module in thirty bees. (It will eventually show up on your list as native!)


  • administrators

    BTW, if you are talking costs. I have noticed that for our store with 3000 products we needed to have at least 8GB of RAM available for ElasticSearch. The best option for us was to rent another VPS (~€50) to make it work, so yeah, that made it just as expensive as Algolia.

    Be sure to research the amount of RAM needed, because you might end up at the same price you would pay for Algolia. A huge advantage of having your own search engine ofc is that you are in full control, like the rest of your software, but for some merchants Algolia might just be both easier and cheaper than running an ElasticSearch server.



  • Thanks for the replies and the information! It’s good to know that I (probably) wasn’t doing something stupid that resulted in the missing filters.

    Many of the Brad files include a “you can do what you want with this” type of pseudo-license message. I’m not sure why they didn’t just go with a known license that would accomplish the same thing while making things clearer for everyone. Perhaps they’ll do this if asked about it… Invertus seems to be heavily wedded to PS and I’m sure they are feeling the pain of 1.7 as well. They might end up being quite interested in TB as a better solution for their own clients.


    The below text is causing a lot of confusion so I have edited it with strikethrough. Elasticsearch for a normal web store does not need anywhere near 8GB. If you have a busy site then you may find a separate server helps but for many sites it will be very possible to run ES on the same server as your web & database are already running. I fully intend to test this as I get sites operational and will add information here as I figure more things out.

    That’s interesting about the Elasticsearch memory requirements, I wasn’t aware that it was that intensive. Looking at the Elasticsearch site it seems they recommend Elasticsearch servers have at least 8GB but not more than 64GB, and they do mention it’s very memory intensive.

    My first goal is to get a site up and running, and then I will work on Brad and see what it can do with an 8GB or 16GB server and a few websites. Algolia is definitely interesting, if it is a big enough time saver without costing too much then it would be the better choice.


  • Global Moderator

    FWIW, PS 1.7 does show results as one types, so one option would be to grab code from there.

    0_1493078298255_PS 1.7 Search.png

    (Theme is Classic with heavily modified CSS)



  • Onsite search is much more complex than most users realise and, to be honest, thirtybees default search is good enough for like 99.9% of merchants.

    Though going beyond that would be welcome, I don’t know if it’s the right time to do so (I think there are some more important subjets for thirtybees developers).

    In the meantime, we can discuss search and how would you like to get integrated into an ecommerce platform :).



  • @Traumflug said in Let's talk about Search!:

    FWIW, PS 1.7 does show results as one types, so one option would be to grab code from there.

    Thirtybees does too, am I missing something ?

    0_1493126655331_Capture d’écran 2017-04-25 à 09.22.46.png


  • Global Moderator

    All the better, @ssimard ! @dynambee wrote about this lack of functionality in the opening post, so I didn’t even try. Now I tried and see this working in TB, too.



  • Hmm, interesting about the “instant search” feature with standard search function.

    I type pretty quickly (~120wpm when I was young, ~80wpm today) and I actually finished typing search terms and hit enter to search before anything came up.

    When I do the same thing in the bradsearch box the results start to appear nearly instantly and then continue to be refined very quickly as I continue to add letters and words. I have the minimum search length set to 3 characters on my dev box and by the time I strike the fourth key the results are there.

    So the feature is there with the standard search, it’s just not all that…instant.



  • I have yet to test with all my products added but should be able to do it soon. (1000 books with their pdf and epub variants so close to 3000 products)

    Hopefully speed can stay decent but I’m a bit worried at the moment…



  • I have just “enabled” my dev domain so people here can try bradsearch.

    Notes:

    1. The server is in Tokyo because I’m in Osaka and it’s really just for my own dev & testing. I have Cloudflare set up but I have no idea what the performance will be like in other parts of the world. I’d be interested in hearing how it feels speedwise, both for page loading times and for the search. (I am working with webservice integration right now so the site itself is bog standard.)

    2. The search box at the top of the display with the blue search button is the standard search. This has not been changed by bradsearch at all.

    3. The search box just below that with the black button is the bradsearch box.

    4. There are only the standard 7 products on the site right now so this isn’t a proper performance test, just a proof of concept. Good test terms will be things like “dress”, “summer”, etc. Stuff that will turn up the standard sample products. Try with spelling mistakes too, in the bradsearch box.

    5. This is a dev box. Dual cores, 2GB memory. Not exactly a fast machine and far from an ideal Elasticsearch system.

    6. Please play nicely. :)

    The domain is https://inxonline.com/

    Edit: I’ll be able to keep the site enabled for a day or so, after that I will have to put it back into maintenance mode for further internal dev work & testing.


  • Global Moderator

    Works nicely and with decent speed from Germany. These pictures in the immediate search result are certainly a good idea!



  • @ssimard said in Let's talk about Search!:

    I have yet to test with all my products added but should be able to do it soon. (1000 books with their pdf and epub variants so close to 3000 products)

    I should also have a site with ~1500 products soonish. A few more things to prepare before I can remove the existing products from the site and start sending my actual products through the webservice. I’ll be sure to try out bradsearch again at that time and see how it performs with more items, as well as if it needs a more powerful server to work.

    Hopefully speed can stay decent but I’m a bit worried at the moment…

    I think you should be okay with ~3000 SKUs. The website for bradsearch has some performance stats in a basic chart and their tests show 480ms for standard search results with 10,000 SKUs, vs 170ms for bradsearch. No idea what sort of hardware they were testing on, unfortunately. Things really start to go haywire at 100,000 SKUs and will go downhill dramatically from there. It will of course also depend on how busy your site is, busier sites will benefit more from a dedicated search system.



  • @Traumflug said in Let's talk about Search!:

    Works nicely and with decent speed from Germany. These pictures in the immediate search result are certainly a good idea!

    Thanks for the feedback!

    I’ll be able to keep the dev box open for bradsearch testing for another day or so and would certainly appreciate more feedback & thoughts if anyone has a moment to try it. Sometime tomorrow I hope to be back to testing webservice stuff and will have to put the website back into maintenance mode.



  • @dynambee said in Let's talk about Search!:

    I think you should be okay with ~3000 SKUs. The website for bradsearch has some performance stats in a basic chart and their tests show 480ms for standard search results with 10,000 SKUs, vs 170ms for bradsearch. No idea what sort of hardware they were testing on, unfortunately. Things really start to go haywire at 100,000 SKUs and will go downhill dramatically from there. It will of course also depend on how busy your site is, busier sites will benefit more from a dedicated search system.

    Sounds good, will report back once I have something online, development is still few weeks away from having a working inventory.



  • A lot of improvement can already been achieved by having a good look at how indexation works in Prestashop.

    If you look in classes\search.php you see that for supplier_reference the field of the ps_product table is used instead of the fields of the ps_product_supplier table. Also - when a product is in more than one category only the name of the default category is used.



  • I have found a live production site that uses an older version of BRAD search that used Elasticsearch 1.7. I tested this older version BRAD search using Elasticsearch 1.7 about a year ago and the biggest difference I noticed was that the instant search results were much slower than with the newer versions.

    For me, the most interesting thing to be seen on this site with the older version of BRAD search is the ability to refine subcategory results, shown in the left column of these pages:
    https://www.deelat.com/vehicle-fleet-maintenance/drones/
    https://www.deelat.com/fasteners/flange-screws/hex-flange/

    The sliders and check boxes can be configured within the BRAD module, but I have no ability to figure out how to get them to show up within a template or how to get different sets of options for different categories.



  • I’ve found another production site using BRAD search but it appears to be a newer version. It’s hosted in (of all places!) Iran:

    Site in Persian

    They use Cloudflare for DNS but they don’t have Cloudflare’s caching turned on, so I could see their IP and hosting location.

    Google Translate does a decent job with the site but BRAD doesn’t seem to work on the post-translate site. Not too surprising really since the translation is passing through Google’s servers which will likely cause problems with AJAX functions.

    Persian is a right-to-left language so the site is a little odd to use but the search performance is impressive, especially considering Iran isn’t thought to have particularly fast Internet. Some sample search terms to try include:

    • kardon

    • portable

    • digital

    • bluetooth

    The site does not have fuzzy search turned on so spelling mistakes will not return results.

    They do have category result options enabled though:

    Original Persian

    Google Translate English

    Again the Google Translate version doesn’t seem to work with AJAX updates but it gives you an idea what the options are. Clicking on various options on the Persian site does give results. There seems to be a bug with the color selection setup, once I chose a color I couldn’t deselect it any more.

    Anyway, it’s interesting to see what’s possible, I hope to have a live sample site I can post later this month. I’ll certainly share whatever I manage to learn about setup and implementation with the community.


  • administrators

    Hmmz, you seem to have gone the ElasticSearch way haven’t you? We are pretty close to modularizing @lesley 's Algolia module for thirty bees. Would you be interested in a beta test? Or is it all ElasticSearch now? :P



  • I don’t have a functioning site yet, but I’m getting close!

    I like the idea of Elasticsearch mostly because of the number of sites I want to launch using the automation I’m currently putting in place. Between automation and a large available catalog, having to pay $$$ to $$$$ per month for any specific aspect of each site will be financially difficult. This makes my situation rather unusual as I’m willing to put in more work to save that $$$ to $$$$ than most other store owners would be. $$$ x 1 isn’t so bad. $$$ x 10+ is too much for me right now.

    On the other hand, I do fully understand that you, @lesley, and other people building 30bz need to make some money off the project at some point or the project will die. I hope to find a way to contribute in both time and with finances as things improve financially for me.



Looks like your connection to thirty bees forum was lost, please wait while we try to reconnect.