Help with importing combinations

[the.rampage.rado]

I'm trying to make a python file that matches two files with information and generates me a CSV to import in my site.

The output is as follows:



But it can not be imported because of the following error:

Errors occurred:
Property ProductAttribute->name is empty

I'm trying the following settings for Combinations import:



Does anybody have some ideas?

Also - what is the purpose of position value in Attribute and Value columns? I believe they should not increase per same product reference (according to the demo csv).

[312erik123]

Hello!

I havn't fully understood the combination import, but I've found a few ways to make it work. First of, when you come to the second stage, i.e after you click import - does all the columns match the data? Secondly, I have found that I sometimes need to put the data in quotes - especially if its data with commas and dots etc. 

Also regarding the position value, i believe thats how it will be shown in your store. So if color has position one, and size position two, it will be shown in that order.

[the.rampage.rado]

@312erik123 thanks, now I'll try your hint with the quotes.

Regarding the position - the default demo csv gives only 0 and 1 for all positions, so I don't understand it at all. First I was thinking it was the position inside the drop down (speaking for Size attribute), later I saw on PS' forum that it is the order in which the attribute groups are shown in FO, which does not make any sense (first color, then Size, or vise versa). Then there was an opinion that they can be removed altogether and rely only on the positions inside the attribute group.

Will make a shorter file and try this too. On one installation a file with ~2k rows throws this error on the 25th row, on another it does on ~1600th row. So not quite sure what is going on.

Logger says only:

[the.rampage.rado]

Quotes don't help - the values get imported with them in the store (when the import succeeds).

I managed to import few combinations but only if the file is very small (under 20 rows or so). I see that large files are split in multiple parts that are imported by ajax but I can't figure out what is breaking. So I looked at my file and found a row where a product had only color but not size associated with it (no stock is available). An easy fix - python script will exclude those and give me a list to remove them from my store.

But then when I try to import the file without those unproperly formatted lines I get this error:

And once again - if I make the file very small (~20 lines) it imports, but if I have 2k lines - this error appears...

[the.rampage.rado]

 

@datakick, I think chatgpt made a working code for me but can you confirm there is something wrong in this mechanism or it's just a bug with my csv file (let's say using Cyrillic and making the encoded ajax call very big for this operation with so many lines, etc).

During debuggint it advised the following is not working:

 

Key Changes:
Handling of crossStepsVariables:

It now checks if crossStepsVars exists in the AJAX request and decodes it correctly.
If crossStepsVars is missing or invalid, it initializes crossStepsVariables with empty arrays for groups, attributes, and deletedProducts.
Extensive Logging:

Logs the raw and decoded crossStepsVars at the start.
Logs the final crossStepsVariables after the import step.
Fallback for Missing Data:

Ensures that crossStepsVariables is never empty or invalid, preventing errors in subsequent processing.
Improved Structure:

Cleaned up logic for handling results and reserved additional POST size for large imports.

(of course I removed the logging from the following functions after debugging)

Working versions for me that import the large file:

public function attributeImport($offset = false, $limit = false, &$crossStepsVariables = false, $validateOnly = false)
    {
        $defaultLanguage = Configuration::get('PS_LANG_DEFAULT');

        // Initialize groups
        $groups = is_array($crossStepsVariables) && array_key_exists('groups', $crossStepsVariables) ? $crossStepsVariables['groups'] : [];
        foreach (AttributeGroup::getAttributesGroups($defaultLanguage) as $group) {
            $groups[$group['name']] = (int) $group['id_attribute_group'];
        }

        // Initialize attributes
        $attributes = is_array($crossStepsVariables) && array_key_exists('attributes', $crossStepsVariables) ? $crossStepsVariables['attributes'] : [];
        foreach (ProductAttribute::getAttributes($defaultLanguage) as $attribute) {
            $attributes[$attribute['attribute_group'].'_'.$attribute['name']] = (int) $attribute['id_attribute'];
        }

        // Initialize deleted products
        $deletedProducts = is_array($crossStepsVariables) && array_key_exists('deletedProducts', $crossStepsVariables) ? $crossStepsVariables['deletedProducts'] : [];

        $this->receiveTab();
        $datasource = $this->openDataSource($offset);

        static::setLocale();

        $regenerate = Tools::getValue('regenerate');
        $shopIsFeatureActive = Shop::isFeatureActive();

        $lineCount = 0;
        for ($currentLine = 0; ($line = $datasource->getRow()) && (!$limit || $currentLine < $limit); $currentLine++) {
            $lineCount++;

            if (empty($line) || !is_array($line) || count(array_filter($line)) == 0) {
                $this->warnings[] = $this->l('There is an empty row in the file that won\'t be imported.');
                continue;
            }

            $info = static::getMaskedRow($line);
            $info = array_map('trim', $info);

            try {
                $this->attributeImportOne(
                    $info,
                    $defaultLanguage,
                    $groups, // by ref
                    $attributes, // by ref
                    $regenerate,
                    $shopIsFeatureActive,
                    $validateOnly,
                    $deletedProducts // by ref
                );
            } catch (PrestaShopException $e) {
                $this->errors[] = $e->getMessage();
            }
        }

        $datasource->close();

        if ($crossStepsVariables !== false) {
            $crossStepsVariables['groups'] = is_array($groups) ? $groups : [];
            $crossStepsVariables['attributes'] = is_array($attributes) ? $attributes : [];
            $crossStepsVariables['deletedProducts'] = is_array($deletedProducts) ? $deletedProducts : [];
        }

        return $lineCount;
    }

public function importByGroups($offset = false, $limit = false, &$results = null, $validateOnly = false, $moreStep = 0)
    {
        // Check if the CSV file exists
        if (Tools::getValue('filename')) {
            $entityType = $this->getSelectedEntity();
            $shopIsFeatureActive = Shop::isFeatureActive();

            // If I am a superadmin, truncate table (ONLY IF OFFSET == 0 or false and NOT FOR VALIDATION MODE!)
            if (!$offset && !$moreStep && !$validateOnly && (($shopIsFeatureActive && $this->context->employee->isSuperAdmin()) || !$shopIsFeatureActive) && Tools::getValue('truncate')) {
                $this->truncateTables($entityType);
            }

            $doneCount = 0;
            $crossStepsVariables = [];

            // Get crossStepsVariables from the previous AJAX call
            if ($crossStepsVars = Tools::getValue('crossStepsVars')) {
                $crossStepsVars = json_decode($crossStepsVars, true);

                if (!empty($crossStepsVars) && is_array($crossStepsVars)) {
                    $crossStepsVariables = $crossStepsVars;
                } else {
                    $crossStepsVariables = ['groups' => [], 'attributes' => [], 'deletedProducts' => []];
                }
            } else {
                $crossStepsVariables = ['groups' => [], 'attributes' => [], 'deletedProducts' => []];
            }

            // Process based on entity type
            if (static::hasEntityType($entityType)) {
                $doneCount += $this->importGroup(static::getEntityType($entityType), $offset, $limit, $crossStepsVariables, $validateOnly, $moreStep);
            } else {
                // Fallback to original implementation
                switch ($entityType) {
                    case static::ENTITY_TYPE_CATEGORIES:
                        $doneCount += $this->categoryImport($offset, $limit, $crossStepsVariables, $validateOnly);
                        $this->clearSmartyCache();
                        break;

                    case static::ENTITY_TYPE_PRODUCTS:
                        if (!defined('PS_MASS_PRODUCT_CREATION')) {
                            define('PS_MASS_PRODUCT_CREATION', true);
                        }
                        $moreStepLabels = [$this->l('Linking Accessories...')];
                        $doneCount += $this->productImport($offset, $limit, $crossStepsVariables, $validateOnly, $moreStep);
                        $this->clearSmartyCache();
                        break;

                    case static::ENTITY_TYPE_COMBINATIONS:
                        $doneCount += $this->attributeImport($offset, $limit, $crossStepsVariables, $validateOnly);
                        $this->clearSmartyCache();
                        break;

                    // Add other cases if needed
                    case static::ENTITY_TYPE_CUSTOMERS:
                        $doneCount += $this->customerImport($offset, $limit, $validateOnly);
                        break;

                    case static::ENTITY_TYPE_ADDRESSES:
                        $doneCount += $this->addressImport($offset, $limit, $validateOnly);
                        break;

                    case static::ENTITY_TYPE_MANUFACTURERS:
                        $doneCount += $this->manufacturerImport($offset, $limit, $validateOnly);
                        $this->clearSmartyCache();
                        break;

                    case static::ENTITY_TYPE_SUPPLIERS:
                        $doneCount += $this->supplierImport($offset, $limit, $validateOnly);
                        $this->clearSmartyCache();
                        break;

                    case static::ENTITY_TYPE_ALIAS:
                        $doneCount += $this->aliasImport($offset, $limit, $validateOnly);
                        break;

                    case static::ENTITY_TYPE_STORE_CONTACTS:
                        $doneCount += $this->storeContactImport($offset, $limit, $validateOnly);
                        $this->clearSmartyCache();
                        break;

                    case static::ENTITY_TYPE_SUPPLY_ORDERS:
                        $doneCount += $this->supplyOrdersImport($offset, $limit, $validateOnly);
                        break;

                    case static::ENTITY_TYPE_SUPPLY_ORDER_DETAILS:
                        $doneCount += $this->supplyOrdersDetailsImport($offset, $limit, $crossStepsVariables, $validateOnly);
                        break;
                }
            }

            // Handle results and progress
            if ($results !== null) {
                $results['isFinished'] = ($doneCount < $limit);
                $results['doneCount'] = $offset + $doneCount;

                if ($offset === 0) {
                    // Compute total count only once
                    $datasource = $this->openDataSource(0);
                    $results['totalCount'] = $datasource->getNumberOfRows() - Tools::getIntValue('skip');
                    $datasource->close();
                }

                if (!isset($moreStepLabels)) {
                    $moreStepLabels = [];
                }

                if (!$results['isFinished'] || (!$validateOnly && ($moreStep < count($moreStepLabels)))) {
                    $nextPostSize = mb_strlen(json_encode($crossStepsVariables));
                    $results['crossStepsVariables'] = $crossStepsVariables;
                    $results['nextPostSize'] = $nextPostSize + (1024 * 64); // Reserve additional size
                    $results['postSizeLimit'] = Tools::getMaxUploadSize();
                }

                if ($results['isFinished'] && !$validateOnly && ($moreStep < count($moreStepLabels))) {
                    $results['oneMoreStep'] = $moreStep + 1;
                    $results['moreStepLabel'] = $moreStepLabels[$moreStep];
                }
            }

            // Final log for current step
            $logMessage = sprintf($this->l('%s import'), $entityType);
            if ($offset !== false && $limit !== false) {
                $logMessage .= ' ' . sprintf($this->l('(from %s to %s)'), $offset, $limit);
            }
            if (Tools::getValue('truncate')) {
                $logMessage .= ' ' . $this->l('with truncate');
            }
            Logger::addLog($logMessage, 1, null, $entityType, null, true, (int) $this->context->employee->id);
        } else {
            $this->errors[] = $this->l('To proceed, please upload a file first.');
        }
    }