magento / magento/inventory

MySQL Query stuck on "Sending Data" while retrieving related products

Open
#2,793 17 comments 2 reactions 0 assignees View on GitHub
Bug
Dominant language
PHP
Stars
357
Forks
262
PR merge metrics
No merged PRs in 30d

Description

Although I can not tell for sure how to reproduce the issue, as it manifests randomly, I suspect the problem is that MySQL is taking too much time in scanning the tables for the result.

Even tho the issue was discovered on Magento Commerce, except the join related to Catalog Staging the same query is executed on Magento Open Source.

### Preconditions (*)

1. Magento Commerce 2.3.1
2. MySQL 5.7
3. Aprox 70k products
4. Single Store
5. Add 2 Inventory Stocks
6. Add ~75 Inventory Sources

### Steps to reproduce (*)

1. Add Simple Product
2. Assign all inventory sources
3. Add 5 related products
4. Enable catalog product flat
5. Stores > Configuration > Catalog > Inventory > Stock Options > Display Out of Stock Products > NO
6. Visit product page on frontend and profile the page

### Expected result (*)

1. The resulting query should look similar to this:
```sql
SELECT COUNT(DISTINCT e.entity_id)
FROM catalog_product_flat_1 AS e
INNER JOIN inventory_stock_2 AS inventory_in_stock
ON e.sku = inventory_in_stock.sku
INNER JOIN catalog_product_index_price AS price_index
ON price_index.entity_id = e.entity_id
AND price_index.website_id = '1'
AND price_index.customer_group_id = 0
INNER JOIN catalog_category_product_index_store1 AS cat_index
ON cat_index.product_id=e.entity_id
AND cat_index.store_id=1
AND cat_index.visibility IN(2, 4)
AND cat_index.category_id=1276
INNER JOIN catalog_product_entity AS product_entity
ON product_entity.entity_id = e.entity_id
AND (product_entity.created_in <= '1579039140' AND product_entity.updated_in > '1579039140')
INNER JOIN catalog_product_link AS links
ON links.linked_product_id = e.entity_id
AND links.link_type_id = 1
WHERE (inventory_in_stock.is_salable = 1)
AND (e.entity_id NOT IN('124079'))
AND (links.product_id = 124079)
AND (e.row_id != '124079')
```

### Actual result (*)

1. The resulting query performs too many joins with the same tables, in this case 2 times for `inventory_stock_2` and 3 times `catalog_product_entity`.

```sql
SELECT COUNT(DISTINCT e.entity_id)
FROM catalog_product_flat_1 AS e

# \Magento\InventoryCatalog\Model\ResourceModel\AddIsInStockFieldToCollection::execute
INNER JOIN inventory_stock_2 AS inventory_in_stock
ON e.sku = inventory_in_stock.sku

INNER JOIN catalog_product_index_price AS price_index
ON price_index.entity_id = e.entity_id
AND price_index.website_id = '1'
AND price_index.customer_group_id = 0
INNER JOIN catalog_category_product_index_store1 AS cat_index
ON cat_index.product_id=e.entity_id
AND cat_index.store_id=1
AND cat_index.visibility IN(2, 4)
AND cat_index.category_id=1276

# \Magento\CatalogStaging\Model\Plugin\ResourceModel\Product\JoinProductsWhenFlatEnabled::beforeLoad
INNER JOIN catalog_product_entity AS product_entity
ON product_entity.entity_id = e.entity_id
AND (product_entity.created_in <= '1579039140' AND product_entity.updated_in > '1579039140')

# \Magento\InventoryCatalog\Model\ResourceModel\AddStockDataToCollection::execute
INNER JOIN catalog_product_entity AS product
ON product.entity_id = e.entity_id
AND (product.created_in <= '1579039140' AND product.updated_in > '1579039140')

# \Magento\InventoryCatalog\Model\ResourceModel\AddStockDataToCollection::execute
INNER JOIN inventory_stock_2 AS stock_status_index
ON product.sku = stock_status_index.sku

# \Magento\Catalog\Model\ResourceModel\Product\Link\Product\Collection::_joinLinks
INNER JOIN catalog_product_link AS links
ON links.linked_product_id = e.entity_id
AND links.link_type_id = 1

# \Magento\Catalog\Model\ResourceModel\Product\Link\Product\Collection::joinProductsToLinks
INNER JOIN catalog_product_entity AS product_entity_table
ON links.product_id = product_entity_table.row_id
AND (product_entity_table.created_in <= '1579039140' AND product_entity_table.updated_in > '1579039140')

WHERE (inventory_in_stock.is_salable = 1)
AND (e.entity_id NOT IN('124079'))
AND (stock_status_index.is_salable = 1)
AND (links.product_id = 124079)
AND (e.row_id != '124079')
```

Contributor guide

Open the contributing guide

Research direction

Start by tracing the collection through the listed entry points: AddIsInStockFieldToCollection::execute, AddStockDataToCollection::execute, JoinProductsWhenFlatEnabled::beforeLoad, and the product-link collection methods. Reproduce or profile the related-products query with the stated inventory and flat-catalog settings, comparing the expected and actual SQL. Done means the redundant joins are addressed and the resulting query no longer exhibits the reported excessive execution time.

Written by the indexing model from the issue text.

Assessment

Tech stack
mysql, php
Domain
backend, databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.