MySQL Query stuck on "Sending Data" while retrieving related products
- Dominant language
- PHP
- Stars
- 357
- Forks
- 262
- PR merge metrics
- No merged PRs in 30d
Description
Although I can not tell for sure how to reproduce the issue, as it manifests randomly, I suspect the problem is that MySQL is taking too much time in scanning the tables for the result.
Even tho the issue was discovered on Magento Commerce, except the join related to Catalog Staging the same query is executed on Magento Open Source.
### Preconditions (*)
1. Magento Commerce 2.3.1
2. MySQL 5.7
3. Aprox 70k products
4. Single Store
5. Add 2 Inventory Stocks
6. Add ~75 Inventory Sources
### Steps to reproduce (*)
1. Add Simple Product
2. Assign all inventory sources
3. Add 5 related products
4. Enable catalog product flat
5. Stores > Configuration > Catalog > Inventory > Stock Options > Display Out of Stock Products > NO
6. Visit product page on frontend and profile the page
### Expected result (*)
1. The resulting query should look similar to this:
```sql
SELECT COUNT(DISTINCT e.entity_id)
FROM catalog_product_flat_1 AS e
INNER JOIN inventory_stock_2 AS inventory_in_stock
ON e.sku = inventory_in_stock.sku
INNER JOIN catalog_product_index_price AS price_index
ON price_index.entity_id = e.entity_id
AND price_index.website_id = '1'
AND price_index.customer_group_id = 0
INNER JOIN catalog_category_product_index_store1 AS cat_index
ON cat_index.product_id=e.entity_id
AND cat_index.store_id=1
AND cat_index.visibility IN(2, 4)
AND cat_index.category_id=1276
INNER JOIN catalog_product_entity AS product_entity
ON product_entity.entity_id = e.entity_id
AND (product_entity.created_in <= '1579039140' AND product_entity.updated_in > '1579039140')
INNER JOIN catalog_product_link AS links
ON links.linked_product_id = e.entity_id
AND links.link_type_id = 1
WHERE (inventory_in_stock.is_salable = 1)
AND (e.entity_id NOT IN('124079'))
AND (links.product_id = 124079)
AND (e.row_id != '124079')
```
### Actual result (*)
1. The resulting query performs too many joins with the same tables, in this case 2 times for `inventory_stock_2` and 3 times `catalog_product_entity`.
```sql
SELECT COUNT(DISTINCT e.entity_id)
FROM catalog_product_flat_1 AS e
# \Magento\InventoryCatalog\Model\ResourceModel\AddIsInStockFieldToCollection::execute
INNER JOIN inventory_stock_2 AS inventory_in_stock
ON e.sku = inventory_in_stock.sku
INNER JOIN catalog_product_index_price AS price_index
ON price_index.entity_id = e.entity_id
AND price_index.website_id = '1'
AND price_index.customer_group_id = 0
INNER JOIN catalog_category_product_index_store1 AS cat_index
ON cat_index.product_id=e.entity_id
AND cat_index.store_id=1
AND cat_index.visibility IN(2, 4)
AND cat_index.category_id=1276
# \Magento\CatalogStaging\Model\Plugin\ResourceModel\Product\JoinProductsWhenFlatEnabled::beforeLoad
INNER JOIN catalog_product_entity AS product_entity
ON product_entity.entity_id = e.entity_id
AND (product_entity.created_in <= '1579039140' AND product_entity.updated_in > '1579039140')
# \Magento\InventoryCatalog\Model\ResourceModel\AddStockDataToCollection::execute
INNER JOIN catalog_product_entity AS product
ON product.entity_id = e.entity_id
AND (product.created_in <= '1579039140' AND product.updated_in > '1579039140')
# \Magento\InventoryCatalog\Model\ResourceModel\AddStockDataToCollection::execute
INNER JOIN inventory_stock_2 AS stock_status_index
ON product.sku = stock_status_index.sku
# \Magento\Catalog\Model\ResourceModel\Product\Link\Product\Collection::_joinLinks
INNER JOIN catalog_product_link AS links
ON links.linked_product_id = e.entity_id
AND links.link_type_id = 1
# \Magento\Catalog\Model\ResourceModel\Product\Link\Product\Collection::joinProductsToLinks
INNER JOIN catalog_product_entity AS product_entity_table
ON links.product_id = product_entity_table.row_id
AND (product_entity_table.created_in <= '1579039140' AND product_entity_table.updated_in > '1579039140')
WHERE (inventory_in_stock.is_salable = 1)
AND (e.entity_id NOT IN('124079'))
AND (stock_status_index.is_salable = 1)
AND (links.product_id = 124079)
AND (e.row_id != '124079')
```
Contributor guide
Research direction
Start by tracing the collection through the listed entry points: AddIsInStockFieldToCollection::execute, AddStockDataToCollection::execute, JoinProductsWhenFlatEnabled::beforeLoad, and the product-link collection methods. Reproduce or profile the related-products query with the stated inventory and flat-catalog settings, comparing the expected and actual SQL. Done means the redundant joins are addressed and the resulting query no longer exhibits the reported excessive execution time.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- mysql, php
- Domain
- backend, databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100