Synapse Intelligent Cache is not working

Open

#155 opened on Nov 10, 2022 0 comments 0 reactions 0 assignees View on GitHub

Description

Env:
--Spark 3.2
--Synapse-premium
--Intelligent caching enabled.

**Intelligent cache enabled**
![image](https://user-images.githubusercontent.com/29754295/200988210-f7ea757f-7560-4b8a-bc4e-9cae859b37ec.png)

**Inital content of CSV file**

![image](https://user-images.githubusercontent.com/29754295/200988353-ccd87a24-ab1b-4770-92da-3f3e9d3a33f2.png)

read successfully

![image](https://user-images.githubusercontent.com/29754295/200988523-b15e51e6-ed48-4ebc-b1cb-ed4a78faa0d4.png)

added more content later

![image](https://user-images.githubusercontent.com/29754295/200988659-53a85395-aecd-4913-84f4-0032bb38895a.png)

the same result was shown even after running df1.take(100) several times instead of displaying the newly records added to it.

![image](https://user-images.githubusercontent.com/29754295/200988815-b8350470-1e16-4f46-a769-877c060747de.png)

The issue was resolved after rerunning the spark.read again

No contributing guide indexed for this repository

This issue has not been assessed yet.