Improve processing of data to be scalable

The SimulatedBaseClusterStateProvider requires an abstract method, process_data, to be implemented by child classes. Some of those child classes have implemented that method by reading the entire csv file. While this works, it is not scalable and can potentially have issues when data contained in *perf_event_log.csv file grows large. For an example, see https://github.com/microsoft/vasim/blob/main/src/vasim/recommender/cluster_state_provider/FileClusterStateProvider.py#L153C25-L153C26.

Task: Improve process_data methods under all the *ClusterStateProvider classes to be scalable as the size of the log files grows. Some techniques to explore scalability are:

• Chunking: Read the CSV file in chunks instead of loading the entire file into memory.
• Parallel Processing: Use parallel processing to read and process multiple chunks simultaneously.
• Efficient Libraries: Use efficient libraries like pandas for data manipulation and dask for parallel computing.

Start with the abstract process_data method on SimulatedBaseClusterStateProvider and the implementation in src/vasim/recommender/cluster_state_provider/FileClusterStateProvider.py around line 153. Inspect the process_data methods in all *ClusterStateProvider classes and compare how they read *perf_event_log.csv files. Done means the implementations remain usable as those logs grow without loading the entire file into memory.