MosaicML-Streaming on Databricks

Hi all, I'm a new user of mosaicml-streaming on Databricks who stumbled upon Mosaic ML (and Petastorm) for loading large data from PySpark to PyTorch tensors. Here is an example jupyter notebook that I'm trying to replicate on my Databricks clusters, however, I have a few questions:

1. The above notebook's requirements say that we need a "Databricks Runtime for ML 15.2 or higher". However, my organization has an earlier version. Can we use mosaic-streaming on earlier runtime versions?

2. This above notebook imports "from petastorm import TransformSpec" but this blog says that Petastorm is deprecated and suggests the usage of mosaic-streaming instead. I checked the code, and it just imports the petastorm but doesn't use it. Can someone confirm if this package was mistakenly imported?

3. My understanding of mosaic-streaming is that it takes a PySpark dataframe as input and provides API for getting a PyTorch dataloader as output which can be used for ML training on the fly without writing the whole data in some MDS format. Is my understanding correct?

PS: I started a discussion on your Slack community but was re-directed to submit an issue here.

Start with the linked Databricks notebook and Petastorm documentation, then compare their runtime requirements and imports with the mosaicml-streaming usage described in the issue. Done would mean confirming the supported Databricks Runtime versions, whether the Petastorm import is needed, and whether the stated PySpark-to-PyTorch workflow is correct.