[Feature] Use Scala 2.13 spark dependencies on the VenicePushJob instead of 2.12

Willingness to contribute

Yes. I can contribute a fix for this bug independently.

Feature Request Proposal

Use Scala 2.13 spark dependencies on the VenicePushJob instead of 2.12

Motivation

What is the use case for this feature?

Scala is moving towards Scala 3 and 2.13 is meant as a stepping stone for migrating from Scala 2 to Scala 3. Moreover, the next spark version might drop support to Scala 2.12. At the moment we are using the 2.12 spark dependancies instead of 2.13. Therefore it is better to have Scala 2.13 dependancies instead of 2.12.

Details

No response

What component(s) does this bug affect?

• Controller: This is the control-plane for Venice. Used to create/update/query stores and their metadata.
• Router: This is the stateless query-routing layer for serving read requests.
• Server: This is the component that persists all the store data.
• VenicePushJob: This is the component that pushes derived data from Hadoop to Venice backend.
• VenicePulsarSink: This is a Sink connector for Apache Pulsar that pushes data from Pulsar into Venice.
• Thin Client: This is a stateless client users use to query Venice Router for reading store data.
• Fast Client: This is a stateful client users use to query Venice Server for reading store data.
• Da Vinci Client: This is an embedded, stateful client that materializes store data locally.
• Samza: This is the library users use to make nearline updates to store data.
• Admin Tool: This is the stand-alone client used for ad-hoc operations on Venice.

Start with the VenicePushJob module and locate its Scala and Spark dependency declarations, then compare the surrounding build configuration for Scala 2.12 references. The work is done when VenicePushJob uses the corresponding Scala 2.13 Spark dependencies and its existing validation or build checks pass.