RDD lineage support

Currently there is no one easy solution to provide lineage for RDDs, but there are several ways how to provide at lease some of it or define it manually. This ticket will group all of them.

Subtasks:
| Task | Ticket |
| --- | --- |
| partial RDD support (Write - LogicalPlan, Read - RDD) | https://github.com/AbsaOSS/spline-spark-agent/issues/498 |
| RDD Read support |
| RDD manual lineage metadata enrichment |

---
_Original message:_

I am using the latest spline version. When I enable spline, I see incomplete lineage, my code reads the file from parquet and creates a temporary view, performs some transformations and creates a new view which is then finally written to a parquet. However I am not able to see lineages for my transformations.


Are there any constraints like all the intermediate RDD's should be in memory? or any specific spark config apart from that mentioned in the documentation which needs to enabled?