Azure / Azure/azure-sdk-for-java

[QUERY] How to use Linked Services with Cosmos DB Spark connector Throughput control in Synapse Analytics

Aperta
#42,855 8 commenti 0 reazioni 1 assegnatario Rivendicata da @TheovanKraay Vedi su GitHub
Client Cosmos customer-reported needs-team-attention question Service Attention
Lingua principale
Java
Stelle
2.6k
Fork
2.2k
Merge medio
2g 7h
PR unite (30g)
211

Descrizione

**Query/Question**

What is the equivalent to `spark.synapse.linkedService` for use with `spark.cosmos.throughputControl.globalControl.database`

I need to use private endpoints because of DEP, and having credentials stored with Linked Services is very handy.

***Why is this not a Bug or a feature Request?***

Feels like I'm missing something really obvious, but the [docs](https://github.com/Azure/azure-sdk-for-java/blob/main/sdk/cosmos/azure-cosmos-spark_3_2-12/docs/configuration-reference.md) don't seem to cover what I'm seeking,

**Setup (please complete the following information if applicable):**
- OS: Spark 3.4 Python 3.10 Scala
- IDE: Synapse Analytics Web based
- Library/Libraries: default Spark Pool config

**PySpark**
```python
(
df_floodre
.withColumn('id', F.col('uprn').cast('string'))
.write
.format("cosmos.oltp")
.mode("Append")
.option("spark.synapse.linkedService", "CosmosDB_GraphQL_databasets")
.option("spark.cosmos.container", 'floodRe')
.option('spark.cosmos.throughputControl.enabled','true')
.option('spark.cosmos.throughputControl.name','floodRe_Loader_Jobs_Group')
.option('spark.cosmos.throughputControl.globalControl.database','CosmosDB_GraphQL_databasets_ThroughputControl')
.option('spark.cosmos.throughputControl.globalControl.container','ThroughputControl-floodre')
.option('spark.cosmos.throughputControl.targetThroughput',7000)
.save()
)
```

I've tried a few likely nonsensical permutations and got 403 or 404 errors. Guidance much appreciated.

Guida per i contributori

Apri la guida per i contributori

Valutazione

Questa issue non è ancora stata valutata.

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.