[BUG] materialize_to_cosmosdb() fails with error at "Storage quota for 'Collection' exceeded.

### Willingness to contribute

Yes. I can contribute a fix for this bug independently.

### Feathr version

v0.7.2

### System information

- **OS Platform and Distribution (e.g., Linux Ubuntu 20.0)**: ubuntu
- **Python version**: v0.7.2
- **Spark version, if reporting runtime issue**: v0.7.2

### Describe the problem

materialize_to_cosmosdb() in CI test fails with error at "Storage quota for 'Collection' exceeded.

Link to failed ci test: https://github.com/feathr-ai/feathr/actions/runs/3046440631/jobs/4909251776

### Tracking information

2022-09-13 15:52:25.701 | ERROR | feathr.spark_provider._databricks_submission:wait_for_completion:210 - Feathr job has failed. Please visit this page to view error message: ***?o=5638037984879289#job/989778037256226/run/6648171
2022-09-13 15:52:25.702 | ERROR | feathr.spark_provider._databricks_submission:wait_for_completion:212 - Error Code: CosmosException: {"innerErrorMessage":"Message: {\"Errors\":[\"Storage quota for 'Collection' exceeded.\"]}\r\nActivityId: e180cd10-16cb-4849-a994-950e6e699259, Request URI: /apps/02081c9d-0df4-4ee1-9d2e-1bcb5b53c4fe/services/7ebe558b-1e49-4fff-a896-36b3f6ff5d49/partitions/5b42a7ad-3414-4249-a47b-9db83694a16f/replicas/133040658095713030p, RequestStats: , SDK: Microsoft.Azure.Documents.Common/2.14.0, StatusCode: Forbidden","cosmosDiagnostics":{"userAgent":"azsdk-java-cosmos/4.32.0-snapshot.1 Linux/5.4.0-1089-azure JRE/1.8.0_302","activityId":"e180cd10-16cb-4849-a994-950e6e699259","requestLatencyInMs":75,"requestStartTimeUTC":"2022-09-13T15:52:00.267Z","requestEndTimeUTC":"2022-09-13T15:52:00.342Z","responseStatisticsList":[],"supplementalResponseStatisticsList":[],"addressResolutionStatistics":{},"regionsContacted":["west us"],"retryContext":{"statusAndSubStatusCodes":null,"retryLatency":0,"retryCount":0},"metadataDiagnosticsContext":{"metadataDiagnosticList":null},"serializationDiagnosticsContext":{"serializationDiagnosticsList":[{"serializationType":"CONTAINER_SERIALIZATION","startTimeUTC":"2022-09-13T15:52:00.267Z","endTimeUTC":"2022-09-13T15:52:00.268Z","durationInMicroSec":1000}]},"gatewayStatistics":{"sessionToken":"0:-1#109","operationType":"Create","resourceType":"DocumentCollection","statusCode":403,"subStatusCode":3090,"requestCharge":0.0,"requestTimeline":[{"eventName":"connectionAcquired","startTimeUTC":"2022-09-13T15:52:00.269Z","durationInMicroSec":0},{"eventName":"connectionConfigured","startTimeUTC":"2022-09-13T15:52:00.269Z","durationInMicroSec":0},{"eventName":"requestSent","startTimeUTC":"2022-09-13T15:52:00.269Z","durationInMicroSec":5000},{"eventName":"transitTime","startTimeUTC":"2022-09-13T15:52:00.274Z","durationInMicroSec":66000},{"eventName":"received","startTimeUTC":"2022-09-13T15:52:00.340Z","durationInMicroSec":1000}],"partitionKeyRangeId":null,"exceptionMessage":"Message: {\"Errors\":[\"Storage quota for 'Collection' exceeded.\"]}\r\nActivityId: e180cd10-16cb-4849-a994-950e6e699259, Request URI: /apps/02081c9d-0df4-4ee1-9d2e-1bcb5b53c4fe/services/7ebe558b-1e49-4fff-a896-36b3f6ff5d49/partitions/5b42a7ad-3414-4249-a47b-9db83694a16f/replicas/133040658095713030p, RequestStats: , SDK: Microsoft.Azure.Documents.Common/2.14.0, StatusCode: Forbidden","exceptionResponseHeaders":"{Content-Type=application/json, Date=Tue, 13 Sep 2022 15:51:59 GMT, lsn=109, Server=Microsoft-HTTPAPI/2.0, Strict-Transport-Security=max-age=31536000, Transfer-Encoding=chunked, x-ms-activity-id=e180cd10-16cb-4849-a994-950e6e699259, x-ms-cosmos-llsn=109, x-ms-cosmos-quorum-acked-llsn=109, x-ms-current-replica-set-size=4, x-ms-current-write-quorum=3, x-ms-gatewayversion=version=2.14.0, x-ms-global-Committed-lsn=109, x-ms-last-state-change-utc=Mon, 05 Sep 2022 08:55:19.961 GMT, x-ms-number-of-read-regions=0, x-ms-quorum-acked-lsn=109, x-ms-request-charge=0, x-ms-request-duration-ms=0.951, x-ms-schemaversion=1.14, x-ms-serviceversion=version=2.14.0.0, x-ms-session-token=0:-1#109, x-ms-substatus=3090, x-ms-transport-request-id=25054, x-ms-xp-role=1}"},"systemInformation":{"usedMemory":"673148 KB","availableMemory":"1269380 KB","systemCpuLoad":"(2022-09-13T15:51:56.936Z 34.8%)","availableProcessors":4},"clientCfgs":{"id":1,"machineId":"vmId:c066d651-9971-480e-84fa-5a4ca0a89d40","connectionMode":"DIRECT","numberOfClients":1,"connCfg":{"rntbd":"(cto:PT10S, nrto:PT10S, icto:PT0S, ieto:PT1H, mcpe:130, mrpc:30, cer:true)","gw":"(cps:1000, nrto:PT1M, icto:PT1M, p:false)","other":"(ed: true, cs: false)"},"consistencyCfg":"(consistency: Eventual, mm: true, prgns: [])"}}}
2022-09-13 15:52:25.702 | ERROR | feathr.spark_provider._databricks_submission:wait_for_completion:214 - at azure_cosmos_spark.com.azure.cosmos.BridgeInternal.createCosmosException(BridgeInternal.java:488)
at azure_cosmos_spark.com.azure.cosmos.implementation.RxGatewayStoreModel.validateOrThrow(RxGatewayStoreModel.java:440)
at azure_cosmos_spark.com.azure.cosmos.implementation.RxGatewayStoreModel.lambda$toDocumentServiceResponse$0(RxGatewayStoreModel.java:344)
at azure_cosmos_spark.reactor.core.publisher.FluxMap$MapSubscriber.onNext(FluxMap.java:[106](https://github.com/feathr-ai/feathr/actions/runs/3046440631/jobs/4909251776#step:8:107))
at azure_cosmos_spark.reactor.core.publisher.FluxSwitchIfEmpty$SwitchIfEmptySubscriber.onNext(FluxSwitchIfEmpty.java:74)
at azure_cosmos_spark.reactor.core.publisher.FluxPeek$PeekSubscriber.onNext(FluxPeek.java:200)
at azure_cosmos_spark.reactor.core.publisher.FluxHandle$HandleSubscriber.onNext(FluxHandle.java:126)
at azure_cosmos_spark.reactor.core.publisher.FluxMap$MapConditionalSubscriber.onNext(FluxMap.java:220)
at azure_cosmos_spark.reactor.core.publisher.FluxDoFinally$DoFinallySubscriber.onNext(FluxDoFinally.java:130)
at azure_cosmos_spark.reactor.core.publisher.FluxHandleFuseable$HandleFuseableSubscriber.onNext(FluxHandleFuseable.java:191)
at azure_cosmos_spark.reactor.core.publisher.FluxContextWrite$ContextWriteSubscriber.onNext(FluxContextWrite.java:[107](https://github.com/feathr-ai/feathr/actions/runs/3046440631/jobs/4909251776#step:8:108))
at azure_cosmos_spark.reactor.core.publisher.Operators$MonoSubscriber.complete(Operators.java:1816)
at azure_cosmos_spark.reactor.core.publisher.MonoCollectList$MonoCollectListSubscriber.onComplete(MonoCollectList.java:128)
at azure_cosmos_spark.reactor.core.publisher.FluxPeek$PeekSubscriber.onComplete(FluxPeek.java:260)
at azure_cosmos_spark.reactor.core.publisher.FluxMap$MapSubscriber.onComplete(FluxMap.java:142)
at azure_cosmos_spark.reactor.netty.channel.FluxReceive.onInboundComplete(FluxReceive.java:400)
at azure_cosmos_spark.reactor.netty.channel.ChannelOperations.onInboundComplete(ChannelOperations.java:419)
at azure_cosmos_spark.reactor.netty.channel.ChannelOperations.terminate(ChannelOperations.java:473)
at azure_cosmos_spark.reactor.netty.http.client.HttpClientOperations.onInboundNext(HttpClientOperations.java:703)
at azure_cosmos_spark.reactor.netty.channel.ChannelOperationsHandler.channelRead(ChannelOperationsHandler.java:93)
at azure_cosmos_spark.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
at azure_cosmos_spark.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
at azure_cosmos_spark.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
at azure_cosmos_spark.io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286)
at azure_cosmos_spark.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
at azure_cosmos_spark.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
at azure_cosmos_spark.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
at azure_cosmos_spark.io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead(CombinedChannelDuplexHandler.java:436)
at azure_cosmos_spark.io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:327)
at azure_cosmos_spark.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:299)
at azure_cosmos_spark.io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:251)
at azure_cosmos_spark.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
at azure_cosmos_spark.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
at azure_cosmos_spark.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
at azure_cosmos_spark.io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1372)
at azure_cosmos_spark.io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:[123](https://github.com/feathr-ai/feathr/actions/runs/3046440631/jobs/4909251776#step:8:124)5)
at azure_cosmos_spark.io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1284)
at azure_cosmos_spark.io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:510)
at azure_cosmos_spark.io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:449)
at azure_cosmos_spark.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:279)
at azure_cosmos_spark.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
at azure_cosmos_spark.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
at azure_cosmos_spark.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
at azure_cosmos_spark.io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
at azure_cosmos_spark.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
at azure_cosmos_spark.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
at azure_cosmos_spark.io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
at azure_cosmos_spark.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166)
at azure_cosmos_spark.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:722)
at azure_cosmos_spark.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:658)
at azure_cosmos_spark.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:584)
at azure_cosmos_spark.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:496)
at azure_cosmos_spark.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986)
at azure_cosmos_spark.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at azure_cosmos_spark.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.lang.Thread.run(Thread.java:748)

### Code to reproduce bug

_No response_

### What component(s) does this bug affect?

- [X] `Python Client`: This is the client users use to interact with most of our API. Mostly written in Python.
- [X] `Computation Engine`: The computation engine that execute the actual feature join and generation work. Mostly in Scala and Spark.
- [ ] `Feature Registry API`: The frontend API layer supports SQL, Purview(Atlas) as storage. The API layer is in Python(FAST API)
- [ ] `Feature Registry Web UI`: The Web UI for feature registry. Written in React