source-revenuecat: API Limitations + Sync Breaks with Default UserIDs like $RCAnonymousID:00000

### Connector Name

RevenueCat

### Connector Version

0.1.25

### What step the error happened?

During the sync

### Relevant information

TLDR: The RevenueCat connector is not currently usable due to limitations of their own API and other reasons unknown. Users should use their export tool instead.

The RevenueCat connector uses cursor pagination. In the response we get something like "next_page": "/v2/projects/projec1a2b3c4d/customers?starting_after=223xx1100", but when a project uses the default anonymous revenuecat user ids which get assigned as $RCAnonymousID:00000 some symbols get url encoded. The current connector extracts this id and url encodes it again, causing it to set something like %2425RCAnon... for the cursor, which causes an infinite loop of loading the same 20 records over and over again.

My current fix for this is to modify the cursor in the customers table to replace some of these values:
{{ response.get("next_page", "").split("starting_after=")[-1].split("&")[0] .replace("%2524", "$") .replace("%253A", ":") .replace("%24", "$") .replace("%3A", ":") if "starting_after=" in response.get("next_page", "") else "" }}

I assume there is a much better way to do this but I am new to Airbyte so this is all I have.

Additionally, the RevenueCat API does not let you sort/filter when pulling in customer lists. This means we need to paginate every single customer, every time, in order to get fresh data. The current connector sets Data Feed API to true, but since this API does not sort by date this should not be on. I believe Client-side Incremental Filtering should be used instead.

Their API also has rate limits set for each endpoint (x requests per minute) and max records per request is 100. In my personal requests my limits are:
/customers - 480/minute
/projects - 60/minute
/apps - 60/minute
/entitlements - 60/minute
/offerings - 60/minute
/packages - 60/minute
/products - 60/minute

Now, even after modifying all of this I still run into issues with sync hanging indefinitely, or just taking an exorbitant amount of time to pull customer data either due to rate limits or other issues. I'm yet to have a successful sync with ClickHouse or Postgres.

IMHO this connector should not be used due to these issues. Instead, the user should use RevenueCat's export to s3/Azure/GCP, which initially exports all data, followed by new exports of fresh data every 24 hours.

If someone is interested in viewing the changes I've made, here is my current yaml:
```
version: 7.0.4

type: DeclarativeSource

description: >-
RevenueCat is a powerful and reliable in-app purchase server that makes it
easy to build, analyze, and grow your subscriber base whether you're just
starting out or already have millions of customers.

check:
type: CheckStream
stream_names:
- projects
dynamic_streams_check_configs: []

definitions:
linked:
HttpRequester:
authenticator:
type: BearerAuthenticator
api_token: "{{ config[\"api_key\"] }}"
error_handler:
type: DefaultErrorHandler
max_retries: 5
backoff_strategies:
- type: ConstantBackoffStrategy
backoff_time_in_seconds: 5
- type: ExponentialBackoffStrategy
factor: 5
SimpleRetriever:
paginator:
type: DefaultPaginator
page_size_option:
type: RequestOption
field_name: limit
inject_into: request_parameter
page_token_option:
type: RequestOption
field_name: starting_after
inject_into: request_parameter
pagination_strategy:
type: CursorPagination
page_size: 100
cursor_value: \
stop_condition: "{{ not response.get(\"next_page\", \"\") }}"
DeclarativeStream: {}

streams:
- type: DeclarativeStream
name: projects
retriever:
type: SimpleRetriever
paginator:
$ref: "#/definitions/linked/SimpleRetriever/paginator"
requester:
type: HttpRequester
url: https://api.revenuecat.com/v2/projects
http_method: GET
authenticator:
$ref: "#/definitions/linked/HttpRequester/authenticator"
error_handler:
$ref: "#/definitions/linked/HttpRequester/error_handler"
record_selector:
type: RecordSelector
extractor:
type: DpathExtractor
field_path:
- items
primary_key:
- id
schema_loader:
type: InlineSchemaLoader
schema:
type: object
$schema: http://json-schema.org/schema#
properties:
created_at:
type: number
id:
type: string
name:
type:
- string
- "null"
object:
type:
- string
- "null"
required:
- id
- created_at
additionalProperties: true
incremental_sync:
type: DatetimeBasedCursor
cursor_field: created_at
is_data_feed: true
start_datetime:
type: MinMaxDatetime
datetime: "{{ config[\"start_date\"] }}"
datetime_format: "%Y-%m-%dT%H:%M:%SZ"
datetime_format: "%ms"
cursor_datetime_formats:
- "%ms"
- type: DeclarativeStream
name: apps
retriever:
type: SimpleRetriever
paginator:
$ref: "#/definitions/linked/SimpleRetriever/paginator"
requester:
type: HttpRequester
url: >-
https://api.revenuecat.com/v2/projects/{{
stream_partition['project_id'] }}/apps
http_method: GET
authenticator:
$ref: "#/definitions/linked/HttpRequester/authenticator"
record_selector:
type: RecordSelector
extractor:
type: DpathExtractor
field_path:
- items
partition_router:
type: SubstreamPartitionRouter
parent_stream_configs:
- type: ParentStreamConfig
stream:
$ref: "#/streams/0"
parent_key: id
partition_field: project_id
primary_key:
- id
schema_loader:
type: InlineSchemaLoader
schema:
type: object
$schema: http://json-schema.org/schema#
properties:
type:
type:
- string
- "null"
created_at:
type: number
id:
type: string
name:
type:
- string
- "null"
object:
type:
- string
- "null"
play_store:
type:
- object
- "null"
properties:
package_name:
type:
- string
- "null"
project_id:
type:
- string
- "null"
app_store:
type:
- object
- "null"
properties: {}
stripe:
type:
- object
- "null"
properties: {}
required:
- id
- created_at
additionalProperties: true
incremental_sync:
type: DatetimeBasedCursor
cursor_field: created_at
is_data_feed: false
start_datetime:
type: MinMaxDatetime
datetime: "{{ config[\"start_date\"] }}"
datetime_format: "%Y-%m-%dT%H:%M:%SZ"
datetime_format: "%ms"
cursor_datetime_formats:
- "%ms"
is_client_side_incremental: true
- type: DeclarativeStream
name: metrics_overview
retriever:
type: SimpleRetriever
paginator:
$ref: "#/definitions/linked/SimpleRetriever/paginator"
requester:
type: HttpRequester
url: >-
https://api.revenuecat.com/v2/projects/{{
stream_partition['project_id'] }}/metrics/overview
http_method: GET
authenticator:
$ref: "#/definitions/linked/HttpRequester/authenticator"
record_selector:
type: RecordSelector
extractor:
type: DpathExtractor
field_path:
- metrics
partition_router:
type: SubstreamPartitionRouter
parent_stream_configs:
- type: ParentStreamConfig
stream:
$ref: "#/streams/0"
parent_key: id
partition_field: project_id
primary_key:
- id
schema_loader:
type: InlineSchemaLoader
schema:
type: object
$schema: http://json-schema.org/schema#
properties:
description:
type:
- string
- "null"
id:
type: string
last_updated_at:
type:
- number
- "null"
last_updated_at_iso8601:
type:
- string
- "null"
name:
type:
- string
- "null"
object:
type:
- string
- "null"
period:
type:
- string
- "null"
unit:
type:
- string
- "null"
value:
type:
- number
- "null"
project_id:
type:
- string
- "null"
required:
- id
additionalProperties: true
transformations:
- type: AddFields
fields:
- type: AddedFieldDefinition
path:
- project_id
value: "{{ stream_partition['project_id'] }}"
- type: DeclarativeStream
name: customers
retriever:
type: SimpleRetriever
paginator:
type: DefaultPaginator
page_size_option:
type: RequestOption
field_name: limit
inject_into: request_parameter
page_token_option:
type: RequestOption
field_name: starting_after
inject_into: request_parameter
pagination_strategy:
type: CursorPagination
page_size: 100
cursor_value: >-
{{ response.get("next_page",
"").split("starting_after=")[-1].split("&")[0] .replace("%2524",
"$") .replace("%253A", ":") .replace("%24", "$")
.replace("%3A", ":") if "starting_after=" in
response.get("next_page", "") else "" }}
stop_condition: "{{ not response.get(\"next_page\", \"\") }}"
requester:
type: HttpRequester
url: >-
https://api.revenuecat.com/v2/projects/proj17e840ef{{
stream_partition['proj_id'] }}/customers
http_method: GET
authenticator:
$ref: "#/definitions/linked/HttpRequester/authenticator"
record_selector:
type: RecordSelector
extractor:
type: DpathExtractor
field_path:
- items
transform_before_filtering: false
partition_router:
type: SubstreamPartitionRouter
parent_stream_configs:
- type: ParentStreamConfig
stream:
$ref: "#/streams/0"
parent_key: id
partition_field: project_id
primary_key:
- id
schema_loader:
type: InlineSchemaLoader
schema:
type: object
$schema: http://json-schema.org/schema#
properties:
first_seen_at:
type:
- number
- "null"
id:
type: string
last_seen_app_version:
type:
- string
- "null"
last_seen_at:
type: number
last_seen_country:
type:
- string
- "null"
last_seen_platform:
type:
- string
- "null"
last_seen_platform_version:
type:
- string
- "null"
object:
type:
- string
- "null"
project_id:
type:
- string
- "null"
required:
- id
- last_seen_at
additionalProperties: true
transformations:
- type: AddFields
fields: []
incremental_sync:
type: DatetimeBasedCursor
cursor_field: last_seen_at
is_data_feed: false
start_datetime:
type: MinMaxDatetime
datetime: "{{ config[\"start_date\"] }}"
datetime_format: "%Y-%m-%dT%H:%M:%SZ"
datetime_format: "%ms"
cursor_datetime_formats:
- "%ms"
is_client_side_incremental: true
- type: DeclarativeStream
name: customers_subscriptions
retriever:
type: SimpleRetriever
paginator:
$ref: "#/definitions/linked/SimpleRetriever/paginator"
requester:
type: HttpRequester
url: >-
https://api.revenuecat.com/v2/projects/{{
stream_partition['project_id'] }}/customers/{{
stream_partition['customer_id'] }}/subscriptions
http_method: GET
authenticator:
$ref: "#/definitions/linked/HttpRequester/authenticator"
loggingOptions:
logLevel: trace
record_selector:
type: RecordSelector
extractor:
type: DpathExtractor
field_path:
- items
partition_router:
- type: SubstreamPartitionRouter
parent_stream_configs:
- type: ParentStreamConfig
stream:
$ref: "#/streams/0"
parent_key: id
partition_field: project_id
- type: SubstreamPartitionRouter
parent_stream_configs:
- type: ParentStreamConfig
stream:
$ref: "#/streams/3"
parent_key: id
partition_field: customer_id
primary_key:
- id
schema_loader:
type: InlineSchemaLoader
schema:
type: object
$schema: http://json-schema.org/draft-07/schema#
properties:
id:
type: string
store:
type: string
object:
type: string
enum:
- purchase
- subscription
status:
type: string
country:
type: string
ownership:
type: string
starts_at:
type: integer
product_id:
type: string
customer_id:
type: string
environment:
type: string
entitlements:
type: object
properties:
url:
type: string
items:
type: array
items:
type: object
properties:
id:
type: string
object:
type: string
enum:
- entitlement
products:
type: object
properties:
url:
type: string
items:
type: array
items:
type: object
properties:
type:
type: string
id:
type: string
app:
type: object
app_id:
type: string
object:
type: string
enum:
- product
one_time:
type: object
properties:
is_consumable:
type:
- boolean
- "null"
created_at:
type: integer
display_name:
type: string
subscription:
type: object
properties:
duration:
type:
- string
- "null"
trial_duration:
type:
- string
- "null"
grace_period_duration:
type:
- string
- "null"
store_identifier:
type: string
object:
type: string
enum:
- list
next_page:
type: string
created_at:
type: integer
lookup_key:
type: string
project_id:
type: string
display_name:
type: string
object:
type: string
enum:
- list
next_page:
type: string
gives_access:
type: boolean
purchased_at:
type: integer
management_url:
type: string
pending_changes:
type: object
pending_payment:
type: boolean
auto_renewal_status:
type: string
original_customer_id:
type: string
total_revenue_in_usd:
type: object
properties:
tax:
type: number
gross:
type: number
currency:
type: string
proceeds:
type: number
commission:
type: number
presented_offering_id:
type: string
current_period_ends_at:
type: integer
current_period_starts_at:
type: integer
store_purchase_identifier:
type: integer
store_subscription_identifier:
type: integer
additionalProperties: true
transformations:
- type: AddFields
fields:
- type: AddedFieldDefinition
path:
- project_id
value: "{{ stream_partition['project_id'] }}"
incremental_sync:
type: DatetimeBasedCursor
cursor_field: starts_at
is_data_feed: false
start_datetime:
type: MinMaxDatetime
datetime: "{{ config[\"start_date\"] }}"
datetime_format: "%Y-%m-%dT%H:%M:%SZ"
datetime_format: "%ms"
cursor_datetime_formats:
- "%ms"
is_client_side_incremental: true
- type: DeclarativeStream
name: customers_purchases
retriever:
type: SimpleRetriever
paginator:
$ref: "#/definitions/linked/SimpleRetriever/paginator"
requester:
type: HttpRequester
url: >-
https://api.revenuecat.com/v2/projects/{{
stream_partition['project_id'] }}/customers/{{
stream_partition['customer_id'] }}/purchases
http_method: GET
authenticator:
$ref: "#/definitions/linked/HttpRequester/authenticator"
record_selector:
type: RecordSelector
extractor:
type: DpathExtractor
field_path:
- items
partition_router:
- type: SubstreamPartitionRouter
parent_stream_configs:
- type: ParentStreamConfig
stream:
$ref: "#/streams/0"
parent_key: id
partition_field: project_id
- type: SubstreamPartitionRouter
parent_stream_configs:
- type: ParentStreamConfig
stream:
$ref: "#/streams/3"
parent_key: id
partition_field: customer_id
primary_key:
- id
schema_loader:
type: InlineSchemaLoader
schema:
type: object
$schema: http://json-schema.org/draft-07/schema#
properties:
id:
type: string
store:
type: string
object:
type: string
enum:
- purchase
status:
type: string
country:
type: string
quantity:
type: integer
ownership:
type: string
product_id:
type: string
customer_id:
type: string
environment:
type: string
entitlements:
type: object
properties:
url:
type: string
items:
type: array
items:
type: object
properties:
id:
type: string
object:
type: string
enum:
- entitlement
products:
type: object
properties:
url:
type: string
items:
type: array
items:
type: object
properties:
type:
type: string
id:
type: string
app:
type: object
app_id:
type: string
object:
type: string
enum:
- product
one_time:
type: object
properties:
is_consumable:
type:
- boolean
- "null"
created_at:
type: integer
display_name:
type: string
subscription:
type: object
properties:
duration:
type:
- string
- "null"
trial_duration:
type:
- string
- "null"
grace_period_duration:
type:
- string
- "null"
store_identifier:
type: string
object:
type: string
enum:
- list
next_page:
type: string
created_at:
type: integer
lookup_key:
type: string
project_id:
type: string
display_name:
type: string
object:
type: string
enum:
- list
next_page:
type: string
purchased_at:
type: integer
revenue_in_usd:
type: object
properties:
tax:
type: number
gross:
type: number
currency:
type: string
proceeds:
type: number
commission:
type: number
original_customer_id:
type: string
presented_offering_id:
type: string
store_purchase_identifier:
type: integer
additionalProperties: true
transformations:
- type: AddFields
fields:
- type: AddedFieldDefinition
path:
- project_id
value: "{{ stream_partition['project_id'] }}"
incremental_sync:
type: DatetimeBasedCursor
cursor_field: purchased_at
is_data_feed: false
start_datetime:
type: MinMaxDatetime
datetime: "{{ config[\"start_date\"] }}"
datetime_format: "%Y-%m-%dT%H:%M:%SZ"
datetime_format: "%ms"
cursor_datetime_formats:
- "%ms"
is_client_side_incremental: true
- type: DeclarativeStream
name: customers_active_entitlements
retriever:
type: SimpleRetriever
paginator:
$ref: "#/definitions/linked/SimpleRetriever/paginator"
requester:
type: HttpRequester
url: >-
https://api.revenuecat.com/v2/projects/{{
stream_partition['project_id'] }}/customers/{{
stream_partition['customer_id'] }}/active_entitlements
http_method: GET
authenticator:
$ref: "#/definitions/linked/HttpRequester/authenticator"
record_selector:
type: RecordSelector
extractor:
type: DpathExtractor
field_path:
- items
partition_router:
- type: SubstreamPartitionRouter
parent_stream_configs:
- type: ParentStreamConfig
stream:
$ref: "#/streams/0"
parent_key: id
partition_field: project_id
- type: SubstreamPartitionRouter
parent_stream_configs:
- type: ParentStreamConfig
stream:
$ref: "#/streams/3"
parent_key: id
partition_field: customer_id
primary_key:
- entitlement_id
schema_loader:
type: InlineSchemaLoader
schema:
type: object
$schema: http://json-schema.org/draft-07/schema#
properties:
object:
type: string
enum:
- customer.active_entitlement
expires_at:
type: integer
entitlement_id:
type: string
additionalProperties: true
transformations:
- type: AddFields
fields:
- type: AddedFieldDefinition
path:
- project_id
value: "{{ stream_partition['project_id'] }}"
- type: AddFields
fields:
- type: AddedFieldDefinition
path:
- customer_id
value: "{{ stream_partition['customer_id'] }}"
incremental_sync:
type: DatetimeBasedCursor
cursor_field: expires_at
is_data_feed: false
start_datetime:
type: MinMaxDatetime
datetime: "{{ config[\"start_date\"] }}"
datetime_format: "%Y-%m-%dT%H:%M:%SZ"
datetime_format: "%ms"
cursor_datetime_formats:
- "%ms"
is_client_side_incremental: true
- type: DeclarativeStream
name: customers_aliases
retriever:
type: SimpleRetriever
paginator:
$ref: "#/definitions/linked/SimpleRetriever/paginator"
requester:
type: HttpRequester
url: >-
https://api.revenuecat.com/v2/projects/{{
stream_partition['project_id'] }}/customers/{{
stream_partition['customer_id'] }}/aliases
http_method: GET
authenticator:
$ref: "#/definitions/linked/HttpRequester/authenticator"
record_selector:
type: RecordSelector
extractor:
type: DpathExtractor
field_path:
- items
partition_router:
- type: SubstreamPartitionRouter
parent_stream_configs:
- type: ParentStreamConfig
stream:
$ref: "#/streams/0"
parent_key: id
partition_field: project_id
- type: SubstreamPartitionRouter
parent_stream_configs:
- type: ParentStreamConfig
stream:
$ref: "#/streams/3"
parent_key: id
partition_field: customer_id
primary_key:
- id
schema_loader:
type: InlineSchemaLoader
schema:
type: object
$schema: http://json-schema.org/draft-07/schema#
properties:
id:
type: string
object:
type: string
enum:
- customer.alias
created_at:
type: integer
additionalProperties: true
transformations:
- type: AddFields
fields:
- type: AddedFieldDefinition
path:
- project_id
value: "{{ stream_partition['project_id'] }}"
- type: AddFields
fields:
- type: AddedFieldDefinition
path:
- customer_id
value: "{{ stream_partition['customer_id'] }}"
incremental_sync:
type: DatetimeBasedCursor
cursor_field: created_at
is_data_feed: false
start_datetime:
type: MinMaxDatetime
datetime: "{{ config[\"start_date\"] }}"
datetime_format: "%Y-%m-%dT%H:%M:%SZ"
datetime_format: "%ms"
cursor_datetime_formats:
- "%ms"
is_client_side_incremental: true
- type: DeclarativeStream
name: customers_invoices
retriever:
type: SimpleRetriever
paginator:
$ref: "#/definitions/linked/SimpleRetriever/paginator"
requester:
type: HttpRequester
url: >-
https://api.revenuecat.com/v2/projects/{{
stream_partition['project_id'] }}/customers/{{
stream_partition['customer_id'] }}/invoices
http_method: GET
authenticator:
$ref: "#/definitions/linked/HttpRequester/authenticator"
record_selector:
type: RecordSelector
extractor:
type: DpathExtractor
field_path:
- items
partition_router:
- type: SubstreamPartitionRouter
parent_stream_configs:
- type: ParentStreamConfig
stream:
$ref: "#/streams/0"
parent_key: id
partition_field: project_id
- type: SubstreamPartitionRouter
parent_stream_configs:
- type: ParentStreamConfig
stream:
$ref: "#/streams/3"
parent_key: id
partition_field: customer_id
primary_key:
- id
schema_loader:
type: InlineSchemaLoader
schema:
type: object
$schema: http://json-schema.org/draft-07/schema#
properties:
id:
type: string
object:
type: string
enum:
- invoice
paid_at:
type: integer
issued_at:
type: integer
line_items:
type: array
items:
type: object
properties:
object:
type: string
enum:
- invoice.line_item
quantity:
type: integer
unit_amount:
type: object
properties:
tax:
type: number
gross:
type: number
currency:
type: string
proceeds:
type: number
commission:
type: number
product_duration:
type: string
product_identifier:
type: string
product_display_name:
type: string
invoice_url:
type: string
format: uri
total_amount:
type: object
properties:
tax:
type: number
gross:
type: number
currency:
type: string
proceeds:
type: number
commission:
type: number
additionalProperties: true
transformations:
- type: AddFields
fields:
- type: AddedFieldDefinition
path:
- project_id
value: "{{ stream_partition['project_id'] }}"
- type: AddFields
fields:
- type: AddedFieldDefinition
path:
- customer_id
value: "{{ stream_partition['customer_id'] }}"
incremental_sync:
type: DatetimeBasedCursor
cursor_field: issued_at
is_data_feed: false
start_datetime:
type: MinMaxDatetime
datetime: "{{ config[\"start_date\"] }}"
datetime_format: "%Y-%m-%dT%H:%M:%SZ"
datetime_format: "%ms"
cursor_datetime_formats:
- "%ms"
is_client_side_incremental: true
- type: DeclarativeStream
name: entitlements
retriever:
type: SimpleRetriever
paginator:
$ref: "#/definitions/linked/SimpleRetriever/paginator"
requester:
type: HttpRequester
url: >-
https://api.revenuecat.com/v2/projects/{{
stream_partition['project_id'] }}/entitlements
http_method: GET
authenticator:
$ref: "#/definitions/linked/HttpRequester/authenticator"
record_selector:
type: RecordSelector
extractor:
type: DpathExtractor
field_path:
- items
partition_router:
type: SubstreamPartitionRouter
parent_stream_configs:
- type: ParentStreamConfig
stream:
$ref: "#/streams/0"
parent_key: id
partition_field: project_id
primary_key:
- id
schema_loader:
type: InlineSchemaLoader
schema:
type: object
$schema: http://json-schema.org/schema#
required:
- id
- created_at
properties:
id:
type: string
object:
type:
- string
- "null"
created_at:
type: number
lookup_key:
type:
- string
- "null"
project_id:
type:
- string
- "null"
display_name:
type:
- string
- "null"
additionalProperties: true
incremental_sync:
type: DatetimeBasedCursor
cursor_field: created_at
is_data_feed: false
start_datetime:
type: MinMaxDatetime
datetime: "{{ config[\"start_date\"] }}"
datetime_format: "%Y-%m-%dT%H:%M:%SZ"
datetime_format: "%ms"
cursor_datetime_formats:
- "%ms"
is_client_side_incremental: true
- type: DeclarativeStream
name: entitlements_products
retriever:
type: SimpleRetriever
paginator:
type: DefaultPaginator
page_size_option:
type: RequestOption
field_name: limit
inject_into: request_parameter
pagination_strategy:
type: CursorPagination
page_size: 100
cursor_value: \
stop_condition: "{{ not response.get(\"next_page\", \"\") }}"
requester:
type: HttpRequester
url: >-
https://api.revenuecat.com/v2/projects/{{
stream_partition['project_id'] }}/entitlements/{{
stream_partition['entitlement_id'] }}/products
http_method: GET
authenticator:
$ref: "#/definitions/linked/HttpRequester/authenticator"
record_selector:
type: RecordSelector
extractor:
type: DpathExtractor
field_path:
- items
partition_router:
- type: SubstreamPartitionRouter
parent_stream_configs:
- type: ParentStreamConfig
stream:
$ref: "#/streams/0"
parent_key: id
partition_field: project_id
- type: SubstreamPartitionRouter
parent_stream_configs:
- type: ParentStreamConfig
stream:
$ref: "#/streams/9"
parent_key: id
partition_field: entitlement_id
primary_key:
- id
- entitlement_id
schema_loader:
type: InlineSchemaLoader
schema:
type: object
$schema: http://json-schema.org/schema#
required:
- id
- entitlement_id
- created_at
properties:
type:
type:
- string
- "null"
id:
type: string
app_id:
type:
- string
- "null"
object:
type:
- string
- "null"
created_at:
type: number
project_id:
type:
- string
- "null"
display_name:
type:
- string
- "null"
subscription:
type:
- object
- "null"
properties:
duration:
type:
- string
- "null"
trial_duration:
type:
- string
- "null"
grace_period_duration:
type:
- string
- "null"
entitlement_id:
type: string
store_identifier:
type:
- string
- "null"
additionalProperties: true
transformations:
- type: AddFields
fields:
- type: AddedFieldDefinition
path:
- entitlement_id
value: "{{ stream_partition['entitlement_id'] }}"
- type: AddFields
fields:
- type: AddedFieldDefinition
path:
- project_id
value: "{{ stream_partition['project_id'] }}"
incremental_sync:
type: DatetimeBasedCursor
cursor_field: created_at
is_data_feed: false
start_datetime:
type: MinMaxDatetime
datetime: "{{ config[\"start_date\"] }}"
datetime_format: "%Y-%m-%dT%H:%M:%SZ"
datetime_format: "%ms"
cursor_datetime_formats:
- "%ms"
is_client_side_incremental: true
- type: DeclarativeStream
name: offerings
retriever:
type: SimpleRetriever
paginator:
$ref: "#/definitions/linked/SimpleRetriever/paginator"
requester:
type: HttpRequester
url: >-
https://api.revenuecat.com/v2/projects/{{
stream_partition['project_id'] }}/offerings
http_method: GET
authenticator:
$ref: "#/definitions/linked/HttpRequester/authenticator"
record_selector:
type: RecordSelector
extractor:
type: DpathExtractor
field_path:
- items
partition_router:
type: SubstreamPartitionRouter
parent_stream_configs:
- type: ParentStreamConfig
stream:
$ref: "#/streams/0"
parent_key: id
partition_field: project_id
primary_key:
- id
schema_loader:
type: InlineSchemaLoader
schema:
type: object
$schema: http://json-schema.org/schema#
properties:
created_at:
type: number
display_name:
type:
- string
- "null"
id:
type: string
is_current:
type:
- boolean
- "null"
lookup_key:
type:
- string
- "null"
object:
type:
- string
- "null"
project_id:
type:
- string
- "null"
required:
- id
- created_at
additionalProperties: true
incremental_sync:
type: DatetimeBasedCursor
cursor_field: created_at
is_data_feed: false
start_datetime:
type: MinMaxDatetime
datetime: "{{ config[\"start_date\"] }}"
datetime_format: "%Y-%m-%dT%H:%M:%SZ"
datetime_format: "%ms"
cursor_datetime_formats:
- "%ms"
is_client_side_incremental: true
- type: DeclarativeStream
name: offerings_packages
retriever:
type: SimpleRetriever
paginator:
$ref: "#/definitions/linked/SimpleRetriever/paginator"
requester:
type: HttpRequester
url: >-
https://api.revenuecat.com/v2/projects/{{
stream_partition['project_id'] }}/offerings/{{
stream_partition['offering_id'] }}/packages
http_method: GET
authenticator:
$ref: "#/definitions/linked/HttpRequester/authenticator"
record_selector:
type: RecordSelector
extractor:
type: DpathExtractor
field_path:
- items
partition_router:
- type: SubstreamPartitionRouter
parent_stream_configs:
- type: ParentStreamConfig
stream:
$ref: "#/streams/0"
parent_key: id
partition_field: project_id
- type: SubstreamPartitionRouter
parent_stream_configs:
- type: ParentStreamConfig
stream:
$ref: "#/streams/11"
parent_key: id
partition_field: offering_id
primary_key:
- id
schema_loader:
type: InlineSchemaLoader
schema:
type: object
$schema: http://json-schema.org/schema#
properties:
created_at:
type: number
display_name:
type:
- string
- "null"
id:
type: string
lookup_key:
type:
- string
- "null"
object:
type:
- string
- "null"
position:
type:
- number
- "null"
offering_id:
type:
- string
- "null"
project_id:
type:
- string
- "null"
required:
- id
- created_at
additionalProperties: true
transformations:
- type: AddFields
fields:
- type: AddedFieldDefinition
path:
- offering_id
value: "{{ stream_partition['offering_id'] }}"
- type: AddFields
fields:
- type: AddedFieldDefinition
path:
- project_id
value: "{{ stream_partition['project_id'] }}"
incremental_sync:
type: DatetimeBasedCursor
cursor_field: created_at
is_data_feed: false
start_datetime:
type: MinMaxDatetime
datetime: "{{ config[\"start_date\"] }}"
datetime_format: "%Y-%m-%dT%H:%M:%SZ"
datetime_format: "%ms"
cursor_datetime_formats:
- "%ms"
is_client_side_incremental: true
- type: DeclarativeStream
name: offerings_packages_products
retriever:
type: SimpleRetriever
paginator:
$ref: "#/definitions/linked/SimpleRetriever/paginator"
requester:
type: HttpRequester
url: >-
https://api.revenuecat.com/v2/projects/{{
stream_partition['project_id'] }}/packages/{{
stream_partition['package_id'] }}/products
http_method: GET
authenticator:
$ref: "#/definitions/linked/HttpRequester/authenticator"
record_selector:
type: RecordSelector
extractor:
type: DpathExtractor
field_path:
- items
partition_router:
- type: SubstreamPartitionRouter
parent_stream_configs:
- type: ParentStreamConfig
stream:
$ref: "#/streams/0"
parent_key: id
partition_field: project_id
- type: SubstreamPartitionRouter
parent_stream_configs:
- type: ParentStreamConfig
stream:
$ref: "#/streams/12"
parent_key: id
partition_field: package_id
primary_key:
- id
- package_id
schema_loader:
type: InlineSchemaLoader
schema:
type: object
$schema: http://json-schema.org/schema#
properties:
eligibility_criteria:
type:
- string
- "null"
product:
type:
- object
- "null"
properties:
type:
type:
- string
- "null"
app_id:
type:
- string
- "null"
created_at:
type:
- number
- "null"
display_name:
type:
- string
- "null"
id:
type:
- string
- "null"
object:
type:
- string
- "null"
one_time:
type:
- object
- "null"
properties:
is_consumable:
type:
- boolean
- "null"
store_identifier:
type:
- string
- "null"
subscription:
type:
- object
- "null"
properties:
duration:
type:
- string
- "null"
grace_period_duration:
type:
- string
- "null"
id:
type: string
package_id:
type: string
created_at:
type: number
project_id:
type:
- string
- "null"
required:
- id
- package_id
- created_at
additionalProperties: true
transformations:
- type: AddFields
fields:
- type: AddedFieldDefinition
path:
- id
value: "{{ record['product']['id'] }}"
- type: AddFields
fields:
- type: AddedFieldDefinition
path:
- package_id
value: "{{ stream_partition['package_id'] }}"
- type: AddFields
fields:
- type: AddedFieldDefinition
path:
- created_at
value: "{{ record['product']['created_at'] }}"
- type: AddFields
fields:
- type: AddedFieldDefinition
path:
- project_id
value: "{{ stream_partition['project_id'] }}"
incremental_sync:
type: DatetimeBasedCursor
cursor_field: created_at
is_data_feed: false
start_datetime:
type: MinMaxDatetime
datetime: "{{ config[\"start_date\"] }}"
datetime_format: "%Y-%m-%dT%H:%M:%SZ"
datetime_format: "%ms"
cursor_datetime_formats:
- "%ms"
is_client_side_incremental: true
- type: DeclarativeStream
name: products
retriever:
type: SimpleRetriever
paginator:
$ref: "#/definitions/linked/SimpleRetriever/paginator"
requester:
type: HttpRequester
url: >-
https://api.revenuecat.com/v2/projects/{{
stream_partition['project_id'] }}/products
http_method: GET
authenticator:
$ref: "#/definitions/linked/HttpRequester/authenticator"
record_selector:
type: RecordSelector
extractor:
type: DpathExtractor
field_path:
- items
partition_router:
type: SubstreamPartitionRouter
parent_stream_configs:
- type: ParentStreamConfig
stream:
$ref: "#/streams/0"
parent_key: id
partition_field: project_id
primary_key:
- id
schema_loader:
type: InlineSchemaLoader
schema:
type: object
$schema: http://json-schema.org/schema#
properties:
type:
type:
- string
- "null"
app_id:
type:
- string
- "null"
created_at:
type: number
display_name:
type:
- string
- "null"
id:
type: string
object:
type:
- string
- "null"
one_time:
type:
- object
- "null"
properties:
is_consumable:
type:
- boolean
- "null"
store_identifier:
type:
- string
- "null"
subscription:
type:
- object
- "null"
properties:
duration:
type:
- string
- "null"
grace_period_duration:
type:
- string
- "null"
project_id:
type:
- string
- "null"
required:
- id
- created_at
additionalProperties: true
transformations:
- type: AddFields
fields:
- type: AddedFieldDefinition
path:
- project_id
value: "{{ stream_partition['project_id'] }}"
incremental_sync:
type: DatetimeBasedCursor
cursor_field: created_at
is_data_feed: false
start_datetime:
type: MinMaxDatetime
datetime: "{{ config[\"start_date\"] }}"
datetime_format: "%Y-%m-%dT%H:%M:%SZ"
datetime_format: "%ms"
cursor_datetime_formats:
- "%ms"
is_client_side_incremental: true

spec:
type: Spec
connection_specification:
type: object
$schema: http://json-schema.org/draft-07/schema#
required:
- api_key
- start_date
properties:
api_key:
type: string
description: API key or access token
name: api_key
order: 0
title: API Key
airbyte_secret: true
start_date:
type: string
order: 1
title: Start date
format: date-time
pattern: ^[0-9]{4}-[0-9]{2}-[0-9]{2}T[0-9]{2}:[0-9]{2}:[0-9]{2}Z$
additionalProperties: true

metadata:
assist:
docsUrl: https://www.revenuecat.com/docs/api-v2
openapiSpecUrl: https://www.revenuecat.com/docs/redocusaurus/plugin-redoc-0.yaml
testedStreams:
apps:
isStale: false
hasRecords: true
streamHash: 87baa796783c41f92cac5119fa0e450b8d5966b2
hasResponse: true
primaryKeysAreUnique: true
primaryKeysArePresent: true
responsesAreSuccessful: true
products:
isStale: false
hasRecords: true
streamHash: d367b3613968301f04cf3df6c99ecdc2928eeb4c
hasResponse: true
primaryKeysAreUnique: true
primaryKeysArePresent: true
responsesAreSuccessful: true
projects:
isStale: false
hasRecords: true
streamHash: 02d955db98f12602d52d590a065059706cf6579f
hasResponse: true
primaryKeysAreUnique: true
primaryKeysArePresent: true
responsesAreSuccessful: true
customers:
isStale: false
hasRecords: true
streamHash: f81371a1eba58a136a4af3eba90b0a9db6bb7e4e
hasResponse: true
primaryKeysAreUnique: true
primaryKeysArePresent: true
responsesAreSuccessful: true
offerings:
isStale: false
hasRecords: true
streamHash: f780fb8c86cf4e6c571f009879f501e5e50575de
hasResponse: true
primaryKeysAreUnique: true
primaryKeysArePresent: true
responsesAreSuccessful: true
entitlements:
isStale: false
hasRecords: false
streamHash: aa76dd52e06feb3bc0c04e2b45e86cdf00a596bf
hasResponse: true
primaryKeysAreUnique: true
primaryKeysArePresent: true
responsesAreSuccessful: true
metrics_overview:
isStale: false
hasRecords: true
streamHash: 9e77e45c772c41f064736ec6b15b46e68b3df46b
hasResponse: true
primaryKeysAreUnique: true
primaryKeysArePresent: true
responsesAreSuccessful: true
customers_aliases:
isStale: false
hasRecords: true
streamHash: fca30db39b73172d05e476607d18f76140e4ded8
hasResponse: true
primaryKeysAreUnique: true
primaryKeysArePresent: true
responsesAreSuccessful: true
customers_invoices:
isStale: false
hasRecords: false
streamHash: b26d5255d23a9af6c7654f7d3326f838aef9e099
hasResponse: true
primaryKeysAreUnique: true
primaryKeysArePresent: true
responsesAreSuccessful: true
offerings_packages:
isStale: false
hasRecords: true
streamHash: 4e2bd4873e5ca19260ca2526939ec806052b4ba5
hasResponse: true
primaryKeysAreUnique: true
primaryKeysArePresent: true
responsesAreSuccessful: true
customers_purchases:
isStale: false
hasRecords: false
streamHash: 7ee39999d0f7457f0214f1311c4fc22b3e7dc481
hasResponse: true
primaryKeysAreUnique: true
primaryKeysArePresent: true
responsesAreSuccessful: true
entitlements_products:
isStale: false
hasRecords: false
streamHash: 893106f7d6cbb432995b421d9e2e62c280f9dc64
hasResponse: false
primaryKeysAreUnique: false
primaryKeysArePresent: false
responsesAreSuccessful: false
customers_subscriptions:
isStale: false
hasRecords: false
streamHash: 83d41ecd19e9e5703e38571da38f3706a0c9f81c
hasResponse: true
primaryKeysAreUnique: true
primaryKeysArePresent: true
responsesAreSuccessful: true
offerings_packages_products:
isStale: false
hasRecords: true
streamHash: 0535bce5170653e9fb73e3b1505a710b7cd00cbd
hasResponse: true
primaryKeysAreUnique: true
primaryKeysArePresent: true
responsesAreSuccessful: true
customers_active_entitlements:
isStale: false
hasRecords: false
streamHash: 67f34eff0593626d961d9f71661e23bfb8e8bf31
hasResponse: true
primaryKeysAreUnique: true
primaryKeysArePresent: true
responsesAreSuccessful: true
autoImportSchema:
apps: true
products: true
projects: true
customers: true
offerings: true
entitlements: true
metrics_overview: true
customers_aliases: false
customers_invoices: false
offerings_packages: true
customers_purchases: false
entitlements_products: true
customers_subscriptions: false
offerings_packages_products: true
customers_active_entitlements: false
applied_migrations:
- migration: HttpRequesterUrlBaseToUrl
to_version: "*"
migrated_at: "2025-12-02T13:07:49.677303+00:00"
from_version: 6.36.3
- migration: HttpRequesterPathToUrl
to_version: "*"
migrated_at: "2025-12-02T13:07:49.679929+00:00"
from_version: 6.36.3

api_budget:
type: HTTPAPIBudget
policies:
- type: MovingWindowCallRatePolicy
rates:
- limit: 480
interval: PT1M
matchers:
- url_path_pattern: /customers(/|$)
- type: MovingWindowCallRatePolicy
rates:
- limit: 60
interval: PT1M
matchers:
- url_path_pattern: /(projects|apps|entitlements|offerings|packages|products)(/|$)
- type: MovingWindowCallRatePolicy
rates:
- limit: 5
interval: PT1M
matchers:
- url_path_pattern: /metrics(/|$)
status_codes_for_ratelimit_hit:
- 429

concurrency_level:
type: ConcurrencyLevel
default_concurrency: 1
```

### Relevant log output

```shell

```

### Contribute

- [ ] Yes, I want to contribute