airbytehq / airbytehq/airbyte

[source-outlook] conversations stream lacks incremental sync and Full Refresh | Overwrite + Deduped creates massive duplicate records

Abierto
#83,476 3 comentarios 0 reacciones 0 asignados Ver en GitHub
area/connectors autoteam community connectors/source/outlook needs-triage team/extensibility team/use type/bug
Lenguaje dominante
Python
Estrellas
22.1k
Forks
5.3k
Métricas de merge de PR
Métricas de PR pendientes

Descripción

### Connector Name

source-outlook

### Connector Version

0.0.26

### What step the error happened?

During the sync

### Relevant information

The Microsoft Outlook source connector is creating massive duplication when syncing the conversations stream.

Although my Outlook account only contains 10–15 unique conversations, the destination table grows to 300,000–400,000 records after repeated syncs.

**Current Behavior:**
Initial sync completes successfully and imports approximately 10–15 conversations.
Every subsequent sync re-fetches the same conversations.
Even though conversationId is configured as the primary key and Overwrite + Deduped is selected, duplicate records continue accumulating.
The destination eventually contains 300,000–400,000 rows despite only 10–15 unique conversation IDs.
I reproduced the same behavior across multiple Airbyte accounts/workspaces, suggesting this is a connector-level issue rather than a workspace-specific configuration issue.
Expected Behavior

**Suspected Root Cause:**
The conversations stream emits one record per message because **$groupby: conversationId** in [its yaml manifest](https://github.com/airbytehq/airbyte/blob/a6e2c7ce/airbyte-integrations/connectors/source-outlook/manifest.yaml#L456-L470) is not a valid Microsoft Graph OData parameter on /me/messages and is silently ignored, so it never actually returns unique conversations.

### Relevant log output

```shell
3 Aug 3 Aug

Active Streams 116,116 records loaded Search

Status Stream name Latest sync in records Data fresh as of
(Sum over attempts)

Syncing conversations 116,011 extracted | 19m elapsed a few seconds ago
Syncing mailboxes 8 extracted | 19m elapsed a few seconds ago
Syncing messages 48 extracted | 19m elapsed a few seconds ago
Syncing messages_details 48 extracted | 19m elapsed a few seconds ago
Syncing profile 1 extracted | 19m elapsed a few seconds ago
```

### Contribute

- [ ] Yes, I want to contribute

---
**Internal Tracking:** https://github.com/airbytehq/oncall/issues/13232

Guía de contribución

Abrir la guía de contribución

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.