ACM-VIT / ACM-VIT/Weave

[Feed delivery] Deduplicate recommendations, track recent exposure, and keep caches coherent

Open
#59 1 comment 0 reactions 0 assignees View on GitHub
bug
Dominant language
TypeScript
Stars
4
Forks
4
PR merge metrics
No merged PRs in 30d

Description

## Problem

The backend stores generated recommendations in a per-user Redis list and appends newly generated batches when the queue becomes small:

- https://github.com/ACM-VIT/gh-social/blob/49073fc71423aa8e10334158d70ac6d41be1dbc0/backend/services/feedService.ts#L74-L123
- https://github.com/ACM-VIT/gh-social/blob/49073fc71423aa8e10334158d70ac6d41be1dbc0/backend/services/feedService.ts#L141-L203

There is no durable recent-exposure filter across replenishments. On the client, duplicate repositories receive distinct keys because the page index is part of `feedId`, and pagination continues as long as any non-empty page is returned:

- https://github.com/ACM-VIT/gh-social/blob/49073fc71423aa8e10334158d70ac6d41be1dbc0/src/app/%28tabs%29/home.tsx#L90-L100

Feedback can invalidate the ML-side cache while stale items remain in the backend Redis delivery queue.

## Proposed change

- Track recently delivered/impressed repository IDs per user with a configurable window.
- Deduplicate before enqueue, before append, and again on the client as a defensive measure.
- Invalidate or selectively prune the Redis queue after preference-changing feedback.
- Return a real cursor/has-more contract rather than a constant boolean page parameter.
- Include a stable recommendation/impression ID separate from repository ID.
- Define behavior for an exhausted eligible catalogue.

## Acceptance criteria

- A repository cannot appear twice within the configured recent-exposure window.
- Concurrent replenishment does not append duplicate repositories.
- Like/save/dislike changes affect the next delivery without waiting for the ten-minute TTL.
- Pagination terminates cleanly when no unseen items remain.
- Tests cover concurrent fetches, cache invalidation, queue replenishment, duplicate ML batches, and small catalogues.

## Cross-repo context

ML-side diversity and seen filtering is tracked in https://github.com/ACM-VIT/gh-social-ml/issues/24.

Contributor guide

Open the contributing guide

Research direction

Start with backend/services/feedService.ts at the linked queue-generation and replenishment sections, then inspect src/app/(tabs)/home.tsx pagination and feed ID handling. Review the cross-repo ML issue for related filtering requirements before defining the cursor, exposure window, cache behavior, and exhausted-catalogue contract. Done means the listed concurrency, invalidation, deduplication, pagination, and small-catalogue cases are covered by tests.

Written by the indexing model from the issue text.

Assessment

Tech stack
redis, typescript
Domain
api, backend, database, frontend
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.