OWASP / OWASP/Nest

Implement Scalable Search API for Domain and Taxonomy Autocomplete

Open
#3,527 3 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Python
Stars
451
Forks
707
Avg merge
22h 59m
Merged PRs (30d)
91

Description

### Problem Statement
The current method for selecting domains, taxonomies, and categories lacks a dedicated backend-driven search mechanism. As the dataset for mentorship modules grows, relying on frontend-side filtering or loading full lists into the client will cause performance bottlenecks and unnecessary payload bloat.

To maintain a professional architecture, the "search responsibility" needs to be shifted from the frontend to a high-performance backend service.

### Proposed Solution
I plan to implement a backend search service that provides a dedicated endpoint for real-time taxonomy and category lookups. This ensures the system remains responsive regardless of how much metadata is added to the portal.

**Backend Engineering (Core Focus):**
* **Optimized API Endpoint:** Develop a GET endpoint supporting `q` parameters for partial string matching using Django's `icontains` logic.
* **Query Optimization:** Utilize `.only()` or `.values()` to minimize database I/O and ensure JSON responses are lightweight.
* **Service Standardization:** Design the logic to be modular so it can be reused for any searchable categorization across the portal.
* **Logic Validation:** Implement unit tests to verify search accuracy, handling of special characters, and empty query states.

**Frontend Integration (Minimal):**
* Replace static selection logic with a debounced autocomplete field that consumes the new API. This keeps the frontend "dumb" and ensures it only handles data display while the backend manages the logic.

### Impact
* **Scalability:** Prepares the system for large-scale data without degrading the user experience.
* **Performance:** Reduces initial page load times by fetching data only when needed.
* **Code Quality:** Centralizes business logic within the backend serializers and services.

### Tasks
- [ ] Audit existing Taxonomy and Category model relationships.
- [ ] Implement the Backend Search ViewSet and URL routing.
- [ ] Apply database query optimizations for low-latency results.
- [ ] Add unit tests for the new search service logic.
- [ ] Connect the frontend autocomplete component to the new endpoint.

#### Are you going to work on this?
- [x] Yes
- [ ] No

Reference - #2701

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.