Azure SDK Review - [Introduction to Azure AI Vision Face API]

New SDK Review meeting has been requested.

**Service Name**: Azure AI Vision Face API
**Review Created By**: nabilat
**Review Date**: 10/07/2024 02:05 PM PT

**Release Plan**: n/a (this is a frontend client SDK that bundles UI components and does not fit any of the existing release plans)
**Languages to review**: Swift, Kotlin, JavaScript
**Hero Scenarios Link**: [Loop containing sample code and API overview](https://microsoft-my.sharepoint.com/:fl:/p/pradiphe/EdIurV6u1vRMtgcHC5t2_fEBj5UV3RFY3WHGoBekr8vYkg?e=LNbfEb&nav=cz0lMkZwZXJzb25hbCUyRnByYWRpcGhlX21pY3Jvc29mdF9jb20mZD1iJTIxSWdzZkRaMkJBRU92ZHl2eXp1VU1IYmVlekxDV3N6SkRwSGdtV0g4OEtTR1hmMG5GMVdHa1RxTUxoTjBSTW1SSiZmPTAxV0czRktDV1NGMldWNUxXVzZSR0xNQllIQk9OWE43UFImYz0lMkYmYT1Mb29wQXBwJnA9JTQwZmx1aWR4JTJGbG9vcC1wYWdlLWNvbnRhaW5lcg%3D%3D)

**Solution Architecture Overview**:


**Core Concepts Doc Link**: [Tutorial from public documentation with core concepts explained](https://learn.microsoft.com/en-us/azure/ai-services/computer-vision/tutorials/liveness?tabs=windows%2Ccsharp%2Cmobile-kotlin)
**APIView Links**:

1. [Swift](https://apiview.dev/Assemblies/Review/346131be3ec94502a5785e548c8c6b48)
2. [JavaScript](https://apiview.dev/Assemblies/Review/20ec79cf4f3c41e0892c1755171715de/6c0e1d29fda0424d9925c0b83ba211f8)
3. [Kotlin](https://ashy-flower-0ed817510.5.azurestaticapps.net/azure-ai-vision-facelivenessdetector/com.azure.android.ai.vision.facelivenessdetector/index.html) (note: have to use a static web site instead of apiview since Kotlin is not supported on apiview yet).

**Description**:

Introduction to the Azure AI Vision Face Liveness Detection feature along with demo videos are available in our release [blog1](https://techcommunity.microsoft.com/t5/ai-azure-ai-services-blog/are-you-alive-enhancing-azure-ai-vision-face-api-with-liveness/ba-p/3984003#:~:text=The%20newly%20released%20Azure%20AI%20Vision%20Face) and [blog2](https://techcommunity.microsoft.com/t5/ai-azure-ai-services-blog/you-are-real-more-secure-identity-verification/ba-p/4146641#:~:text=The%20integration%20of%20liveness%20detection).

This feature relies on the frontend application to generate signals from the camera’s field of view to determine whether the person in front of it is real or spoofed. Currently, we use techniques like flashing the screen, asking the user to smile, and having them move their head in random directions to collect the necessary data, which is then processed by AI models on the backend for classification.

In previous SDK versions, the UI was not bundled into it, so developers had to build it themselves by following our sample code. However, we’ve seen a lot of variation in how the UI was implemented, which has often led to issues with the classification accuracy. Each time, we’ve had to request access to the developer’s app for debugging, which isn’t a scalable solution. Moreover, as we’ve added more complex UI flows that are closely tied to our algorithms, it’s become increasingly difficult for developers to implement these correctly.

To address this, we’ve decided to bundle the UI directly into our SDK. This reduces the complexity of the API and makes it easier and more efficient for developers to integrate. This approach also aligns with what our competitors are offering, as they provide SDKs with built-in UI as well.

Detailed meeting information and documents provided can be accessed [here](https://web.powerapps.com/apps/f0a98ca4-117c-4d4b-a06a-3e2cdfe46753?view=edit&event=2b4f8302-4880-ef11-ac21-00224804e9b4)

Start with the linked APIView reviews for Swift and JavaScript and the Kotlin API documentation, then read the hero scenarios and core concepts links. Done means completing the Azure AI Vision Face API SDK review across Swift, Kotlin, and JavaScript, including the bundled UI and documented review findings.

Written by the indexing model from the issue text.