Azure / Azure/azure-search-vector-samples

Entity recognizer skill doesn't extracts output.

Open
#183 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
910
Forks
377
PR merge metrics
No merged PRs in 30d

Description

I have tried using EntityRecognizer skill to extract the entities combined with SplitText, EmbeddingSkill.

On the output I am not getting extracted entities, it is showing null values for people, skills, locations etc.

PFB code:
fields = [
SearchField(name="parent_id", type=SearchFieldDataType.String, sortable=True, filterable=True, facetable=True),
SearchField(name="chunk_id", type=SearchFieldDataType.String, key=True, sortable=True, filterable=True, facetable=True, analyzer_name="keyword"),
SearchField(name="chunk", type=SearchFieldDataType.String, sortable=False, filterable=False, facetable=False),
SearchField(name="vector", type=SearchFieldDataType.Collection(SearchFieldDataType.Single), vector_search_dimensions=1536, vector_search_profile_name="myHnswProfile"),
SearchField(name="title", type=SearchFieldDataType.String),
SearchField(name="url", type=SearchFieldDataType.String),
SearchField(name="last_modified", type=SearchFieldDataType.DateTimeOffset),
SearchField(name="people", type=SearchFieldDataType.String),
SearchField(name="skills", type=SearchFieldDataType.String)
]

entity_skill=EntityRecognitionSkill(
description="Skill Used to detect entities from the document",
context= "/document/content/",
categories=["Person",
"Email",
"Location",
"Organization",
"PhoneNumber",
"Address",
"Skill"
],
inputs=[
InputFieldMappingEntry(name="text", source="/document/content"),
InputFieldMappingEntry(name="languageCode", source="/document/language")
],
outputs=[
OutputFieldMappingEntry(name="persons", target_name="people"),
OutputFieldMappingEntry(name="skills", target_name="skills"),
],
)

index_projections = SearchIndexerIndexProjections(
selectors=[
SearchIndexerIndexProjectionSelector(
target_index_name=index_name,
parent_key_field_name="parent_id",
source_context="/document/pages/*",
mappings=[
InputFieldMappingEntry(name="chunk", source="/document/pages/*"),
InputFieldMappingEntry(name="vector", source="/document/pages/*/vector"),
InputFieldMappingEntry(name="title", source="/document/metadata_storage_name"),
InputFieldMappingEntry(name="people", source="/document/content/people"),
InputFieldMappingEntry(name="skills", source="/document/content/skills")
],
),
],
parameters=SearchIndexerIndexProjectionsParameters(
projection_mode=IndexProjectionMode.SKIP_INDEXING_PARENT_DOCUMENTS
)

indexer = SearchIndexer(
name=indexer_name,
description="Indexer to index documents and generate embeddings",
skillset_name=skillset_name,
target_index_name=index_name,
data_source_name=data_source.name,
# Map the metadata_storage_name field to the title field in the index to display the PDF title in the search results
field_mappings=[FieldMapping(source_field_name="metadata_storage_name", target_field_name="title"),
FieldMapping(source_field_name="metadata_storage_path", target_field_name="url"),
FieldMapping(source_field_name="metadata_storage_last_modified", target_field_name="last_modified")
] ,
output_field_mappings=[FieldMapping(source_field_name="/document/content/people", target_field_name="people"),
FieldMapping(source_field_name="/document/content/skills", target_field_name="skills")]
)

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by tracing the EntityRecognitionSkill output paths through the index projection selectors and the indexer's output_field_mappings. Verify whether the people and skills outputs are produced under the skill's context and whether those paths match the projected page context. Done means the indexed people and skills fields contain extracted values instead of null.

Written by the indexing model from the issue text.

Assessment

Tech stack
azure, python
Domain
backend, search
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.