Algolia search improvement

Prerequisites

• I have searched for duplicate or closed feature requests
• I have read the contributing guidelines

Proposal

Group all the crawlers and the indexes in one Algolia crawler.

• Editor of the crawler should be:

new Crawler({
  appId: "***",
  indexPrefix: "",
  rateLimit: 8,
  startUrls: ["https://web.unified-design-system.orange.com/orange/"],
  renderJavaScript: false,
  maxDepth: 10,
  maxUrls: null,
  schedule: null,
  sitemaps: ["https://web.unified-design-system.orange.com/sitemap-0"],
  ignoreCanonicalTo: false,
  discoveryPatterns: ["https://web.unified-design-system.orange.com/**"],
  actions: [
    {
      indexName: "ouds-web-index",
      pathsToMatch: ["https://web.unified-design-system.orange.com/**"],
      recordExtractor: ({ helpers }) => {
        return helpers.docsearch({
          recordProps: {
            lvl1: ["header h1", "article h1", "main h1", "h1", "head > title"],
            content: ["article p, article li", "main p, main li", "p, li"],
            lvl0: {
              selectors: "",
              defaultValue: "Documentation",
            },
            lvl2: ["article h2", "main h2", "h2"],
            lvl3: ["article h3", "main h3", "h3"],
            lvl4: ["article h4", "main h4", "h4"],
            lvl5: ["article h5", "main h5", "h5"],
            lvl6: ["article h6", "main h6", "h6"],
          },
          aggregateContent: true,
          recordVersion: "v3",
        });
      },
    }
  ],
  safetyChecks: { beforeIndexPublishing: { maxLostRecordsPercentage: 30 } },
  initialIndexSettings: {
    orange: {
      attributesForFaceting: ["type", "lang"],
      attributesToRetrieve: [
        "hierarchy",
        "content",
        "anchor",
        "url",
        "url_without_anchor",
        "type",
      ],
      attributesToHighlight: ["hierarchy", "content"],
      attributesToSnippet: ["content:10"],
      camelCaseAttributes: ["hierarchy", "content"],
      searchableAttributes: [
        "unordered(hierarchy.lvl0)",
        "unordered(hierarchy.lvl1)",
        "unordered(hierarchy.lvl2)",
        "unordered(hierarchy.lvl3)",
        "unordered(hierarchy.lvl4)",
        "unordered(hierarchy.lvl5)",
        "unordered(hierarchy.lvl6)",
        "content",
      ],
      distinct: true,
      attributeForDistinct: "url",
      customRanking: [
        "desc(weight.pageRank)",
        "desc(weight.level)",
        "asc(weight.position)",
      ],
      ranking: [
        "words",
        "filters",
        "typo",
        "attribute",
        "proximity",
        "exact",
        "custom",
      ],
      highlightPreTag: '<span class="algolia-docsearch-suggestion--highlight">',
      highlightPostTag: "</span>",
      minWordSizefor1Typo: 3,
      minWordSizefor2Typos: 7,
      allowTyposOnNumericTokens: false,
      minProximity: 1,
      ignorePlurals: true,
      advancedSyntax: true,
      attributeCriteriaComputedByMinProximity: true,
      removeWordsIfNoResults: "allOptional",
    },
  },
  apiKey: "***",
  exclusionPatterns: ["**migration**"],
});

• Add a new facet brand to the newly created index, and add <meta name="docsearch:brand" content={getConfig().brand} /> in the Head.astro
• Add in search.js a facetFilter with: \brand:${getBrand}``
• Add the fact to manually run a crawling while releasing a version.

Motivation and context

Avoid having too much indexes and crawler when we will have many brands to handle.

Locate the Algolia crawler configuration, then inspect Head.astro and search.js for the metadata and brand facet-filter changes. Verify that the unified index supports brand filtering and that the release process documents how to run crawling manually.