Understanding AEM Smart Tags & Confidence Scores

Ankur Ahlawat • Aug 18, 2022

About the Author: Ankur Ahlawat is a Technical Architect for Hoodoo Digital (now Rightpoint). He has over 12 years of industry experience - 7 of which have been working directly with Adobe Experience Manager. Ankur maintains his own technical blog AEM CQ5 Tutorials and also contributes content to the Adobe Experience League. He has a number of AEM certifications, including AEM 6 Certified Developer, AEM 6 Certified Lead Developer, AEM 6 Certified Architect, Adobe Workfront Project manager Certified Professional, and AWS Certified Cloud Practitioner. Don't worry, he doesn't spend all his time working. Ankur is also an avid runner and loves to play tennis.



Confidence Score is one of the key criteria for deciding which assets need to be shown to end users based on your search query in Adobe Experience Manager (AEM). Assets tagged with high Confidence Scores are displayed first followed by assets tagged with a low Confidence Score. 


This article will discuss the following: 

  • Why AEM Smart tagging is required 
  • Understand how AEM search behaves with Smart Tags. 
  • Update Confidence Score for Smart Tags. 

 

Why Smart Tagging is Required:  

Enterprise organizations deal with enormous amounts of digital assets, and these assets only grow over time. Given this continuous growth, it’s important that digital assets have proper metadata and tags; without this information, assets become dark assets, rendering them unusable. 


Dark Assets are assets uploaded to your Digital Asset Manager (DAM) without any metadata, or without the normal metadata your organization uses to find assets. Without this information, these assets cannot be searched or located by your internal teams, rendering them useless to your organization as a whole. 


However, manually tagging every asset can be inefficient and time-consuming. To resolve this problem with dark assets, AEM provides automated smart tagging for assets, using Adobe’s Artificial Intelligence (AI) platform, Sensei. Automated smart tagging with Sensei allows businesses to improve the overall experience for their business teams with minimal effort. 


However, this does come with some caveats. Smart Tagging is a one-way process, in which we cannot send feedback to Adobe Sensei to avoid specific tags on similar assets. Sensei learns using assets uploaded to Adobe Stock Photography, not necessarily by the content uploaded by your business specifically. That said, developers can curate automatically applied Smart Tags to avoid inaccurate tagging, or they can create customer-specific tags with Enhanced Smart Tagging. 


Understanding AEM search with Smart Tags: 

By default, AEM omni-search combines the search terms with an AND clause. Using Smart Tags does not change this default behavior. However, Smart Tags also add an OR clause to find any of the search terms in the applied Smart Tags. 


For example, consider searching for man jumping. When the system does not use Smart Tags, assets with just man or just jumping keywords in the metadata will not appear in the search results by default. However, using Smart Tags, an asset that is tagged with either man or jumping will appear in the search query. So, the search results are a combination of: 


  • Assets with man and jumping keywords in the metadata. 
  • Assets smart tagged with either of the keywords man or jumping

 

Order of Precedence how search results are evaluated:  

The search results that match all search terms in metadata fields are displayed first, followed by the search results that match any of the search terms in the Smart Tags. In the above example, the approximate order of display of search results is: 

  • Matches man and jumping term in various metadata fields. 
  • Matches man and jumping term in Smart Tags. 
  • Matches man or jumping term in Smart Tags. 

 

Understand Confidence Score for Smart Tags: 

Confidence Score, in simple terms, means how confident Adobe Smart Tag Sensei is about tagging a specific tag to the asset. A higher Confidence score means Sensei is highly confident that this tag is most relevant for the asset. 


AEM applies a minimum confidence threshold for Smart Tags to avoid having too many tags for each asset (which slows down indexing). Your asset search results are ranked based on the Confidence Scores; this helps improve search results by placing more weight on specific tags for that asset, meaning that rather than viewing all asset tags as equally relevant, weighted tags will be more relevant. For example, an image of a bowl of strawberries might be tagged with strawberries, fruit, berries, and food. Smart Tags would apply a higher Confidence Score to strawberries than it would to food. This helps improve search results beyond what a standard inspection of the assigned tags would be able to deliver. Inaccurate tags often have low Confidence Scores, so they seldom appear at the top of the Smart Tags list for assets. 


The default value for Smart Tag for an image asset is 0.5 and for a video asset is 0.7 (It should be between 0 and 1). The default value might not hold good for all customers, and as an Architect, you should discuss it with business users to understand what threshold is appropriate for your customer. 


Note:  Manual Tags are always assigned with 100% (maximum Confidence Score). Hence, if any manual tag matches the search term it will be listed at the top of the search results. 

 

Update Smart Tag OSGI config for updating Confidence Score: 

To update Confidence Score OSGI Config to the project deployed to Adobe Experience Manager as a Cloud Service through Cloud Manager. Follow the steps listed below: 

Note: Please add/update only those JSON properties that you wish to update in AEMasCloud. For example, if you want to update Confidence Score value only for images then below JSON is enough. 

To validate your changes to the local environment after code deployment, navigate to Felix console, and search for OSGI configuration "Adobe CQ Sensei SDK (Software Development Kit) Service" as shown below. 

To update Confidence Score OSGI Config to the project deployed to Adobe Experience Manager 6.5 or below, include a config file named com.day.cq.dam.similaritysearch.internal.impl.SimilaritySearchServiceImpl with the following contents: 

Options available to configure AEM 6.5 and the instances below are:  

  • max.tags: Maximum number of tags. The default value is 25. 
  • min.tag.confidence: Minimum Confidence Score below which tags will not be added. The default value is 0.5 
  • training.namespace: Separate training namespace for storing tags under predictedTags node. For Ex: - "custom_namespace.tag_name1" 

 

Testing Smart tag Confidence Score on Development or Higher environment: 

Because Smart tag service integration is not available in local development environment. We need to deploy our code to a higher environment to complete our final testing. 


Below screenshot states applying Smart Tags prior to updating Confidence Score: 

Below, the screenshot shows applying Smart Tags after updating Confidence Score: 

As you can see now, the number of Smart Tags applied to an asset is reduced and asset is automatically tagged with only the higher Confidence Score tag. 


Note: Do not keep Confidence Score too high because Smart Tag service is still in the learning phase and continuously evolving; if it is not able to identify an asset then it will tag it with low Confidence Score and due to the high threshold, that smart tag will not be applied, and we will eventually face the same dark asset issue again. 


Hoodoo The Next Evolution: Rightpoint
16 Mar, 2023
Hoodoo is now Rightpoint, and we couldn’t be more excited to have a new name, a new look, and new capabilities.
By Kim Melton 29 Nov, 2022
Google is sunsetting Google Analytics - and a lot of people are left wondering what to do next. Don't worry - we have a plan (and a team) that can help.
By Sara Wetmore 22 Nov, 2022
A recent Forrester report evaluated enterprise marketing software - from Adobe to SalesForce and more. Find out how Adobe fared against their competitors across 25 different categories.
Show More
Share by: