Cognitive Toolkit May 2022/2.7.1 Release Notes

Product: Cognitive Toolkit

Version: 2.7.1

Date: May 2, 2022

• Content Server Delta Option: 

  • --delta for scanning/crawling your Content Server has been added to better detect file changes for delta scans/crawls in OpenText Content Server

    • Use the --delta argument after you have done a complete scan/crawl of your data.

    • The delta option is only compatible with database credentials

    • Supports SQL Server and Oracle databases at this time

    • Detected changes will be marked with either outdated-binary: true and/or outdated-meta: true depending on the situation

      • outdated-binary: true will be set for files that have been modified

      • outdated-meta: true will be set for files whose metadata has changed (ex. categories & attributes, classification)

      • Detecting renamed classifications is not supported at this time

    • The delta option uses the Content Server audit table DAUDITNEWCORE to detect changes based on the settings in the admin panel (Event Auditing → Set Audit Interests). A list of events that we support can be found here

    • The delta option also uses the --modified-after date/time with a default of the current day (12:00a - 11:59:59p)

• FileSystem Tag-Updates Option

  • The --tag-updates option for the filesystem crawl is used to detect any document changes when scanning

    • Note: If you have an index that was crawled with a version of Cognitive Toolkit < v2.6.0, all files crawled previously will be tagged with outdated-binary: true due to the new field created in this version called crawlableId. To resolve this issue:

      • Run CrawlFileSystem on Cognitive Toolkit 2.7.1 without --tag-updates

      • After, run CrawlFileSystem on Cognitive Toolkit 2.7.1 with --tag-updates

    • Use the --tag-updates argument after you have done a complete scan/crawl of your data.

    • Detected changes will be marked with outdated-binary: true

    • LastAccessTimeUTC will no longer be tracked

    • Optimized to run without the --after-last-modified option

    • Running CrawlFileSystem --tag-updates on indexes that were scanned with versions prior to 2.7.1 will cause false-positive tagging. Please run WITHOUT --tag-updates for the first scan to ensure the schema is up to date.