Setting Up Kerberos authentication for Shinydocs Pro

What is Kerberos?

Kerberos is a network authentication protocol designed to provide strong authentication for client/server applications. Shinydocs Pro and Shinydocs Enterprise Search use Kestrel and Windows Authentication with Kerberos to authenticate a user from their browser.

The following guide walks through how to set up a Windows Server environment to support Kerberos pass-through (2 hop authentication).

Setup

Pre-Requisites

In order to setup Kestrel to authenticate using Kerberos, the process needs to run as a named user from the domain controller. This can be either a regular user account on the domain, or a group managed service account (For more information, see: Group Managed Service Accounts Overview | Microsoft Learn)

Setting the spns

Open a command prompt or powershell prompt as a user that has the ability to modify spns. An active directory admin has the correct rights, but the right can be delegated by a domain admin to a user that has lesser rights.
Determine the hostnames of the systems that Shinydocs Enterprise Search will be running on. This may be one machine or multiple machines, if running a load balanced cluster. If running a load balanced cluster, ensure that you include the DNS name that will be used for the load balancer in the list of host names.

e.g: When running a single instance, the host name may be search.example.local

e.g. When running a clustered instance, the host names may be search01.example.local, search02.example.local and search.example.local for the load balancer DNS entry.

Run the following commands to add the spns to the service account created above for each of the hostnames. Note, if using a group managed service account, the account name will always end with a $.

CODE

setspn -S HTTP/<hostname> <domain>\<serviceAccount>
setspn -S HTTP/<fqdn> <domain>\<serviceAccount>

e.g: single instance with the service account called searchService on the domain example
setspn -S HTTP/search example\searchService
setspn -S HTTP/search.example.local example\searchService

e.g: load balanced instance with the service account called searchService on the domain example
setspn -S HTTP/search01 example\searchService
setspn -S HTTP/search01.example.local example\searchService
setspn -S HTTP/search02 example\searchService
setspn -S HTTP/search02.example.local example\searchService
setspn -S HTTP/search example\searchService
setspn -S HTTP/search.example.local example\searchService

Verify that the spns were set correctly by running the command
CODE
```
setspn -l <domain>\<serviceAccount>
```
Shinydocs Pro requires that you run the Shinydocs Search windows service. As a result, Kestrel needs to be configured. Please refer to the Kestrel Configuration section below.

Kestrel Configuration

Ensure the pre-requisites (above) have been met prior to configuring Kestrel with the service account.

Open the Services panel on the servers that search is being configured on.
Navigate to the Shinydocs Search service.
Open the properties for the Shinydocs Search service.
Select the Log On tab for the service.
Change the radio button from Local System account to This account:
Enter the username and password for the service account. Note, if you are using the group managed service account, enter the domain\serviceAccount followed by a $, and blank out the password fields.
Click Apply.
You may be prompted that the user has been given Logon as a Service right.
Start / Restart the service for the login user to take effect.
Attempt to login to the website from another machine to ensure kerberos authentication is working.

Source Specific Settings

OpenText Content Server

Active Directory User Setup

Find the user configured for the Shinydocs application in Active Directory Users and Computers
- Right click on the user and select Properties
- Select the Delegation tab
- Select Trust this user for delegation to the specified services only
- Select Use Kerberos only
- Select Add -> new window pops up
- Select Users or Computers
- Search for the configured content server user above. It will show you the service type, domain name for the spn set on the content server user
- Select the spns set for the content server user above, and click Add.
- Select Apply to save the settings.

Authorization to OTDS

Ensure there is a user account created for Content Server / OTDS on the domain controller
Ensure OTDS is running from the user account mentioned above. Either tomcat (External OTDS) or Content Server (internal otds) needs to run as the content server user created above.
Set spns for the host name(s) that the content server is running on. If using a load balancer, ensure the spns are set for all machines in the cluster, as well as the load balancer DNS entry for the virtual ip.

setspn -S HTTP/otdsHostName domain\contentserveruser

setspn -S HTTP/otdsHostName.fqdn.com domain\contentserveruser

setspn -S HTTP/otdsHostName2 domain\contentserveruser

setspn -S HTTP/otdsHostName2.fqdn.com domain\contentserveruser

setspn -S HTTP/otdsloadbalancer domain\contentserveruser

setspn -S HTTP/otdsloadbalancer.fqdn.com domain\contentserveruser

File Share: Authorization

You must authorize the discoveryserverHostName as Principals that the FileShareServerName can delegate to.
To do this, check to see if there is any existing authorizations already set for the FileShareServerName using this command in PowerShell (as an Administrator). Any existing authorizations must be put into the next query otherwise they will get deleted when the new authorizations are set.
CODE
```
Get-ADComputer FileShareServerName -Properties PrincipalsAllowedToDelegateToAccount
```
Next, set the discoveryserverHostName as well as any results returned from the above query as Principals that the FileShareServerName can delegate to with the following command. If no results are returned from the above query then you only need to pass into the below query the discoveryserverHostName.
CODE
```
$Server1 = Get-ADComputer -Identity discoveryserverHostName
$Server2 = Get-ADComputer -Identity optioncomputer1fromabovequery
$Server3 = Get-ADComputer -Identity optioncomputer2fromabovequery
Set-ADComputer FileShareServerName -PrincipalsAllowedToDelegateToAccount @($Server1, $Server2, $Server3)
```
Note: If Shinydocs Enterprise Search’s discovery server is load balanced, then all nodes in the load balanced cluster must be added.

Initial Testing

After configuring the above settings, you should test from your desktop and search for documents that you know you already have access to. From your desktop, you should be seamlessly authenticated to the Shinydocs Enterprise Search server and be presented the default search screen. In the upper right hand corner you should see that you are logged in as your network account and not be presented with an authentication dialogue.

If you are not logged in automatically, check the following items first:

You are logged into your computer as a network user that is on the same network as the Shinydocs Enterprise Search server.
For the Enterprise Search server URL, you are using either the computer name, fully qualified computer name or DNS entry name and that there is an HTTP SPN registered for the service account running Kestrel or IIS for this name (you cannot use localhost if you are on the server directly)
The name used above is part of your trusted sites. To check this, go to Internet Options in the Control Panel and under the Security Tab, select Local Intranet and then click on sites and click on Advanced and ensure that the Enterprise Search server either added or matches one of the wild card entries in the list
If you are still prompted to login, ensure that you prefix the username with the domain you are logging into ( in the format of <DOMAIN>\<USER NAME>)

Troubleshooting

Kerberos logging can be enabled by editing the following registry key:

CODE

 HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Lsa\Kerberos\Parameters

Registry Value: LogLevel

Value Type: REG_DWORD

Value Data: 0x1

If the Parameters subkey does not exist, create it.

Further details at:https://docs.microsoft.com/en-us/troubleshoot/windows-server/identity/enable-kerberos-event-logging
The logging events will show in the System event log, in the windows event viewer.
Ensure that the OTDS server is running as the content server user account created above, and negotiate authentication is enabled within otds.
Review the OTDS logs, otds.log, directory_access.log for any errors during authentication.
Ensure the spns have been replicated though the domain.