Configure Shinydocs Pro Control Center access for Microsoft Sources (OneDrive, Teams, SharePoint Online, Exchange Online)

By integrating with Microsoft Azure, the Shinydocs Pro software can operate with Microsoft SharePoint Online and/or Exchange Online — maintaining secure and compliant access to this cloud application with policy-based access controls.

The following document describes how to enable Azure authentication for Microsoft SharePoint Online.

Note that this process does require certificates, which are used by Azure to prove the Shinydocs application’s identity when requesting a token. You need two files, a .cer file with the public key which you upload to Azure, and a .pfx file with the private key that you add to the Shinydocs software.

These files are often provided by an organization’s IT or Network team.

A self-signed certificate can be used but is not advised depending on your organization’s infrastructure and security policies.

For the Microsoft PowerShell script to create a self-signed certificate, please visit https://learn.microsoft.com/en-us/sharepoint/dev/solution-guidance/security-apponly-azuread

Requirements

Internet access to reach Microsoft Online
- If the server cannot have direct internet access, ensure firewall rules are setup to allow HTTPS traffic to the following on port 443:
  - login.microsoftonline.com
  - http://graph.microsoft.com
Azure/Entra permission to create application registrations and grant admin consent
A certificate used to authenticate (Azure requires the .cer and Shinydocs Pro requires the .pfx)

Create the self signed certificate for Azure

If you already have a process internally for creating these certificates, please follow your organizations best practices. If not, you can use the script below.

PowerShell script for certificate generation

This script is from Microsoft (https://learn.microsoft.com/en-us/sharepoint/dev/solution-guidance/security-apponly-azuread)

Save this script in notepad as GenerateCertificate.ps1

POWERSHELL

#Requires -RunAsAdministrator
<#
.SYNOPSIS
Creates a Self Signed Certificate for use in server to server authentication
.DESCRIPTION
.EXAMPLE
.\Create-SelfSignedCertificate.ps1 -CommonName "MyCert" -StartDate 2015-11-21 -EndDate 2017-11-21
This will create a new self signed certificate with the common name "CN=MyCert". During creation you will be asked to provide a password to protect the private key.
.EXAMPLE
.\Create-SelfSignedCertificate.ps1 -CommonName "MyCert" -StartDate 2015-11-21 -EndDate 2017-11-21 -Password (ConvertTo-SecureString -String "MyPassword" -AsPlainText -Force)
This will create a new self signed certificate with the common name "CN=MyCert". The password as specified in the Password parameter will be used to protect the private key
.EXAMPLE
.\Create-SelfSignedCertificate.ps1 -CommonName "MyCert" -StartDate 2015-11-21 -EndDate 2017-11-21 -Force
This will create a new self signed certificate with the common name "CN=MyCert". During creation you will be asked to provide a password to protect the private key. If there is already a certificate with the common name you specified, it will be removed first.
#>
Param(

[Parameter(Mandatory=$true)]
   [string]$CommonName,

[Parameter(Mandatory=$true)]
   [DateTime]$StartDate,

[Parameter(Mandatory=$true)]
   [DateTime]$EndDate,

[Parameter(Mandatory=$false, HelpMessage="Will overwrite existing certificates")]
   [Switch]$Force,

[Parameter(Mandatory=$false)]
   [SecureString]$Password
)

# DO NOT MODIFY BELOW

function CreateSelfSignedCertificate(){

#Remove and existing certificates with the same common name from personal and root stores
    #Need to be very wary of this as could break something
    if($CommonName.ToLower().StartsWith("cn="))
    {
        # Remove CN from common name
        $CommonName = $CommonName.Substring(3)
    }
    $certs = Get-ChildItem -Path Cert:\LocalMachine\my | Where-Object{$_.Subject -eq "CN=$CommonName"}
    if($certs -ne $null -and $certs.Length -gt 0)
    {
        if($Force)
        {

foreach($c in $certs)
            {
                remove-item $c.PSPath
            }
        } else {
            Write-Host -ForegroundColor Red "One or more certificates with the same common name (CN=$CommonName) are already located in the local certificate store. Use -Force to remove them";
            return $false
        }
    }

$name = new-object -com "X509Enrollment.CX500DistinguishedName.1"
    $name.Encode("CN=$CommonName", 0)

$key = new-object -com "X509Enrollment.CX509PrivateKey.1"
    $key.ProviderName = "Microsoft RSA SChannel Cryptographic Provider"
    $key.KeySpec = 1
    $key.Length = 2048
    $key.SecurityDescriptor = "D:PAI(A;;0xd01f01ff;;;SY)(A;;0xd01f01ff;;;BA)(A;;0x80120089;;;NS)"
    $key.MachineContext = 1
    $key.ExportPolicy = 1 # This is required to allow the private key to be exported
    $key.Create()

$serverauthoid = new-object -com "X509Enrollment.CObjectId.1"
    $serverauthoid.InitializeFromValue("1.3.6.1.5.5.7.3.1") # Server Authentication
    $ekuoids = new-object -com "X509Enrollment.CObjectIds.1"
    $ekuoids.add($serverauthoid)
    $ekuext = new-object -com "X509Enrollment.CX509ExtensionEnhancedKeyUsage.1"
    $ekuext.InitializeEncode($ekuoids)

$cert = new-object -com "X509Enrollment.CX509CertificateRequestCertificate.1"
    $cert.InitializeFromPrivateKey(2, $key, "")
    $cert.Subject = $name
    $cert.Issuer = $cert.Subject
    $cert.NotBefore = $StartDate
    $cert.NotAfter = $EndDate
    $cert.X509Extensions.Add($ekuext)
    $cert.Encode()

$enrollment = new-object -com "X509Enrollment.CX509Enrollment.1"
    $enrollment.InitializeFromRequest($cert)
    $certdata = $enrollment.CreateRequest(0)
    $enrollment.InstallResponse(2, $certdata, 0, "")
    return $true
}

function ExportPFXFile()
{
    if($CommonName.ToLower().StartsWith("cn="))
    {
        # Remove CN from common name
        $CommonName = $CommonName.Substring(3)
    }
    if($Password -eq $null)
    {
        $Password = Read-Host -Prompt "Enter Password to protect private key" -AsSecureString
    }
    $cert = Get-ChildItem -Path Cert:\LocalMachine\my | where-object{$_.Subject -eq "CN=$CommonName"}

Export-PfxCertificate -Cert $cert -Password $Password -FilePath "$($CommonName).pfx"
    Export-Certificate -Cert $cert -Type CERT -FilePath "$CommonName.cer"
}

function RemoveCertsFromStore()
{
    # Once the certificates have been been exported we can safely remove them from the store
    if($CommonName.ToLower().StartsWith("cn="))
    {
        # Remove CN from common name
        $CommonName = $CommonName.Substring(3)
    }
    $certs = Get-ChildItem -Path Cert:\LocalMachine\my | Where-Object{$_.Subject -eq "CN=$CommonName"}
    foreach($c in $certs)
    {
        remove-item $c.PSPath
    }
}

if(CreateSelfSignedCertificate)
{
    ExportPFXFile
    RemoveCertsFromStore
}

Run the script:

Open PowerShell as administrator.
Navigate to the directory you saved the script to. (cd)
Run the script by entering .\GenerateCertificate.ps1
Follow the prompts in the console to generate the certificate for Azure

Register Shinydocs Software with Azure

The first step of the process is to register your Shinydocs software with Azure, so that it can access SharePoint Online or Exchange Online data.

You only need to register one application for Shinydocs software. Permissions to supported repositories can be configured the one registration.

Open Microsoft Azure.
In the upper left corner dropdown menu, navigate to Azure Active Directory.
From the sidebar menu, select App registrations
Select + New registration
Enter a name for the application. In this case, we used “Shinydocs Pro” as the application name.
Select who, within your organization, can use or access the application. In most cases, the first option (Single tenant) will be selected (see below).
The optional Redirect URl is not needed in this case.
Select Register to continue.

Upload Authentication Certificate to Azure

These steps require a .cer file.

From the sidebar menu, select Certificates & secrets
Select Certificates
Select Upload certificate
Select the folder icon to browse for your certificate (.cer). Optionally, add a description to let other administrators know what this certificate is used for.
Select Add
Verify that the certificate was uploaded successfully by confirming the certificate is shown Certificates
Still in Certificates & secrets, click the Client secrets tab.
Click + New client secret
1. In the Add a client secret panel, add a description of your choice
2. Set it to expire in 12 months or longer.
  This is a good time to schedule a reminder for the expiry, as Shinydocs Pro will also need the new secret key when this one expires.
3. Click Add.
4. Copy the secret key to a safe location, as you will not be able to retrieve it later.
In the left-hand menu, open Overview
Note/save the following information for configuring Shinydocs Pro in later steps.

Application (client) ID: ___________________________________________________
Directory (tenant) ID: ___________________________________________________
Secret key: ___________________________________________________

Authentication

Still in the Azure portal Shinydocs application registration, now you will set up the redirect URIs for authentication.

Navigate to the Authentication tab in the application registration
Web > Redirect URIs
1. This set up will be for the server running Shinydocs Pro Search using it’s hostname. You can update these settings later when/if you set up your fully qualified domain name in your DNS.
2. Click Add URI, and add the following
  1. https://<shinydocs pro server host name>/azuread/success
    1. e.g. https://shinyvm/azuread/success
  2. https://<shinydocs pro server host name>/api/v1/oidc/signin-callback
    1. e.g.https://shinyvm/api/v1/oidc/signin-callback
  3. https://localhost/azuread/success
  4. https://localhost/api/v1/oidc/signin-callback

Source-specific permissions

If you connect to more than one Microsoft content source, there will be overlapping permissions due to the design of SharePoint, Teams, etc.

Your Shinydocs Control Center app registration only needs one copy of the permission.

Permission to access SharePoint Online Content

Permissions for SharePoint Online Content

Now that the Shinydocs Application has been registered with Azure, it’s time to apply permissions to access content within SharePoint Online.

From the sidebar menu, select API permissions
Select + Add a permission
Select Microsoft Graph:
Then, select Delegated permissions
Add the following delegated permissions
1. User.Read
  Type: Delegated
2. offline_acccess
  Type: Delegated
3. openid
  Type: Delegated
4. profile
  Type: Delegated
5. Sites.Read.All
  Type: Delegated
Click Add permissions
Back in API permissions page, click + Add a permission again. Select SharePoint:
Then select Delegated permissions
Add the following permission:
1. Allsites.Read
  Type: Delegated
Click Add permissions
Now, we are going to set application level permission. Back in API permissions page, click + Add a permission again. Select SharePoint
Then, select Application permissions
Add the following permission:
Sites.Read.All
Type: Application
Next, select Grant admin consent for [Tenant Name].
Select Yes to grant consent for the requested permissions for all accounts in [Directory Name].
At the top of the page, there will be a notification that admin consent for the requested permissions was successfully granted.

Permission to access Exchange Online

Permissions for Exchange Online

Now that the Shinydocs Application has been registered with Azure, it’s time to apply permissions to access content within Exchange Online.

From the sidebar menu, select API permissions
Select + Add a permission
Select Microsoft Graph:
Select Application permissions
Add the following Application permissions:
1. User.Read.All
  Type: Application
2. Mail.Read
  Type: Application
Click Add permissions at the bottom of the page.
The API permissions should now be updated to include Graph.
Click Add permissions again > Graph > Delegated
Add the following Delegated permissions:
1. Directory.AccessAsUser.All
  Type: Delegated
2. email
  Type: Delegated
3. EWS.AccessAsUser.All
  Type: Delegated
4. Files.Read.All
  Type: Delegated
5. Group.Read.All
  Type: Delegated
6. Mail.Read.All
  Type: Delegated
7. Mail.Read.Shared
  Type: Delegated
8. offline_access
  Type: Delegated
9. openid
  Type: Delegated
10. profile
  Type: Delegated
11. User.Read
  Type: Delegated
Click Add permissions at the bottom of the page.
Next, select Grant admin consent for [Tenant Name].
Select Yes to grant consent for the requested permissions for all accounts in [Directory Name].
At the top of the page, there will be a notification that admin consent for the requested permissions was successfully granted.

Permission to access Teams

Permissions for Teams

Now that the Shinydocs Application has been registered with Azure, it’s time to apply permissions to access content within Teams.

From the sidebar menu, select API permissions
Select + Add a permission
Select Microsoft Graph:
Select Application permissions
Add the following Application permissions:
1. Channel.ReadBasic.All
  Type: Application
2. ChannelMember.Read.All
  Type: Application
3. ChannelMessage.Read.All
  Type: Application
4. ChannelSettings.Read.All
  Type: Application
5. Chat.Read.All
  Type: Application
6. Chat.ReadBasic.All
  Type: Application
7. ChatMessage.Read.All
  Type: Application
8. Files.Read.All
  Type: Application
9. Team.ReadBasic.All
  Type: Application
10. User.Read.All
  Type: Application
Click Add permissions at the bottom of the page.
Click + Add a permission again > Graph > Delegated
Add the following Delegated permissions:
1. offline_access
  Type: Delegation
2. openid
  Type: Delegation
3. profile
  Type: Delegation
4. User.Read
  Type: Delegation
5. Files.Read.All
  Type: Delegation
6. Sites.Read.All
  Type: Delegation
7. Directory.Read.All
  Type: Delegation
8. Channel.ReadBasic.All
  Type: Delegation
9. ChannelMember.Read.All
  Type: Delegation
10. Chat.Read
  Type: Delegation
11. Group.Read.All
  Type: Delegation
12. Team.ReadBasic.All
  Type: Delegation
13. TeamSettings.Read.All
  Type: Delegation
Click Add permissions at the bottom of the page.
Next, select Grant admin consent for [Tenant Name].
Select Yes to grant consent for the requested permissions for all accounts in [Directory Name].
At the top of the page, there will be a notification that admin consent for the requested permissions was successfully granted.

Permission to access OneDrive

Now that the Shinydocs Application has been registered with Azure, it’s time to apply permissions to access content within OneDrive.

From the sidebar menu, select API permissions
Select + Add a permission
Select Microsoft Graph:
Select Application permissions
Add the following Application permissions:
1. Files.Read.All
  Type: Application
2. Directory.Read.All
  Type: Application
Click Add permissions at the bottom of the page.
Click + Add a permission again > Graph > Delegated
Add the following Delegated permissions:
1. offline_access
  Type: Delegation
2. openid
  Type: Delegation
3. profile
  Type: Delegation
4. User.Read
  Type: Delegation
5. Files.Read.All
  Type: Delegation
6. Directory.Read.All
  Type: Delegation
Click Add permissions at the bottom of the page
Next, select Grant admin consent for [Tenant Name].
Select Yes to grant consent for the requested permissions for all accounts in [Directory Name].
At the top of the page, there will be a notification that admin consent for the requested permissions was successfully granted.

Configure Shinydocs Control Center

The following configurations are performed in the Shinydocs Control Center’s + Add source feature.

SharePoint Online

These steps can be followed once Shinydocs Pro has been installed.
You will need the .pfx file from your certificate.

Moving the .pfx after setting the Certificate file location will cause any related tasks to fail.

If the .pfx file is moved, you will need to update the Certificate file location to the new path.

In Shinydocs Control Center (either in quick-start or + Add source):

Select Microsoft SharePoint Online as your new or existing source
1. Tenant URL: enter the root URL of the Sharepoint site (https://acmecorp.sharepoint.com/)
2. Application ID: enter the Application (client) ID previously noted
3. Tenant ID: enter the Directory (tenant) ID previously noted
4. Certificate file location: enter the path of the .pfx file
  1. Do not use double quotes around the path
5. Certificate password: enter the password for the .pfx file. If your .pfx file does not have a password, leave this field blank
6. Search Authentication Type: select Protected - OAuth2
  1. Enter the Client ID from your application registration
  2. Enter the Client Secret (key) from your application registration
  3. For Login/Authorize Endpoint, replace “common” with your Tenant ID
    1. e.g. https://login.microsoftonline.com/42abc123-a76a-4j03-bf1e-4e51c696d65d/oauth2/v2.0/authorize
  4. For Token Endpoint, replace “common” with your Tenant ID
    1. e.g. https://login.microsoftonline.com/42abc123-a76a-4j03-bf1e-4e51c696d65d/oauth2/v2.0/token
Click Next. A validation check is performed to make sure the information entered is correct. If an error occurs, the error should indicate the issue at hand.
Site, if you want to crawl specific sites, enter the URL for the site like this:
CODE
```
https://acmecorp.sharepoint.com/sites/ACMEhome
```
Otherwise leave it blank to analyze all sites.
Click Start Analysis

Congratulations! You should now be crawling your organization’s SharePoint content.

Exchange Online

These steps can be followed once Shinydocs Pro has been installed.
You will need the .pfx file from your certificate.

Moving the .pfx after setting the Certificate file location will cause any related tasks to fail.

If the .pfx file is moved, you will need to update the Certificate file location to the new path.

In Shinydocs Control Center (either in quick-start or + Add source):

Select Microsoft Exchange Online as your new or existing source
1. Application ID: enter the Application (client) ID previously noted
2. Tenant ID: enter the Directory (tenant) ID previously noted
3. Certificate file location: enter the path of the .pfx file
  1. Do not use double quotes around the path
4. Certificate password: enter the password for the .pfx file. If your .pfx file does not have a password, leave this field blank
5. Search Authentication Type, select Protected - OAuth2
  1. Enter the Client ID from your application registration
  2. Enter the Client Secret (key) from your application registration
  3. For Login/Authorize Endpoint, replace “common” with your Tenant ID
    1. e.g. https://login.microsoftonline.com/42abc123-a76a-4j03-bf1e-4e51c696d65d/oauth2/v2.0/authorize
  4. For Token Endpoint, replace “common” with your Tenant ID
    1. e.g. https://login.microsoftonline.com/42abc123-a76a-4j03-bf1e-4e51c696d65d/oauth2/v2.0/token
Click Next. A validation check is performed to make sure the information entered is correct. If an error occurs, the error should indicate the issue at hand.
Email addresses, if you want to crawl a specific account(s), enter the email addresses like this:
CODE
```
sketchum@shinydocs.com
```
Otherwise leave it blank to analyze all accounts.
Click Start Analysis

Congratulations! You should now be crawling your organization’s Exchange email content

Teams

When crawling a specific user(s), only the users Direct Messages (DMs) will be analyzed. For a complete analysis, leave the user field empty.

These steps can be followed once Shinydocs Pro has been installed.
You will need the .pfx file from your certificate.

Moving the .pfx after setting the Certificate file location will cause any related tasks to fail.

If the .pfx file is moved, you will need to update the Certificate file location to the new path.

In Shinydocs Control Center (either in quick-start or + Add source):

Select Microsoft Teams as your new or existing source
1. Application ID: enter the Application (client) ID previously noted
2. Tenant ID: enter the Directory (tenant) ID previously noted
3. Certificate file location: enter the path of the .pfx file
  1. Do not use double quotes around the path
4. Certificate password: enter the password for the .pfx file. If your .pfx file does not have a password, leave this field blank
5. Search Authentication Type: select Protected - OAuth2
  1. Enter the Client ID from your application registration
  2. Enter the Client Secret (key) from your application registration
  3. For Login/Authorize Endpoint, replace “common” with your Tenant ID
    1. e.g. https://login.microsoftonline.com/42abc123-a76a-4j03-bf1e-4e51c696d65d/oauth2/v2.0/authorize
  4. For Token Endpoint, replace “common” with your Tenant ID
    1. e.g. https://login.microsoftonline.com/42abc123-a76a-4j03-bf1e-4e51c696d65d/oauth2/v2.0/token
Click Next. A validation check is performed to make sure the information entered is correct. If an error occurs, the error should indicate the issue at hand.
User Ids or Teams Ids, if you want to crawl a specific account(s), enter the email address like this:
CODE
```
sketchum@shinydocs.com
```
Otherwise leave it blank to analyze all accounts.
Click Start Analysis

Congratulations! You should now be crawling your organization’s Microsoft Teams content

OneDrive

These steps can be followed once Shinydocs Pro has been installed.
You will need the .pfx file from your certificate.

Moving the .pfx after setting the Certificate file location will cause any related tasks to fail.

If the .pfx file is moved, you will need to update the Certificate file location to the new path.

In Shinydocs Control Center (either in quick-start or + Add source):

Select Microsoft OneDrive as your new or existing source
1. Application ID: enter the Application (client) ID previously noted
2. Tenant ID: enter the Directory (tenant) ID previously noted
3. Certificate file location: enter the path of the .pfx file
  1. Do not use double quotes around the path
4. Certificate password: enter the password for the .pfx file. If your .pfx file does not have a password, leave this field blank
5. Search Authentication Type: select Protected - OAuth2
  1. Enter the Client ID from your application registration
  2. Enter the Client Secret (key) from your application registration
  3. For Login/Authorize Endpoint, replace “common” with your Tenant ID
    1. e.g. https://login.microsoftonline.com/42abc123-a76a-4j03-bf1e-4e51c696d65d/oauth2/v2.0/authorize
  4. For Token Endpoint, replace “common” with your Tenant ID
    1. e.g. https://login.microsoftonline.com/42abc123-a76a-4j03-bf1e-4e51c696d65d/oauth2/v2.0/token
Click Next. A validation check is performed to make sure the information entered is correct. If an error occurs, the error should indicate the issue at hand.
User Id, if you want to crawl a specific account(s), enter the email address like this:
CODE
```
sketchum@shinydocs.com
```
Otherwise leave it blank to analyze all accounts.
Click Start Analysis

Congratulations! You should now be crawling your organization’s Microsoft OneDrive content.

Table of Contents

Requirements

Create the self signed certificate for Azure

Register Shinydocs Software with Azure

Upload Authentication Certificate to Azure

Authentication

Source-specific permissions

Permissions for SharePoint Online Content

Permissions for Exchange Online

Permissions for Teams

Permission to access OneDrive

Configure Shinydocs Control Center

SharePoint Online

Exchange Online

Teams

OneDrive