Creating and Updating Indexes

Dovetail Seeker comes with a console and a Windows service that can both create and update search indexes. These indexing applications are the Seeker Windows Service and the Seeker Console application.

Seeker Windows Service

Dovetail Seeker includes a Windows service called Seeker Indexer Service which makes it easy to keep your search index up-to-date. Seeker Indexer Service polls the Dovetail CRM database for new or updated entities and updates the index. Additionally the service will poll directories for your file specifications. The Seeker Indexer Service behavior is equivalent to running the Seeker Console indexer on a recurring basis but with less performance impact to your application server.

Before starting the Seeker Indexer Service you will need to edit the application settings file. However, you may want to test that your application settings are configured correctly by first using the Seeker Console to create your index.

Note: You can control how long the Seeker Indexer Service waits 
between index updates in the application settings file.
To start the Dovetail Seeker Service
  1. Open the Services Control Manager. Control Panel -> Administrative Tools -> Services
  2. Select the Dovetail Seeker Indexer Service
  3. Click on Start link, click the Start button or Right Click -> Start
To Install Additional Dovetail Seeker Windows Services Instances on the Same Server
  1. Make a copy of the seeker install directory.
  2. Open a command prompt
  3. Navigate to the indexer directory in the seeker install directory
  4. At a command line run SeekerService.exe install /instance: <new instance name>
  5. You should see output that looks like this containing your windows service instance name which will make this service unique.
    >SeekerService.exe service install /instance:NumeroDos
2009-09-24 14:02:43,948 INFO  Topshelf.Runner - Starting Host
2009-09-24 14:02:44,026 INFO  Topshelf.Internal.Actions.InstallServiceAction - Received service install notification
Running a transacted installation.
Beginning the Install phase of the installation.
2009-09-24 14:02:44,044 INFO  Topshelf.Internal.Hosts.HostServiceInstaller - Installing Service DovetailSeekerNumeroDos
Installing service DovetailSeekerNumeroDos...
Service DovetailSeekerNumeroDos has been successfully installed.
Creating EventLog source DovetailSeekerNumeroDos in log Application...
The Install phase completed successfully, and the Commit phase is beginning.
The Commit phase completed successfully.
The transacted install has completed.
To Uninstall Additional Dovetail Seeker Windows Services
  1. At a command prompt change directories to the location of your alternate Dovetail Seeker application directory.
  2. Run SeekerService.exe uninstall /instance:<instance name>
Add a new web application for each additional Dovetail Seeker Windows Service

Create a Application Pool

  1. Open IIS Manager: select Start - type IIS, and select Internet Information Services (IIS) Manager
  2. Click on your computer host name to expand the top-level node.
  3. Click on Application Pools
  4. Right-click on Application Pools and choose Add Application Pool
  5. Enter a name, such as Seeker-{instance}
  6. Set the .NET Framework Version to version 4.
  7. Set the Managed Pipeline Mode to Integrated.

Create the web application

  1. Right-click Default Web Site and select Add Application
  2. In the Alias text box, type Seeker-{instance}
  3. Enter the path to $newSeekerInstance\webservice
  4. Select the Seeker-{instance} Application Pool that you created earlier
  5. Select OK.

Update the Web.config file for the web application

  1. Navigate to the webservice directory in the seeker install directory.
  2. Edit the Web.config file.
  3. Update the <add key="serviceInstanceName" value=""/> entry, setting its value to "{instance}".

Live Document Specification Change Detection

The Dovetail Seeker Indexer Windows Service monitors the documentSpecifications.xml file in the configuration directory for changes. The Windows Service is smart enough to detect the addition, removal, and modification of document specifications for both Dovetail and file document specifications.

Specification changes cause the affected search domain to be re-indexed When there is a change in an existing document specification, the Windows Service will perform a re-index of that specification to make sure all changes to the specification are reflected by the search index.

Why?

Re-indexing the search index on changes is important in-order to keep the search index fields consistent. If this did not happen search documents already indexed would be in the old format with new documents added to the index reflecting the current configuration.

Removing Specifications

When dovetail or file document specifications elements are removed from the documentSpecification.xml file the search documents for that domain will be removed from the search index.

Adding Specifications

When specifications are added to the documentSpecification.xml file they will be begin indexing during the next Dovetail or File polling interval.

How Indexing Work Is Produced And Consumed

The primary goal of the Dovetail Seeker Indexer Windows Service is to keep the search index up-to-date. This is accomplished by splitting the job between work producers (what needs to be done) and consumers (content indexers) using a Microsoft Message Queue to communicated between them.

Producers - What's New or Updated?

The indexer has work producers which monitor the database and file system for changes. When a producer detects that an object in the database or file system has changed it publishes a message to a queue which contains instructions on what items need to be added or updated in the index. The job of the producer is then done. The producers use timers to wake up at configurable intervals to produce index update messages.

Database Producer Example

The database monitor detects that 3 cases have been modified. It will then push a message onto the work queue which basically says: Please update the search index with the details from the Dovetail database's case 42, 43, and 44.

Consumers - Update the index

When a message is published to the queue a message consumer will wake up and start processing the message. Doing whatever needs to be done be it updating the index for a case or indexing a new word document that just showed up in a watched directory. One big advantage of this approach is that many consumers can be created to chew through the work queue. In fact, when there is a lot of work to do the number of consumers is equal to the total of 2 times the number of processors on the host server. This allows Dovetail Seeker to process a lot of search content simultaneously.

One Down Side

When there is a lot of work queued up Dovetail Seeker may appear to not be doing what you've told it to do. For example if you tell Dovetail Seeker to re-index all of your case documents and the queue is already full of other work it is doing you make think your request didn't work when it was really just added to the end of the work queue.