Case study – Investigating problems with Sitecore Experience Analytics 8.2 in a CM/CD environment

Despite working with Sitecore in depth for a couple of years I had never had the opportunity to delve into the Marketing and Analytics parts of the XP. The perfect opportunity came up recently when the client upgraded their Sitecore 6.6 installation to 8.2. The Analytics functions were not displaying data through the Apps on the Launchpad (Experience Analytics, Experience Profile etc) so I set to task.

When dealing with Sitecore in general it pays dividends to take a patient, analytical approach. Sitecore produces a lot of documentation on how your installation should be configured, however take this with a big pinch of salt; The documentation tends to fall down slightly when you move into the CM/CD arena. Setting descriptions are often brief with minimal examples so you will more than often need to turn to the community to help. Stack Overflow and Slack can be invaluable tools picking up where the Sitecore docs left off.

To briefly explain the production environment I was dealing with, the client had:

  • One CM server also doing processing and reporting
  • 6 load balanced CD servers
  • Sitecore 8.2 Update 4 (170614)

To diagnose issues with analytics it helps to have an understanding of the fundamentals of how it works. In simple terms:

  • A user browses the site
  • Data about the user’s habits and preferences are stored and information that might identify them is stored in session until the user has ended their browsing session
  • This Analytics data is then stored in the Collection database, that is, the MongoDB database specified in your Sitecore connection strings. 
  • The Sitecore Processing server (the CM server in our case) uses Aggregators to then curate the Collection data and write it to the Reporting database at predefined intervals. The default interval is 15 seconds but this can be adjusted in the configuration. The sitecore_analytics_index is also updated.
  • The Reporting service then queries the reporting database to display data in applets such as Experience Analytics

The first issue to investigate were the errors being reported in the Analytics interface

Problem 1: Errors in Experience Analytics interface

Errors appearing in the Analytics interface

In our case the following files were enabled on CM which should be disabled according to the documentation from Sitecore for a CM+processing server:

Config file nameType Search Provider UsedContent Delivery (CD)Content Management (CM)ProcessingCM + ProcessingReporting
Sitecore.Xdb.Remote.Client.config.disabledconfig DisableEnableDisableDisableDisable
Sitecore.Xdb.Remote.Client.MarketingAssets.config.disabledconfig DisableEnableDisableDisableDisable

The accompanying note from the Sitecore documentation (below) took a few reads through before it was clear in my mind!

Note All Remote.Client.config files are disabled by default. You should enable these files if you have a reporting server in another location. Disable these files on a content delivery server and enable them on a content management server. 

Advice from Sitecore

Since our config is set via Octopus deploy, it was a case of correcting any mismatches in the config enabled/disabled states inside the package deployment steps.

Problem Solved!

Problem 2: No data was appearing at all in the Experience Analytics 

There can be many reasons why no data is being displayed but there are a few common avenues you can use for investigating Analytics / Reporting issues:

Enable detailed logging and check the logs

Often the first port of call in diagnosing issues is to check the Sitecore logs. Log viewers such as the Sitecore Instance Manager (SIM) Dynamic Log Viewer or the Log Analyzer tool make light work of analysing any errors. To enable debug level logging for analytics change the configuration setting as per below:

<setting name="Analytics.LogLevel" value="Debug" />

Check the Browser Console for Errors

Occasionally Javascript errors can appear in the browser developer tools that help point you in the right direction.  

Check the Sitecore configuration

With so many working parts and the multiple ways in which your production system can be deployed, its important to get the Sitecore configuration correct. The official documents for configuration at the time of writing are located below:

Each document provides a link to a spreadsheet detailing the list of all the configuration files that should be enabled or disabled for each version of Sitecore 8.2 e.g. Update 1 – Update 6 at the time of writing. Since in our case we have one CM server doing the processing and reporting we use the CM+Processing columns and the Content Delivery columns. 

This in itself is confusing as their is no mention of reporting in that column. Sitecore don’t provide an explanation of exactly what each column means which is a cause for confusion so I am left to infer the following:

  • Content Delivery – Each customer facing CD server
  • Content Management – A Content Management server only 
  • Processing – A Processing server only
  • CM + Processing – A combined CM, Processing AND Reporting server (Reporting is not specified in the column)
  • Reporting – A Reporting server only

It is important to get this right or it can create spurious errors that are hard to track down. However as we will read later, sometimes Sitecore can get this information wrong!

Shorten the feedback loop

To aid testing and shorten the time it takes to analyse and debug Analytics issues you can create a quick and dirty aspx page that will abandon the session. Drop the following code into a new file with the aspx extension and drop it temporarily into the site. e.g.

<% Session.Abandon();%>
<h1>Session killed to death</h1>

Navigating to this page when testing should flush the data from the current session into Mongo DB. The alternative is to reduce the session timeout in Web.config to a lower value to avoid waiting around for long period of time during testing:

<sessionState mode="InProc" cookieless="false" timeout="1">

Checking the MongoDB database

Now the user’s session has ended, data should be written to the Interactions collection in MongoDB. To check this we can use a tool such as RoboMongo to connect to the Mongo instance in question and browse the data.
By issuing a query we can check that site interactions are getting:

 logged:db.getCollection('Interactions').find({}).sort({'StartDateTime': -1})

In our case the interactions returned confirm that data are indeed being written correctly. If there are no interactions with a datetime of around the time you killed the session, it suggests this part of the process is not working. With the collection database checked it’s time to move on to the next step in the process: the SQL Server Reporting Database.

Checking the Reporting DB

By connecting to the Reporting database in SQL Server using the analytics connection string, you can check whether the data are being aggregated by runing a query:

SELECT TOP 1000 [SegmentRecordId],
FROM [<AnalyticsDatabaseName>].[dbo].[ReportDataView]

In our case again the data itself was being aggregated correctly to the SQL database but if yours isn’t try the steps detailed here.

Experience Analytics Caching

After the aggregation process completes, the data should be available in Experience Analytics. However in my experience the page is often cached which means closing the browser window and logging into Sitecore again to refresh the data. To combat this the cache can be disabled by altering the cache settings in Sitecore.ExperienceAnalytics.WebAPI.config as this Stack overflow post by Jacob Neilsen details.

Rebuild the reporting database

A final last ditch option, if you’re still experiencing issues is to rebuild the Reporting database via the relevant Sitecore admin page: /sitecore/admin/RebuildReportingDB.aspx

This was enough to get our instance of Experience Analytics displaying data.

Problem 3: Experience Profile not logging contacts or Empty 

Image result for experience profile no contacts

This issue of no Contacts appearing in the Experience Profile is quite often accompanied by many errors in the log such as:

ERROR Error during aggregation.Exception: System.NullReferenceExceptionMessage: Object reference not set to an instance of an object.

ERROR General error when submitting contact.Exception: System.NotImplementedExceptionMessage: The current data adapter provider does not support renumbering interactions.Source: Sitecore.Analytics.DataAccess   at Sitecore.Analytics.DataAccess.DataAdapterProvider.RenumberInteractions(ID contactId)   at Sitecore.Analytics.Data.ContactRepository.ReconcileContact(Contact contact, LeaseOwner owner, TimeSpan duration)   at Sitecore.Analytics.Tracking.ContactManager.SubmitContact(Contact contact, ContactSaveOptions options, Contact& realContact)

Checking the Sitecore documentation for configuration maintains that the following files should be disabled for Content Management and enabled for Content Delivery:

  • Sitecore.Analytics.Tracking.Aggregation.config
  • Sitecore.Analytics.Tracking.Database.config
  • Sitecore.Analytics.Tracking.RobotDetection.config

However I had proof that following this advice and disabling the above files on CM prevented Contacts being created and in fact no Experience Analytics data appeared. Enabling the files caused the contacts to start getting logged!

Kam Figy to the rescue! 

An answer on Stack Exchange from the man himself indicates that the configuration documentation is, in fact incorrect, and the files need to be enabled.

Moral of the story: Don’t disable these files on a CM if you’re identifying contacts.

Kam Figy

Happily this solved our issue and the client could start making use of Sitecore 8.2’s great Analytics features.

Sitecore User Groups

In the past I’ve not put much onus on visiting user groups. I once visited a .NET user group for the first time when I lived in Sussex and the experience was heavily tarnished by trying to interact with developers possessing zero social skills or even the ability to converse in basic terms. After talking to a few guys there it felt like talking to a brick wall, they seemed more interested in the free pizza than interacting with the community. After receiving one word answers time and again I quickly got bored, gained very little from the experience and was put off for years. However in the past year I decided to give it another try – I visited a few Sitecore user groups and found them much more useful. Everyone there has been approachable and friendly with few egos despite the breadth of talent that attends. You also tend to see the same faces over and over again and people you’ve worked with before. Maybe its something to do with the small community feel of Sitecore which still remains a fairly specialised skill compared with the vastness of .NET. Traditionally Sitecore documentation is one of the company’s weaker points and lags behind so it helps to see how people are using it in the wild and how they solve real world issues. Meeting the community also helps with networking, and staying up to date with the rapid pace of Sitecore.

To find your nearest User Group, Google is often your best bet, but you can also use the dedicated channel on Sitecore Slack or the website host regular Sitecore events.

Sitecore Certification

Having worked with Sitecore for a couple of years, thrown in at the deep end and self taught – I decided it was time to do the certification and get a firm grounding in the software from scratch. Sitecore 8.2 was the latest version at the time, though I realised later that if I’d waited a month, the Sitecore 9 training would have been available! :doh
As of late 2017 there were four eLearning options

  • Sitecore Developer Foundations
  • Sitecore Experience Solution Developer
  • Sitecore Context Marketing Fundamentals
  • Sitecore Web Experience Management

The first one is free and it made sense for me to take advantage. According to Sitecore: “The free Developer Foundations course teaches students how to create data infrastructure, build editable pages, and publish content. During the course, you will build a basic website using the powerful and easy-to-use Sitecore XP tools. Students also build simple MVC components using a text editor and minimal coding.
After completing this short course, which is pretty rudimentary, covering templates, publishing and presentation it was time to shell out for the Sitecore Experience Solution Developer course, which at the time of writing was $1500.
Although I would heartily recommend the training as it does cover a range of topics, obviously it can never furnish you with a command of the whole of the Sitecore ecosystem – the system is far too broad to condense down. Rather see it as a stepping stone to gaining further knowledge and exploring the system. There is no substitute for working on a Sitecore project at a client or partner.

My own experience with the training was mixed. Although it filled in a lot of gaps where I was hazy, especially on the marketing features, the training materials were littered with errors and mistakes which made reading hard going especially when the example wasn’t working despite following the instructions to the letter.

Despite the problems it was good to see references to real world concerns like the use of Glass, Content Search API, IoC and Helix patterns in the training.
The approach to the training is down to the individual. As with any form of study people learn in different ways and what is effective for one person will not work for another.  To get the most out of the training I took the following approach – YMMV:

  • Made my own typed notes throughout each chapter condensing it down into a text file
  • Wrote down every question and answer at the end of each chapter for revision later
  • Carried out every practical workbook exercise locally including the optional ones

Despite the course notes saying it will take approximately twelve hours to complete, the above took me around 2 weeks of evenings studying – so probably around 30 hours. I’m not sure if their estimates includes doing the workbook exercises because if it does I find it very optimistic! The exam consists of 70 multiple choice questions but the time period of 2 hours to complete it is more than enough.
Although this approach is not necessary to pass the exam bit does give you a better grasp of the system overall. And since in any form of life I can’t approach something half-arsed I had to do it this way 😛 The pay off was a good resulting grade but it was still annoying to get some questions wrong and even more annoying to not find out which ones! 
I believe the pass mark is 80%, which does allow you to get 14 questions wrong, however you could probably pass the exam without training if you’ve been using Sitecore for any time at all.

As with a lot of closed-book exams it seems more of a test of memory than competency; you would rarely carry out Sitecore work without access to the Internet or other resources. However there is something to be said IMO for committing the basics to memory so you’re not overly dependent on Google. It also means employers/clients can feel comfortable that in taking you on as a certified developer, they can expect a certain level of knowledge for their money.

Here’s looking forward to taking the Sitecore 9 exam just as v10 is released!