Sep
29

Response Time SLA Management in Service Manager

By

Updated 2012-02-17

  • The performance of the script has been greatly improved.
  • When comparing time I had missed to convert to UTC which is what is stored in the database.

A big THANK YOU goes out to Andrew Gault for testing and finding the UTC bug.


A common request for Service Manager is to be able to notify upon Incidents breaching
their “Response Time SLA”, the time between registration of an incident and when
the incident is assigned to someone who starts working on it. In this post I’ll
describe a solution for reacting on a “Response Time Breaches” and using Incident
Event Workflows triggered by the breach to send notifications. The solution I describe
below will be configured to send a notification to the “primary owner” of an Incident
if the Incident has been registered for more than 30 minutes without being assigned
to anyone. The solution consists of the following components:

  • Response Time Breached Property: A class type extension of the
    Incident class type consisting of one property Name: RTBreached Type: Integer DefaultValue:
    0
  • Scheduled workflow: A re-occurring workflow created using the Authoring
    Tool that runs every other minute to locate all Incidents that has breached the
    “Response Time Threshold” using the following criteria: Status equal Active CreatedDate
    less than (“Now – 30 minutes”) AssignedTo equal Null RTBreached not equal 0
  • Notification Template: Defines how the email will look like when
    the notifications are sent regarding the breach.
  • Incident Event Workflow: A workflow created directly in the console
    that will be indirectly triggered for each Incident detected by the re-occurring
    workflow.

You’ll need the following PowerShell module to make things work:

http://smlets.codeplex.com

Extending the Incident Class Type

  1. Create a new strong name key using sn.exe (if you don’t already
    have a suitable one) In case you aren’t using the “Visual Studio Command Prompt”
    (Start >> Microsoft Visual Studio 2010 >> Visual Studio Tools >>
    Visual Studio Command Prompt (2010)) you can locate sn.exe at %ProgramFiles%\Microsoft
    SDKs\Windows\v7.0A\bin\sn.exe
  2. Start the Service Manager Authoring Tool
  3. Create new Management Pack called Litware.Demo.ResponseTimeSLA.Library
  4. In the new MP, right click “Classes” and select “Create
    other class…”
  5. Choose Incident as base class
  6. Name the new class Litware.Incident.ResponseTimeExtension
  7. Set class properties as follows: Extension: True
  8. In the list of class type properties, delete the automatically added property (called
    something like “Property26”).
  9. Add a new property as follows: Internal name: RTBreached Data type: Integer Default
    value: 0
  10. Save the MP
  11. Right click Litware.Demo.ResponseTimeSLA.Library in the
    Mangement Pack Explorer
    , select Seal Management Pack
  12. In the Seal Management Pack dialog
    1. Choose choose a suitable output directory
    2. Browse to and select your strong name key
    3. Enter your company name
    4. Click Seal then Close

Creating the re-occurring workflow

  1. Back in the Authoring Tool, create a new Management Pack called Litware.Demo.ResponseTimeSLA.Config
  2. In the new MP, right click Workflows, select Create
  3. Enter the following information then click Next Name: DetectResponseTimeBreaches
    Description: “Used to detect incidents breaching their Response Time SLA”
  4. Select “Run at a scheduled time or at scheduled intervals” then
    click Next
  5. Select “Other Interval”, specify the frequency “2 minutes”
    and click Next
  6. Verify the summary and click “Create” then “Close”
  7. Drag a Windows PowerShell Script activity onto
    the design surface from the Activities ToolBox
  8. Change the name property of the activity to “MarkRTBreachedIncidents”
  9. In the Script Body of the PowerShell Activity
    paste the following script:
  10. ?View Code POWERSHELL
        # Author: Patrik Sundqvist
    # Author blog: http://blogs.litware.se
    # Requires: SCSM PowerShell Cmdlets (Beta 1 or later)
    # Update 2012-02-15 - Major imp. on performance.
     
    Import-Module -Name smlets -Force
    # Get the type projection
    $IncidentProjection = Get-SCSMTypeProjection System.WorkItem.Incident.View.ProjectionType$
    # Get id for Incident Status "Active"
    $IncidentActive = (Get-SCSMEnumeration IncidentStatusEnum.Active$).Id
    # Set the target response time to 30 minutes
    $RTSLA = New-Object TimeSpan 0,0,30,0,0
     
     
    # Define the required ObjectProjectionCriteria
    $CTYPE = "Microsoft.EnterpriseManagement.Common.ObjectProjectionCriteria"
     
    # After defining the XML based criteria, I had to replace the following charcters, "<>$" with their ASCII representation 
    $lt = [char]60 
    $gt = [char]62 
    $dollar = [char]36 
     
    $C = @"
    $($lt)Criteria xmlns="http://Microsoft.EnterpriseManagement.Core.Criteria/"$($gt)
      $($lt)Reference Id="System.WorkItem.Incident.Library" PublicKeyToken="31bf3856ad364e35" Version="7.0.6555.0" Alias="CoreIncident" /$($gt)
      $($lt)Reference Id="System.WorkItem.Library" PublicKeyToken="31bf3856ad364e35" Version="7.0.6555.0" Alias="CoreWorkItem" /$($gt)
      $($lt)Expression$($gt)
        $($lt)And$($gt)
          $($lt)Expression$($gt)
            $($lt)SimpleExpression$($gt)
              $($lt)ValueExpressionLeft$($gt)
                $($lt)Property$($gt)$($dollar)Context/Property[Type='CoreIncident!System.WorkItem.Incident']/Status$($dollar)$($lt)/Property$($gt)
              $($lt)/ValueExpressionLeft$($gt)
              $($lt)Operator$($gt)Equal$($lt)/Operator$($gt)
              $($lt)ValueExpressionRight$($gt)
                $($lt)Value$($gt){0}$($lt)/Value$($gt)
              $($lt)/ValueExpressionRight$($gt)
            $($lt)/SimpleExpression$($gt)
          $($lt)/Expression$($gt)
          $($lt)Expression$($gt)
            $($lt)SimpleExpression$($gt)
              $($lt)ValueExpressionLeft$($gt)
                $($lt)Property$($gt)$($dollar)Context/Property[Type='CoreIncident!System.WorkItem.Incident']/CreatedDate$($dollar)$($lt)/Property$($gt)
              $($lt)/ValueExpressionLeft$($gt)
              $($lt)Operator$($gt)Less$($lt)/Operator$($gt)
              $($lt)ValueExpressionRight$($gt)
                $($lt)Value$($gt){1}$($lt)/Value$($gt)
              $($lt)/ValueExpressionRight$($gt)
            $($lt)/SimpleExpression$($gt)
          $($lt)/Expression$($gt)
          $($lt)Expression$($gt)
            $($lt)UnaryExpression$($gt)
              $($lt)ValueExpression$($gt)
                $($lt)GenericProperty Path="$($dollar)Context/Path[Relationship='CoreWorkItem!System.WorkItemAssignedToUser' SeedRole='Source']$($dollar)"$($gt)Id$($lt)/GenericProperty$($gt)
              $($lt)/ValueExpression$($gt)
              $($lt)Operator$($gt)IsNull$($lt)/Operator$($gt)
            $($lt)/UnaryExpression$($gt)
          $($lt)/Expression$($gt)
          $($lt)Expression$($gt)
            $($lt)SimpleExpression$($gt)
              $($lt)ValueExpressionLeft$($gt)
                $($lt)Property$($gt)$($dollar)Context/Property[Type='CoreIncident!System.WorkItem.Incident']/RTBreached$($dollar)$($lt)/Property$($gt)
              $($lt)/ValueExpressionLeft$($gt)
              $($lt)Operator$($gt)NotEqual$($lt)/Operator$($gt)
              $($lt)ValueExpressionRight$($gt)
                $($lt)Value$($gt)1$($lt)/Value$($gt)
              $($lt)/ValueExpressionRight$($gt)
            $($lt)/SimpleExpression$($gt)
          $($lt)/Expression$($gt)
        $($lt)/And$($gt)
      $($lt)/Expression$($gt)
    $($lt)/Criteria$($gt)
    "@
     
    $cc = [string]::Format($C, $IncidentActive, (Get-Date).Subtract($RTSLA).ToUniversalTime())
    $criteria = new-object $CTYPE $cc,$IncidentProjection.__Base,$IncidentProjection.managementgroup
     
    # Get incidents matching the criteria
    $BreachedIncidents = get-scsmobjectprojection -criteria $criteria|%{$_.Object}
     
    # Marking all located incidents to trigger notification workflow
    if ($BreachedIncidents -ne $null)
    {
        $BreachedIncidents|Set-SCSMObject -Property RTBreached -Value 1
    }
  11. Save the Management Pack
  12. Go to the directory where you created the Management Packs
  13. Copy the assembly (dll) called DetectResponseTimeBreaches.dll to
    %ProgramFiles%\Microsoft System Center\Service Manager 2010\
  14. In the Service Manager console, go to Administration – Management Packs
    and import the MPs in the following order

    1. Litware.Demo.ResponseTimeSLA.Library.mp
    2. Litware.Demo.ResponseTimeSLA.Config.xml

You might need to restart the Service Manager services after deploying the workflow
assemblies.

Creating the notification template

  1. In the console, go to Administration – Notifications – Templates
  2. Create a template with the following characteristics: Name: Response
    Time Breached Template Description: Notify the primary user of
    an unassigned incident when Response Time SLA is breached Target class:
    Incident Management pack: Litware.Demo.SLAResponseTime.Config
    Subject
    : Response Time SLA Breached Message body:
    The Response Time has breached for the incident described below.Name: $Context/Property[Type='System!System.Entity']/DisplayName$
    Priority: $Context/Property[Type='CustomSystem_WorkItem_Library!System.WorkItem.TroubleTicket']/Priority$
    Primary Owner: $Context/Path[Relationship='CustomSystem_WorkItem_Incident_Library!System.WorkItem.IncidentPrimaryOwner'
    TypeConstraint='System!System.User']$?$DisplayName$? Please act now!

Creating the Incident Event Workflow

  1. Go to Administration – Workflows – Configuration
  2. Create an Incident Event Workflow with the following characteristics : Name:
    Response Time Breached Notification Check for event: When an incident
    is updated Management pack: Litware.Demo.ResponseTimeSLA.Config
    Criteria – Changed From: “RTBreached equals 0” Criteria – Changed
    To
    : “RTBreached equals 1” Enable notification: “True”
    (Checked) Add a notification: User – Primary Owner, Message Template
    – Response Time Breached Template

To try things out – create an incident, assign yourself as the primary owner and
leave it for 30 minutes then check your mailbox :)

Comments

  1. Nice Patrik! Are you going to add this to the SLA Management CodePlex project now? :)

  2. [...] This post was mentioned on Twitter by Travis Wright, Maarten Goet. Maarten Goet said: Response Time SLA Management in Service Manager: http://j.mp/blrwo3 #SystemCenter [...]

  3. Patrik Sundqvist says:

    Yes Travis, I aim to do that as soon as possible. :)

  4. Sam Erskine says:

    Hi Patrik, will this perfom the same functionality as the one Travis posted in the resource kit?
    I am have the date function problem with the Zipped download from codeplex (same error as others report).
    do you have this available to share as a downloadable MP?

    Cheers

    Sam

  5. Ozge Ozkaya says:

    The script doesnt set the property to 1 for the unassigned incidents after the defined period of time has past. Still remains 0. I can see the script works every 5 minutes successfully but not the workflow.

  6. Patrik Sundqvist says:

    Hi Ozge,

    Are you sure the script is really being execute?

  7. Ozge Ozkaya says:

    I used a subscription. Some workflows succeed som not. Some workflow errors.

    System.Workflow.ComponentModel.WorkflowTerminatedException: The workflow instance exceeded the maximum running time.

    Import-Module -Name smlets -Force
    $IncidentProjection = Get-SCSMTypeProjection System.WorkItem.Incident.ProjectionType$
    $IncidentActive = Get-SCSMEnumeration IncidentStatusEnum.Active$

    $RTSLA = New-Object TimeSpan 0,0,30,0,0
    #Locate all incidents that needs attention

    $BreachedIncidents = Get-SCSMObjectProjection $IncidentProjection|?{$_.Status -eq $IncidentActive -and $_.CreatedDate -lt (Get-Date).Subtract($RTSLA) -and $_.AssignedTo -eq $null -and $_.RTBreached -ne 1}|%{$_.Object}

    $BreachedIncidents

    #Marking all located incidents to trigger notification workflow
    if ($BreachedIncidents -ne $null)
    {
    $BreachedIncidents|Set-SCSMObject -Property RTBreached -Value 1
    }

  8. Andrew says:

    I’m also seeing the Timeout issue that Ozge reports. If I run the script manually in PS it takes 8-9 minutes to complete successfully and the Service Manager SDK service CPU spikes during this time.

  9. Patrik Sundqvist says:

    Hi Andrew and Ozge,

    I’ll try to find some time to go over this again and improve this a bit. Bare with me though, finding time is hard work :)

  10. Andrew says:

    Thanks Patrik!

    FY: I’ve tried it with database that has around 300 active incidents and another instance with about 3500 active incidents. Both seem to take around 7-8 minutes. I tried a demo environment with the latest CU3 update applied too.

  11. Patrik Sundqvist says:

    Hi Guys,

    First of all, apologies for the terrible performance, need I say it wasn’t my main concern when I first wrote the post.

    If we need high performance we need to replace the client side filter “|?{….” by using the “-Filter” property in the get-scsmobjectprojection command or by using a “projection criteria”. Using a client side filter basically means that we’re retrieving all objects of specified type and then looking at each one of them seeing if they match the criteria. By using the “-Filter” property or a projection criteria we hand the filtering part over to the Service Manager SDK service an only retrieve the objects matching the criteria.

    Also, I noticed that I was using the “heaviest” out-of-the-box projection available for incidents. This is totally unnecessary since we’re only in need of one component in the filter, the “AssignedUser” component. Hence, we can use a lighter type projection to improve performance even more.

    To summarize, I’ve made some major improvements on this. You should expect an update of post within the next couple of days.

  12. Patrik Sundqvist says:

    There it is guys, updated and ready to go!
    The performance is GREATLY improved.

Leave a Reply