Archive

Author Archive

How to: SharePoint Search PDF Document Preview using Acrobat

June 12, 2014 Leave a comment

Introduction

This is what we are looking to build, a SharePoint 2013 search experience where PDFs can be previewed within Search.

pdf_preview_demo

 

On a recent project I built a solution which made extensive use of the SharePoint 2013 search features. Microsoft have been pushing building solutions with SharePoint 2013 and I have to say it does give a great experience in terms of performance and functionality.

One of the main reasons for selecting search based solution is that it scales and this solution needed to provide quick access to documents. The complexity was that due to the nature of the documents they each had unique security permissions. The solution stored a few hundred thousand documents, this would have been a killer for performance if using traditional data querying methods.

95% of the documents were PDFs and I thought it would be cool if we could provide a preview of the PDF in the search results. Unfortunately we didn’t have Office Web Apps and so I started to think about how I could build something similar. The preview would work in a similar way to the search visualisation for Office when Office Web Apps is available.

At the time Office Web Apps wasn’t available and so we could use its ability to view PDFs.

One of the important goals was to ensure that the solution would show the PDF preview window in the hover panel but would also fail gracefully when a user didn’t have the necessary components installed on their machine.

 

The solution relies on the following:-

  • SharePoint 2013
  • SharePoint Search Centre
  • Custom Display Template
  • Acrobat Reader / Acrobat
  • Acrobat Reader / Acrobat’s Web Browser Browser Plugin
  • the PDF Object JavaScript library

 

Adobe Acrobat Reader Pre-requisite Configuration

As mentioned we need a few pre-requisites for this solution to work.

The user must have Acrobat or Acrobat Reader installed and configured to use the PDF Web Browser plugin. You can see my previous post, which explains how to enable the PDF Web Browser plugin.

http://blog.ithinksharepoint.com/2011/02/21/open-pdf-files-from-a-document-library-in-a-new-window/

Adobe Acrobat Reader can be downloaded from: https://get.adobe.com/uk/reader/

 

SharePoint 2013 Setup

So the SharePoint 2013 Search configuration requires the following:-

  • custom Search Item Display Template
  • custom Search Hover Panel Display Template
  • configured Search Result Type

The Search Result type is required to actually configure search to use our custom display templates to render PDF results with our preview hover panel.

We are expecting that you are using a search centre for the sake of this blog post but the solution could be deployed into any site collection.

 

Setting up display templates

Lets set this thing up. We will need to have a SharePoint 2013 Visual Studio project created so that we can put the display templates into source control, plus we need to be able to deploy them too!

  • Start Visual Studio 2012 / Visual Studio 2013
  • Create a new project using the SharePoint 2013 Empty Project
  • Give it a suitable name and location

pdf_vs_newproject

  • Provide the URL to the SharePoint search centre site collection that will be used for development.
  • Choose a Sandbox Solution
  • Click Create

Once we have a Visual Studio project setup we need to create our display templates. The best way to learn is to take an existing display template and hover panel and make a new set. Display Templates are found in the master page document library. This can be found in a site collection’s root web, for example. http://sharepoint/search/_catalogs/masterpages.

Within the Master Pages document library is a sub folder called DisplayTemplates, this is where the Display Templates for the site collection are stored. I suggest that you create your own folder within this so that you can differentiate your changes from the out of the box display templates.

The deployment of display templates is a little tricky because they are made up of two parts. A display template is made up of an html file which SharePoint parses and creates a JavaScript representation of automatically. This occurs each time the .html file is saved.

Unfortunately if you deploy a Display Template via a feature the JavaScript representation is not created automatically. To get around this, I used the following process:-

As we mentioned earlier, we will create our display templates by copying an existing display template, renaming and modifying. This is achieved by the following:-

  • Browse to your SharePoint site.
  • Click on the cog icon in the top right hand section of the page. Click on Site Settings.
  • Click on display page layouts and master pages Pages link under “Look and Feel”.
  • Browse to the Display Templates folder
  • Download the following html display templates
  • Item_PDF.html and Item_PDF_HoverPanel.html

Now that we have a copy of the display templates, we need to add them to our Visual Studio project Create a new feature in Visual Studio by clicking on the Features node in the Solution Explorer window and choosing new feature.

  • Rename the feature folder to an appropriate name such as:-
    • PDFPreviewTemplateWithAcrobatimport
  • Open the new feature and fill in the feature’s information.

pdf_vs_featuredef

Next we will add a SharePoint Project Item to deploy the Display Templates.

  • Right-Click on the project and choose new item
  • From within the SharePoint project items choose the Module project item and call it something appropriate such as “PDFDisplayTemplates”
  • Drag the files into the newly created Project Item

If you cannot drag them then you are likely running Visual Studio as Administrator and you will need to copy the files into the folder created for PDFDisplayTemplates project item and then use the View All Files icon (found at the top of solution explorer) and add them by right clicking on each item and choosing “Include in Project” from the menu.

  • Next, rename the files to something appropriate, like this
    • item_pdf.html –> item_pdf_acrobat.html
    • item_pdf_hoverpanel.html –> item_pdf_acrobat_hoverpanel.html

Once you have uploaded your files into Visual Studio then we need to edit the files and configure Visual Studio to deploy the files as part of the solution by changing the file properties in Visual Studio solution explorer to change your deployment type from “No Deployment” to “Element File”.

We also need to update the display templates so that the name and description allows a user to identify the template. These descritions are modified using the metadata for the display template which isactually provided by a set of headers in the .html file.

Please see the screenshots below, we have modified the name and description to:-

  • Name: PDF Item with Acrobat
  • Description: Displays a result tailored for a Portable Document Format (PDF) document and displays a preview using Adobe Acrobat.

 

Before:-

pdfitem_properties_before

After:-

pdfitem_properties_after

The last step is to add the PDFObject.js file to Visual Studio so that its deployed with the solution. We could do this in a number of ways, I am going to include it within the PDF Display Template feature so that we only have to activate the one feature.

 

Finally I am going to clean up the the feature’s element.xml file so that it is deployed into the correct location within SharePoint, if you remember it should be deployed into the masterpage document library.

So here is the element.xml before:-

pdffeature_elementsbefore

and here is the element.xml after:-

pdffeature_elements_after

Once you have updated your display template’s name then they are ready to be deployed to SharePoint via the visual studio solution. Install your solution and activate the feature that was created.

 

Modifying the display templates to show PDFs

Hopefully now you have the display templates uploaded into SharePoint, though they are just copies of an existing display templates, so we’ll need to update the display template with the code to show PDFs.

Next we need to modify the contents of the display template, I have already create updated versions which you can download but to lets give you an overview of what I have changed.

 

Item_Pdf_Acrobat.html

This has been modified so that the line reading

var hoverUrl = "~sitecollection/_catalogs/masterpage/Display Templates/Search/Item_PDF_HoverPanel.js";

is updated to the use are version of the hover panel.

var hoverUrl = "~sitecollection/_catalogs/masterpage/Display Templates/Search/item_pdf_acrobat_hoverpanel.js";

Item_Pdf_Acrobat_HoverPanel.html

This has also been modified to update the name and description metadata elements. The entire html has been replaced and rather than explain all the changes, download the solution zip file and take a look at the file within Visual Studio.

The standout piece is the following section, which is called after the display template is rendered using the AddPostRenderCallback() function. This checks to see if the file is a pdf, and if it is will try and create the Acrobat Web Viewer window, if it fails for whatever reason the preview div is hidden.

<!--#_
AddPostRenderCallback(ctx, function()
{
var csrId = ctx.CurrentItem.csr_id;

if(ctx.CurrentItem.SecondaryFileExtension=="pdf")
{
try
{
var employeePDFDocument = new PDFObject({
url: $urlHtmlEncode(ctx.CurrentItem.OriginalPath),
pdfOpenParams: {
navpanes: 0,
toolbar: 0,
statusbar: 0,
view: "FitV"
}

}).embed($htmlEncode(id + HP.ids.viewer));
}
catch(e)
{
var viewDivId = $htmlEncode(id + HP.ids.viewer);
var viewDiv = document.getElementById(viewDivId);
if(viewDiv!=null)
{
viewDiv.style.display = 'none';
}

}
}

});

_#-->

Next the display templates need to be updated, I will show you the following approach which is the way that Microsoft recommend when developing display templates.

  • browse to the master page catalogue using http://sharepoint/search/_layouts/15/masterpages
  • click on the library tab in the ribbon toolbar
  • click on open in windows explorer
    • if you get an error make sure that IE is setup to include the web site in the Local Intranet zone and also if you are developing on Windows Server 2008/2012 make sure you have the Desktop Experience feature installed.
  • Once you have the windows explorer folder open you can now copy and paste the files from Visual Studio into Display Templates folder. SharePoint will take care of updating the JavaScript representation of your html file

Now we have our updated display templates we need to configure them to be used!

 

Configure search result: Tell SharePoint about the display templates

The approach to configuring search to use our display templates is by setting up a result type. Search Result types are the new Search Scopes but with SharePoint 2013 they also allow you to specify how each search result is displayed. This is really powerful and one of my favourite features of SharePoint 2013!

This feature allows you to have a list of search results and if a search result for example had a custom content type with custom metadata that result could be displayed differently to show that additional information.

Anyway, on to setting up the custom search result type:-

  • browse to your SharePoint search centre site collection
  • click on the cog icon and then choose site settings
  • Click Search Result Types (under Site Collection Administrator heading)
  • Click new search result type
    • Name: PDF using Acrobat
    • Sources: All Sources
    • Type of Content: PDF
    • Display Template: PDF using Acrobat
    • Tick optimize for frequent use

Lets try it out!

Deploy the solution to your SharePoint environment and add some PDF content to your SharePoint sites. Perform an incremental crawl or wait for your continuous crawl to pick up the content.

If you perform a search for  pdf, you should get some search results with the content uploaded, as you hover over the PDF search result you should get a preview of the PDF!

pdf_preview_demo

 

Troubleshooting the solution

Then the solution was being built then there were a couple of problems that I had. To be honest most of the issues were down to the Adobe Acrobat configuration.

Please take a look at my post Opening PDFs in a New Window and the section

 

Solution Files and resources

The Visual Studio Project, PowerShell script and Display Templates can be found here:

ITSP.SP.PDFSearchPreviewWithAcrobat.zip

 

 

PowerShell: Deleting SharePoint List Items

June 5, 2014 Leave a comment

Introduction

Whilst I love SharePoint Workflows and how versatile they can be, they can generate quite a bit of data. Well mine do as I like to log plenty of information so that the support / admin teams can find out what’s going on with the workflow.

Unfortunately when you log plenty of information this means that the workflow history list can get quite large.

One of the workflows that we built over a ten month period has processed a couple of hundred thousand list items and has created about 3 million list items in the workflow history list.

We wanted to clear down this list and so PowerShell came to the rescue.

Solution

We built the following PowerShell script which you provide the following parameters:0

  • Url – Url of web hosting the workflow history list
  • AgeOfItemsToDelete – days of logs that you wish to keep
  • ListName – the display name of the workflow history list
  • NumberOfItemsInBatch – the number of items that should be returned in each query.

The original script looked like this:-

param
(
	[Parameter(Mandatory=$false, HelpMessage='System Host Url')]
	[string]$Url = "http://sharepoint",
	[Parameter(Mandatory=$false, HelpMessage='List Name')]
	[string]$ListName = "Workflow Tasks",
	[Parameter(Mandatory=$false, HelpMessage='Age of items in list to keep (number of days).')]
	[int]$AgeOfItemsToKeepInDays = 365,
	[Parameter(Mandatory=$false, HelpMessage='What size batch should we delete the items in?')]
	[int]$NumberOfItemsToDeleteInBatch = 1000
	
)

$assignmentCollection = Start-SPAssignment -Global;

$rootWeb=Get-SPWeb $Url -AssignmentCollection $assignmentCollection;

$listToProcess = $rootWeb.Lists.TryGetList($ListName);
if($listToProcess -ne $null)
{
	$startTime = [DateTime]::Now;
	$numberOfDaysToDelete = [TimeSpan]::FromDays($AgeOfItemsToKeepInDays);
	$deleteItemsOlderThanDate = [DateTime]::Now.Subtract($numberOfDaysToDelete);
	$isoDeleteItemsOlderThanDate = [Microsoft.SharePoint.Utilities.SPUtility]::CreateISO8601DateTimeFromSystemDateTime($deleteItemsOlderThanDate);
	$numberOfItemsToRetrieve = $NumberOfItemsToDeleteInBatch;
	
	$camlQueryString = [String]::Format("<Where><Leq><FieldRef Name='Modified' /><Value IncludeTimeValue='TRUE' Type='DateTime'>{0}</Value></Leq></Where>", $isoDeleteItemsOlderThanDate);
	$camlQuery = New-Object -TypeName "Microsoft.SharePoint.SPQuery" -ArgumentList @($listToProcess.DefaultView);
	$camlQuery.Query=$camlQueryString;
	$camlQuery.RowLimit=$numberOfItemsToRetrieve;
	
	$deletedItemCount=0;
	
	do
	{
		$camlResults = [Microsoft.SharePoint.SPListItemCollection] $listToProcess.GetItems($camlQuery);
		$itemsCountReturnedByQuery = $camlResults.Count;
		Write-Host "Executed Query and found " $camlResults.Count " Items";
		
		$listItemDataTable = [System.Data.DataTable]$camlResults.GetDataTable();
		foreach($listItemRow in $listItemDataTable.Rows)
		{
			$listItemIdToDelete = $listItemRow["ID"];
			$listItemModifiedDate = $listItemRow["Modified"];
			Write-Host "Deleting Item $listItemIdToDelete - Modified $listItemModifiedDate";
			$listItemToDelete = $listToProcess.GetItemById($listItemIdToDelete);
			$listItemToDelete.Delete();
			$deletedItemCount++;
		}
	}
	while($itemsCountReturnedByQuery -gt 0)
	
	$totalSecondsTaken = [DateTime]::Now.Subtract($startTime).TotalSeconds;
	Write-Host -ForegroundColor Green "Processing took $totalSecondsTaken seconds to delete $deletedItemCount Item(s).";
}
else
{
	Write-Host "Cannot find list: " $ListName;
}

Stop-SPAssignment -Global -AssignmentCollection $assignmentCollection;

Write-Host "Finished";

However, whilst this worked ok for a list that was quite small. When we went to use it on the Production environment it performed like a dog. Fortunately the script was run out of hours so didn’t impact the environment too much. Though the memory that it consumed was quite large (4GB) after deleting the second item.

There was something seriously wrong with approach being taken, so after a bit of investigation it was obvious what was going on.

Look at the script again, there is a line of code that is:-

$listToProcess.Items.DeleteItemById($listItemIdToDelete);

Well it turns out that this call, updates the collection after the DeleteItemById function is called. So we made a small modification and the offensive line became:-

$listItemToDelete = $listToProcess.GetItemById($listItemIdToDelete);
$listItemToDelete.Delete();

This change meant that the PowerShell session now only consumed 270Mb (I say only!) and memory usage did not rise. The deletion of the items was much quicker too, probably by a few 1000%!

Here is the final script for completeness.

param
(
[Parameter(Mandatory=$false, HelpMessage='System Host Url')]
[string]$Url = "<a href="http://sharepoint&quot;">http://sharepoint"</a>,
[Parameter(Mandatory=$false, HelpMessage='List Name')]
[string]$ListName = "Workflow Tasks",
[Parameter(Mandatory=$false, HelpMessage='Age of items in list to keep (number of days).')]
[int]$AgeOfItemsToKeepInDays = 365,
[Parameter(Mandatory=$false, HelpMessage='What size batch should we delete the items in?')]
[int]$NumberOfItemsToDeleteInBatch = 1000

)

$assignmentCollection = Start-SPAssignment -Global;

$rootWeb=Get-SPWeb $Url -AssignmentCollection $assignmentCollection;

$listToProcess = $rootWeb.Lists.TryGetList($ListName);
if($listToProcess -ne $null)
{
$startTime = [DateTime]::Now;
$numberOfDaysToDelete = [TimeSpan]::FromDays($AgeOfItemsToKeepInDays);
$deleteItemsOlderThanDate = [DateTime]::Now.Subtract($numberOfDaysToDelete);
$isoDeleteItemsOlderThanDate = [Microsoft.SharePoint.Utilities.SPUtility]::CreateISO8601DateTimeFromSystemDateTime($deleteItemsOlderThanDate);
$numberOfItemsToRetrieve = $NumberOfItemsToDeleteInBatch;

$camlQueryString = [String]::Format("&lt;Where&gt;&lt;Leq&gt;&lt;FieldRef Name='Modified' /&gt;&lt;Value IncludeTimeValue='TRUE' Type='DateTime'&gt;{0}&lt;/Value&gt;&lt;/Leq&gt;&lt;/Where&gt;", $isoDeleteItemsOlderThanDate);
$camlQuery = New-Object -TypeName "Microsoft.SharePoint.SPQuery" -ArgumentList @($listToProcess.DefaultView);
$camlQuery.Query=$camlQueryString;
$camlQuery.RowLimit=$numberOfItemsToRetrieve;

$deletedItemCount=0;

do
{
$camlResults = [Microsoft.SharePoint.SPListItemCollection] $listToProcess.GetItems($camlQuery);
$itemsCountReturnedByQuery = $camlResults.Count;
Write-Host "Executed Query and found " $camlResults.Count " Items";

$listItemDataTable = [System.Data.DataTable]$camlResults.GetDataTable();
foreach($listItemRow in $listItemDataTable.Rows)
{
$listItemIdToDelete = $listItemRow["ID"];
$listItemModifiedDate = $listItemRow["Modified"];
Write-Host "Deleting Item $listItemIdToDelete - Modified $listItemModifiedDate";
$listItemToDelete = $listToProcess.GetItemById($listItemIdToDelete);
$listItemToDelete.Delete();
$deletedItemCount++;
}
}
while($itemsCountReturnedByQuery -gt 0)

$totalSecondsTaken = [DateTime]::Now.Subtract($startTime).TotalSeconds;
Write-Host -ForegroundColor Green "Processing took $totalSecondsTaken seconds to delete $deletedItemCount Item(s).";
}
else
{
Write-Host "Cannot find list: " $ListName;
}

Stop-SPAssignment -Global -AssignmentCollection $assignmentCollection;

Write-Host "Finished";

Hope that helps someone who has the same problem. Please let me know if you have an alternative solution!

Links to the scripts:-

Delete-ListItemsOlderThan-Slow.txt

Delete-ListItemsOlderThanV2.txt

PowerShell, SharePoint and Memory Leaks (Start-SPAssignment)

June 2, 2014 Leave a comment

Introduction

Over the last couple of years I have been using PowerShell to do more & more within my SharePoint environments.

Recently I have been writing a script to move a large number of SharePoint Web objects from one site collection to another.

Whilst this was being tested, it was obvious that something wasn’t quite right as I kept seeing the following messages in the ULS logs:-

Potential excessive number of SPRequest objects (60) currently unreleased on thread 10. Ensure that this object or its parent (such as an SPWeb or SPSite) is being properly disposed. This object is holding on to a separate native heap. This object will not be automatically disposed…

 

Now I am used to seeing these “Object Not disposed messages” in logs when writing code and the majority are easily fixed. Please see Stefan Goßner’s excellent article on Disposing of SPWeb and SPSite objects for more information.

The reason for the errors being displayed are that the objects are not being correctly disposed after they are used. This leads to the memory slowly being consumed and not released. Eventually if there are enough objects created, then memory pressure is placed on the server and performance is impacted.

With C# code this is easily sorted out by either calling .Dispose() or wrapping the code that creates the disposable object with a using() statement around the assignment line.
An example is shown below.

For example:-


using(SPWeb web=_site.OpenWeb(“http://sharepoint"))

{

if(web.Exists)

{

//do something

}

}

However PowerShell objects such as the SPWebPipeBind object or SPSitePipeBind object don’t have a .Dispose() function. So how the heck do you stop them from leaking memory?

Well the secret is that you make use of the command Start-SPAssignment and Stop-SPAssignment cmdlets.

Start-SPAssignment / Stop-SPAssignment Cmdlets

Whilst using this commands I came across a number of blog posts but the approach that they were taken didn’t really work and I would still see the same memory leaks.
The set of Cmdlets should be used in the following fashion:-


$assignmentCollection = Start-SPAssignment;

$allWebs = Get-SPWeb –Site <a href="http://sharepoint">http://sharepoint</a> –AssignmentCollection $assignmentCollection

foreach($web in $allWebs)

{

Enable-SPFeature –Identity “Feature” –Url $web.Url –AssignmentCollection $assignmentCollection;

}

Stop-SPAssignment $assignmentCollection;

Walking through the code you will see that the Start-SPAssignment is used to return an SPAssignmentCollection object. We keep a reference to this SPAssignmentCollection so that we can use it to collect any objects that need disposing.

So the next question is how do we use the SPAssignmentCollection object to collect the objects?

Well a large number of the SharePoint PowerShell functions have an optional parameter called AssignmentCollection. The variable that we created using Start-SPAssignment should then be passed into these SharePoint PowerShell cmdlets.

For example, Get-SPWeb has the a set of parameters (content taken from TechNet):-

Parameter Required Description
Identity Optional Specifies the name or full or partial URL of the subsite. If you use a relative path, you must specify the Site parameter.A valid URL in the form http://server_name or a relative path in the form of /SubSites/MySubSite.
AssignmentCollection Optional Manages objects for the purpose of proper disposal. Use of objects, such as SPWeb or SPSite, can use large amounts of memory and use of these objects in Windows PowerShell scripts requires proper memory management. Using the SPAssignment object, you can assign objects to a variable and dispose of the objects after they are needed to free up memory. When SPWeb, SPSite, or SPSiteAdministration objects are used, the objects are automatically disposed of if an assignment collection or the Global parameter is not used.
Confirm Optional Prompts you for confirmation before executing the command. For more information, type the following command: get-help about_commonparameters
Filter Optional Specifies the server-side filter to use for the specified scope.The type must be a valid filter in the form {filterName <operator> “filterValue”}.
Limit Optional Limits the maximum number of subsites to return. The default value is 200. To return all sites, enter ALLThe type must be a valid number greater than 0 or ALL.
Site Optional Specifies the URL or GUID of the site collection from which to list subsites.The type must be a valid URL, in the form of http://server_name; a GUID, in the form 1234-5678-9807, or an SPSite object.
WhatIf Optional Displays a message that describes the effect of the command instead of executing the command. For more information, type the following command: get-help about_commonparameters

When the AssignmentCollection parameter is passed then the objects that are created by that function are then stored in this AssignmentCollection.

When the objects are no longer required then they can be released using:-

Stop-SPAssignment –AssignmentCollection $assignmentCollection

This will free the memory and the associated objects.

SPAssignment Modes of Operation

The SPAssignment cmdlets can be used in a couple of ways:-

  • Simple mode, which is used by passing the –Global switch which seems to start monitoring the objects being created and will ensure that they are all disposed of when you call Stop-SPAssignment.
  • Advanced mode, which is where you create an AssignmentCollection using Start-SPAssignment and then manage the entries added to the collection.

I prefer the advanced mode as you are in control of when objects are being disposed of and also after running your PowerShell script a few times you can see where the appropriate calls to release memory should be placed.

When to use Stop-SPAssignment

One of the mistakes that I made was not calling the Stop-SPAssignment at the right time.

For example, say we have a script which is looping through all the Site Collections in a Web Application and then looping through all the Webs in a Site Collection. For each web object an operation is performed such as enabling a feature.

Well I had put the Stop-SPAssignment function being called at the end of the function. Guess what happened?

The PowerShell script would eat loads of RAM. Actually the memory would be freed but with large web applications the PowerShell script would slowing down considerably as the memory allocated started to impact the server.

The fix was relatively simple, move the Stop-SPAssignment function to the end of each loop which processes each site collection. This kept the PowerShell script from consuming too much RAM and it  performed well.

Not too often

I could have moved the Stop-SPAssignment function so that it is called after each web has been processed, however my testing found that this slowed down the script too much as Stop-SPAssignment was called.

Conclusion

Using the SPAssignment cmdlets are essential to building production ready PowerShell scripts. They will ensure that the servers in your SharePoint farm are not affected by over zealous PowerShell scripts consuming huge quantities of RAM!

PowerShell Script: Add user as Site Collection Admin to all sites in Web Application

May 30, 2014 Leave a comment

 

Introduction

 

A few days ago I had a request from one of the SharePoint team. Could we give him Site Collection Admin rights to all site collections in a web application.

Now one method would be to use a Web Application User Policy (see below) and give them the full access permission.

WebApplicationPolicy

However, we didn’t want to take that approach.

So we looked at using a PowerShell script. The important point was that we did not want to assign the permission using the Owner or SecondaryContact properties of the SPSiteobject. Instead we just wanted to add the user as the site collection admin.

When using the user interface this is achieved by doing the following:-

  • Browse to the site collection
  • Click Site Settings
  • Click Site collection administrators (Under Users and Permissions)
    • Add the user to the list and click OK.
      After a bit of investigation using PowerShell I could see how this permissions was set.

 

Solution

So to the solution, how is a user configured as a Site Collection Admin?

Well it turns out that its based on the following property, SPUser.IsSiteAdmin. Site Collection Administrators have the IsSiteAdmin property set to true.

 

Once that information had been understood, then the script was relatively easy and the following script was created:-

param
(
	[Parameter(Mandatory=$true, HelpMessage='username in format DOMAIN\username')]
	[string]$Username = "",
	[Parameter(Mandatory=$true, HelpMessage='url for web application e.g. http://collab')]
	[string]$WebApplicationUrl = ""

)

Write-Host "Setting up user $Username as site collection admin on all sitecollections in Web Application $WebApplicationUrl" -ForegroundColor White;
$webApplication = Get-SPWebApplication $WebApplicationUrl;

if($webApplication -ne $null)
{

foreach($siteCollection in $webApplication.Sites){
    Write-Host "Setting up user $Username as site collection admin for $siteCollection" -ForegroundColor White;
    $userToBeMadeSiteCollectionAdmin = $siteCollection.RootWeb.EnsureUser($Username);
    if($userToBeMadeSiteCollectionAdmin.IsSiteAdmin -ne $true)
    {
        $userToBeMadeSiteCollectionAdmin.IsSiteAdmin = $true;
        $userToBeMadeSiteCollectionAdmin.Update();
        Write-Host "User is now site collection admin for $siteCollection" -ForegroundColor Green;
    }
    else
    {
        Write-Host "User is already site collection admin for $siteCollection" -ForegroundColor DarkYellow;
    }

    Write-Host "Current Site Collection Admins for site: " $siteCollection.Url " " $siteCollection.RootWeb.SiteAdministrators;
}
}
else
{
	Write-Host "Could not find Web Application $WebApplicationUrl" -ForegroundColor Red;
}

The PowerShell script accepts the following parameters:-

  • -UserName – the user to add as a site collection admin (DOMAIN\username)
  • -WebApplication – the URL to the Web Application that should be updated

The script tries to get resolve the Web Application. The script then runs through each site collection in the web application and ensures that the user can be found in the site collection.

If the user is not already a site collection admin then the property is updated and the user object is saved.

The script is not perfect and could have a bit more exception handling, for example the Get-SPWebApplication call does not check the return value.

 

Anyway the link to the PowerShell script is below (just rename the file extension from .txt to ps1):-

Set-UserAsSiteCollectionAdminOnWebApplication.txt

SharePoint Development: Speed up development with Visual Studio CKSDEV Keyboard Shortcuts

February 19, 2014 Leave a comment

Introduction

Firstly if you are not using the CKSDEV Visual Studio extensions for SharePoint. Then please do yourself a favour and download them NOW!

Wes Hackett and the team have been doing an amazing job to save us time when developing SharePoint solutions.

You can download the Visual studio extension via Visual Studio’s gallery.

Keyboard Shortcuts

I have to say that I was a little late to the 1.2 release which I updated to in November even though it was out in August! Anyway this release is the best yet for SharePoint 2013 and Visual Studio 2012. Finally it includes a SharePoint solution deployment profile to update/upgrade your solutions. Finally, I can throw away my version!

Anyway, I noticed that there were lots more keyboard shortcuts (I am pretty sure these have been there for a while) but I wanted to start making use of them as it really makes life much easier.

Fortunately the CKSDEV development team have a strategy for the shortcut conventions so remembering them is made a little easier.

To recycle a process then the two stage shortcut will always use Alt+R

To attach the debugger to a process then the two stage shortcut will always use Alt + A

So here are the shortcuts which I use the most:-

  • Recycle SharePoint Application Pools
    • Alt+R, A
  • Recycle SharePoint Timer Service
    • Alt R. T
  • Attach debugger to SharePoint Application Pools
    • Alt A, S
  • Attach debugger to SharePoint Timer Service
    • Alt A, T

    This makes life so much easier when you are in the code, debug, fix, code, debug, fix cycle!

Issue applying Workflow Associations to Content Types

November 21, 2013 2 comments

 

Introduction

 

First, let’s set the scene. We have built a solution for a customer with a site collection and within that site collection a sub-web which has a large number of document libraries in it.

For the next release we were looking to setup records management and use the out of the box disposition workflow to allow the customer to decide whether to delete the document or not. The workflow is triggered by an information policy.

The upgrade went well all the feature upgrades worked nicely however then we hit a snag setting up the workflows.

The process of applying a workflow to a content type is pretty straight forward and we did not have any problems with the development, staging or UAT environments. Of course its only when we go to setup in production did we hit a problem.

Applying the workflow to a content type was achieved by doing the following:-

  • Browse to the root of your site collection
  • Click on Site Content –> Settings
  • Click on Site Content Types
  • Click on your content type that you wish to configure
  • Click “Workflow Settings”
  • Click Add a Workflow which takes you into the following screen

addingworkflowtocontenttype

If you look at the image at the bottom of the page you will see the option “Update all content types”, so we set this and clicked Ok.

The page just hung there processing, we left it for a while but still it was hung there. Knowing SharePoint we thought that is fine we’ll leave it. Two hours later still nothing so we started to check the SharePoint Unified Logs and unfortunately we found an error

 

SharePoint Foundation   Database        fa46    High    at Microsoft.SharePoint.SPSqlClient.ExecuteQueryInternal(Boolean retryfordeadlock)     at Microsoft.SharePoint.SPSqlClient.ExecuteQuery(Boolean retryfordeadlock)     at Microsoft.SharePoint.Library.SPRequestInternalClass.GetListsWithCallback(String bstrUrl, Guid foreignWebId, String bstrListInternalName, Int32 dwBaseType, Int32 dwBaseTypeAlt, Int32 dwServerTemplate, UInt32 dwGetListFlags, UInt32 dwListFilterFlags, Boolean bPrefetchMetaData, Boolean bSecurityTrimmed, Boolean bGetSecurityData, Boolean bPrefetchRelatedFields, ISP2DSafeArrayWriter p2DWriter, Int32& plRecycleBinCount)     at Microsoft.SharePoint.Library.SPRequestInternalClass.GetListsWithCallback(String bstrUrl, Guid foreignWebId, String bstrListInternalName, Int32 dwBaseType, Int32 dwBaseTypeAlt, Int32 dwServerTemplate, UInt32 dwGetListFlags, UInt32 dwListFilterFlags, Boolean bPrefetchMetaData, Boolean bSecurityTrimmed, Boolean bGetSecurityData, Boolean bPrefetchRelatedFields, ISP2DSafeArrayWriter p2DWriter, Int32& plRecycleBinCount)     at Microsoft.SharePoint.Library.SPRequest.GetListsWithCallback(String bstrUrl, Guid foreignWebId, String bstrListInternalName, Int32 dwBaseType, Int32 dwBaseTypeAlt, Int32 dwServerTemplate, UInt32 dwGetListFlags, UInt32 dwListFilterFlags, Boolean bPrefetchMetaData, Boolean bSecurityTrimmed, Boolean bGetSecurityData, Boolean bPrefetchRelatedFields, ISP2DSafeArrayWriter p2DWriter, Int32& plRecycleBinCount)     at Microsoft.SharePoint.SPListCollection.EnsureListsData(Guid webId, String strListName)     at Microsoft.SharePoint.SPListCollection.GetListByName(String strListName, Boolean bThrowException)     at Microsoft.SharePoint.Workflow.SPWorkflowAssociation.EnsureUtilityList(SPWeb web, String workflowName, String listTitle, Guid Id, SPListTemplateType templateType, Boolean forceListCreation, String alternateNameRes, String descriptionRes)     at Microsoft.SharePoint.Workflow.SPWorkflowAssociation.EnsureTaskList(SPWeb web, String workflowName, String createTaskListTitle, Guid createTaskListGuid, Boolean forceListCreation)     at Microsoft.SharePoint.Workflow.SPWorkflowAssociationCollection.SetUtilityLists(SPWorkflowAssociation wa, Boolean forceUtilityListCreation)     at Microsoft.SharePoint.Workflow.SPWorkflowAssociationCollection.AddCore(SPWorkflowAssociation wa, Guid id, SPList list, Boolean forceUtilityListCreation)     at Microsoft.SharePoint.Workflow.SPContentTypeWorkflowAssociationCollection.AddCoreCT(SPWorkflowAssociation wa)     at Microsoft.SharePoint.Workflow.SPContentTypeWorkflowAssociationCollection.PushDownAssociation(SPWorkflowAssociation associationTemplate, Boolean bUpdateIfExisting, MethodBase mbChangeEntry)     at Microsoft.SharePoint.Workflow.SPContentTypeWorkflowAssociationCollection.UpdateOrAdd(SPWorkflowAssociation associationTemplate)     at System.RuntimeMethodHandle.InvokeMethod(Object target, Object[] arguments, Signature sig, Boolean constructor)     at System.Reflection.RuntimeMethodInfo.UnsafeInvokeInternal(Object obj, Object[] parameters, Object[] arguments)     at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)     at System.Reflection.MethodBase.Invoke(Object obj, Object[] parameters)     at Microsoft.SharePoint.SPChangeMonitor.ApplyChangesCore(Object ct, Boolean applyAll, Type typeFilter, Boolean bFilterInclude)     at Microsoft.SharePoint.SPContentType.PushDownWorkflowChangesToDerivedCTCore(SPContentType ct, Boolean fCloseWebAsNecessary)     at Microsoft.SharePoint.SPContentType.PushDownWorkflowChangesToListCTs(SPContentTypeCollection cts, IEnumerable`1 cids, Boolean throwOnSealedOrReadOnly)     at Microsoft.SharePoint.SPContentType.PushDownChanges(CodeToPushDownChangesToDerivedCT derivedCTPushdownCode, CodeToPushDownChangesToListCTs listDerivedCTsPushdownCode, Boolean throwOnSealedOrReadOnly, IList`1 exceptions)     at Microsoft.SharePoint.SPContentType.UpdateWorkflowAssociationsOnChildren(Boolean bGenerateFullChangeList, Boolean bPushdownDerivedCT, Boolean bPushdownListCTs, Boolean bThrowOnSealedOrReadOnly)     at Microsoft.SharePoint.WorkflowServices.ApplicationPages.WrkSetngPage.OnClick_Update(Object sender, EventArgs e)     at System.Web.UI.WebControls.LinkButton.RaisePostBackEvent(String eventArgument)     at System.Web.UI.Page.ProcessRequestMain(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint)     at System.Web.UI.Page.ProcessRequest(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint)     at System.Web.UI.Page.ProcessRequest()     at System.Web.UI.Page.ProcessRequest(HttpContext context)     at System.Web.HttpApplication.CallHandlerExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute()     at System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean& completedSynchronously)     at System.Web.HttpApplication.PipelineStepManager.ResumeSteps(Exception error)     at System.Web.HttpApplication.BeginProcessRequestNotification(HttpContext context, AsyncCallback cb)     at System.Web.HttpRuntime.ProcessRequestNotificationPrivate(IIS7WorkerRequest wr, HttpContext context)     at System.Web.Hosting.PipelineRuntime.ProcessRequestNotificationHelper(IntPtr rootedObjectsPointer, IntPtr nativeRequestContext, IntPtr moduleData, Int32 flags)     at System.Web.Hosting.PipelineRuntime.ProcessRequestNotification(IntPtr rootedObjectsPointer, IntPtr nativeRequestContext, IntPtr moduleData, Int32 flags)     at System.Web.Hosting.UnsafeIISMethods.MgdIndicateCompletion(IntPtr pHandler, RequestNotificationStatus& notificationStatus)     at System.Web.Hosting.UnsafeIISMethods.MgdIndicateCompletion(IntPtr pHandler, RequestNotificationStatus& notificationStatus)     at System.Web.Hosting.PipelineRuntime.ProcessRequestNotificationHelper(IntPtr rootedObjectsPointer, IntPtr nativeRequestContext, IntPtr moduleData, Int32 flags)     at System.Web.Hosting.PipelineRuntime.ProcessRequestNotification(IntPtr rootedObjectsPointer, IntPtr nativeRequestContext, IntPtr moduleData, Int32 flags)

 

The reason for this error was explained in the next log entry which was a little more concise:-

Getting Error Message for Exception System.Web.HttpException (0x80004005): Request timed out

 

So a bit of head scratching and we thought well we should really make this change via PowerShell but the problem is how to do that?

 

Solution

 

Well its funny some of the method calls you see on SharePoint objects. Obviously the guys at Microsoft have seen this issue before. When I was updating the SPContentType I remember seeing the function UpdateWorkflowAssociation() but then there was the much more useful UpdateWorkflowAssociationsOnChildren()!

Thanks to the team at Microsoft – that is just what we were looking for!

The fix was the following piece of PowerShell:-

 

$web = Get-SPWeb [your site collection root web url]; $contentTypeToUpdate = $web.ContentTypes | ?{$_.Name –eq “[Your Content Name Here]”}; $contentTypeToUpdate.UpdateWorkflowAssociationsOnChildren($true, $true, $true, $false);

I used ULSViewer to watch the function running SPSqlClient sessions to see that the process was working correctly and more importantly it wasn’t timing out!

 

To talk through the PowerShell we are doing the following:-

  • Getting the SharePoint web for the root of our site collection which contains the content type to update
  • Get the content type that we need to update
  • Call the UpdateWorkflowAssociationsOnChildrenUpdate()
    • mark all the content as changed = true
    • push down to all derived content types = true
    • push down to all content types that are associated to lists = true
    • throw an error if you encounter a read-only or sealed content type = false

Lessons Learned

 

When you are doing upgrades always try and use a similar sized data set as production!

When it does not work through the UI – bring on the PowerShell!

 

Anyway I hope that helps someone else get themselves out of trouble!

Experiences with SharePoint Disposition Approval Workflow

November 14, 2013 Leave a comment

 

Introduction

 

Recently I have been working on document lifecycle management project. One of the targets for the project was to try and use out of the box (OOB) SharePoint features rather than develop additional components. I am all for this, it reduces support and maintenance overhead and the technical debt of the solution.

The solution ended up implementing a multiple content types with information management policies for auditing and retention policies.

The retention policy had a number of stages which were:-

  • Created date + 1 day make the document into a record
  • Created date + X years execute the “Disposition Approval Workflow”

The number of years would vary depending on the type of document.

Before I go any further I should talk about the Disposition Approval Workflow, this is a workflow which has been in SharePoint since SharePoint 2007 and is made available using the Disposition Workflow feature.

The workflow when run against a document has one step. The step creates a task which presents the user with the option to either delete the document or retain it. The user can also provide comments.

What should happen is that if the user chooses to delete the document then the workflow will delete the document.

However, this fails if the document is declared as a record. Unfortunately the out of the box disposition approval workflow which has been designed to provide a mechanism to delete a document cannot delete a document if its a record.

The workflow displays the following error or words to the effect of “Cannot delete the document”

This was a little frustrating so what do we do?

 

Options

 

After some thought about the problem I came up with two approaches:-

  • redesign the Disposition Approval Workflow to work how we want it to
  • add a step to undeclare the document as a record

Thinking back to one of the key targets, keep the amount of custom development to a minimum, I decided to go for the second option.

Unfortunately, there is no expiration action which allows you to undeclare a record. So I looked at the options to create one.

Fortunately there are a couple of reasonable examples, they might be a little lacking in detail but there is enough to understand the process.

 

Solution

 

So the following solution was built, the solution is made up of the following parts:-

  • A class which implements the IExpirationAction interface and perform the action to undeclare a document as a record
  • A feature which has a feature receiver which adds the IExpirationAction to the PolicyTemplates
    A class UndeclareRecordExpirationAction is created which implements the IExpirationAction interface as follows:-
    The IExpirationAction interface has a function OnExpiration which passes in the SPListItem to delete and the date that the document has expired on.
    The function implementation would be to check that the document is not already on hold and also that the document isn’t a record already.
    if the document is on hold then an exception is raised so that the process goes no further.

Two static functions were created which register / unregister the ExpirationPolicyAction with SharePoint.

 

A feature was created which implemented FeatureActivatation and FeatureDeactivation methods. These call into the appropriate static functions in the UndeclareRecordExpirationAction class.

 

Once the solution is installed and the feature activated then a new policy action appears here:

image

 

To configure the policy do the following:-

  • Browse to the site collection
  • Browse to the site settings page
  • Click Site Content Types
  • Click on the content type that you wish to apply the policy to
  • Click on “Information Management Policy settings”
  • Click “Define a policy..”
    • Click Ok
    • Click Enable Retention
    • Add a retention stage…

    The information policy was modified so that it has the following steps:-

    Created + 1 day = Declare record (reoccurrence 1 month)

 

    Created + 6 Years = Undeclare Record (reccurrence 1 year)

 

    Created + 6 Years = Start Workflow (reoccurence 1 year)

The option to specify how to manage retention on records was set that the same policy was used for both records and non-records.

I set the reoccurrence for the Declare record action to one month so that if a document was undeclared and then the record managers decided not to delete the document through the workflow it would then picked up again.

 

Testing the solution

The process of testing these expiration policies was a little tricky so I will cover the method that used. Information policies are applied by timer jobs. The following timer jobs are used:-

  • Information Management Policy Timer Job
  • Expiration Timer job

The expiration timer job does the actual work of applying/running the various steps of a retention policy. By default it runs once a week in SharePoint 2013.

To speed things up I did the following to test/debug the solution.

  • Compile the assembly hosting our UndeclareRecordExpirationAction class as a debug build.
  • Deploy the solution
  • Restart SharePoint Timer Service
  • Attach the Visual Studio debugger to the “owstimer.exe” process
  • From Central Admin, start the timer job through the monitoring->review job definintions
  • Browse to the Expiration Policy Job Definitions
  • Click on the “Expiration policy” link for the one associated to the Web Application where your application is hosted and click “Run now”
  • Wait for the visual studio debugger to hit the breakpoint

If you are having problems getting the debugger to hit the breakpoint then:-

  • Start the SharePoint Management Shell
  • From within the PowerShell session
  • restart the owstimer service using Restart-Service sptimerv4
  • restart IIS using iisreset

Improvements

 

The only thing that I may look to do is rather than have two stages for “Undeclare record” and “Starting a workflow” is merge them into two and have a “undeclare record and start workflow action” so that there is less delay with the processing.

 

Conclusion

 

I hope people find this useful, please let me know if you do.

 

Here are links to the code samples:-

ITSP.SP.Policy.UndeclareRecordAction.zip

%d bloggers like this: