Problems with SharePoint Governance

Now hang on, this post shouldn’t be misconstrued as attacking the concept of governance as a whole, it is more rantish. I am not going to portray that “Buenz Thinks Governance is Bad”, I do violently believe that cultivating long-term value from an arbitrary SharePoint deployment requires a specified level of governance. I spoke on the exact subject at the last SPC conference, I eat the Wheaties. I just think there are a lot of tribulations with conceptualization, use, and implementation which leads to the prostitution of the notion which is maddening.

And I am of the firm belief that I can’t be the only one.

SharePoint can curiously be subject to colossal content sprawl which fundamentally equates to a failed implementation, and governance is generally put into place to curve that, otherwise, there are business-class nightmares. This is due to the fact that SharePoint can be considered as naturally organic and amorphous, empowering users which in turn can lead to content chaos. This can be multiplied by the fact that current business process problems can be replicated in SharePoint, and because of its reach, be even more of a problem than before. This is 100% the space that a governance strategy seeks to fill, and can productively accomplish when acknowledged as a series of piece-meal cogs that construct a complete machine. Loosely defined, let’s just consider SharePoint governance as a strategy to ensure quality of information architecture and associated content taxonomy while keeping in mind defined business objectives, ensuring procedures are defined for support of the aforementioned. Ahhh, nice and ambiguous.

I see more and more people doing three things in regards to SharePoint governance:

1) Downloading random governance templates / processes. Implementing them without taking into account the organization, then when failure occurs putting said policies in the same place as the old company BetaMax.

2) Thinking that a series of policies is actually going to be effective for an enterprise SharePoint governance strategy automagically. Governance tooling is required, and third party components will never provide a holistic closed-loop governance solution.

3) Using the word governance to characterize 50,000 things. Then using it incongruously because it sounds super nifty and is so ambiguous that hey, it should apply. Right? Yeah let’s do that, buzzwords are fun.

I Found This On A Blog. Let’s Do This. Like Nowish and Be Done With it.

Firstly, let’s start off with the misconception that SharePoint governance is solely policy definition and implementation. A lot of current organizations are downloading, briefly examining, and implementing governance policy definitions with little or no foresight into the auditing and enforcement of said governance policies. They just sit there; though can without difficulty be referenced in order to defend the deprecated state of a SharePoint implementation when such information is solicited. But lest us forget that standard, bare-bones policy management by definition is composed of three things:




Most people I see starting and stopping with the first one, which brings up the question, how the hell do you actually measure the effectiveness of the governance strategy when you have no means to gather indicator data? Sure, you can build some basic indicators through the internal use of SharePoint auditing and associated concepts; however that is woefully lacking in what is required for an inclusive governance strategy. Closed-loop SharePoint governance software is required in order to actually mine and massage required data (which I will get to in the next segment, small deviation), meaning that all three concepts lie in the same domain, and relationships between the three are inherently procured as part of the system. I mean, part of the standard definition of governance is verifying performance! While documents can provide benchmarks in order to empirically study some segments, translation of the information cannot be inherently provided with ease.

Furthermore, when defining the governance policies it is necessary to contextualize them, and this requires actually reading through the policies you are considering and tailoring them to an organization. If you don’t its literally like talking to a go to a car dealership and saying Hey you, slimy sales dude. You can have this blank check. I signed it. I just want something that gives the impression of having a four wheels and movement. A cardboard picture of an engine being under the hood as opposed to the real engine sounds pretty awesome. Lightweight! I’ll do that.

In the same respect as this post where best practices were discussed, governance requires taking into account all sides of the industry shape, not just piece-meal portions that seem attractively easily to put into place. This is something referred to as a Fragmented Governance Implementation, since you really have only hit aspects that seem on the surface as imperative without bearing in mind the relational nature of governance concepts.

I am NOT alluding to that broad governance concepts have no applicability; they can actually be useful and educational when leveraged in the right way. But they can also have severe implications, and be very, very dangerous when used in an erroneous way. You don’t run around blindly signing contracts, and implementing a governance policy is the *exact same thing* as a contractual obligation. It is effectively a SLA between an IT entity and itself in order to maintain platform sanity and ensure information architecture maintenance.

Governance Requires Tooling. Custom Tooling At That.

And if you think it can be solved with a one-size-fits-all third party component for everything (traditional architecture governance, data cleansing governance, etc.) since it looks like a low hanging fruit solution, you will speedily find yourself in a pigeon hole. A really small one that is uncomfortable and has glass shards glued all over it. Do you really think that there can exist a SharePoint governance application that allows IT organizations to create or adopt IT governance frameworks and structure, manage, and maintain the processes and activities required for meeting ANY governance objective? Puu-leaaaze.

Proper governance requires vertical sensitive tooling in the same way that applications are developed with industry understanding in mind. Assuming that the same series of governance steps will provide the same governance results is 100% false, analogous to someone claiming that a Task Management System software package can be used OOB in every sector with no friction. I can’t even count how many Task Management Systems I have written for SharePoint, and there were all severely dissimilar. And while they share broad concepts, the guts of the software was very different and would not translate well in-between each project. This is something that should be ported to the concept of governance, and why there can never be universally applied policies or a shipped software package that provides closed loop governance, deployments are unique and individual to organization. Like I said, I am not alluding to that broad governance concepts have no applicability, nor are there no available inbuilt mechanisms within the shipped SharePoint software that procure some tooling. However it woefully lacks the granular components required for a holistic closed loop governance solution and is effectively ignorant to organizational particulars.

For all intents and purposes, organizations that are serious about governance need to inspect their current business state and develop the governance strategy around it. This will lead to tooling that can use the overall governance concepts as a base to inherit more defined governance procedures around that are sensitive to the company. I have always purported that it is the poorest decision an architect can make to try to tailor an entity to a product; governance is no exception and should be treated accordingly.

Governance Means Everything!
This is seriously how I feel about this:

[kml_flashembed movie=”” width=”400″ height=”320″ /]”

but replace every time Bob Dole is saying Bob Dole with the word governance. People just like this drive me up the frickin wall because every 20 minutes it’s SOMETHING about governance. Governance this, governance that. Governance, governance, governance.

I don’t really want to argue semantics which I am sure I am going to hear about anyways, but I know I am not the only one in this camp that grows weary of the repetition and assumption of immediate applicability of word governance. Overuse of the word has actually made me do the RCA dog look to people in meetings that actually bring it up now. It’s becoming difficult to take seriously.

This blog post is too long so I am going to cut it off. I might continue it later but my fingers are going to need Band-Aids soon from all the typing.


Cloning SharePoint List Security (Freeware Security Tool)


I have run up into a cloning issue on other occasions, quite recently actually, since the project I am currently slated for has a really complex scheme in order to control user access to our local SharePoint instance and its related objects, as such sometimes rudimentary tasks like making two lists have identical user sets can prove to be a pain in the ass, so it would help to provide an automated way to do such tasks. Since this is always coming up, particularly during the initial provisioning process.

What lead to its creation was I was building a task management system that I am using some helper lists for, just storing some basic data that is pulled in by some sister WebParts. I was creating, deleting, recreating the lists and it became very, very tedious to keep setting the permissions. Although I could have changed the inheritance of the parent and just relied on feeding permissions, this unfortunately did not work for my particular situation due to some interesting trimming components that rely on certain security attributes from the parent. To complicate matters further, with the initial content provisioning that followed needed to ensure that sub-items (SPListItem’s and SPFolder’s) would if the names matched clone the item(s) security as well.

To make a long story short, it had to be done manually, there was really no native way to tackle the issue. And manually hurts.
From a project type standpoint, what I needed was a tiny WinForms application that would allow me to take a baseline stamp of existing permission sets as they are currently were bound to an SPList object, then take that and apply it to a new SPList object with just a couple clicks instead of having to wade through the plethora of SharePoint pages I otherwise would have to go through. This shouldn’t imply that it couldn’t be done in a WebPart or some other medium, however this was the quickest method. I will most likely look into a conversion of it later.
So, the process flow that I was looking to solve was a relatively simple one.

1) Locate / select a pre-existing list as it exists in SharePoint. This list should have the appropriate permissions on it representative of the set you would like to clone.

2) Find the secondary list that has a malformed / not yet set permission set.

3) Ensure that the permissions that are provided in the secondary do indeed represent the set that you would like to clone.

4) Begin the cloning taking into account checks like whether both objects inherit from the parent, which would definitively not represent an actionable function (other checks on inheritance are performed as well).
5) During the cloning, iterate through all SPListItem / SPFolder objects that are found. If the name matches, clone the permissions of the item as well.

So, without further hesitation, the SharePoint List Security Cloner, which is a very simple application taking to view URL’s that coordinate to the baseline list as well as the target list. All you have to do is put these two parameters into the application, and then following execute the application, the tracing of the operations will be written to the TextBox, following the adjustments to the list security should be accessible from the web interface.

Firstly, the main interface of the application:

As you can see in the above image, you are afforded to inputs into the application. The first is the source URL, or the SPList View URL of the list that you would like to use as a baseline. The second TextBox is the destination URL, which represents the target for the cloning as explained previously.Following, I am going to choose two URL’s of two different lists, cloning my Tasks list security attributes to my Calendar list. This is what the security of them looks like currently.Firstly, the baseline list permission settings:

Secondly, my Calendar list permission settings:

We can see that the variation between the two lists are development Members, Robert Lyon, and Viewers. I am going to now use the SharePoint List Security Cloner, and place the AllItems.aspx URL into the appropriate slots.

Once the execution starts, you will see a progress bar at the bottom of the main form that will disappear again when the execution is complete:

You can track the operations progress in the “Cloning Operations” section as well that will inform you of the specific progress that the application is making.

Afterwards, when I look at the permissions for the Calendar, I see that the security attributes were successfully cloned over!

Anyways, it was written for fun and my own utility more than anything, so I really can’t guarantee its going to work. Read the software disclaimer on the main site before downloading please! If you do run into errors or have feature suggestions to enhance it (posting such things in the comments generally motivates me to) please do post in the comments. I think my next version is still going to be a windows application but I am going to add the ability to toggle between sites, lists, etc.
Download SharePoint List Security Cloner (the interface might have moved around a little bit from the above screenshots but the functionality is consistent).


Why Microsoft Data Protection Manager Will Replace Your SharePoint Tape Backups

* This article was written in the context of System Center Data Protection Manager 2006 (SCDPM), a technology now considered deprecated with the introduction of System Center Data Protection Manager 2007. Variations may exist. *

Why Microsoft Data Protection Manager Will Replace Your SharePoint Tape Backups
Typically, within organizations it is common to have a backup strategy where your critical SharePoint data is backed up to tape, and either taken to secure on-site locations or to a designated off-site sheltered faculty. Tape backups have been a reliable way to backup SharePoint data for an extended period of time, however this type of disaster recovery, although typically reliable, tends to be slow for restoration of crucial business processes.
The Three Types of Backups Processes
There are three main types of backups that exist for SharePoint (there are obviously several others that can exist, however in the context of this particular article):
  • Disk-to-Tape (DtT)
  • Disk-to-Disk (DtD)
  • Disk-to-Disk-to-Tape (DtDtT)
The latter of the three is the most advanced, and relevant to a DPM implementation protecting a SharePoint environment. Although legacy networks are most familiar with DtT backups, this method alone is not advantageous to a SharePoint environment which needs a more agile disaster recovery framework so the business processes and the environment that information workers are used to can be ensured.
The second of the three, Disk-to-Disk backups, are much different that Disk-to-Tape backups for one overlying reason. Instead of populating backup material to a tape directly, it is copied to another server within your network, typically a network/file share. Similarly, within a Disk-to-Disk-to-Tape strategy, your SharePoint data is backed up into a network shared, and then pulled off that share onto a tape for offsite storage, while maintained on the file share for agile backups.
Why Combine Tapes with a Disk-To-Disk Strategy
Why are these two methods being combined anyways? It seems that in the long run, with a Disk-to-Disk-to-Tape strategy, there is a mixture of steps that could otherwise be handled with a simple Disk-to-Tape backup strategy. While this is true, one of the benefits of implementing Microsoft Data Protection Manager is that it allows automation of these steps in order to protect your SharePoint environment.
Picture first your SharePoint environment. Assume that you are involved with a medium sized company, around 5,000 employees each of which is heavily dependent on your SharePoint implementation for line of business applications and facilitating communications and collaborations within virtual teams in your organization. Your arbitrary SharePoint implementation is a medium server farm consisting of two front-end web servers, a separate server that facilitates indexing and job functions, and a backend SQL server. Within your SharePoint implementation are several file shares exposed as well which house certain content which don’t necessitate the need of revision controls which are provided by SharePoint such as .iso and .exe installation files (maintained in the blocked files list to protect the portal from malware). Within your SharePoint environment you also have 1 server dedicated to DPM processes that help to facilitate disaster recovery within your environment providing full fidelity backups for your 250 SharePoint site collections.
These site collections are critical for your business operations for multiple reasons, including however not limited to document repositories, revision controls, task management, and integration with a Team Foundation Server implementation providing your developers and program managers insight into your Software Development Lifecycle (SDLC) and work item tracking.
In the legacy backup strategy, your environment database files are placed on tape and moved off-site every morning at 2:00 a.m. in order to harvest the most recent data and not interfere with user activity.
Your CEO just uploaded a critical document to a document library whose subject is the quarterly fiscal budget, also including a PowerPoint presentation that is going to be shown to shareholders. Without these vital metrics, there will be less interest in the company and it is feasible that some of the shareholders may pull their funding and throw the company into a financial disarray.
And Then, a Catastrophe Occurs
Disaster strikes. Another user accidentally uploaded a document infected with a piece of malware that essentially turned your SharePoint server into a large paper weight, corrupting several pieces of functional SharePoint data and brining down your farm. Your CEO is in a state of panic because of the implications of not having the presentation and document available, and he is holding you responsible.
You tell him not to worry because as the SharePoint administrator you have rights to gain access to the tape backups. However, the CEO loaded the document at 9:00 a.m. this morning working on it feverishly all evening, making it not feasible for you to actually reload the document, so his work has been lost and now there is a possibility of shareholders not observing relevant metrics, and losing interest in the organization.
With DPM, this situation could be avoided. Using DPM, you can make a full backup of your SharePoint data (after export) and file stores so that if any relevant data is lost at anytime during the day, it can be restored, even in hourly increments. Once that data has been modified in coordination with the synchronization schedule, it will be pushed block by block into your backup files, ensuring that business critical data can immediately be pushed back into your environment.
This means that the CEO will be able to bring up the corporate portal during his meeting with shareholders, and even though his file was uploaded to the document library at 9:00 it can still be restored in enough time that he will have all of his relevant assets he needs to ensure his shareholders that they are making the right investment. Even better, since the CEO should have access to the relevant backups, he can even invoke the DPM UI and restore the backup himself. Other users can take advantage of this feature as well, depending on permissions that you set up. If the CEO didn’t have access, assuming he is not incredibly tech savvy and therefore his access is restricted to certain resources, you will most likely be responsible for restoring his relevant system status. This is easily done through the DPM UI, which is easy to facilitate through a Windows Explorer type snap-in, for both you as the administrator of the SharePoint environment as well as your users.
The Shrinking Window of Data Backups
Eliminating this 2:00 a.m. restoration process is eradicating the shrinking window of database backup. More and more data is needed to be backed up relating to your SharePoint environment, and there is less and less time during a 24 hour window for you to create these backups. The shrinking window isn’t large concern when you have an implementation of DPM since the data is constantly backed up for you to restore whenever problems may occur with your portal, which is quite useful for proper disaster recovery.
As described before, as the SharePoint administrator responsible or your network, you are responsible for proper disk allocations and how your backups are stored. Having space for multiple versions of large SharePoint environments might seem to be not advantageous for an environment based on disk-to-disk data storage, it would take up a fair amount of space if you have multiple SharePoint site collections with large content repositories!
Adaptive Copies Within Microsoft Data Protection Manager
DPM handles this type of allocation quite nicely by using adaptive copies, only moving the changes so that you can save disk space for an environment that can have incredibly large backups already pegged with network bandwidth allocations issues since users are typically relying heavily on SharePoint for virtual team environments. Even more relevant to SharePoint is that while users are currently hitting your portal environment backups can still be made of the file stores, which is crucial for a communications and collaborations platform which is typically under constant use.
Unhealthy Storage Limits Within A Disaster Recovery System
DPM will also warn you if you are exceeding an unhealthy storage limit, which by DPM standards is if you hit a threshold of 75%. This is an atypical situation, and should realistically cause two courses of actions.
1. What are my physical storage options, do I have proper disk allocation?
2. Would my DPM configurations be causing this issue?
DPM has several inherit calculations built into it that will help you as the SharePoint administrator. Using a disk-to-disk-to-tape backup solution further should emphasize why you should not be getting these types of messages, since your legacy data should eventually work its way off the disk-to-disk portion of the backup solution and should eventually move to a tape for off-site storage.
Errors Due To Lack of Space
Within a SharePoint environment, since the data is constantly changing, the cause of these types of errors is because within a platform that promotes virtual teams that data is changing constantly, and DPM upon initialization will make intelligent configuration options as to how fast your backup data will change. If the data changes over this threshold, the shadow copies of the data will grow to quickly and will cause DPM to become confused.
Solving this issue quickly is easy, by adding more space. The two options for adding more space are:
1. Add more disks
2. Increase the storage allocation of the DPM server