Introducing Free SharePoint Governance Software – Riadenia SharePoint Governance Automation – Part 1

Disclaimer – This post is simply an introduction to SharePoint governance software, Riadenia, which will be released shortly. The software will be released shortly after being QAed.

SharePoint governance has been a subject that people have discussed forever, but it really didn’t seem to become such an important buzzword until the 2007 platform was released. I don’t know why that was, but I never heard it come up previously. There has been a lot written about it, from scripted guidance to tooling. Interestingly though, SharePoint governance, as well as computing governance, is for the most part super arbitrary so standards that attempt to define any “best practice” tend to fall woefully short. They don’t even make sense most of the time in terms of pragmatic application. For any meaningful progress in regards to SharePoint governance, the objective of reform must firstly be defined having regard to the standards that an organization wishes to achieve. As I see it, any undertaking would only be of value to an organization if its ultimate aim were to be the establishing of a framework that would allow for rules of governance. By this I mean a system in which everyone is subject to however remains sheltered from arbitrary governance standards.

In this way it should be stressed that firms should look at governance tooling and guidance not really ever as a completed solution but as a means of enabling them to better apply a SharePoint governance framework. This framework, importantly, would need to remain mutable. SharePoint governance, and it’s related tooling, is neither a project nor a technology. It sponsors a control framework for safeguarding your organization at a level that strikes a balance between business needs and protection needs. Basically your firm needs to have a solid framework in place before any governance automation technology could make a difference. These tools are created to enhance your systems not develop them.

Some might argue that implementing governance in SharePoint is as simple as setting basic IT SLA’s in place, pointing to the existence of some of the inherent features that constitute the wider system of the administration of collaborative (SharePoint) software, an honest and objective assessment would make it patently clear that this is no longer the case. Serious doubts have been cast over the competence and integrity of leveraging such basic features. No less significant is the very low level of confidence in the system as a whole, such confidence being a necessary prerequisite to its effectiveness. In view of this, an intention to address those factors have led to the belief that governance is arbitrary for the sake of there being no effective governance system.  It also becomes apparent that the governance reform initiative must be approached on at least two equally important levels; the SharePoint framework and, for the sake of a better term, the human resource.

So what’s the problem? The real crux of SharePoint governance issues arise from the fact that people take canned SharePoint governance advice, and attempt implementation without tailoring towards very crucial enterprise aspects such as SharePoint deployment intention, company culture, and industry. Rarely will SharePoint governance aspects, outside of the most generic counsel, translate well. These needs are not met satisfactorily by a method tailored around informal recommendations behind closed doors. These factors underscore a need for a mechanism that in my view would best be embodied in an independent commission, automated and managed within the framework itself and ultimately in an automated fashion.

So what does all this esoteric crap actually mean? It entails balancing the practical, with the not so practical. There must be pragmatic objects for each object being governened, and this must in turn contains relevant thresholds that define the characteristic, and in a larger sense the limits, of that object. In terms of SharePoint, this is pretty easy to graft what this should be shooting for, for each object, within the context of the Riadenia – SharePoint Governance Automation,  this means sites (SPWeb‘s) limits must be placed. The reason that SPWeb‘s are a practical target is because they represent a good middle tier proxy object, it isn’t as vast and untargetable as a SPFarm, SPWebApplication or SPSite but it isn’t as specific and narrow as a collection of governance-worthy objects like an SPList. Roping this back in, this problem, and the overall advised approach has nothing to do with the version of the software. Rather, this problems spans multiple version of it, even the objects being mentioned in the above are consistent with those present in the current, and last release (2003 didn’t have, for example, the SPWebApplication and SPFarm objects).

The thresholds themselves are nothing fantastic and mind-blowing. Ideally, to build profiles a definitive model can be built that tells you, for example, to set your thresholds x site administrators / securable site object is “good” in which would allow you to average these metrics and add (eg) 1 standard deviation above the average to help you identify better. Through application of the central limit theorum (conditions under which the mean of a sufficiently large number of independent random variables, each with finite mean and variance, will be approximately normally distributed) you can adjust your threshold metrics to select different sets. However, this is beyond the scope of my simple application! I will take it there one day though when I get the chance to get some feedback on current state.

One important piece of adaptive governance procedures is the introduction of some methods of forecasting. The main problem with this effort in large-scale SharePoint projects is the existence of optimism bias and strategic misrepresentation with project promoters. A consequence of such bias is a high incidence of cost overruns and benefit shortfalls in projects. Thus number of measures aimed at eliminating, or at least reducing, optimism bias and strategic misrepresentation in governance development must be introduced. The measures include changed governance structures and better planning methods. The aim is to ensure that decisions on whether to build projects or not are based on valid information about costs and benefits, instead of being based on misinformation as is often the case today. This is not to say that costs and benefits are or should be the only basis for deciding whether to build large projects. Clearly, forms of rationality other than economic rationality are at work in most projects and are balanced in the broader frame of public deliberation and decision making. But the costs and benefits of large-scale projects often run in the hundreds of millions of dollars, with risks correspondingly high. Without knowledge of such risks, decisions are likely to be flawed. When contemplating what planners can do to improve decision making, we need to distinguish between two fundamentally different situations: (1) planners and promoters consider it important to get forecasts of costs, benefits, and risks right, and (2) planners and promoters do not consider it important to get forecasts right, because optimistic forecasts are seen as a necessary means to getting projects started. The first situation is the easier one to deal with and here better methodology will go a long way in improving planning and decision making. The second situation is more difficult. Here changed incentives are essential in order to reward honesty and punish deception, where today’s incentives often do the exact opposite. Thus two main measures of reform will be considered below: (1) better forecasting methods, and (2) improved incentive structures, with the latter being more important.

Thus there are four types of forecasting for each SharePoint object under the governance umbrella introduced.

Naïve / Bayes

The Naive Bayes algorithm is based on conditional probabilities. It uses Bayes’ Theorem, a formula that calculates a probability by counting the frequency of values and combinations of values in the historical data.

Bayes’ Theorem finds the probability of an event occurring given the probability of another event that has already occurred. If B represents the dependent event and A represents the prior event, Bayes’ theorem can be stated as follows.

Prob(B given A) = Prob(A and B)/Prob(A)

To calculate the probability of B given A, the algorithm counts the number of cases where A and B occur together and divides it by the number of cases where A occurs alone.

Simple Moving Average

A simple moving average is the easiest and most popular technical indicator.

The simple moving average is calculated by taking the arithmetic mean of a given set of data values. For example, the basic 5-day moving average of 5, 6, 7, 8, 9 is (5+6+7+8 +9)/5 =35/5 =7.0

As new values become available, the oldest data points must be dropped from the set and new data points must come in to replace them. For example, the basic 5-day moving average of 4, 5, 6, 7, 8, 9 is (4+5+6+7+8)/5 =30/5 =6.0

4 is the newest data point that has come to replace 9. Thus, the data set is constantly “moving” to account for new data as it becomes available. This ensures that only the current information is being accounted for.

Weighted Moving Average

A weighted moving average is simply a moving average that is weighted so that more recent values are more heavily weighted than values further in the past.

The commonest type of weighted moving average is exponential smoothing. The calculation is quite simple:

P0 + αP1 + α2P2 + α3P3 + ⋅⋅⋅+ αnPn + ⋅⋅⋅

where α, the smoothing factor, is more than zero and less than one, P0 is the latest value on which the moving average is being calculated and Pi is the value i periods previously (usually i days ago).

Exponential Smoothing

This is a very popular scheme to produce a smoothed Time Series. Whereas in Single Moving Averages the past observations are weighted equally, Exponential Smoothing assigns exponentially decreasing weights as the observation get older.

In other words, recent observations are given relatively more weight in forecasting than the older observations.

In the case of moving averages, the weights assigned to the observations are the same and are equal to 1/N. In exponential smoothing, however, there are one or more smoothing parameters to be determined (or estimated) and these choices determine the weights assigned to the observations.

This smoothing scheme begins by setting S2 to y1, where Si stands for smoothed observation or EWMA, and y stands for the original observation. The subscripts refer to the time periods, 1, 2, …, n. For the third period, S3 = y2 + (1- ) S2; and so on. There is no S1; the smoothed series starts with the smoothed version of the second observation.

Adaptive Rate Smoothing

Statistical forecasting technique that takes variations into account through a coefficient. This coefficient is allowed to fluctuate with time to reflect significant changes in the pattern of the activity or phenomenon being studied. Adaptive exponential smoothing is an extended version of exponential smoothing.

All these types of averages, for software to be complete, must and are baked into the final code. By keeping the forecasting approaches ambiguous, each of the concerning SharePoint governable objects can be targeted.

Next Post In Series >> Leveraged Metric Constraints And Building Governance Profiles (coming soon!)

Upcoming Posts In Series >> Using Riadenia™SharePoint Governance Automation (coming soon!)

Read More About Intial SharePoint Governance Software Experiments


Free Software – SharePoint Kerberos Buddy – Detect And Repair Kerberos Issues

The next best thing to a SharePoint security consultant! Kind of.

Kerberos authentication with SharePoint, and some of the middle-tier issues it presents (particularly when examining orthodox double-hope scenarios) can become both arduous, and frankly redundant to fix. A lot of the time it is typical that a SharePoint consultant is presented with a remediation project where the Kerberos environment is malformed, and the specific issue can be attributed to a wide variety of components, on different machines with different roles etc. As such, it makes sense to provide some level of automation for the detection of such misconfiguration since the actions taken for detection are pretty rhythmic. Since Kerberos is a pretty black-and-white technology (either works or it doesn’t!), pushing recommendations for a fix based on a large set of data capture from all subscribed tiers is pretty feasible. Obviously not going to be 100% since environments are also particular, but it can be pretty close and still give relevant, useful advice.

In itself the data capture presents a fundamental issue, since the machines that are involved in a SharePoint/Kerberos mixed-tier environment can potentially be exclusive (meaning, not on the same box so flowing authentication through), delegation problems can arise on four (this can be a lot more, but from a bare-bones BI environment perspective with full break-out of roles) main tiers: the client, the SharePoint WFE, the SSRS machine (obviously when not in integrated mode), and the SSAS machine. The latter two of these becoming vital when implementing a business intelligence solution where PowerPivot and PerformancePoint are relevant issues to consider and certain services like SSAS are off-loaded. Considering account break-out to isolate services to particular identities, the delegation scheme becomes even more complex. So there are a lot of things to consider, and a lot of things that can go wrong. Therefore all tiers have to be considered and data provided from each:

After examining what I feel is a reasonable breadth of Kerberos issues over the past few years there are a lot of common things that can be automatically checked, and solution advice automatically written that help to solve those issues. Some of these are as small as tolerated machine time differences and others as complex as port checks for clustered or balanced SSAS instances in SharePoint/BI environment. For example, consider 20 of the things that the tool will automatically check:

  1. Client Internet Explorer Settings
  2. Client Delegations
  3. Machine Time Differentiation Between Tiers
  4. Proper Domain Trusts
  5. OLE DB Provider (SSRS, SSAS) Types And Versions
  6. Required Data Warehouse Instance 
  7. Provider Versioning Checks
  8. Malformed Provider Strings
  9. HTTP Host Name Checks
  10. IP Address Conflicts
  11. Duplicate SPN’s
  12. Malformed SPN’s (Both Those That Are Causing Errors As Well As Unnecessary Ones)
  13. SharePoint Application Pool Account Delegation
  14. Authentication Provider Types
  15. Configuration Files (SSRS, SSAS, SharePoint)
  16. Connection String Verification
  17. Named Instance Pre-Req’s
  18. SQL Browser Settings
  19. Cluster/Port Resolution
  20. Kerberos.dll Bug-Fix Existence

So, it’s pretty robust as that is just a subset of the checks, there are others that I am forgetting. I am sure I will be adding onto it at a later time as well.

When you first open the tool, you will be presented with the primary analysis screen that will offer very little enabled controls. However notice the fine icon use that is sorta relevant to the word Kerberos, but I think it’s actually just three dogs looking different directions. A majority of the interface buttons will be not enabled since no analysis files are present yet, firstly the tool must be run on all relevant tiers within the environment.

SharePoint Kerberos Main Form

Holistic analysis cannot occur unless properly formatted *.sharepointkerberos files (nothing fancy about the file type just the name I choose when going between a BinaryFormatter) have been generated on all tiers, as you will see shortly when present these files will enable the Analyze option in the primary analysis form. 

Firstly, you will select which role that the machine you are running the SharePoint Kerberos Buddy on (it is possible to run remotely, but it introduces a bunch of possible problems that are not handled currently within the tool) is targeting, either the Client, SharePoint WFE, stand-alone SSRS, or SSAS role. This is accomplished by selecting the Configure Profiling button at the top of the application which will display a new setup screen to adjust role targetting. Once this button is selected, you will be presented with the Kerberos Delegated Environment Role Selection screen:

For the Client option, no further information is required however do not run the tool on the SharePoint WFE *as the client test*. Meaning, collect the data for the client analysis results on a domain authenticated (the tool will tell you if it detects it is running under a local account) client machine that is used to access SharePoint during normal business operations. Kerberos testing for the delegated machine will heavily skew the results if collected on the SharePoint machine. Once the IE Client option is selected, simply selecte the Initialize button. This will close the current configuration form and enable the Client Results button on the main form.

Selecting the Client Results button will display the collated, formatted result in a seperate form:

For the SharePoint tier, a URL must be provided. Best case scenario your most commonly accessed URL is also the default path in your SharePoint AAM settings, and this is the one you should use in this instance. The application doesn’t really dig it when the AAM settings are all over the map. Since your SPN registration will follow certain service protocols based on the bindings you configured for SharePoint, this has to coordinate to the appropriate URL. Select the Configure Profiling option and enter the URL in the SharePoint URL box:

Click the  Intialize  button and then the SharePoint Results button will become enabled. Then you can view the holistic results in the common informational display form:

For a Stand-Alone SSRS instance, the selection will be provided for you and the SSAS settings for how Kerberos functions require a well-formed data source connection string. Once the role is selected and the prerequisite information is populated, simply click the Initialize button.

At this point, I would imagine that someone is wondering what the hell is the Initialize button is actually doing. All it is doing is generating files that you will have to bring over into one, cohesive Analysis directory:

These files also sponsor the pooling of information from the ad-hoc result button clicks. For example final warehouse analysis provides the following information that gathers the required information from the aforementioned:

Once the data capture is initialized, a series of prompts will be visible while the SharePoint Kerberos Buddy is collecting information from both custom routines in itself as well as using a bunch of applications that are distributed natively with the OS and associated server platforms. This information is written to standard output for information purposes, however importantly is dumped into the Analysis directory located in the program installation folder.

These files are the information required for final analysis to occur, and why the Analyze button in the applications primary interface is not available until all requisite files are present.

The file types that are created are pretty standard, the important ones are built as custom file type *.sharepointkerberos. Complete analysis might not be required depending on whether the exact error is caught beforehand by examining the ad-hoc output from the tool, which may or may not point to your direct error, usually it won’t.

After the tool is run, the MSFT reference articles will be pooled and a list of the potential errors (both information, warnings and operation blocking errors) will be written out. These errors will have both a suggested resolution as well as the link to the MSDN/TechNet support site which verifies that this is indeed a practical action to take. I really wanted to make the hyperlinks clickable, but I forgot the whole time I was using a TextBox control instead of a RichTextBox, that will probably be the first thing I change around.

If you want to read more about why you are experiencing the error or the brief resolution path I am suggesting through the tool isn’t enough, just follow the MSDN link! The tool is available on the following CodePlex site:


Free SharePoint WebPart – Simple, Generic Rollup WebPart

The client I am presently at has a comparatively large internal .NET shop since they are a .NET SaaS firm, and a lot of the traditional .NET developers are starting to migrate over to SharePoint for an internal product basis. So we figured the best way to train was to pick a pretty generic WebPart requirement, everyone builds their own, see who does what how, integrate the best approaches from the different projects, and see what the final product looks like.

Thus was born the Simple Rollup WebPart. It is named as such because it is literally the most generic WebPart that I have ever seen. And the requirement that was given was to make the World’s most generic WebPart. So this was successful!

In the spirt of being super generic, it is also really easy to use with a nominal amount of properties requiring configuration. Namely, there is:

  1. A URL
  2. A List Name
  3. A View Name
  4. Amount Up To Query
  5. Amount Down To Query

Noticably, the approach being used is much faster than traditional SharePoint rollups I have seen in the past, way past what the CQWP (Content Query WebPart) performs at.

Ok, so now to the screenshot-tastic demonstration of how the WebPart works. The WebPart is written in a Feature controlled format, following the same approach as nearly every other WebPart available on this site. So, firstly add the SharePoint Solution to the Solution store by running the installation .bat file:

10-22-2009 11-54-48 AM

After, the Solution will be available in the Solution store for deployment:

10-22-2009 11-53-40 AM

Once the solution is deployed, you have to active the Rollup Feature to populate the WebPart description file into the gallery so that you can add it to the page. Navigate to the Site Collection Features, and Active The Rollup WebPart Feature. 

10-22-2009 11-58-16 AM


Once Activated, you will find the WebPart available under the ARB Security Solutions group heading in the WebPart gallery:

10-22-2009 12-00-33 PM

Add it to the page, and you will be presented with the notification that no items have been found (because you haven’t configured the Rollup properties yet!)

10-22-2009 12-00-09 PM

Following, configure the properties for the WebPart. I have two Task lists on the primary and a subsite, so I am going to use that for testing. The properties are explained below:

10-22-2009 12-03-36 PM

  1. The URL to start the rollup at. If you leave this blank then it will use the current context as the starting point.
  2. The list name you want to query for, like “Tasks”, “Discussions”, etc.
  3. The levels up you want the rollup to go.
  4. The levels down you want the rollup to go
  5. The SharePoint view you want to target in the rollup, this allows you to target the data display to particular users since you have use the inherit view replacement variables, like [Me] and [Today].

Following, you will be presented with the columns and values based on those available in the configured view. The grouping is provided in order to split up the values by the site they are gathered from.

10-22-2009 12-08-37 PM

 And that’s it! The WebPart is free, but as always comments about use / backlinks are appreciated.


Link Will Be Back Up Shortly

Let me know if you have problems in the comments :)