The List View Threshold In SharePoint 2010

There are several things to take into consideration with the list view threshold in SharePoint 2010. There are several operations here including non-indexed queries that allow for columns to be added. They can be used to make a list and to give time for resources that are proportional to the number of items in the list. When you have a small list you don’t have to add items quickly. The bigger the list gets though the more the related operations is going to take up your available resources.

The list view threshold is a safety in place to let you know when you should change the query. It also lets you know that the data can be accessed for performance when the farm usage is low. The list view threshold is the maximum number of items that a database can hold at once. The default is set to 5,000 items and it will impact how our holistic system operates so don’t increase it.

You don’t want to prevent a query from being filtered on columns that aren’t indexed. When that happens there is a filter in place and the correct data set will be affected. You need to allow the default value for the limit to be based on farm and list performance. This is how the SQL Server is able to manage locks, so it is a good idea to leave it just the way it is.

You want to minimize database contention and the best way to do so is with the SQL Server row level locking. This is a strategy that allows for accurate updates to occur without affecting users when they have access to other rows. If a read or a write database operation including queries is in place it can mean that 5,000 rows are all locked at the same time. It will be handled more efficiently when the SQL Servers in place to serve as the lock. This means the entire table will be complete. It is going to stop other users from being able to access that table.

The mixture of queries allows for the return of all the items in the list and then more items can be returned later on. When the limits are changed from the default of 5,000 to 10,000 there is a huge impact of the overall performance. It is a good idea though to focus on high performance from the queries.

With list view threshold there are some exception due to the fact that there are operations to perform. They may not do well though if you reconfigure them. They have to be raised to the limit you will need so that they can sufficiently operate as you perform them. The worst that can happen is that you will need to change the enable throttle setting for a given list to false. Then you can ignore the list view threshold.

However, this can only be accomplished at the list level. You can’t do it for a site. You should only be doing this when you want to allow a list access while changes are being made to fix poor performance operations that are being blocked. As soon as you are done with all of that the enable throttle setting needs to be changed back right away.

Both farm and local computer administrators on the web end of things where the server is and the query starts won’t be blocked by the list view threshold. They are users that are browsing large lists and it could be found that some of them aren’t properly configured. They have to be careful when testing them so that the data is normal when other users see it. This is why some operations are prevented by the list view threshold.

There are time services that can be run with an account that isn’t protected by the list view threshold. This allows for various scenarios to be in place including a creation of an index for a large list later on. The use of general will apply so that there aren’t problems during such a scenario playing out.

With the list view threshold for auditors and administrators, the list view threshold for service account is in place. There is a limit for caching the results of a large query and therefore it saves resources. The custom code allows for a request to use the higher limit for running an account that is going to comply with the web application security policy.

Object model override determines if the service accounts can be part of the list view threshold for auditors and administrators. With a farm administrator there is the ability for the object model to be overridden. This is a program specific ability that has some exceptions.

The programmers with authorization can request for a query or a list and then benefit from it. They can change the value so that custom codes can be used to override what is already in place. It is a good idea to leave this setting at the default.

There is a daily time window that can be set for the operations to be performed. This is allowed to occur without being subject to the list view threshold. The time can be changed by 15 minute increments for a period of up to 24 hours. The database operation or query will start with the daily time window where it continues until it is completed so that it doesn’t finish in the time specified.

By default the daily time window won’t be configured because of the fact that off peak hours can vary. We suggest that you only have a daily time window in place to be specified if you have reasonable off hours. The time frame to use would be when very few people will be using a given Web application. The users are able to perform administrative operations for large lists. This includes creating indexes. All of this is best to do during those periods of time when farm usage is lower than normal.

Share

TFS Proxy Server Unexpected Shutdowns

TFS Proxy Servers are essential for my current client’s TFSenvironment because they allow the disparate SharePoint development environment to experience improved network performance by caching copies of VC files. Since this particular environment is geo-distributed, this is a necessary architectural requirement in order to maintain appropriate developer efficiency.

Recently, a strange issue was occurring with my clients geo-environment where the proxy servers would start shutting down repetitively. The exact error you may run into is:

The VSTF Proxy Server stopped at [server]. The application is being shutdown for the following reason: HostingEnvironment. For more information …..”

Now this can happen for a variety of reasons, but first thing is you should enable proxy server tracing to get some more relevant error information by opening the web.config in the VersionControlProxy folder by setting the traceDirectoryName to a familiar storage location and changing traceWriter to true. For this particular error, one of the error returns can be:

Detailed Message: TF53002: Unable to obtain registration data for application VersionControl.
TF30055: Visual Studio could not find or read the Team Foundation Server server name in the configuration file. Contact your Team Foundation Server administrator. (type VstfNotConfiguredException)

If you get this error, the TfsNameUrl appsetting is not configured in the web.config file for the proxy server. Locate the:
[xml]

[/xml]

element and change it. After, check your IIS app pool setting and check that the recycle interval or memory limit. After, you should be good to go!

Share

The Definitive Guide To MOSS Pluggable Authentication Providers

Want To Skip Directly to Implementation? Check out the Universal Provider Framework (free of course) for Universal Membership, Role, and Profile provider schemas, getting you up and running with nearly any custom database type in less than 30 minutes. The following membership providers where defined using the framework for your convenience:

There are a total of six classes that make up the Universal Provider Framework.

Download Visual Studio Project File


SharePointMembershipProvider.cs – View Online | Download Class File


SharePointProfileProvider.cs – View Online | Download Class File


SharePointRoleProvider.cs View Online | Download Class File


SharePointUsersProvider.cs – View Online | Download Class File


GeneralUtilities.cs – View Online | Download Class File


UserData.cs View Online | Download Class File

SQL Pluggable Provider Management
If you are looking for a way to interact with the SQL provider database, you can find the ASP.NET 2.0 Provider Manager and the business layer classes that it uses here.

Restricting Security Features in Previous Versions of SharePoint and Improvements
In previous versions of SharePoint, there were many built-in security mechanisms and features that allowed a granular collaboration environment with varying types of application architecture. Within this antiquated security framework, one of the most frustrating, foremost restrictions was that SharePoint user accounts were required to resolve back to a Windows identity, severely impacting the application extendibility, particularly with perimeter facing deployments. There were workarounds to adapt to the restriction, creating multiple Active Directory trees and local user accounts was exceedingly common, however for large extranets this was not only a management nightmare but lead to poor security protocol management.


MOSS 2.0 Security Request Flow and IIS Handshaking

The IIS / ASP.NET 2.0 security process flow is relatively straightforward from when a client initiates the authentication handshake for proper verification and routing to the relevant MOSS web application and MOSS zone.

  1. When the request first begins the handshaking process, IIS will ensure that the incoming client request comes from an IP/host that is allowed access to the domain. If this condition the packets are rejects and the request is rejected by the MOSS server.
  2. Assuming that there are relevant IIS authentication routines (such as basic, integrated, digest, etc.) the MOSS IIS instance will perform that specific authentication.
    1. If you select anonymous authentication, IIS does not perform any authentication by default. There is further configuration required to get anonymous authentication to work with MOSS>
    2. If you select basic authentication, users must provide a Windows username and password to connect. This information is sent across the network in clear text, by which it is natively insecure. Therefore, this typically means that SSL will be used.
    3. If you select digest authentication, users must still provide a Windows username and password to connect. The difference that exists between this and bsic authentication is that the password presented will still be hashed after it is sent by the user. Since this is still an IIS authentication routines, the user Windows accounts wil still need to be stored in a network accessible Active Directory.
    4. If you select Windows integrated authentication, users still have a Windows username and password, however the authentication routine will depend Kerberos or typical challenge/response (NTLM). There are some further settings that are needed in Internet Explorer to make the user experience seamless when leveraging Integrated Windows Authentication.
  3. If there is no configuration done in IIS it will leverage anonymous access so client authentication handshake requests will be automatically passed through and authenticated as legit users (although this will not allow access to MOSS, since the Windows membership provider will be used by default). These types of authentication are set on a per virtual server basis, whereas the MOSS authentication providers can be set on a per web application basis (via zones), and multiple site collections can exist in varying web applications, on n number of virtual servers (it is important to remember that the concept of always having to resolve to a Windows identity doesn’t necessarily have to exist in MOSS).
  4. The first thing that MOSS will do when the authentication request is passed to it will see whether impersonation is enabled.This is how MOSS functions with the varying authentication providers, which allows ASP.NET to acts as an authenticated user.
  5. Finally, the identity from the previous step is used to authorize resources from the Microsoft Office Server System. This is based on the authentication providers that are configured as well as varying assets that can affect the authentication providers such as zones and web application policies. The MOSS resources obtained don’t even has to be restricted to the MOSS webforms, since Code Access Security (CAS) can enable exposure to such things such as keys, disks, and various other server resources.

One of the chief improvements in the new revision of SharePoint (Microsoft Office Server System, or MOSS) is the membership and user model which builds off the revamped ASP.NET 2.0 membership model providing user credentials and user roles functionality (with the addition of seven new server-side controls, membership classes to retrieve and update user information within a database, and role management functionality). This new SharePoint /ASP.NET 2.0 membership model presents state-of-the-art provider APIs that allow a SharePoint environment to talk with a variety of backend corporate user account systems, some providers being provided by default whereas others require creating a custom provider, or simply downloading an already created one from sharepointsecurity.com. Forms-based authentication, the subject of another article, integrate extremely well with this pluggable model, however they are not dependent on each other and can work as independent pieces of technology depending on requirements.

The membership architecture can depicted well visually, as shown in the following diagram:


So How Does This Membership Model Exactly Work?


There are three main pieces that build the application architecture of a membership provider in ASP.NET 2.0. There is the membership API, the membership provider, and the provider specific storage. The actual logic process of the membership model is very simplistic because of its relatively straight-forward design pattern that provides a high layer of abstraction. Instead of being restricted when using the provider API by simply providing methods to tap into a data store through it, the API is flexible and can be molded by a developer, along with definition of the user member storage mechanism.
[csharp]
public class MembershipProvider : ProviderBase
{
// Public properties
public abstract string ApplicationName { get; set; } public abstract bool
EnablePasswordReset { get; } public abstract bool EnablePasswordRetrieval {
get; } public abstract bool RequiresQuestionAndAnswer { get; }
// Public methods
public override void Initialize (string name, NameValueCollection config);
public abstract bool ValidateUser (string name, string password);
public abstract bool ChangePassword (…);
public abstract MembershipUser CreateUser (…);
public abstract bool DeleteUser (string name, bool deleteAllRelatedData);
public abstract string GetPassword (string name, string answer);
public abstract MembershipUser GetUser (string name, bool userIsOnline);
public abstract string ResetPassword (string name, string answer); public
abstract void UpdateUser (MembershipUser user);

}

[/csharp]

The main class within the membership API is the membership class. The membership class only contains static methods, and it doesn’t require an object instance. Though the controls handle the majority of desired functionally, ASP.NET 2.0 provides public methods of the Membership class to expand the developer’s control. A few of these include:

CreateUser
Adds an arbitrary user to the MOSS membership data store
DeleteUser
Removes an arbitrary user from the MOSS membership data store
GeneratePassword
Generates a random password of a specified length for access into MOSS
GetAllUsers
Retrieves a collection of MembershipUser objects representing all currently registered users for the pluggable based MOSS environment
GetUser
Retrieves a MembershipUser object representing a user
UpdateUser
Updates information for a specified user
ValidateUser
Validates logins based on user names and passwords
ChangePassword
Changes MOSS user’s password
ChangePassword- QuestionAndAnswer
Changes the MOSS users question and answer used for password recovery
Get Password
Retrieves a MOSS user password
ResetPassword
Resets a MOSS user password by setting it to a new random password
These members reference the MembershipUser class, which represents a user stored in the Membership data system. As stated before, this MembershipProvider class can be extended to create a custom providers for a variety of systems, allowing you extend the way that users can authenticate to your SharePoint environment.
As an example of how these methods work, lets look at how one could programmatically create a user for your MOSS environment to consume. This method could be hooked to any number of pluggable providers, depending on your backend membership data store:
For the first portion of the code, you simply need to call the CreateUser method and fill in the appropriate string entries to populate the membership database with relevant information about the user:
[csharp]
try {
Membership.CreateUser (“Adam”, “Buenz”, adam@sharepointsecurity.com);
}

[/csharp]
Following, you should allow there to be some flexibility to be procured for handling if a user does not meet any of the standard criteria. This can be accomplished by leveraging MembershipCreateUserException.StatusCode along with a switch case.

[csharp]
catch (MembershipCreateUserException e)

[/csharp]
Following, build the switch case then to handle the various types of exceptions that may occur when you are creating the user. These can vary heavily so it is best to be as robust as possible whilst writing them:
For example, we could do the following starting off with the switch case to parse the status code:
 [csharp]
switch (e.StatusCode)

[/csharp]
Following we can build out the various types of exceptions that may occur, some of them are the more obvious:

Check whether there is a duplicate username that exists
MembershipCreateStatus.DuplicateUsername

Check whether there is a duplicate email address that exists
MembershipCreateStatus.DuplicateEmail

Check whether there is an invalid password entered
MembershipCreateStatus.InvalidPassword

The above exception, where we are checking the user password, also has to be implemented programmatically, and can be bound to some of the initial sign on events to validate the password format as users enter into the MOSS instance. For example, it is common for passwords with an organization to require a certain length of characters. This can be achieved by using some regular expressions. Regular expressions are just a way to implement pattern matching within your code, and are supported by all .NET compliant languages.

 

[csharp]
void OnValidateCredentials (Object sender, CancelEventArgs e)

[/csharp]
Then, just create a if statement to start the regex pattern:
[csharp]
if (!Regex.IsMatch (LoginControl.Password, “[a-zA-Z0-9]{8,}”) )

[/csharp]
If they don’t meet the requirements, and use a cancel, e.Cancel = true; (whose full setting is CancelEventArgs.Cancel), you can simply display them a message using LoginControl.InstructionText, such as:
[csharp]
LoginControl.InstructionText = “Passwords must be 8 characters at least”

[/csharp]
Then setting the e.CancelEventArgs.Cancel to true:
[csharp]
e.Cancel = true;

[/csharp]
Along with multiple other types of various exceptions that could occur depending on your implementation.

The schema when using the SQL membership provider provides some great insight into how the database is structured with a pluggable membership provider:



In relation to the membership model for MOSS under pluggable authentication schemes, there are some unique events that are fired in order to facilitate the login process (this events can all be manipulated if you want to provide a custom login procedure to users):

Authenticate

Event that is triggered when a user trips the login process in order to authenticate the user by validating arbitrary passed credentials to MOSS

LoggedIn
Event that can optionally be tripped following a login to MOSS

LoggingIn
Prevalidation of user credentials in order to implement properly formed entry

LoginError
Event that is tripped if a user does not enter appropriate credentials to MOSS

For your login page, you will get a default one by MOSS if you use the internet presence template. However, if you need to create your own there are a variety of Visual Studio controls that are available for you to exploit in order to build your own, and a variety that will allow you to customize and enhance a very standard login procedure.

ChangePassword
User Interface for altering MOSS pluggable passwords

CreateUserWizard
User Interface for creating new MOSS pluggable user accounts

Login
User Interface for entering and validating MOSS pluggable user names and passwords

LoginName
Displays authenticated MOSS user names

LoginStatus
User Interface for logging in and logging out out of the MOSS instance

LoginView
Displays different views based on MOSS login status and roles

PasswordRecovery
User Interface for recovering forgotten MOSS instance passwords  

The Role ManagementProvider
The Role Management provider offers three built-in providers:

  • AuthorizationStoreRoleProvider
  • SqlRoleProvider
  • WindowsTokenRoleProvider

The role provider can be depicted visually as well:


These providers produce a lot more functionality over anything available within previous versions of SharePoint, as well as traditional .NET 1.1 applications (roles typically required the use of the Application_AuthenticateRequest method), which still required developers to leverage the custom roles passing through the ASP.NET HTTP pipelines. The purpose of roles is fairly straightforward, it allows a developer the option of creating rules that can control access to various pieces of content within your MOSS environment. A user does not have to be bound to just one role, but can in fact belong to n number of roles as they are defined by a developer. Programmatically, it is straightforward in adding a user to your MOSS role provider.

Firstly, you have to declare the user in a string in order to enable storage of the unique name that will later be passed into the AddUserToRole method. This is done by just declaring a string as such:
[csharp]
string name = Membership.GetUser (“Adam Buenz”).Username;

[/csharp]
In the above we are declaring a string with the name, name. Then we are using the GetUser method in and passing in the username of “Adam Buenz” to get that user. Following, we have to add that user to the MOSS role provider by using the AddUserToRole method:
[csharp]
Roles.AddUserToRole (name, “I Love SharePoint Security”);

[/csharp]
In the above we are simply passing in the string that we declared before, then declaring what group we are going to add the user to, in this case the role “I Love SharePoint Security”)
They are not a required feature, however when using a pluggable membership provider are easy to implement and manage, allowing you very granular control over your access security.
In order to get the role manager working, it is necessary to enable it. With your MOSS installation, since it will initially target itself to Windows Identities (even the initial authentication provider will be as such), this is disabled. To enable it, locate this section:

Just change the “false” node in the role manager section to true, as depicted below:

[xml]
< configuration >
< system.web >
< roleManager enabled=”false” / >
< /system.web >
< /configuration >
enable it so it is true:
< configuration >
< system.web >
< roleManager enabled=”true” / >
< /system.web >
< /configuration >

[/xml]

 

This can either be done through the WAT (Website Administration Tool) or through directly editing the web.config.

 

One of the features talked about in the overview article is that role cookies can be encrypted. This cookie comes from using the roles class which produces a role caching property. Assuming that the user roles are too large to fit in the cookie or there are some other restrictions, the cookie will store the most frequently used roles.

There are several role methods that related to the new functionality that can be exposed in SharePoint now through the role provider:

AddUserToRole
Adds a MOSS user to a role

CreateRole
Creates a new MOSS role (see example for instance of using this)

DeleteRole
Deletes an existing MOSS role

GetRolesForUser
Gets a collection of MOSS roles for which a user resides

GetUsersInRole
Gets the collection of MOSS users belonging to a specified MOSS role

IsUserInRole
Indicates whether a MOSS user belongs to a specified MOSS role

RemoveUserFromRole
Removes a MOSS user from the specified MOSS role

When a user enters a SharePoint environment where there is role checking being used, it will firstly check whether the encrypted cookie mentioned before is available for consumption. This is because frequent checks for the role manager service will result in performance issues, if instead all the relevant roles of the user is held in a tamper-proof cookie (the cookie must remain encrypted and tamper-proof otherwise someone could spoof it), it is much quicker for SharePoint to access this information. By default, this cookie will be held for 30 minutes (as shown in the below attributes), however how long it is held onto can be manipulated through the web.config file of the arbitrary SharePoint site.
Within the web.config, there are several settings related to how the SharePoint cookie will interact with the user, some of these are fairly self-evident, however explanation of each have been included for brevity:

[csharp]
< roleManager enabled=”true” cacheRolesInCookie=”true” / >
[/csharp]

To enable the cookie, you must enable the cacheRolesInCookie directive, as shown above. There are also the following attributes that are available for you to use for your MOSS cookies: 

  • cookieName=”.MOSSROLES” – what is the name of this cookie?
  • cookieTimeout=”30″ – how long should this cookie last (what is the cookie lifetime?)
  • cookiePath=”/” – what is the path to the cookie?
  • cookieRequireSSL=”false” – does this cookie require SSL
  • cookieSlidingExpiration=”true” – should expired cookies be renewed?
  • createPersistentCookie=”false” – should we implement persistence for the MOSS cookies?
  • cookieProtection=”All” – what is the cookie protection level?

If this cookie is not located on the client machine, SharePoint will instead make calls through the API the role provider leverages in order to determine the role of the user, and match whether the request matches the role in the environment. If the request is successful, the request will be written back to the encrypted (if the cookie is specified to be encrypted, this again is not a requirement) with the most recently requested role. If so, it will replace the last role stored in the cookie with the most recently requested role. Using this model, the role lookup processes is automated and can be immediately consumed by a SharePoint environment with no custom development needed, only configuration and implementation.

Going Beyond Using The Provider For Authentication
Once you have the provider setup, you can begin to build some pretty neat administration tools that can access the data layer that you can wrap to consume the data. One really neat use of this is building management applications that take advantage of the user options that exist in the environment. For example, sometimes you want to give other administrators within your environment the option of managing the pluggable users in your environment within an easy to use client. For example, lets assume that you are going to build a WinForms client to accomplish this task.

Here is the data access layer class files that I used to access the SQL pluggable provider that build out the WinForms client that allow management of all the custom users that are added to the SharePoint environment.

Once you have these classes define that build the data access layer, you can start to do some pretty neat management from within a WinForms client that ease the amount of visibility that you have over the users in your SharePoint environment.

Share