List View Thresholds And Blocked Operations In SharePoint 2010

There have been several past posts that deal with the list threshold, such as here. A list may exceed the list view threshold and then some operations will be blocked. The big problem with this is that the default list view can’t be used to access the list, bad news bear! They have to be properly configured before they can work with a large list. The list view threshold blocks the database operations that affects more items than that threshold allows. It won’t just affect the number of items that have been returned.

There are two classifications that come into the picture when you have large lists: “List Exceeds The List Value Threshold” and “Container Exceeds The List View Threshold”.

There are operations that can be blocked when the size of the entire list exceeds the list view threshold. This occurs even if the items are placed into folders. These operations include managing and checking versions, operations of all items, and recursive queries. The views that return all items without folders can also be prevented. There are operations that affect a complete list too including adding a column or deleting indexes and they can be blocked.

There can be operations prevented due to the folder for the list containing more items than the list view threshold allows. You won’t be able to rename it or to delete it so you do need to be careful. The list view threshold can prevent you from performing some common actions when you setup your list. This is why you should configure the columns and indexes for a list before the size is greater than the list view threshold.

Should a list exceed what the list view threshold allows, then you need to plan to configure it correctly. You need to configure view and navigational options well in advance. However, lists can grow beyond the list view threshold and that will require some action from you. For example when you create a column or index that column in a list you need to be prepared for it to take time. The operations are prevented by the live view threshold. They can be performed during the daily time window. They can also be performed by the farm and computer administrators.

The operations need to be planned well in advance. The list may be too big so you will need to use a daily time window. An administrator with the right privileges may be needed in order to perform the necessary operations. It is possible for a list to become so large that some of the operations can time out when they are used in associated with a Web browser.

List Exceeds The List Value Threshold

  • Add/Remove/Update a list column All of the columns including lookup and calculated columns. There are updates such as name change. They aren’t blocked due to the fact that they won’t affect all of the items in the list.
  • Add/Remove/Update a list content type Every item in the list is affected so it is blocked if the list has more items than the list view threshold.
  • Create/Remove indexes it is blocked for any list that has more items than the list view threshold has it affects each item in the list.
  • Mange files The non indexed query fails for any list that has more items than the list view threshold.
  • Non indexed queries This includes filters and various sorts. The operation will fail if the list size is larger than the list view threshold. There isn’t an index so a full scan of the list occurs. The items will all be returned but the folders will be ignored.
  • Cross list query This includes the various queries by the Content Query Web Part. It follows the list view threshold setting for auditors and administration. The default for it is 20,000. For operations above that threshold it will fail.
  • Lookup columns This refers to those that enforce relationship behavior. You can’t offer lookup columns like this though if the list references content for more items than the list view threshold.
  • Delete a list This is blocked if the list has more items than the list view threshold due to the fact that it affects every item in the list.
  • Delete a site This affects all of the items in the list so it’s blocked for any list if there are more items in it than the list view threshold.
  • Save the list as a template This affects all of the items in the list so it is blocked for any list with more items than the list view threshold.
  • Show totals in list views This performs a query against all of the items in the list. It is blocked for a list that has more items than the list view threshold.
  • Enable/Disable attachments for a list This affects all of the items in the list so it will be blocked when the list offers more items than the list view threshold.
    Container Exceeds The List View Threshold

  • Delete/Rename/Copy a folder This fails if the folder contains more items than the list view threshold as too many of the rows will be affected.
  • Queries to filter non indexed columns This fails if the folder or list has more items than the list view threshold. It performs a full scan against the entire folder because there isn’t an index.
  • Fine grain security permissions This fails when the list or the folder that is being set has fine grained permissions that contain more items than the list view threshold as too many rows are affected. You can use the fine grain permissions on documents in a large list. However, you can’t set the permissions of the list or of the folders if they contain more than the list view threshold.
  • Open with Explorer This won’t show any items if a container has more items than the list view threshold, other than in reference to sub folders. If the root list contains more items than the list view threshold the “Open with Explorer” won’t show anything. In order to use Open with Explorer the list needs items to be organized into folders. The amount needs to be less than the list view threshold in the root for a given container.

SharePoint Claims Based Authentication Architectures Explained Part 2 Claims Architecture Priming

You can use one of many different types of approaches in order to create a claims based application. Both Web applications and SOAP Web Services can accomplish the same thing but they take different approaches for making it happen. Yet the overall structure that is there is the same as it’s the overall goal. The purpose is to create claims that can communicate with each other and that are secure.

We are going to take a close look at how you can evaluate the different types of architecture that can be used. Taking variables into consideration including different perspectives, the experience of the user, opportunities for optimizing, the performance of the different applications, and even how the claims get passed from the application to the issuer all need to be closely looked at. Only then will you see the entire picture of what is offered. I will also give you some advice about how to create your claims and how to know your users.

The overall purpose of the different architectures is to allow for either an active for passive type of federation to be implemented. With an active federation you will have the WS-Trust and WS-Federation Active Requestor Profile in place. They help to describe the way in which the communication between the clients and the services go about requesting a token from the issuer. It also covers how that token is sent for authorization.

With passive federation, the WS-Federation Passive Requestor Profile describes the same type of communication flow between the web application and the browser. In order for tokens to be requested and authorized the web browser has to redirect those requests.


First Steps In Web Service Security

The notion of the Web Services framework and Services Oriented Architectures are gaining momentum as an approach to collaborative business systems by supporting the creation, deployment, and dynamic discovery of various channels of information. The Web Services evolution is made possible in part by the adaptation of universally accepted standard protocols, these include:

  • HTTP (Hyper Text Transfer Protocol)
  • XML (Extensible Markup Language)
  • SOAP (Simple Object Access Protocol)
  • WSDL (Web Services Description Language)
  • WSFL (Web Services Flow Language)
  • UDDI (Universal Description, Discovery and Integration)

These concepts are discussed more exhaustively in terms of protocol here.

What are the portions that are involved in Web Service security

  • Authentication – Determine the identity or role of a party attempting to perform some action such as accessing a resource or participating in a transaction. A role may be appropriate to many parties, for example “Human Resources Person”.
  • Authorization – Determine whether some party is allowed to perform a requested action, such as viewing a web page, changing a password, or committing an organization to a 10 million dollar transaction.
  • Integrity – Ensure that information is not changed, either due to malicious intent or by accident. This may be information transmitted over a network, such as from a web browser to a web server, information stored in a database or file system, or information passed in a web services message and processed by intermediaries, to give a few examples.
  • Signature – Produce or verify an electronic signature intended to be the equivalent of a handwritten signature. Such a signature may be used for different purposes such as approval, confirmation of receipt, acceptance or agreement.
  • Confidentiality – Ensure that content may only be viewed by legitimate parties, even if other access control mechanisms are bypassed. Confidentiality is generally associated with encryption technologies, although other approaches such as steganography (information hiding) might serve a similar purpose.
  • Privacy – Personally identifiable information is required by individuals and companies in order to perform services for the individual. An example is a Doctor’s office that requires medical records to track a patient’s health. Privacy relates to control over what is done with this information and whether it is redistributed to others without the individual’s knowledge or consent. Privacy may be managed by a combination of technical and legal means. Confidentiality technology may be used to protect privacy, but cannot prevent inappropriate sharing of information.
  • Digital Rights Management – Ensure that content is used according to license agreements. Generally access rules are incorporated with the content, and enforcement controls are integrated with the clients needed to use the content.

This in turn break down to the actual protocols involved:

  • XML Digital Client Signatures for LOB signing solutions
  • XML Encryption for confidentiality and verifiable integrity
  • XML Key Management (XKMS) for encryption key buckets
  • Security Assertion Markup Language (SAML) for conveying authentication and authorization
  • XML Access Control Markup Language (XACML) for defining ACL related information
  • Platform for Privacy Preferences (P3P) for defining privacy actions and associations
  • Digital Rights Management (eXtensible Rights Markup Language 2.0 – XrML)

XML Security standards provide a set of technical standards to meet security requirements. These standards are designed to conform to common XML paradigms. The XML Security standards leverage existing XML standards and also enhance XML standards as follows:

  1. The XML Security standards define XML vocabularies for representing security information, using XML technologies, such as XML Schema, for definition. An example is the element defined in the XML Digital Signature recommendation for carrying signing or encryption key information. This definition is used in a number of the specifications. The specifications define a shared meaning for the XML vocabularies.
  2. The XML Security standards use other existing XML standards where possible to leverage current XML efforts. For example, XML Digital Signature allows XPath expressions to extract portions of XML for processing. (Defined in XMLDigSig and extended in XPathFilter).
  3. The XML Security standards are designed to offer the flexibility and extensibility aspects of XML. They allow security to be applied to XML documents, to XML elements and element content, as well as to arbitrary binary documents. They support extending the XML vocabularies through the use of XML namespaces and extensible XML Schema definitions.
  4. XML Security technologies may be applied to end-end security, which is especially important when XML messages are routed through a number of processing intermediaries. Persistent security is associated with the content, rather than with a transport pipe. The security remains with the content. XML Security technologies may be used in conjunction with transport security technologies, such as SSL/TLS, as well.
  5. XML Security technologies reuse existing cryptographic and security technologies whenever possible, without reinventing the wheel. For example, X.509 V3 certificates [ X509Cert ] are used without redefinition when needed – they are simply encoded in a text format. Existing algorithms, such as the SHA1 digest algorithm, are also brought into the XML Security standards world by associating unique URI identifiers with them and defining how they may be used in the XML Security processing models.

Although there is a mixture of these protocols, it does not necessarily mean that these protocols have to be used in order for one to exist within a Service Oriented Architecture.

For example, SOAP is a protocol for remote procedure calling and messaging with XML-encoded application data. However, SOAP does not require the use of XML. In fact, SOAP supports remotely referenced data such as objects provided by third parties that are produced or consumed at separate hosts. SOAP also specifies various usage scenarios, such as one-way message passing, single and multiple request-response invocations, as well as routing.

It is also important to note the differences that exist in XML protocols and although it is a set amount of standards, there are vast types that exist. Most XML protocols that are going to be consumed across varying businesses are based on DTDs rather than XML schemas and lack XML namespace and extensibility properties that others may have. The expressiveness of these protocols is restricted to a set of pre-defined data types offered by the protocol. This is an important note to take when attempting to consume various types of web services into a SharePoint environment that may come from various sources.

The security of web services involves many other asepcts. With the growing acceptance of XML technologies for documents and protocols, it is logical that security should be integrated with XML solutions. The XML Security standards define XML vocabularies and processing rules in order to meet security requirements. These standards use legacy cryptographic and security technologies, as well as emerging XML technologies, to provide a flexible, extensible and practical solution toward meeting security requirements.