22 czerwca 2011

SharePoint 2010 upgrade failed - Feature upgrade failed for Feature

Test-SPContentDatabase went fine


Upgrading content database from MOSS/WSS 2007 to SharePoint 2010 is a challanging task in itself. It gets even tricker when overtime number of customizations has been deployed (custom features, event receivers etc) to SharePoint 2007 installation. Microsoft has provided really helpfull Powershell command Test-SPContentDatabase that performs analysis of SharePoint 2007 content database against configured SharePoint 2010 installation. If there are any references to custom components in content database that are not installed on the SharePoint 2010 farm warning and errors will appear in the log.

It was Mount-SPContentDatabase that failed


On the other side Test-SPContentDatabase command cannot discover all inconsistencies that may exist in the content database and may fail the upgrade. After resolving all the problems uncovered by the Test-SPContentDatabase, I run Mount-SPContentDatabase to attach SharePoint 2007 content database.

Mount-SPContentDatabase -Name [ContentDBName] -DatabaseServer [SqlInstance] -WebApplication [WebAppUrl] 

I wasn't expecting flawless upgrade during attach process, but I was really suprised when I finished with 56 errors. All errors where listed in the upgrade log and one of them was Feature upgrade failed.

[powershell] [SPSiteWssSequence2] [INFO] [6/17/2011 9:41:22 AM]: SPSite Url='http://sitecollectionadress here
[powershell] [SPSiteWssSequence2] [ERROR] [6/17/2011 9:41:22 AM]: Feature upgrade failed for Feature 'Mulawa.FailedFeature1' (Id: 'FeatureId here') in Web 'http://sitecollectionadress here'.
[powershell] [SPSiteWssSequence2] [INFO] [6/17/2011 9:41:22 AM]: SPSite Url=http://sitecollectionadress here
[powershell] [SPSiteWssSequence2] [ERROR] [6/17/2011 9:41:22 AM]: Exception: Object reference not set to an instance of an object.
[powershell] [SPSiteWssSequence2] [INFO] [6/17/2011 9:41:22 AM]: SPSite Url=http://sitecollectionadress here
[powershell] [SPSiteWssSequence2] [ERROR] [6/17/2011 9:41:22 AM]:    at Microsoft.SharePoint.SPFeature.HandleUpgradeException(Exception e, Boolean continueUpgrade, Boolean markAsUpgraded, Boolean force)
   at Microsoft.SharePoint.Administration.SPFeatureDefinition.Upgrade(SPFeature feature, SPWebService webService, SPWebApplication webApp, SPSite site, SPWeb web, Boolean fForce)
   at Microsoft.SharePoint.SPFeature.Upgrade(SPSite site, SPWeb web, Boolean fForce)
   at Microsoft.SharePoint.Upgrade.SPSiteWssSequence2.DoUpgrade()


Feature upgrade failed  error caused the whole upgrade to be marked as failed, which made me somehow unhappy. As the error message didn't ring the bell, actually I didn't know where to start. So, I looked into SharePoint log, it was Verify upgrade and review upgraded sites (SharePoint Foundation 2010) article that encouraged me to check if there are any hints hidden there (dump from SharePoint 2010 log).

PowerShell.exe (0x21DC)  0x0860 SharePoint Foundation          Feature Infrastructure         9fnu High     Feature Upgrade: Exception thrown updating content database with new version number for Feature 'Mulawa.FailedFeature1' (Id: 'FeatureId here') in Web 'http://sitecollectionadress here': Object reference not set to an instance of an object. 
PowerShell.exe (0x21DC)  0x0860 SharePoint Foundation          Feature Infrastructure         g0gz High     Feature Upgrade: Error exception thrown upgrading Feature 'Mulawa.FailedFeature1' (Id: 'FeatureId here') in Web 'http://sitecollectionadress here': System.NullReferenceException: Object reference not set to an instance of an object.     at Microsoft.SharePoint.SPFeature.UpdateFeatureVersion(Boolean fForce)     at Microsoft.SharePoint.Administration.SPFeatureDefinition.UpgradeCore(SPFeature feature, SPWebApplication webApp, SPSite site, SPWeb web, Boolean fForce)     at Microsoft.SharePoint.Administration.SPFeatureDefinition.Upgrade(SPFeature feature, SPWebService webService, SPWebApplication webApp, SPSite site, SPWeb web, Boolean fForce) 
...........................
Microsoft.SharePoint.Upgrade.SPUpgradeSession.Upgrade(Object o, Boolean bRecurse)     at Microsoft.SharePoint.Upgrade.SPUpgradeSession.ReflexiveUpgrade(Object o, Boolean bRecurse)     at Microsoft.SharePoint.Upgrade.SPUpgradeSession.Upgrade(Object o, Boolean bRecurse... 
PowerShell.exe (0x21DC)  0x0860 SharePoint Foundation          Topology                       8xqz Medium   ...)     at Object state)     at System.Threading.ThreadHelper.ThreadStart()   ff2e5232-6422-4b57-bab9-588bb64e4cc6




Let's spy on SharePoint


Stack trace from SharePoint log was much more detailed and showed that it was the SPFeature class UpdateFeatureVersion method that had failed. After running ILSpy (can't afford .NET Reflector any more as it's not free ;)) I noticed that the problem lies in updating Features table in Content database and missing FeatureDefinition. On the screenshot from ILSpy presented below, FeatureDefinition property is higlighted in read. It was set to NULL value when UpdateFeatureVersion was called.


In SharePoint 2010 Feature table in the content database has now Version column and updating it with the Version field from the FeatureDefinition object caused the whole problem.

So, why FeatureDefinition is null. Failing Mulawa.FailedFeature1 feature was installed on SharePoint 2010 farm, so FeatureDefinition should not be null. However, Mulawa.FailedFeature1 used to have Scope set to site collection (Site in feature.xml) and it has been changed to site (Web in feature.xml). Before Mulawa.FailedFeature1 feature scope changed it has not been deactivated on site collection level. So, when upgrade occurred it couldn't locate Mulawa.FailedFeature1 feature definition on the site collection level. As it has been installed with scope set to the site (web) level now. It's important to note that it is not the feature definition that is failing the upgrade, it is activated instances of the feature that are missing feature definition.

To asses the scale of the problem, you can run following query against the SharePoint 2007 content database to get the list of feature instances that are activated on both Web and Site scopes.

SELECT f.FeatureId, f.SiteId, f.WebId
  FROM Features f
 WHERE EXISTS (SELECT *
                 FROM Features f2
                WHERE f2.FeatureId = f.FeatureId
                  AND f2.WebId = '00000000-0000-0000-0000-000000000000')
   AND EXISTS (SELECT *
                 FROM Features f2
                WHERE f2.FeatureId = f.FeatureId
                  AND f2.WebId <> '00000000-0000-0000-0000-000000000000')
AND f.WebId = '00000000-0000-0000-0000-000000000000'
ORDER BY  f.FeatureId, f.SiteId, f.WebId

To resolve the problem, Mulawa.FailedFeature1 feature has been installed on the SharePoint 2007 farm with scope set to site collection level (site).

stsadm -o installfeature -name Mulawa.FailedFeature1 -force

Feature has been deactivated on all site collections in the farm using code below.

using System;
using Microsoft.SharePoint;
using Microsoft.SharePoint.Administration;

class Program
{
    static void Main()
    {
        Guid featureId = new Guid("Feature Id Here");
        SPFeatureScope siteCollectionScope = SPFeatureScope.Site;
        SPWebApplicationCollection webApplications = 
            SPWebService.ContentService.WebApplications;
        foreach (SPWebApplication webApp in webApplications)
        {
            foreach (SPSite siteCollection in webApp.Sites)
            {
                try
                {
                    if (siteCollectionScope == SPFeatureScope.Site)
                    {
                        if (siteCollection.Features[featureId] != null)
                        {
                            //deactivate feature with force option
                            siteCollection.Features.Remove(featureId, true);
                        }
                    }
                }
                finally
                {
                    if (siteCollection != null)
                    {
                        siteCollection.Dispose();
                    }
                }
            }

        }
        
    }
}


After that, I've tried attaching SharePoint 2007 content database again (now with Mulawa.FailedFeature1 being deactivated on the site colleciton level) to SharePoint 2010 web application and it finally worked . Upgrade has finished without any errors in the upgrade log.

Unfortunatelly, to fix this problem you cannot use FeatureAdmin tool, as it can't deactivate the feature on the site collection level without matching scope in feature.xml (SharePoint API stops working here). Of couse after changing the scope in the feature.xml and installing it on the farm   will allow FeatureAdmin  to deactivate it manually.

Lessons learned.

When changing the scope of already deployed and activated custom feature, always deactivate it on the old scope first. Uninstall it and install with the new scope. Of course if you want to avoid problems like that in the future.

Brak komentarzy:

Prześlij komentarz

Uwaga: tylko uczestnik tego bloga może przesyłać komentarze.