Extending with Science! ... or files

Hey, say you've got like 40,000 files on your hands.  AND you've got a public, Enterprise SharePoint 2010 portal.  AND you've got a mandate to publish each one of those files to the world.  AND you just got kicked off your previous web host.  Watcha gonna do?


Well my friend, use SCIENCE!


OK, not really.  BUT, SharePoint is a Microsoft.NET application, hosted in Internet Information Services, so we can just add a virtual directory to the SharePoint web application and serve away.  Yes, but...


I tried doing just the above once, but didn't discover the secret sauce until just yesterday.  Here's the breakdown:

• The 40,000 files are served anonymously (this is actually a good thing)
• Directory access performance isn't really an issue, there aren't a million hits a month let alone a minute.
• There's already a structured navigational approach to finding the files so there's no need to browse them or build a new catalog.

What do you need to do to spin up a virtual directory inside the SharePoint web app?  Well here's what I did and it's working find, thank you very much.

1. Establish a virtual directory in your SharePoint web app.  If you've got multiple AAMs and/or multiple WFE's you'll have to repeat the following process for each.
2. Once you've built the VDIR, double click it's Authentication option.   We only needed anonymous so I turned off everything else (More Later).

That's it.  You're up and running.  But wait you say, I tried that and it didn't work. Well it didn't work for me the second time either.  My first test I had access to the directory that was being access and was using an IE client on the same host as the server.  That three-headed-dog be sent home to Hades!
 
The change?  This time around, instead of using pass-through security on the VDIR, I used the connect as option and connect using an account that has access to the files.  That solved the Anonymous access problem I was having.  Another solution would be to grant the "Everyone" user access to the files.  But that's a problem in and of itself, plus if you're using a remote VDIR it probably won't work unless you make the file share wide open.

The other change?  Well you need to make sure you establish a web.config file in the directory  you're serving.  Just inheriting from the base file in SharePoint's home dir won't work.  Until I established a custom web.config, SharePoint tried to intercede and bump the URL up against the content database.  Not-found is the same as access denied in that world.  So in summary:

• Use a delegation account to access the VDIR
• Establish a web.config that explicitly sets the Anonymous access properties

 Well we just turned SharePoint into a regular IIS web application, and my client is happy.

SharePoint Powershell -- Awesomesauce

Introduced with SharePoint 2010 (but actually possible with older versions) we were given direct access to the SharePoint object model in a interpreted scripting environment.

With a simple script like this, I can list all of the lists and libraries in a SharePoint web application:

$wa = get-spwebapplication http://site.domain.com
foreach($site in $wa.Sites) {
    foreach($web in $site.AllWebs) {
        foreach($list in $web.Lists) {
            write-host $wa.Name $web.Title $web.Url $list.Title
        }
    }
}

Great, so what you can list a bunch of the content.  Tell me something I can't do with other tools...

OK, here's one.  I had a WSP that installed custom page layouts.  One of the page layouts got modified by a user using SharePoint designer, and even if I returned the page layout to the original state, upgrading or re-installing the WSP wouldn't overwrite the page layout in the content database.

Solution: Detach the each of the publishing pages from the page layout and  make a file that lists each of the publishing pages that were attached to the page layout.  Then remove all of the bad stuff using what ever tool you like and reattach the page layouts based on the data we saved in the file.

Check out this script, the first half of the process of detaching the page layouts. $detLog gets set outside the function and is the full page to the change log.  Just point the function at an SPWeb and a reference to new page layout (it needs to be a Microsoft.SharePoint.Publishing.PageLayout) and away it goes, cycling through the entire SPWeb and its children.  Oh, $comment gets set outside the function too.  It could be something snappy like "Detaching Page Layout for Upgrade."

 Function BFS-PubPage($web, $newlayout) {
    $web.Lists | foreach-object -process {
        $l = $_
        $l.Items | foreach-object -process {
            $i = $_
            if ([Microsoft.SharePoint.Publishing.PublishingPage]::IsPublishingPage($i)) {
                $pp = [Microsoft.SharePoint.Publishing.PublishingPage]::GetPublishingPage($i)
                if ($pp.Layout -ne $null) {
                
                    if ($pp.Layout.ServerRelativeUrl -eq "/_catalogs/masterpage/OffendingLayout1.aspx" `
                       -or $pp.Layout.ServerRelativeUrl -eq "/_catalogs/masterpage/OffendingLayout2.aspx") {
                        write-host -ForegroundColor DarkBlue ($web.Url + "/" + $pp.Url)
                        
                        $cl = $pp.Layout
                        
                        $pp.CheckOut()
                        $pp.Layout = $newlayout
                        $pp.Update()
                        $pp.CheckIn($comment)
                        $pf = $pp.ListItem.File
                        $pf.Publish($comment)
                        $pf.Approve($comment)
                        
                        add-content $detLog ($web.Url + "`t" + $i.Name + "`t" + $pp.Url + "`t" + $cl.ServerRelativeUrl)
                    }
                     else {
                        write-host -ForeGroundColor DarkYellow `
                             ($web.Url + "/" + $pp.Url +"`t" + $pp.Layout.ServerRelativeUrl)
                    }
                }
            }
            
        }
    }
    
    $web.Webs | foreach-object -process {
        if ($_ -ne $null) {
            BFS-PubPage $_ $newlayout
        }
    }
}

You probably noticed, but this script uses a different way to implement the for-each loops.  Instead of using the C# style of the loop, it uses the object pipe method.  I'm guessing one method is probably more efficient than the other, but  you never know.

Windows Installer Don'ts

Ok, so I'm a software professional (or at least I thought I was).

A few weeks back my laptop started to complain that I was running out of disk space.  I've only got 500 GB on board, and with a couple of VM's for SharePoint, SQL Server 2012, and various FOSS OSes, I really was running out of diskspace.

Know that Image of Windows Server 2003 running SQL Server 2005 and MOSS 2007?  Well that was too important to purge, so what did I do?  I dug out the handy du command and started building up a hog report.  Where was all of that space going...

• c:\MSOCache (hidden) - 2.1 GB
• c:\Windows\Installer (not hidden, but not shown either) - 17.2 GB
• c:\users\mbattey\AppData\Local\Temp (AppData hidden) - 1.85 GB
• c:\ProgramData\Microsoft (ProgramData hidden) - 4.7 GG

Hey, I'm a smart guy, right?  What the heck are all of those Windows Installer files doing still hanging around on my computer?   I'll just delete those.  MSOCache? Stupid Microsoft leaving crap everywhere.  Gone!  Temporary files?  Fried!

Ya bad idea.

Windows Update was immediately broken.  There were a handful of updates to Visual Studio (2005, 2008, 2010) waiting to be installed as well as updates for Office.  None of these would go.  Every time they ran, the MSI tool would ask for a GUID named directory off the root of one of the drive partitions, which of course didn't exist.  Trying to run a Repair from Programs and Features failed miserably with the same result.  In fact, the icons for all of the Office Documents, Adobe Reader PDFs, and a bunch of others disappeared as well.  (Methinks that Adobe and MS got lazy and weren't moving all of the DLL's out to "Program Files" like they tell everyone else to do).

So after manually removing all of the remnants from Office following a nice showcase from Microsoft (the girl reading the script sounded nice, but had trouble pronouncing RegEdit and Suite, which came out more like "reg-it" and suit), writing a custom registry cleaner to delete stuff from HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Microsoft\Windows\CurrentVersion\Uninstall, HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Uninstall, and HKEY_LOCAL_MACHINE\SOFTWARE\Classes\Installer\Products, and finally correcting individual packages with MSIZap, I'm kinda sorta back to normal.

Normal as in I'm back up to all that disk use, and Office and Visual Studio will run updates, mostly.  I do highly recommend MSIZap when you've got the GUID of a a package that won't install, because it's gone missing.  It will root out all of the registry and local files related to the product and in the cases I needed it let a patch go through unperturbed.