Part 1 - Part 2 - Part 3 - Part 4 - Part 5 - Part 6 - Part 7 - Part 8

I was scouring over my feed reader this morning and came across this post on the MSDN forums asking:

"Any one please give me a step by step implementation of deploying a SharePoint application...(Modifying the SharePoint website content and deploying it to the SharePoint server)?"

Now, all platforms are tricky when it comes to development and deploying. I've worked with Interwoven, BEA and numerous other "Enterprise Level" platforms. I took some time to think how I would answer this guy entering this space.

I currently training a graduate programmer with no C#, ASP.NET or SharePoint experience from scratch and it has been a great experience for both of us. It's solidified my understanding of the platform and also challenged him to come up to speed and get as much out of the training as possible. He's come a long way in a short time, and this has a lot to do with me having already spent a lot of time overcoming the hurdles in the first few instances.

SharePoint's biggest problem is that it is so damn easy to be an Owner of a SharePoint Site (or Web if you're in the API) and go nuts creating SharePoint Lists, Content Types, Fields all from within the browser interfaces. Then give them SharePoint Designer and watch them go nuts with creating Workflows, Pages, Page Layouts, Style Sheets, Master Pages, Images etc. - I could go on. The problem with all this is that if all this is done in Development environment ... how do you deploy it to UAT and then through to Production?

Methods of deployment

• STSADM - Backup/Restore
  • Great for releasing v1.0, but will overwrite entire site if you release v2.0 so not a solution unless you copy items across and delete old site and rename new one to existing.
• Solution Package
  • VSeWSS
    • Solution Generator
    • VS Projects
  • STSDEV
    • Couldn't live without it for quick redeployments of solution packages and auto generation of ddf and wsp files!
    • Seems to have a current limitation to one Solution per project, but that's no big deal.
  • WSPBuilder
  • Alternative to STSDEV. Autogenerates manifests, ddf and wsp.
  • MOSS Feature Generator
    • Got a bug when clicking new Item and only outputting title
  • Field Explorer
    • Great for grabbing ContentType and Custom Field schema xml – needs some modifying to get it to actually work but most is just removing bits
  • BDC Meta Man
    • The only way to script BDC without going insane!
• Third Party Product
  • Echo Technology
    • Great for comparing and deploying Content from Site to Site (see backup/restore above)

Cleaning up

The biggest problem with all of these techniques is that this will get you from one environment to the other, but SharePoint's paradigm is that whatever is deployed and activated by default will not be removed when deactivated. So if you have a <ListInstance> element in your manifest in your solution package, if you deactivate the feature it WILL NOT delete this instance. You can add code into your Feature Event Receiver code behind which adds further complexities and duplication (interestingly Content Types do this OOTB).

Iterative Development

I've been putting together a Document Management solution on top of SharePoint (yes that's right folks, SharePoint isn't a Enterprise Document Management system without some development so please don't fall into that trap). I've built various Content Types (with Document Templates and Document Information Panels), Custom Fields Types and Custom Fields with all sorts of inheritance and furthermore created ListTemplate's to package instances up.

One thing I have found even in development is that deploying new versions of your scripted Solution packages is not straight forward. An example of this is typically you don't sit there and churn up your manifest file and schema files for all your lists and he-presto it all works first go when you deploy the solution – so you work iteratively one thing at a time to make finding the cause of the vague errors easier.

Versioning

If you have scripted up your ContentType up and deploy this once and it's all happy, then go back and add further elements to your manifest and some tweaks to your ContentType and "redeploy" this again (deactivate, delete, add, activate). You will notice that these changes are not taken on IF the ContentType was in use in a SharePoint List. So biggest hint, is when developing and deploying new versions...make sure you have full clean up scripts...otherwise you'll end up in a mess!

Change Sets

The other concern around this is obviously you can't simply clean up what's in your Production environment when you want to release the next version of your application. This seems to be the big show stopper for our team to date...what do you do? As motioned above you can:

1. create a copy of the existing site,
2. deactivate the feature which cleans everything up
3. reactivate it with the new version of the Feature
4. copy all the SharePoint List Items from old Site version to new Site version
   1. remembering that to get the metadata, versioning, security etc. you'll need to use API scripting OR a third party tool such as echoTechnology.

This obviously has serious overheads if you have an enormous site with say 20 lists with 2000 odd Items in each list and various versions. It also requires scripting the migration for every deployment to Production.

A simpler solution would be to do an incremental update to objects, a scenario for this would be adding a Workflow to a Content Type or switching on Versioning in a SharePoint List. So possibly you go towards scripting change sets and having new Features to move from each change sets. This in itself can get messy...I'm not even sure that SharePoint will be manageable with thousands of Features available in the Site Features collection. Then you find yourself permanently coding change set differences, rather than automatically generating schemas etc. using Solution Generator.

Continuous Integration

Before moving into the SharePoint development space I was a ASP.NET/WinForms C# developer. We were really into our VSTS Team development with Builder Server projects. Moving into a SharePoint environment where build servers really are making all this above seem incredibly more scary also needs to be resolved. It's not just being able to deploy one Solution Feature package to a build server, but also to have multiple interdependent ones to prove they all sing happily in tune together. A great example of this is ListTemplate Type IDs.

Nice to haves

1. Further explanation of the uses of the Version Id in the Feature file and Solution File.
2. The ability when defining objects in the solution manifest to have an attribute that states whether on deactivation that it is "undeployed" e.g. ListInstances
3. A way of pointing at an entire Site and clicking a button and creating a whole Solution package similar to what STSDEV does but all for you.
4. An interface to visually see the schema changes made for every operation done in the browser interface – this would be a great way to learn what's going on in the code!
5. The ability to publish InfoPath forms via Solution package rather than directly in stsadm
6. The ability to activate Features from within Feature files e.g. for standard things like Publishing Feature etc.
   1. <RANT> try finding out what Feature ID it is by scouring through 12 hive FEATURE folder feature.xml files...they left the Feature name to an existing name (most likely for backwards compatibility) but changed the Title the user sees in the interface and there's no easy way to guess what's what!</RANT>
7. A Script engine which will generate a DIFF script for moving between one version of a Solution Feature package to another.
8. Some more sample Solution packages for common things:
   1. Custom Field Types, Custom Fields, Content Types, SharePoint List
   2. Workflow (Content Types)
   3. Master Pages, Style Sheets, Images
   4. Navigation
   5. InfoPath Form Libraries, Content Types, InfoPath forms
9. A solution for generating build scripts to test solutions and their interdependencies.
10. Some simple ways of populating lists with test data intelligently by inspecting the List columns
11. The ability to have VSeWSS in Visual Studio 2008 and on a 64 bit machine!
    1. Intellisense in VS2008 on Schema, Manifest, Feature and Solution files.

Talk in the paddock

There is a lot going on around this subject at the moment:

• MSDN has come a long way since it first came out in November 2006, but there is still a long way to go. Re-skinning aside, it's gotten better. One tip, use Google to find anything on the site ;-)
• Glenn Block over at CodePlex has a survey he's put together to collect other people's opinions on what are the big issues about development. The Questionnaire is a bit rigid and I think this is because he expected more people to answer, I think more open questions and some time processing these at the end may have got better results.
• WCM Guru, Andrew Connell, has put some notes together around developing solutions too.
• The SharePoint Team Blog has some great articles on it too.
• ThorProjects got me thinking along these lines also
• Joel Olsen also just posted on another product that packages up deployments.
• Bill Simser has some radical thinking about using Nant with SharePoint
• Bill Baer also does a good job of explaining Solution packages
• Paul Andrew is also pushing the campaign to get ASP.NET devs to SharePoint devs.
• Joris Poelmans also has a good post on getting started with Web Parts
• Sahil Malik has some great articles around this also
• Paul Culmsee has a good set of articles that use WSP Builder

"Do It and stop whinging Jeremy!"

Yes, and it has crossed my mind to start writing some of these things I've mentioned above...the problem is that I'm concerned that my efforts in spending so much time will be wasted when Microsoft release vNext of the development tools or vNext of the platform itself which makes my solution outdated. This again, comes back to Microsoft not keeping all their cards to their chest in terms of the direction they're going in. Fair play to echoTechnologies for taking the punt, risk and investment to come up with a product.

Next steps

I hope people find this informative and I hope I'm not the only one in the same boat...please feel free to leave comments and get a bit of a debate going!

Anyway, keep your eyes peeled as I'll be writing a series of posts now showing how to use these tools and best approaches...