Integrating SharePoint On Premises With BPOS and Exchange Online: Part 2 – Inbound

A few weeks ago I posted an article on how to get on premises SharePoint working with BPOS for mail delivery (alerts, etc.). Historically, inbound email is something that is significantly trickier than outbound, but with hosted Exchange, I’d suggest that the two roles are switched in terms of difficulty. There are however still a couple of extra hoops that have to be jumped through, and I’ll try to guide you through them here.

Firstly, allow me to say that SharePointGeorge has an excellent article out there on setting up incoming email when everything is on premises. In addition, BPOS Tutor had an article on using distribution lists that I was able to make use of while preparing this.

1. Set Up the SMTP service

For the purposes of this article, I’m going to assume that you’ve already done this when you set up outgoing mail. If not, I’ll refer you to my article linked above, or SharePoint George that will walk you through the requisite steps. Once it’s done for outgoing email, you don’t need to touch it for incoming.

2. Configure the SharePoint Farm to Accept Incoming email

First,you’ll need to navigate to Central Administration,and get into the System Settings section. Once there, select “Configure incoming e-mail settings” in the E-Mail and Text Messages section.

image

There are a number of settings here that will change a bit from what is the typical guidance out there. I’ll try to explain each configuration item, and what it means. Firstly, I’ll show you a completed configuration:

image

Enable Incoming E-Mail – Well, that’s pretty straightforward, do I turn on incoming email or not? When you turn it on, SharePoint simply monitors an SMTP drop folder for any messages. If it sees one, it will pick it up, and if the destination name matches a list, it will get delivered. It’s really that simple.

The settings mode lets you choose where the drop folder is. The Automatic setting is normally fine, but if you wanted to use a drop folder in a non default location, or on another server, you would select advanced and enter the desired folder. When the configuration is saved, SharePoint will also try to set the appropriate file system rights on that folder (see George’s blog for more details). I set advanced just so I see the path explicitly.

Directory Management Service – This one normally takes a fair bit of configuration to get working, but when we’re using BPOS, it’s easy – we just set it to no. This is a service that sets up contacts and distribution groups in Exchange, and although we’re using Exchange, it’s hosted, and don’t have access to that feature. We will be creating these manually.

Incoming E-Mail Server Display Address – This is the domain that the list email addresses will use. We’re going to change this. It will default to servername.domain.com. However, even if that address is available externally, we don’t want to be accepting mail from everyone. The IIS SMTP service has no real spam or virus protection, so we want all of our email to go through our hosted Exchange server. The best approach is to use the same domain as your other BPOS users.

E-Mail Drop Folder – As mentioned above, this is the folder that will be monitored for incoming email. If you don’t know if you should change this, then don’t… the default is likely fine.

Once you’re done, click OK to save the configuration. SharePoint is now set up to configure incoming email. Steps 3 and 4 will need to be repeated for every list/library that will accept email.

3. Configure Library to Accept Incoming E-Mail

Navigate to a library that you want to have accept incoming email. From the ribbon, select “Library” (or List..), and then select Library Settings.

image 

Next, under the Communications Column, click the “Incoming e-mail settings” link. You should see a screen similar to the following:

image

Most of the options are self explanatory, so I won’t go into detail here. The most important ones are of course in the Incoming E-Mail section, which lets you turn it on or off, and lets you specify the address of the list. The address is important, as it will need to match what we do in BPOS in step 4, and it is also important that it is global across the farm (and of course the domain). That name can’t be repeated, so choose wisely. A naming policy is a good idea here.

Once you have the settings the way you want them, click OK, and your list is ready to go. Now it’s on to BPOS.

4. Configure the Address in BPOS

This is where it gets interesting. What we want to do is to have BPOS accept email from internal (and possibly external) senders, and then turn around and deliver them to out IIS SMTP service. Usually, we could set up a contact in Exchange and use mail forwarding to do this for us, but there is no mail forwarding capability in BPOS. So how do we accomplish this? Instead of using mail forwarding, we’ll set up a distribution list with one member, and let it work its magic that way.

The first thing that we need to do is to log into the admin portal at http://admin.microsoftonline.com. Once in click on the Service Settings tab, and then click on the Exchange Online subtab. From the right hand Actions section, click the “Add new contact” link. You then need to add your contact, which in effect is the library that we enabled in 3 previously:

image

The Email Alias used here must match the one used in 3 above, and. The display name is relatively unimportant, but again will be available to the GAL. Once you save this screen, you should be ready to go.

It’s worthwhile to describe the flow of what happens. When an email is sent from a user, external or internal, the originating server will look for an MX record for the address to the right of the @ symbol. That MX record will point to your BPOS server. The BPOS server will accept the name, as it matches the distribution list that you created in step 4. The message will then be distributed to the members of the list, in this case one member at the precise SMTP address of the server farm. BPOS will send the message to the SMTP server running on the farm, where it will be deposited to the drop folder. Finally, the timer process in SharePoint will pick up the message and deposit it into the appropriate library.

Nothing to it…. Smile

Windows Live Essentials “Wave 4” And Windows Phone 7 – Why You Should Care

A few weeks ago Microsoft made available the latest Beta version of Live Essentials. Most people I know use Windows Live Messenger (formerly MSN Messenger), and that’s all that Windows Live is to them. However, it’s much more than that. If you’ve installed Windows 7, you may have noticed that it no longer ships with a number of productivity applications (for example Movie Maker), All of the missing applications are available through Windows Live. There is a big difference though, in that these applications are all very much “Live Aware”, which is to say that they’re tightly coupled with your Windows Live profile and Live ID. I’ll dive into why that’s a good thing below.

To start with, Essentials Wave 4 consists of 9 Primary Components:

  1. Messenger – This is of course the one most are familiar with. However, it’s very much new and improved, and I’ll talk about this in a bit more detail below.
  2. Photo Gallery – Photo Gallery is the Microsoft application for organizing, tagging and cleaning up photos. This, to me is the absolute standout product of the suite,and I’ll explain why below
  3. Mail – This replaces Windows Mail,which no longer ships with Windows. It allows you to hook in multiple email boxes (of course Hotmail is one option). If you currently use Outlook, you likely won’t use this, but it does work well, and it’s free for the non Office users.
  4. Movie Maker – This application allows you to put together pictures and videos into a video presentation. It’s rudimentary (I personally use Premiere Pro from Adobe – but that’s WAY overkill for most users, not to mention difficult). It’s easy, slick, and will do the job in most cases.
  5. Writer – This is the best blog authoring tool that I’ve come across. I’m using it right now to write this. It can author blog content for a very wide variety of blog providers, and this version brings in the (now) familiar ribbon interface. Connecting to Flickr, YouTube, Facebook, etc is an absolute snap now, as it benefits from the new integrated features of your Live profile.
  6. Family Safety – From the site: “Manage and monitor your children”s Internet activity so they can surf the web more safely”. I personally don’t use it, so I have no comment, but it’s there.
  7. Bing Toolbar – I hate toolbars – they’re allowed nowhere near my PC. If you like them, I’m sure this one is wonderful, but I wouldn’t know.
  8. Messenger Companion – This is a little plug in to IE that lets you know when any of your friends share a link (they don’t need to be Messenger or Live friends). It’s also a quick way of sharing a page that you happen to be viewing.
  9. Sync – If you’ve ever hear of Live Mesh, this is it. This allows you to take a folder on your PC, and keep it synchronized with a SkyDrive folder and/or a folder on another PC that you may use. This works seamlessly in the background, and is excellent for sharing with teams, working with multiple computers, or just making sure that you always have access to current data wherever you are. It is however limited to 2 GB, which to me, is pretty low. I would expect to see that increased in the future. SkyDrive itself allows for 25 GB, so why can’t I use some of that allocation?

These applications are great, in and of themselves, but the real power lies with their tight integration with your live account, and correspondingly, its tight integration with other social networks. Windows Live is Microsoft’s consumer facing social networking offering, but they seem to have taken a different approach than you may have expected from them in the past. They know that they’ll never get as many subscriptions as Facebook, and that the value of a network lies primarily with the number of its nodes, so they seem to have taken an “embrace, not replace strategy. Sure, all of the basic social network capabilities are there, a friends list, news feed, photos, etc. However if your friends use Facebook, no problem – we’ll just incorporate them. MySpace? Flickr, Linked in? No problem, they’ll come in too, and you get one big friends list, and feed that is relatively source agnostic.

Windows Messenger hooks right into that list. So now, instead of a relatively dead list of names, here’s what the new Messenger screen looks like:

image

You can seen that your friends news feed is there, from every network that you are connected to. You can update your status, which again gets broadcast to all connected networks. You have access to all of your Live content via the Social menu at the top, and all of your friends are brought in on the right, and if they use Live Messenger, you can see their status or initiate an IM session, just like you used to.  

 

Another stand out application is the new Photo Gallery. Yes it gets the nice ribbon interface, but it’s got a few VERY nice features. I’ve always struggles with getting my photos tagged with people efficiently (I’m currently working with a base of about 10,000 pictures), but this makes it a snap. Photo Gallery contains built in facial recognition algorithms, so that it can detect that a picture has faces in it, and that they need to be tagged. It will then extract the faces, and prompt you for who those people are.

image

Where does the list of available people come from? Why your amalgamated friends list of course. One interesting thing to note is that internally, if your friends names are slightly different between networks, it maintains an internal map to keep everything straight, so when you post to pictures to Facebook for example, users are all tagged correctly.

The real power though comes from the fact that not only does it recognize faces, it recognizes particular faces. Once you tag the same name a few times, the software can offer suggestions, if you go into batch people tagging mode

image

The recognition is amazing, and while not perfect, it nails it most of the time. It’s interesting to see it recognize the same face over a number of ages, or to see it get confused by look alike relatives.

Tagging is a breeze this way, and all of the tags are respected when sending to any of the social networks. Which networks? Well, any of the ones that you have linked your Live profile to that support pictures. You can really see the power of the integration features here, and the addition of another service will only bring that much more value to th
e platform. This is the beauty of the embrace not replace philosophy. Windows Live is really a solid social mashup platform, filling in gaps where any exist.

To take it one step further, Microsoft will be introducing its new Windows Phone 7 platform later this year. It promises to be an innovative product that changes the way we work with our content, and the way that your personal and business lives integrate. Many of the same concepts discussed above apply to the way that the Windows Phone 7 operates, and its primary means of integration will be your Windows Live profile. Paul Thurott of the SuperSite for Windows is currently writing a book on Windows Phone 7, and has shared his experiences of working with a development prototype. Simply logging in with your Windows Live ID brings all of the content discussed above right down to your phone, no muss, no fuss.

I don’t think that the new Live features, and the new Phone capabilities are a coincidence.I really like what I see developing in this space, and I’m very excited about trying out one of these new phones as soon as I can. In the meantime though, I have a few photos to tag and to post.

Moving To Cloud Based Email–My BPOS Story

When I first stuck out on my own (OK…some time before I struck out on my own..), I knew that I was going to need to come up with a good email solution. My requirements extended beyond those of the consumer market, and ultimately I needed the power and control that commercial email system would offer. I really didn’t know Exchange very well, and I wasn’t about to set up a Domino server (which I knew very well) as it was no longer the direction I was heading in.

I signed up with a hosted Exchange provider. This worked quite well, and was very reliable, but I quickly bumped into size limitations and integration problems. I think that at the time the maximum size mailbox was 25 MB.  I also wanted to gain experience with Exchange, so I bit the bullet and setup up a full domain with Exchange 2003 (including a Blackberry BES server) in my basement. That setup ran (in various guises) from mid 2006 to this past weekend. Initially it was comprised of multiple Exchange servers on virtual machines (required for remote Exchange access with 2003) to a single Exchange server without the BES after upgrading to Exchange 2007.

Hosting my own Exchange server was instructive, but ultimately a pain. My home internet connection is a consumer plan, and my service provider implemented multiple approaches to prevent any server hosting. This initially included blocking SMTP traffic inbound and ultimately (at a particularly bad time) blocking outbound SMTP. I quickly found workarounds to these problems (if you’re interested, I’ve used DynDNS for years, and I find their service to be exceptional. I’d recommend them in a heartbeat), but each one of these represented a significant drag on my time,and I’m not getting any younger.

In addition to the active blocking attempts,consumer ISV service isn’t exactly industrial grade. To be fair, they don’t claim that it is. In fact, ISPs typically go out of their way to not promise uptime reliability. Far too frequently after an outage, communication or power, my automatic DNS synchronizer wouldn’t update quickly enough and mail flow would be interrupted. Backup was another maintenance headache – yes it was getting done, but I had to have the infrastructure to support it, etc. All of this, and a few other things have prompted me to keep an eye open for alternatives.

My company is a Microsoft Online partner. We initially signed up to this program in the early days because of our extensive work with SharePoint, and recently, we have targeted online services as a significant growth area. One of the packages offered in Online Services is BPOS – The Business Productivity Online Suite. Simply put, this is hosted Exchange, SharePoint, Unified Messaging, and Live Meeting. All of this is offered at a very reasonable rate – $12.50 per user per month.

I decided last week to take my home Exchange system and migrate it to BPOS. The process went incredibly smoothly. The BPOS portal lays out all of the steps, but it can be a little confusing. I’ll quickly summarize them below.

1. Sync the Active Directory with BPOS

This sets up a one way synchronization between your Active Directory, and your BPOS Active directory. To be sure these are 2 different directories, and this just allows for simple user maintenance in the cloud. This step is not required for operation, but it is required for mailbox migration. One annoyance here – the synchronization tool must run on a domain joined Windows server running a 32 bit (!!!) OS. Since I only have 64 bit server set up, I had to spin up a new one. Ultimately, I would hope this was replaced by some sort of claims based model.

2. Set up your domain records

There are a number of steps here that are well documented in the setup section. These steps will allow your Outlook clients to auto discover your hosted Exchange mailboxes.

3. Migrate mailboxes

There is a tool that sets all of the appropriate user records, migrates mailbox content, and sets up email forwarding for the migrated users. It’s a VERY good idea to clean up all of your old junk before migrating. I, of course didn’t. That said, my largest mailbox (~2GB) took only about 6 hours to migrate. During the migration period, mail is still delivered to the on premises server, and it is kept both locally and in the cloud for migrated users. If a migration fails, it can be rerun and will pick up from where it left off. Once a user is migrated, and tested to be working, you use the tool to remove the mailbox from the on premises server, which will also remove forwarding. All mail will be delivered to the hosted mailbox.

3.5. Optionally, set up handheld connections to the hosted mailboxes.

4. Set Domain Records

Once all mailboxes have been migrated, set your domain’s MX record to now point to the hosted server, and use the administration portal to set it as authoritative, and to allow incoming mail. Once this is done there will be a lag while the changes propagate through the internet. Mail will not flow for a period of time, so don’t be alarmed.

5. Shut down your on premises Exchange server

…and rest peacefully.

Performance on the BPOS system has been great, and there appear to be no capacity issues. The per user mailbox limit can be set on a per person basis and the maximum is 25 GB. My mailbox is less than 2GB, and I do next to nothing to keep it cleaned out.

The only potential problem I see with it is integration. The Hosted server IS out in the cloud in a different domain, and therefore can’t reach back into the internal systems when necessary. For example, if running in a coexistence mode, free/busy time searches won’t work between the two groups of users. Also, on premises servers that need to send email won’t be able to use the hosted server to do so. Again, I hope that the promise of claims based authentication will help to alleviate these issues going forward.

BPOS is still using the 2007 Suite of products… Exchange 2007 and SharePoint 2007. They are slated to be moved to 2010 this fall, and I’m anxious to see what that will bring. When I know, I’ll certainly be posting back here.

I’m very happy with the results I’ve achieve, and heartily recommend it to any small-medium sized business. In fact, given the cost savings that can be achieved, I can’t see any reason why you wouldn’t want to go this route.

Storing Data In The Cloud

Last week, my colleague Ed Senez posted a very good article about cloud computing, and it’s benefits.Our company has been making moves toward the cloud for a couple of years now, with both Microsoft’s BPOS offering, and our own SharePoint Extranet Accelerator. While companies struggle with the benefits and risks of moving pieces of their business to the cloud, I can see a huge role for the cloud in the consumer space, primarily because it is so cost effective. I have been moving a lot of my personal data to the cloud for the past little while, and I thought that I would share my current observations.

Photos and Videos

Almost any Facebook user is familiar with posting pictures. The social functionality is great – tagging people lets all their friends know that they are in a new picture (maybe not so great if you don’t like the picture, but I digress….). YouTube is of course great for uploading and sharing videos, but both of these services have one drawback – they convert the files on upload resulting in a loss of fidelity. If you care about the quality of your source content, you can’t rely on these services for backup.

This fact led me a few months back to Flickr. At first look, Flickr had a lot of limitations too – a maximum file size,and a maximum upload rate per month,which initially caused me to dismiss it. What I found out was that with the subscriber version there are no limits at all – you can upload to your hearts content, and it will store the images in their true source format. I have been doing just that when I could for the past few weeks, and currently have over 2000 pictures in my photostream. Just 8000 or so to go.

Flickr also allows you to share your pictures publicly, with family and friends, or just keep them private. However, Flickr doesn’t have Facebook’s ubiquity, so I use it for purely public pictures only, and continue to rely on Facebook primarily for sharing and people tagging. Flickr does allow for videos as well, but it does have some size limits, so I will be relying on YouTube for sharing my videos, along with a separate backup strategy (see below) as I get my videos organized.

So how much does this cost? For $25 per year, I know that all of my personal pictures are backed up. Pictures are quite literally irreplaceable. Documents can be recreated, but you’ll never have a chance to capture those precise moments again. The fact that I can use the services to share picture (in full source quality) is really just a bonus.

Simple Storage with SkyDrive

Did you know that you have 25 GB of storage in the cloud that you can use free of charge? If you have a Windows Live ID (also free..) then you do. It’s called Sky Drive, and it’s extremely handy. Simply upload the files you wish to private, shared, or public folders and they’re safely secured away and accessible from any machine with a web browser. Because SkyDrive also uses WebDAV, you can map your SkyDrive folders directly to folders on your computer.

When you are navigating through your SkyDrive, you also have access to the recently released Office Web Applications. These are light, browser only versions of Microsoft Word, Excel, PowerPoint, and One Note, and they’re completely free of charge. You can create a new document using these apps, or edit anything that you upload. These apps are very handy for occasional use, for viewing purposes, or just for accessing an Office document that may have been sent to you when you don’t have the Office applications readily available.

Sky drive should pretty much eliminate the need for FTP servers, certainly for personal use. Given the cost of the service ($0.00), I really don’t see why someone wouldn’t want to take advantage of it.

Backup

I think that everyone that has used a computer for any amount of time has at some point lost data. Afterwards, there is a mad rush to back up the systems, and then make sure that there is a system in place to back everything up. Corporations typically have solid backup strategies in place (that aren’t tested frequently enough, in my opinion), but personal users are often too busy to ensure that their data is backed up in a timely fashion. There are a ton of consumer backup product out there, but they all often have one fatal flaw. They require the user to actually do something to make it work.

This is where the cloud can be of great help. If we can assume that the machine will typically have a connection to the internet, then for all intents and purposes, our backup destination is always available. All that is needed is a good service to make this painless and automatic for the end user. There are a number of such providers out there, and I’m going to briefly discuss the one that I’ve settled on – Carbonite.

With Carbonite, you download a small application that runs in the background, and is constantly ensuring that your files are being backed up. For most users it is as simple as a next – next install, which will backup all standard data folders. If you want to back up a non standard folder, just right click on it and choose to add it to the backup. You can always see what the backup status is from the console, but carbonite also (optionally) places a small indicator over the icon for each file that you have to let you know its backup status. The backed up files are also browser accessible from any internet connected PC, allowing you to access your files in a pinch, and one of the nicest features is that it not only keeps a mirror image of your system off site, it maintains file versioning, so when you make a change to a file and later decide that it wasn’t such a good idea, you can retrieve a previous version.

Given most end users’ bandwidth constraints, the initial backup can take a little while. Mine took two weeks, but that’s me. After initial backup, it all goes very rapidly. So what’s the cost of all of this storage? You can back up as much as you want from a single machine for $55 US per year. To me, that’s a no-brainer.

 

I spend about 5-10% of my time inside my company firewall. Tools to help with remote connectivity are crucial, and I really see a place for cloud based services to provide a lot of these tools. They’re safe, they’re easy, they’re useful, and they’re highly cost effective. In storage alone, I now back up all of my important personal data (redundantly I might add) and enhance my convenience in accessing it. All for less than $100/year.

I’m sold.