Tuesday, October 1, 2013

Out of Office (OOF / OOO) extensive troubleshooting procedures


1.       Determine scope of the issue.
a.       whether the issue affects users on a single server or multiple servers
b.      whether the issue affects users on multiple databases
c.       whether it works for a new user
d.      whether it is everyone or just a few users
e.      Whether it affects internal OOF / external OOF or both
 
2.       Basic troubleshooting steps:
a.       If it only affects a single server then moving the mailbox to another server may resolve the issue
b.      If it only affects a single database then moving the mailbox to another database may resolve the issue
c.       If it works for new users then it may be corruption in the mailbox – running a repair (New-MailboxRepairRequest) may resolve the issue
d.      If it affects everyone then it may be an organisation, site, or server level configuration issue such as Autodiscover.
e.      If it only affects external OOF then it may be an organisation configuration preventing external OOF.
 
3.       How OOF works under the covers:
a.       Note that the below screenshots are taken using MFCMAPI.  Outlook 2010 has been installed on the same computer to allow it to create mailbox profiles. MAPICDO is another option to allow profile creation however it was throwing errors when trying to access certain parts of the mailbox.
b.      When you create enable or disable OOF, it sets a property in the root of the mailbox PR_OOF_STATE to True (if enabled) or False (if disabled)
 


c.       When you enable OOF, you will also create an OOF message – generally both an internal OOF message and an external OOF message.  These should be automatically generated and stored as hidden messages in your inbox.  In MFCMAPI you will see these by browsing down Root Container à Top of Information Store à Right click Inbox à Open Associated Contents Table.  In the below screenshots you will see 3 rules.  Each has been created for a different purpose.  The one of class IPM.Note.Rules.ExternalOofTemplate.Microsoft contains the message that will be sent to users outside the organisation.  The IPM.Note.Rules.OofTemplate.Microsoft class will be sent to users within the organisation.  If you double click on one of these messages, you should see a copy of the OOF message that will go out. 



I am not sure why there are two of the internal OOF message class – possibly an additional one that is created for legacy support purposes.
d.      Another effect of enabling OOF is that rules will be created in the inbox which facilitate the out of office to actually send the message when the condition is triggered (i.e. receiving an email message).  As per below screenshot you may have up to three rules created for OOF.  The one highlighted below is the MSFT:TDC OOF rule and this will exist whether OOF is turned on or off.  The other two “IPM.Rule.Message” messages above it represent the internal OOF and external OOF rules respectively.  If only internal OOF is enabled then you will only see one of these rules.  If no OOF is enabled then you will not see either of these rules.



e.      You can also see these rules by right clicking on Inbox à Other tables à Rules table… 



 
4.       Try toggling OOF in both Outlook and OWA. 
a.       If it works in OWA but not Outlook then try creating a new Outlook profile on the same computer.
                                                               i.      If this fails then try creating a new Outlook profile on another computer
                                                             ii.      If the issue is resolved then no more troubleshooting steps are required
b.      If it does not work in either Outlook or OWA then continue to the next step.
 
5.       Try toggling OOF using the Exchange Management Shell (Exchange 2010 only)
a.       Set-MailboxAutoReplyConfiguration "mailbox name" –AutoReplyState Disabled
b.      Set-MailboxAutoReplyConfiguration "mailbox name" –AutoReplyState Enabled
c.       Check whether OOF is sent when set by the above method.
d.      If this method is successful in toggling OOF on and off, then we know there is not likely a mailbox level issue.  You should probably check your Exchange organisation configuration, autodiscover configuration, and for known Exchange bugs.
e.      You can also try the remaining troubleshooting steps in this document.
 
6.       If you determined in the previous step that it works in OWA but not Outlook then investigate the Autodiscover URLs to ensure they are correct.
a.       If it works in OWA but not Outlook then investigate Autodiscover – specifically the https://domain.com/EWS.Exchange.asmx URL.
b.      Open Outlook and do an Autodiscover connection test



c.       Check for errors – especially ones that may relate to delegates.  If there are errors relating to delegates then you may have a faulty delegate – e.g. maybe a delegate that has been disabled or had the mailbox removed from the account.
d.      If you find delegate errors, following the steps below on resolving delegate issues is your best bet.
 
7.       Check for problematic delegate links.
a.       On a user that is experiencing the OOF issue (User A), run Get-ADUser “User A” –Properties msExchDelegateListBL
b.      If you believe one of the users in the returned list may be problematic (e.g. User B) then open User B in AD Users and Computers and locate the msExchDelegateListLink attribute.  Delete “User A” from the list.
c.       If this doesn’t resolve the issue then go to the next troubleshooting step.
 
The following resolution attempts require you to use MFCMAPI to enter the mailbox in raw mode and delete hidden messages from the mailbox.  If you are not comfortable with doing this then you should find someone who is or log a call with Microsoft.
 
8.       Attempt to delete hidden OOF messages from the mailbox.  An OOF message contains the template of the email that will be sent out if OOF is turned on.  If you have multiple OOF message (one for internal and one for external) then you may have two of these.  The message class of these templates is IPM.Note.Rules.OofTemplate.Microsoft.
a.       Go into Outlook or OWA and disable OOF for the user.
b.      Log in with MFCMAPI (use with Outlook 2010, not MAPICDO) à Default store à Root Container à Top of Information Store à Right click Inbox à Open Associated Contents table.
c.       Delete any OOF message of class:
                                                               i.      IPM.Note-Rules.OofTemplate.Microsoft
                                                             ii.      IPM.Note.Rules.ExternalOofTemplate.Microsoft
d.      Go back into Outlook or OWA and enable OOF.
e.      Refresh MFCMAPI and check whether the templates are recreated.  If they are then try sending an email to the mailbox and check whether you receive an OOF message in reply.
f.        If it is still not working then continue to the next step.
 
9.       Check for OOF rules in the mailbox.
a.       Go into Outlook or OWA and disable OOF for the user.
b.      Log in with MFCMAPI (use with Outlook 2010, not MAPICDO) à Default store à Root Container à Top of Information Store à Right click Inbox à Other Tables à Rules Table…
c.       Delete any rules with a name similar to following (these should be automatically regenerated when OOF is turned on):

d.      Go back into Outlook or OWA and enable OOF.
e.      Refresh MFCMAPI and check whether the rules are recreated.  If they are then try sending an email to the mailbox and check whether you receive an OOF message in reply.
f.        If it is still not working then go to the next step.
 
10.   Check whether OOF has been enabled at the top level.
a.       Log in with MFCMAPI (use with Outlook 2010, not MAPICDO) and select the problematic mailbox from the list (do not double click!)
b.      In the bottom window pane you will see a property PR_OOF_STATE which will be set to either true (OOF is enabled) or false (OOF is disabled).
 
c.       Go into Outlook or OWA and disable OOF for the user.
d.      Check that PR_OOF_STATE is false.
e.      Go into Outlook or OWA and enable OOF for the user.
f.        Check that PF_OOF_STATE is true.
g.       If the state is not changing when OOF is turned on and off, then please go to the next troubleshooting step.
h.      Note: you can make up your own troubleshooting steps by manually enabling the PF_OOF_STATE (double click and put a check in the Boolean checkbox).
i.    Note that when the PF_OOF_STATE is manually enabled, internal users will get the mailtip appear when adding the problematic user to he "To" or "CC" fields.  If you send the email however, you will not receive an OOF reply if the OOF rules and templates mentioned earlier are not there.
 
11.   Restart the Microsoft Exchange mailbox Assistants service
a.       This step is included as it was required following the previous steps that were performed in MFCMAPI for some mailboxes to fix the OOF settings.  A mailbox server restart would likely have the same effect.
b.      Log into the Exchange mailbox server that hosts the problematic mailbox.
c.       Start Services.msc
d.      Locate the Microsoft Exchange mailbox Assistants service



e.      Restart this service.  Microsoft have confirmed that this should not have any impact on end users (even if they are in online mode) so we were able to perform this during the day.
f.        Retry your OOF tests and verify whether it is now working.
g.       If it is still not working then continue to the next step.
 
12.   Turn on diagnostic logging.
a.       Open Exchange Management Console à Server Configuration à Select the mailbox server that hosts the problematic mailbox.
b.      Select “Manage Diagnostic Logging Properties” and turn to High or Expert the following:
                                                               i.      MSExchangeMailboxAssistants\OOF Assistant
                                                             ii.      MSExchangeMailboxAssistants\OOF Library
                                                            iii.      MSExchangeMailboxAssistants\Service
                                                           iv.      MSExchangeIS\9000 Private\Rules
c.       Monitor the Application Log on the mailbox server while toggling the OOF on and off to see if there are any errors or warnings reported.
d.      Turn off diagnostic logging.
e.      If this does not turn up any issues then continue to the next step.
 
13.   Attempt to repair mailbox
a.       Perform a New-MailboxRepairRequest with –DetectOnly to check for errors in the application log on the mailbox server.
b.      If errors are detected then run the above command against the mailbox without the DetectOnly switch.
c.       Perform testing.
d.      If this does not turn up any issues then continue to the next step.
 
14.   Attempt to move the mailbox
a.       Move the mailbox to another server, or if not possible then to another database on the same server.
b.      Perform testing.
c.       If this does not resolve the issue then continue to the next step.
 
15.   Ensure there is no legacy hangover from Exchange 2003
a.       Run Set-Mailbox “Problematic User” –ApplyMandatoryProperties
b.      Note that this command is pretty safe to run, and if everything is correct it will return a message to the effect that no settings were changed.
c.       If changes were made then perform testing.
d.      If this does not resolve the issue then continue to the next step.
 
16.   Run Outlook with Clean Rules switch
a.       Backup any rules and delegates configuration in the mailbox.
b.      Run Outlook /cleanrules against the mailbox.
c.       Perform testing.
d.      If this does not resolve the issue then continue to the next step.
e.      Note: at this stage you might want to put back the configuration you just cleared.
 
17.   Check if the rules Quota has been exceeded (possible if the user has a lot of rules or delegates).
a.       Use Microsoft knowledge base articles to research increasing the rules quota and allow up to 2 hours to take effect (due to caching of quota information).
b.      Perform testing.
c.       If this does not resolve the issue then continue to the next step.
 
18.   If you have reached this step then you really are quite unique.  Please go with one of the following options:
a.       Log the issue with Microsoft. They will perform traces from Best Practices Analyser along with more advanced troubleshooting steps.
b.      Recreate the mailboxes – backup all attributes, email addresses, legacy Exchange DN, delegates configuration, rules, full mailbox permissions, send as permissions, send on behalf of permissions, folder permissions (can get these using the Exchange Management Shell), etc and importantly export all the data inside the mailbox to a PST.
 
19.   Good luck!  If any of these steps were useful in resolving your issue then please leave a quick comment to let me know.