Technology in Government – Cloud Computing

Executive Brief

A number of governments have implemented roadmaps and strategies that ultimately require their ministries, departments and agencies to default to Cloud computing solutions first when evaluating IT implementations. In this article, we evaluate the adoption of cloud computing in government and discuss some of the positive and negative implications of moving government IT onto the cloud.

Latest Trends

In this section, we look at a number of cloud initiatives that have been gaining leeway in the public sector:

  • Office Productivity Services – The New Zealand Government has identified office productivity services as the first set of cloud-based services to be deployed across government agencies. Considered to be low hanging fruit and fueled by successes in migrating perimeter services like anti-spam onto the cloud, many organizations see email and collaboration as a natural next step of cloud adoption. Vendors leading the charge include Microsoft’s Office 365 for Government, with successful deployments including Federal Agencies like the USDA, Veterans Affairs, FAA and the EPA as well as the Cities of Chicago, New York and Shanghai. Other vendor solutions include Google Apps for Government which supports the US Department of the Interior.
  • Government Cloud Marketplaces – A number of governments have signified the need to establish cloud marketplaces, where a federated marketplace of cloud service providers can support a broad range of users and partner organizations. The UK  government called for the development of a government-wide Appstore, as did the New Zealand Government in a separate cabinet paper on cloud computing in August 2012. The US government has plans to establish a number of cloud services marketplaces, including the GSA’s info.apps.gov and the DOE’s YOURcloud, a secure cloud services brokerage built on Amazon’s EC2 offering. (link) The image below lists the initial design for the UK government App store.
    03 UK App Store
  • Making Data publicly available  – The UK Government is readily exploiting opportunities to make available the Terabytes of public data that can be used to develop useful applications. The recent release of Met Office UK Weather information to the public via Microsoft Azure’s cloud hosting platform. (link)
  • Government Security Certification – A 2012 Government Cloud Survey conducted by KPMG listed security as the greatest concern for governments when it comes to cloud adoption and that governments are taking measures to manage security concerns. For example, the US General Services Administration subjects each successful cloud vendor to a battery of tests that include an assessment of access controls.

01a Canada Mappings

Canadian Government Cloud Architectural Components

Strategic Value

The strategic value of cloud computing can be summed up into a number of key elements in government. We’ve listed a few that appear on the top of our list:

  • Enhancing agility of government – Cited as a significant factor in cloud adoption, cloud computing promises rapid provisioning and elasticity of resources, reducing turnaround times on projects.
  • Supporting government policies for the environment – The environmental impact due to reduced data center spending and consumption of energy on cooling has tangible environmental benefits in terms of reduced greenhouse gas emissions and potential reductions in allocations of carbon credits.
  • Enhancing Transparency of government – Cloud allows the developed of initiatives that can make government records accessible to the public, opening up tremendous opportunities for innovation and advancement.
  • Efficient utilization of resources – By adopting a pay-for-use approach towards computing, stakeholders are encouraged to architect their applications to be more cost effective. This means that unused resources are freed up to the common pool of computing resources.
  • Reduction in spending – Our research indicated this particular element is not considered to be a significant aspect of moving to cloud computing according to technology decision makers, however some of the numbers being bandied about in terms of cost savings are significant (Billions of dollars) and can appeal to any constituency.

Positive Implications

We’ve listed a number of positive points towards cloud adoption. These may not be relevant in every use case, but worthwhile for a quick read:

  • Resource Pooling – leads to enhanced efficiency, reduced energy consumption and more economical cost savings from scale
  • Scalability – Unconstrained capacity allows for more agile enterprises that are scalable, flexible and responsive to change
  • Reallocation of human resources – Freed up IT resources can focus on R&D, designing new solutions that are optimized in cloud environments and decoupling applications from existing infrastructures.
  • Cost containment – Cloud computing requires the adoption of a ‘you pay for what you use’ model, which encourages thrift and efficiency. The transfer of CAPEX to OPEX also smoothes out cash-flow concerns  in an environment of tight budgets.
  • Reduce duplication and encourage re-use – Services designed to meet interoperability standards can be advertised in a cloud marketplace and become building blocks that can be used by different departments to construct applications
  • Availability – Cloud architecture is designed to be independent of the underlying hardware infrastructure and promotes scalability and availability paradigms such as homogeneity and decoupling
  • Resiliency – The failure of one node of a cloud computing environment has no overall effect on information availability

Negative Implications

A sound study should also include a review of the negative implications of cloud computing:

  • Bureaucratic hinderances – when transitioning from legacy systems, data migration and change management can slow down the “on demand” adoption of cloud computing.
  • Cloud Gaps – Applications and services that have specific requirements which are unable to be met by the cloud need to be planned for to ensure that they do not become obsolete.
  • Risks of confidentiality – Isolation has been a long-practiced strategy for securing disparate networks. If you’re not connected to a network, there’s no risk of threats getting in. A common cloud infrastructure runs the risk of exploitation that can be pervasive since all applications and tenants are connected via a common underlying infrastructure.
  • Cost savings do not materialize – The cloud is not a silver bullet for cost savings. We need to develop cloud-aligned approaches towards IT provisioning, operations and management. Applications need to be decoupled and re-architected for the cloud. Common services should be used in order to exploit economies of scale; applications and their underlying systems need to be tweaked and optimized.

05 Cloud Security concerns

Security was cited as a major concern (KPMG)

Where to start?

There is considerable research that indicates government adoption of cloud computing will accelerate in coming years. But to walk the fine line of success, what steps can be taken? We’ve distilled a number of best practices into the following list:

00 USG Roadmap

  1. Develop Roadmaps:  Before Cloud Computing can reap all of the benefits that it has to offer, governments must first move along a continuum towards adoption. For that very purpose, a number of governments have developed roadmaps to aid in developing a course of progression towards the cloud. Successful roadmaps featured the following components:
    • A technology vision of Cloud Computing Strategy success
    • Frameworks to support seamless implementation of federated community cloud environments
    • Confidence in Security Capabilities – Demonstration that cloud services can handle the required levels of security across stakeholder constituencies in order to build and establish levels of trust.
    • Harmonization of Security requirements – Differing security standards will impede and obstruct large-scale interoperability and mobility in a multi-tenanted cloud environment, therefore a common overarching security standard must be developed.
    • Management of Cloud outliers – Identify gaps where Cloud cannot provide adequate levels of service or specialization for specific technologies and application and identify strategies to deal with these outliers.
    • Definition of unique mission/sector/business Requirements (e.g. 508 compliance, e-discovery, record retention)
    • Development of cloud service metrics such as common units of measurement in order to track consumption across different units of government and allow the incorporation of common metrics into SLAs.
    • Implementation of Audit standards to promote transparency and gain confidence
  2. Create Centers of Excellence: Cloud Computing Reference Architectures; Business Case Templates and Best Practices should be developed so that cloud service vendors should map their offerings to (i.e. NIST Reference Architecture) so that it is easier to compare services.
  3. Cloud First policies: Implementing policies that mandate all departments across government should consider cloud options first when planning for new IT projects.

Conclusion

The adoption of cloud services holds great promise, but due to the far reaching consequences necessitated by the wide-spread adoption of cloud to achieve objectives such as economies of scale, a comprehensive plan compounded with standardization and transparency become essential elements of success.

We hope this brief has been useful. Ook!

Useful Links

Microsoft’s Cloud Computing in Government page
Cisco’s Government Cloud Computing page
Amazon AWS Cloud Computing page
Redhat cloud computing roadmap for government pdf
US Government Cloud Computing Roadmap Vol 1.
Software and Information Industry updates on NIST Roadmap
New Zealand Government Cloud Computing Strategy link
A
ustralian Government Cloud Computing Strategic Direction paper
Canadian Government Cloud Computing Roadmap
UK Government Cloud Strategy Paper
GCN – A portal for Cloud in Government
Study – State of Cloud Computing in the public sector

Exchange 2013 Sample Architecture Part 3: Design Feature Overview and Virtualization Considerations

Overview

In this part of the Sample Architecture Series, we will hone in on several elements of the Exchange Solution design, namely a description of the overall Exchange 2013 solution design, followed by some basic system configuration parameters as well as virtualization considerations.

Design Features

Exchange 2013 Design

The Exchange 2013 Environment for Chimp Corp features the following design elements:

  • Internal Client Access: Internal clients can automatically locate and connect to available CAS Servers through the Availability and Autodiscover services. CAS Servers are configured in arrays for high-availability and the locations of the CAS servers are published through Service Connection Points (SCPs) in Active Directory.
  • External Client Access:  External clients can connect to Exchange via Outlook Web Access (OWA), Outlook Anywhere and Exchange ActiveSync. Exchange 2013 now supports L4 load balancing for stateless failover of connections between CAS servers in the same Array. Client traffic arrives at the Network Load Balancer, which uses Service Connection Points to locate the internal Mailbox servers and distribute load accordingly.
  • Single Domain Name URL: Exchange 2013 relies on a feature in the TCP/IP protocol stack of client computers that supports the caching of multiple IP addresses that correspond to the same name resolved from DNS. In the event of an individual site failure, the IP address corresponding to the CAS array in that site will become unresponsive. Clients automatically connect to the next cached IP address for the CAS Array in order to reestablish client connections. This IP address corresponds to the CAS Servers in the alternative site and failover occurs without any intervention.
  • Mailbox High availability: This feature is  provided by implementing Database Availability Groups (DAG). A single DAG will be configured to protect the messaging service.  It is preferred to deploy a high number of smaller mailbox databases in order to reduce mailbox restoration or reseed times in the event of a failure of a database copy.
  • Message Routing: All External SMTP traffic will be routed securely via Microsoft’s Exchange Online Protection (EOP) cloud-based services and the Internet. Inter-site messages between the premise and online users will also be routed via EOP. Internal messages between on-premise users in either datacenter site will be routed automatically via the transport service on the on-premise Mailbox servers.
  • Hybrid Deployment: The Exchange 2013 environment will be deployed in tandem with an Exchange Online Organization. The purpose of the Exchange Online Organization will be to host mailbox accounts that have been flagged as non-compliance sensitive and reduce the costs of the on-premises deployment. The hybrid implementation will feature a seamless experience between users in the on-premise and online environments, including Single Sign-on for users through the configuration of trusts between the Microsoft Online ID and the on-premises Active Directory Forest; unified GAL access and the ability for online and on-premise users to share free/busy information through the configuration of a Federation Trust with the Microsoft Federation Gateway; as well as secure encrypted message transport between on-premise and online environments, encrypted, authenticated and transported via Transport Layer Security (TLS)
  • Message Archiving: All Messages will be transferred to the Exchange 2013 via the Exchange Online Archiving Service. The existing on-premises archiving solution will be decommissioned after existing message archives are ingested into the Exchange Online Archive.

Exchange 2013 Virtualization

All Exchange 2013 server roles are fully supported for virtualization by Microsoft. Virtualization can assist an organization in consolidating its computing workload and enjoying benefits from cost reduction and efficient hardware resource utilization. According to Microsoft recommended Best Practices, load calculations when provisioning Exchange deployments in a virtual environment must accommodate for additional overheads from the virtualization hypervisor. Therefore, this solution design has factored in an additional resource overhead of 12% to accommodate virtualization.

The following server roles will be virtualized:

  •     Exchange 2013 Mailbox Servers
  •     Exchange 2013 CAS Servers

Microsoft provides further guidance on implementing Exchange Server 2013 in a virtualized environment. Relevant factors have been listed below:

  1. Exchange Servers may be combined with virtual host-based failover clustering migration technology, provided that the virtual machines are configured to not save and restore disk state when moved or taken offline. Host-based failover must result in a cold boot when the virtual machine is activated on a target node.
  2. The root machine should be free of all applications save the virtual hypervisor and management software.
  3. Microsoft does not support taking a snapshot of an Exchange virtual machine.
  4. Exchange supports a Virtual Processor to Physical Processor ratio of no greater than 2:1 and Microsoft recommends an ideal processor ratio of 1:1. Furthermore, virtual CPUs required to run the host OS should be included in the processor ratio count
  5. The disk size allocated to each Exchange Virtual machine must use a disk that is of size equal to 15GB plus the size of virtual memory allocated to the virtual server.
  6. The storage allocated for Exchange data can either be virtual storage of a fixed site, such as fixed Virtual Hard Disks (VHDs), SCSI pass-through storage or iSCSI storage.
  7. Exchange 2013 does not support NAS storage. However, fixed VHDs that are provisioned on block level storage and accessed via SMB 3.0 on Windows Server 2012 Hyper-V are supported.
  8. Exchange 2013 is designed to make optimal usage of memory allocations and as such, dynamic memory features for Exchange are not supported.

Conclusion

Subsequent sections of this series will focus on the Exchange Mailbox Design and CAS Design, as well as the Hybrid Implementation and additional features.

Please click here for the next part: Exchange 2013 Mailbox Server Role Design.

Exchange 2013 Sample Architecture Part 2: High-level Architectural Design Document

Overview:

This section provides an introduction into the key elements of the Exchange 2013 Architectural solution. It provides high-level solution overview and is suitable for all technical project stakeholders. The excerpts of the final design document are listed under this post and the full High Level Design document can be downloaded here: RoadChimp Sample Architectural Doc (ADOC) v1.1

1. Messaging Infrastructure function

The Messaging Infrastructure serves the primary function of providing electronic mail (E-mail) functionality to Chimp Corporation. The messaging infrastructure supports E-mail access from network connected computers and workstations as well as mobile devices. E-mail is a mission critical application for Chimp Corp and it serves as an invaluable communications tool that increases efficiencies and productivity, both internally to an organization, and externally to a variety of audiences. As a result, it is of paramount importance for the Chimp Corp to maintain a robust infrastructure that will meet present and future messaging needs.

Key requirements of the messaging infrastructure are as follows:

  • Accommodate service availability requirements
  • Satisfy archiving requirements
  • Satisfy growth requirements
  • Provide required business continuity and disaster recovery capabilities

1.A. About IT Services (ITS)

The IT Services Organization is responsible for managing the IT environment for the Chimp Corp as well as ensuring adherence to published standards and operational compliance targets throughout the enterprise.

1.B. Service Level Agreement (SLA)

The Email infrastructure is considered mission critical and, therefore, has an SLA requirement of 99.99% availability.
The full SLA for the messaging environment can be found in the document <Link to SharePoint: Messaging SLA> 

1.C.  Locations

The messaging infrastructure is hosted from two separate datacenters being at:

  • Datacenter A (DCA)
    Chimp Center Prime
    1 Cyber Road,
    Big City
  • Datacenter B (DCB)
    Chimp Center Omega
    10 Jungle Way,
    Banana Town

The messaging infrastructure is supported by the IT Services Support Organization located at:

  • Chimp Corp Headquarters
    Chimp Center Prime
    Bldg 22, 1 Cyber Road,
    Big City

1.D.     E-mail User Classifications

The primary users of the messaging system are Chimp Corp employees. The user base is divided in two groups as follows:

  •     Exec: users performing Senior or Critical corporate functions
  •     Normal: the rest of the user population

2. Existing Platform

This section of the Asset document provides an overview of the present state of the asset, as well as a chronological view of changes based on organizational or technological factors.

2.A.     Existing Exchange 2003 design

A third-party consulting company performed the initial implementation of the messaging environment in 2000. The messaging platform was Microsoft Exchange 2003 and Windows 2003 Active Directory. The diagram below provides a representation of the existing design blueprint.

Exchange 2003 Environment

Fig. 1 Existing Messaging Environment

A single unified Active Directory Domain namespace chimpcorp.com was implemented in a Single Domain, single Forest design.

2.B. Change History

Over the years the Chimp Corp messaging environment has undergone various changes to maintain service level and improve functionality. The timeline below shows the changes over time.

Chimpcorp Timeline
Fig. 2 Chimp Corp Messaging Infrastructure Timeline

2.B.1   Initial Implementation

The Exchange 2003 messaging infrastructure was implemented by IT Services in 2005 and the entire user base was successfully migrated over to Exchange 2003 by September 2005.

2.B.2   Linux Virtual Appliances deployed for Message Hygiene

A decision was made by IT to deploy a Message Hygiene environment for the company in Windows 2013.

 This change was scheduled as maintenance and was executed early 2009.

2.B.3   Additional Datacenter Site (Omega)

In order to improve infrastructure availability and to support additional growth of the corporate environment, a second datacenter site, codenamed Omega was commissioned and fully completed by March of 2009.

2.B.4   Two Exchange Mailbox Clusters (A/P) deployed in the Omega Datacenter Site

To improve the availability of e-mail for users and also to meet plans for storage and user growth, two additional Exchange Mailbox Servers were deployed in Datacenter Omega (DCB).

2.B.5   Third-party archiving solution

A third party archiving solution was deployed by IT Services in 2010 as part of efforts to mitigate growth of the Exchange Information Stores, based on recommendations from their primary technology vendor. The archiving solution incorporates a process known as e-mail stubbing to replace messages in the Exchange Information Stores with XML headers.

2.B.6   Acquisition by Chimp Corp

After being acquired by Chimp Corp in September 2011, immediate plans were laid out to perform a technology refresh across the entire IT infrastructure.

2.B.7   Active Directory 2008 R2 Upgrade

The Windows Active Directory Domain was updated to version 2008 R2 in native mode in anticipation of impending upgrades to the Messaging infrastructure. The replacement Domain Controllers were implemented as Virtual Machines hosted in the Enterprise Virtual Server environment running VMWare vSphere 5. This change was completed in March 2012.

2.C. Existing Hardware Configuration

The current hardware used in the messaging platform consists of the following elements:

2.C.1   Servers

Existing server systems comprising the messaging environment include:

    • 12 x HP DL 380 G4 servers at DCA with between 2 – 4 GB of RAM
    • 10 x HP DL 380 G4 servers at DCB with between 2 – 4 GB of RAM

2.C.2   Storage characteristics

Exchange storage used for databases, backups, transaction logs and public folders have been provisioned on:

    • 2 TB of FC/SAN attached storage provisioned for 5 Exchange Storage Groups and 21 Databases and Transaction Logs
    • 2 TB ISCSI/SAN attached storage Archiving

2.D. Network Infrastructure

The Chimp Corp email infrastructure network has two main physical locations at the DCA and DCB datacenter sites. These are currently connected via the Chimp Corp LAN/WAN. The core switches interconnecting all hardware are Cisco 6500 Series Enterprise class switches.

2.E.  Present Software Configuration

Software and licenses presently in use include:

  • Microsoft Windows 2003 Standard
  • Microsoft Windows 2003 Enterprise
  • Microsoft Exchange 2003 Standard
  • Microsoft Exchange 2003 Enterprise
  • Third Party SMTP Appliances
  • A Stub-based third-party Email Archiving Tool

3. Messaging Infrastructure Requirements

The design requirements for the Exchange 2013 messaging environment have been obtained from the project goals and objectives, as listed in the Project Charter for the E13MAIL Project.

The primary objective for the E13MAIL Project is to ensure continued reliability and efficient delivery of messaging services to users and applications connecting to Chimp Corp from a variety of locations. Stated design goals are to increase performance, stability and align the operational capabilities of the messaging environment with Industry Best Practices.

The requirements/objectives for the messaging infrastructure are:

  • Redundant messaging solution deployed across 2 datacenter locations; DCA and DCB.
  • Capable of Audit and Compliance requirements
  • High Availability (99.99%)
  • Monitoring of services and components
  • Accurate configuration management for ongoing support
  • Adherence to Industry Best Practices for optimal support by vendors and service delivery organizations
  • Reliable Disaster-Recoverable backups, with object level recovery options
  • Message Archiving functionality with a maximum retention period of 7 years

4. Design Components

The primary messaging solution is to deploy new Exchange 2013 environment that spans Chimp Corp’s physical data center locations and extends into Microsoft’s Office 365 cloud to take advantage of the latest user productivity and collaboration features of Microsoft Office 2013.

The main goals for this solution are:

  • Minimize end-user impact: Minimizing the end-user impact is a key goal for Chimp Corp. Significant effort must be made to ensure that the transition of all e-mail related services are seamless to the end-user.
  • Reliable delivery of services: The messaging environment is a mission critical component of Chimp Corps IT infrastructure and adheres to strict Change Management practices. The solution must be able to integrate with existing Operational and Change processes.
  • Longevity of solution: The new messaging solution must endure beyond the initial implementation as it evolves into a production state. This requires the necessary attention to ensuring that operational knowledge is transferred to IT Services technical teams such that they can maintain uptime requirements.

The individual design components were subjected to a stringent evaluation process that included the following design criteria:

  •     Costs of Ownership
  •     Technological engineering quality
  •     Scalability
  •     Fault Tolerance / Reliability
  •     Industry best practices
  •     Supportability
  •     Ease of administration
  •     Compatibility with existing systems
  •     Reliability
  •     Vendor specifications

4.A. Hardware technology

IT Services researched the server solutions from a number of hardware vendors and made a final decision in favor of HP, Brocade and Cisco vendor equipment.

4.A.1   Server hardware

The server platform used is the eighth generation (G8) HP Blade 400-series server. This is an Intel based server system. The CPUS’s in these systems are standardized to Intel Xeon E5-2640 processors; these are hex-core processors with a 2.5 GHz speed. The servers are equipped with 128 GB of memory to accommodate their specific functions. The blade servers are provisioned in HP Blade C 7000 class enclosures.

4.A.2   Storage hardware

To accommodate the storage requirements two storage arrays are implemented. The primary array is an HP EVA 6400 class Storage Area Network. This array is equipped with 25TB of RAW storage and is used for on-line, active data. The secondary array is an HP P2000 G3 MSA class storage area network. This array is equipped with 15TB of RAW storage and is used for secondary storage like archives, backups etc.

4.A.3   Interconnect technology

HP’s Virtual Connect technology is used to accommodate network connectivity to both the storage network and the data networks. The virtual connect technology acts as a virtual patch panel between uplink ports to the core switching infrastructure and the blade modules. The virtual connect backplane will connect the network connections into a Cisco based core network. The storage area network is interconnected via a Brocade switch fabric.

4.A.4   Server Operating Systems technology

The majority of the messaging infrastructure components will be deployed onto the Microsoft Windows Server 2012 Operating System platform, licensed to the Enterprise version of the operating system. For systems that do not support Windows Server 2012, Windows Server 2008/R2 will utilized.

4.A.5   Messaging platform technology

A pristine Microsoft Exchange Server 2013 will be implemented in a hybrid configuration, featuring two major components:

  • On-premise Exchange 2013: The on-premise environment to support core business functions that cannot be moved to the cloud due to compliance reasons.
  • Office 365: All non-compliance restricted users will be migrated onto the Office 365 cloud.

The hybrid deployment will feature full interoperability between on-premise and cloud-based users, featuring single sign-on, sharing of calendar Free/busy information and a single unified OWA login address.

4.A.6   Back-end Database technology

Microsoft SQL Server 2012 was selected as the database platform to support all non-Exchange application requirements. The selection criterion for this product was partly dictated by the usage of technologies that depend on the SQL server back-end. As part of simplification and unification, it is preferred to keep all back-end databases in the messaging infrastructure on the same database platform.

4.A.7   Systems Management Solution

Due to the diversity of software applications and hardware in this infrastructure, a mix of management tools and products are used to manage all aspects of the messaging infrastructure. Major components are listed below:

(a)    Server hardware management: Vendor provided HP System Insight Manager hardware tools are used in combination with Microsoft System Center Operations Manager (SCOM) to provide hardware-level monitoring and alerting.

(b)    Server event management: Microsoft Systems Center Operations Manager (SCOM) 2012 is used for server event consolidation, management and alerting.

(c)     Server Applications management: Server software management comprises of systems patch management and server provisioning.

    • Systems patch management: Windows Systems Update Server (WSUS) integrated into Systems Center Configurations Manager (SCCM) provides patch management of all Windows Server Operating Systems in the messaging environment.
    • Server Provisioning: Server Provisioning for both bare metal and virtual server deployments are managed via the HP rapid deployment pack (HP/RDP)

4.A.8   Message Security and Protection technology

The following Security and Protection products have been selected:

  • Server Virus protection: McAfee Antivirus has been selected to protect the server operating system.
  • Message hygiene: Microsoft Exchange Online Protection (EOP) will be used for message hygiene and protection.
  • Security events auditing: Microsoft SCOM has been selected to capture information such as security auditing and alerting events that are generated from the server platforms.

4.B. Functional Blueprint

The blueprint below illustrates the desired messaging infrastructure:

Exchange 2013 Design

Figure 3 Chimp Corp Functional Messaging Design 

Conclusion

In the next section we will cover more detailed aspects of the Exchange 2013 design, as well as Server Virtualization Considerations for deploying Exchange 2013.

For the next part of this post, please click here.

Exchange 2013 Sample Architecture Part 1: Implementation Scenario

Scenario Overview:

Chimp Corp has recently completed the acquisition of a competitor, Bananas Inc. As part of the core infrastructure architecture team, you have been brought in to design and implement Exchange 2013 as part of a large Enterprise systems refresh. The Project Charter has been signed off by senior stakeholders with the objective of upgrading the existing messaging environment from Exchange 2003 SP2 to Exchange 2013. Senior Management has expressed a desire to migrate the messaging environment to the cloud in order to take advantage of cost benefits, however the compliance department has mandated that specific components of the messaging environment must stay on-premises in order to meet regulatory requirements.

Management has decided to deploy a Hybrid Exchange 2013 environment in a new Active Directory Forest that is Federated to an Exchange Online organization. The on-premise environment will host approximately 60% of the Organization’s mailboxes and the remaining 40% of the Organization’s mailboxes are considered to be non-sensitive and the compliance department has approved their migration onto the cloud. This scenario represents the path of least resistance, as Microsoft Exchange 2013 does not support direct upgrade path from Exchange 2003 to Exchange 2013 and due to the considerable size of the corporate messaging environment (15,000 Mailboxes), a swing migration to Exchange 2007/2010 and then to Exchange 2013 was considered to be impractical.

Existing Environment:

Exchange 2003 Environment

The messaging environment features Exchange 2003 SP2 with Active Directory 2008, featuring four Clustered Exchange Mailbox Servers implemented across two datacenters with dedicated Network Load Balanced Exchange Front End Servers in each location. Third-party SMTP Message Hygiene appliances were configured in each site to provide Spam Filtering and Anti Virus Scanning and in addition, a number of applications were configured to relay SMTP messages via one of the SMTP appliances. A third-party Archiving tool was deployed across both sites and client access was provisioned primarily via Outlook RPC, OWA; Blackberry Enterprise Servers and Microsoft ActiveSync.

Requirements:

The following solution requirements were distilled from theRequest for Proposal (RFP) document. The solution must:

  • Conform to Microsoft Best Practices
  • Accommodate 99.99% high availability standards
  • Adhere to Disaster Recovery, High Availability and Business Continuity standards
  • Provide a fully redundant and load balanced design
  • Accommodate 9,000 Mailboxes across 2 datacenters
  • Accommodate 6,000 Mailboxes on an Exchange Online organization
  • Average Mailbox Size 1 GB
  • Anticipated Storage Growth per year of 20%
  • Store Archived Emails for 7 years
  • Adhere to Retention,  Legal Hold and eDiscovery requirements
  • Perform Email Archiving whenever a mailbox reaches 1GB or whenever messages are 1 year old.
  • Network Access Bandwidth: 1 Gbps
  • Storage Access: Minimum bonded Gigabit Connections or Fibre Channel
  • Client Access: Outlook 2007 and later, Internet Explorer, Safari and Firefox using OWA, PDA access from RIM Blackberries, IP Phones and Microsoft Windows Mobile 6 and later.

Proposed Solution:

Based on stipulated requirements in the RFP, the Proposed Solution must include a number of components including the following:

  1. Methodology used to implement Exchange 2013 and related architectural components
  2. Related hardware and software builds
  3. Related costs of implementing the solution
  4. Annual cost of ownership
  5. A high level project plan detailing the responsibilities of various stakeholders

The final solution proposed an implementation of Exchange 2013 configured as a Hybrid environment. The Exchange 2013 environment would feature the following benefits:

  • Scalability and flexibility of moving users to the cloud
  • Virtualization of on-premises Exchange environment
  • Migration of Archives to Exchange Online Archiving
  • Deployment of High Availability using native Microsoft technologies
  • Unified Management via Microsoft Exchange Administration Center
  • Systems Management via Microsoft System Center Configuration Manager (SCCM) 2012

Solution Components:

The solution featured a high-level design that was broken into the following components:

  • Exchange Server
  • Infrastructure Services
  • Email Archiving
  • Storage Design
  • Backup and Recovery
  • Systems Management
  • Client Access

Conclusion:

Successive sections in this series will provide you with the various design components of the final solution, as well as related Project Implementation plans.

For Part 2: High-level Architectural Design Document, click here.

Exchange 2013 Architecture Samples

I’m posting a set of Sample Architectural Design documents that were adapted from a real-world Multi-site Hybrid Exchange deployment. The documents are built entirely on Best Practices and I’ve taken the liberties of updating elements of the design to reflect changes in the Exchange 2013 Architecture (and of course to remove and sensitive confidential information).

This was a fairly large implementation and took the combined efforts of a large team of engineers to complete all of the deliverables, who inspired me and continue to do so. You may not need all of these document components, but it’s good to see how a large Messaging Environment can be broken down into its constituent components and architected in detail.

Read First: These design documents were derived from detailed research and consulting with expert engineers. Feel free to use as a reference, but always verify the requirements of your project against the data in these guides. Roadchimp takes no responsibility for any implementation issues that you encounter. Make sure that you implement licensed copies of Microsoft Software with valid Microsoft Support in place prior to making any changes to a production environment. Further more, make sure that you consult with Microsoft’s  resources to ensure that your hardware is fully supported by Microsoft for deploying Exchange 2013, Windows Active Directory and other architectural components.

I will start posting links to these templates here:

Exchange 2013 Brief – Hybrid Deployments

Executive Overview

The cloud offers consumers more options for deploying their applications and is attractive from the perspective of predictable costs, reliability and scalability. However, not every component of an Organization’s environment may be fully suited for the cloud due to a variety of reasons including confidentiality and compliance. With the increasing trend of organizations to move parts of IT onto the cloud and retain core aspects of their business within their datacenters, it becomes important for us to understand how Exchange 2013 interoperates between on-premises and cloud. Exchange 2013 is designed from the ground up to support coexistence with the cloud. From both the administrator and end-user’s perspective, Exchange 2013 and Office 365 provide a seamless and feature rich experience. We will explore some of these features in this post.

Notable Features

  • Secure mail routing
  • Mail routing with the same domain space
  • Unified GAL and Free/Busy sharing
  • Centralized Egress of Messages
  • Unified OWA login
  • Centralized Management
  • Mailbox Migrations
  • Cloud-based Message Archiving

Architecture/Components

  • Architecture Components: A hybrid Exchange 2013 environment comprises of the following components.
    • Exchange servers: You may have a combination of Exchange 2013, Exchange 2010 or earlier Exchange Servers and roles deployed on-premises. You will need a minimum of one Exchange 2013 Client Access and one Exchange 2013 Mailbox Server if you deploy Exchange 2013 on-premises in your organization.
    • Microsoft Office 365: This is Microsoft’s feature-rich cloud based service that includes cloud-based email, instant messaging and online conferencing, Office Web Apps including Word, Excel, Powerpoint and OneNote and Email Archiving. You will need the Midsize Business and Enterprise Plan (E3) in order to configure Active Directory Synchronization with your on-premises environment. You will also need to configure an Exchange Online organization to enable hybrid deployments.
    • Exchange Online Protection (EOP): EOP is included in all Office 365 Enterprise tenant subscriptions. EOP enables secure message delivery between cloud and on-premises Exchange Organizations and can also be configured to manage message routing between the Internet and your on-premises Exchange Organization.
    • Hybrid Configuration wizard: The Hybrid Configuration wizard is used to manage the hybrid configuration through the Exchange Administrative Center (EAC). The Hybrid Configuration Wizard first performs prerequisite and topology checks, tests account credentials between on-premise and Exchange Online organizations and then subsequently performs the necessary configuration changes to create and enable the hybrid deployment, this includes adding the HybridConfiguration object in the on-premise Active Directory environment.
    • Microsoft Federation Gateway: On-premises Exchange Organizations must configure a federation trust with the Microsoft Federation Gateway before they can enable a hybrid configuration with an Exchange Online organization. The Microsoft Federation Gateway acts as a trust broker between the on-premises Exchange and the Online Exchange organizations and federation trusts can be configured manually or via the Hybrid Configuration Wizard. A Federation Trust is necessary for your on-line and on-premise users to be able to share free/busy information.
    • Active Directory Synchronization: AD synchronization enables a unified GAL across Online and on-premises users in your Exchange deployment. AD Sync feature requires you to download and install the tool on a separate server (Physical or Virtual) in your on-premises environment. Note that the default limit of 20,000 objects that can be replicated between on-premises Active Directory and the online organization can be increased by contacting the Microsoft Online Services team.
    • Active Directory Federation Services (Optional): the AD FS server implementation will enable users in your organization to use their existing network credentials for logging on to the on-premises and Exchange Online organizations using “Single Sign-on”. This is facilitated by configuring trusts between the on-premises Active Directory Forest and the Microsoft Online ID.
    • Certificates: To support secure communications between the on-premises and Online environments, Microsoft recommends that you purchase a Subject Alternative Name (SAN) SSL certificate that can be used to secure access to the following services:
      • Primary shared SMTP domain: This is your primary email domain and needs to be installed on local Client Access and Mailbox Servers. ie. chimpcorp.com
      • Autodiscover: The autodiscover services supports the configuration of remote clients (Outlook and Exchange Active-sync), is installed on your CAS servers and should be provisioned according to the external Autodiscover FQDN of your Exchange 2013 CAS server. ie. autodiscover. chimpcorp.com
      • Transport: This is installed on your Exchange 2010 SP3 Edge Transport Servers and matches the external FQDN of your edge transport servers. ie. edge.chimpcorp.com
      • AD FS (optional): A certificate is required to establish trust between web clients and federation server proxies and to sign and decrypt security tokens.
      • Exchange Federation: A self-signed certificate is required to establish a secure connection between the on-premises Exchange 2013 servers and the Microsoft Federation Gateway.
      • Client Access: An SSL certificate is required for use by clients such as OWA and Exchange ActiveSync and Outlook Anywhere. ie. webmail.chimpcorp.com
  • Message Transport: Messages between the on-premises and online organizations are encrypted, authenticated and transferred via Transport Layer Security (TLS). Depending on how you choose to configure your hybrid environment, messages can flow either one of the following ways:
    • Centralized Mail Transport: All Internet-bound email is delivered via the on-premises Exchange Organization. The Exchange on-premises organization is responsible for message transport and relays all Internet messages from the Exchange Online organization. This configuration is preferable if your organization has compliance or regulatory requirements and must monitor a single point of egress for all messages outside of your organization. Ensure that you provision sufficient bandwidth between the on-premises and online environments to process all outbound messages.
    • Online-centric Transport: All Internet-bound email in the Organization is delivered via the Exchange Online organization. In this case, all external outbound messages from the on-premises Exchange Organization are relayed to servers in the Exchange Online organization. This is preferable if you wish to use Microsoft’s Exchange Archiving and Exchange Online Protection (EOP) solutions, as it supports the most efficient flow of messaging traffic.
    • Independent message routing: All Internet-bound email from recipients in the Exchange Online organization are delivered directly to the Internet, taking an independent path from your on-premises Exchange 2013 Organization.
    • Edge Routing: On-premises endpoint for Exchange and Exchange Online organizations must be an Exchange 2013 CAS Server, or Exchange 2010 SP3 Edge Transport Server. Communications between Exchange Online and older versions of Exchange, SMTP hosts or appliances  are not supported.
  • Client Access: In Exchange 2013 client access is supported from Outlook via RPC/HTTP and Outlook Web App. Clients connecting to the on-premises Client Access server are redirected to either the on-premises Exchange 2013 Mailbox Server or provided with a link to logon to the Exchange Online organization.

Common Administrative Tasks

  1. Set up an Office 365 account: Via the Office 365 online portal here.
  2. Enabling a Hybrid Deployment: Use the Hybrid Deployment Wizard in the EAC.
  3. Configure  or modify the Hybrid Deployment Options: Via the Hybrid Deployment Wizard in the EAC or Powershell
    Set-HybridConfiguration -Features OnlineArchive,MailTips,OWARedirection,FreeBusy,MessageTracking
  4. Verify the configuration was successful: Via PowerShell
    Get-HybridConfiguration
  5. Sharing Free/Busy information: Steps on how to configure Federation Trusts
  6. Configuring Active Directory Synchronization: Steps to download the AD Synchronization tool from the Office 365 portal.

Top PowerShell Commands/Tools:

– Set|Update|Get-HybridConfiguration

Click here to read more briefs on Exchange 2013.

References/Links

PowerShell Command Reference for Hybrid Configuration
Technet: Article on the Hybrid Configuration Wizard
Technet: Article on Hybrid Certificate Requirements
Technet: Article on configuring message routing
Labs on AD Synchronization

Exchange 2013 Configuration Guides

A warm Ook and hello from your banana loving primate friend! I’ve decided to put up a list of configuration guides for Exchange 2013 in an easy to access part of this blog. The configuration guides will help you to perform (hopefully) some tasks that you may find useful. I will post links to various guides on this page.

1. Exchange 2013 in Windows Azure

2. Configuring a Hybrid Exchange 2013 Deployment

 

I hope to get more posts out there. Thanks for all your comments and likes!

Road Chimp saying Ook!

 

 

Exchange 2013 Architecture Series – Part 3: Mailbox and Storage Design

Hello all, in this third section on Exchange 2013 Architecture, we will look into the Exchange storage subsystem and build around some best practices on Exchange 2013 design.

Exchange Storage Types

Before you start provisioning your Exchange Server 2013 environment, it’s useful to think about the different types of storage that a healthy Exchange environment uses. Each storage classification has its own unique performance requirements, and a well-architected Exchange storage architecture will be designed in order to support these needs.

  • Operating System and Page File Volumes: At the most fundamental level, all Exchange Servers run on top of an Operating System. In addition to storing the OS Kernel, the Operating System volume manages all I/O operations on the Exchange Server, as well as memory allocation and disk management.
  • Exchange Binaries: These volumes contain the actual application files that Exchange needs to run and follows the path: <Drive:>Program FilesMicrosoftExchange ServerV15. Microsoft requires at least 30GB of free space on the drive you wish to install the Exchange binaries on.
  • Exchange Database Volumes: These volumes store the actual Exchange Database files, which follow the format ‘.edb’. Enhancements in the Exchange Server database engine have resulted in reductions in disk resource requirements. However, database volumes should still be optimized for high performance and commonly use RAID striped volumes with parity to support high IOPS.
  • Exchange Database Log Volumes: Exchange 2013 uses a relational database technology that utilizes transaction logs to record changes to the Exchange Databases. The database log volumes are write intensive in nature.
  • Exchange Transport Database Volumes: Changes to how Exchange 2013 manages mailflow have resulted in the creation of several new features known as Shadow Redundancy and Safety Net. Read my previous post for more information on these new features. For Shadow Redundancy, the transport server makes a redundant copy of any messages it receives before it acknowledges successfully receiving the message back to the sending server. The Safety Net feature is an updated version of the Transport Dumpster and retains copies of retained messages in a database for a default of 2 days, via the SafetyNetHoldTime parameter. You should design your storage to accommodate two full days of additional e-mails within a high-availability boundary.
  • Backup/Restore Volumes: With database replication and resiliency features of Exchange now providing fault tolerance and high availability, backup and restore services are less crucial. However, they must be considered in the event of restoring historical or archived data. Most organizations consider less expensive storage types such as WORM (Write one, read many)

Storage hardware technologies

Exchange 2013 supports the following storage hardware technologies:

  • Serial ATA (SATA): SATA disks are cost effective, high capacity storage options that come in a variety of form factors. Microsoft recommends that you do not store your Exchange databases across a spanned volume comprising of multiple SATA drives.
  • Serial-attached SCSI: SCSI is a mature disk access technology that supports higher performance than SATA, but at a higher cost.
  • Fibre Channel (FC): Fibre Channel (note the spelling) support high performance and more complex configuration options such as connectivity to a SAN at a higher cost. FC disks are typically used in a collection of disks known as an Array and support the high-speed transmission of data (up to 16Gbps and 3200 MBps) and potentially require expensive fibre-channel infrastructure (known as switch-fabric) supporting single-mode or multi-mode fiber cables. The advantages of Fibre-channel are that the disks can be colocated a significant distance away from the actual servers (hundreds of meters up to Kilometers) without experiencing any loss in performance, which means that an organization can consolidate the disks used by numerous applications into one part of the datacenter and configure them optimally with high-redundancy features. This is the basic justification for a SAN.
  • Solid-state Drive (SSD): SSD drives use flash-type memory to store data and have a number of advantages of conventional SATA and SCSI based disk technologies which still employ rotary mechanical operations to access and write data on spinning platters. SSD drives are a relatively newer technology and currently support lower disk capacities but feature very high performance, boasting low disk access times (0.1ms compared to SCSI 4-12ms times). Due to their high cost, it is common for organizations to build servers with a combination of disk types, using SSD for Operating System partitions as well as volumes that benefit from write-heavy access, such as transaction log volumes for relational database systems.

Logical Storage Architectures

Exchange 2013 has changed how it addresses storage architecture from the ground up. Since the Extensible Storage Engine was rewritten via managed code and optimized for multiple threading, the storage design for Exchange has had to change as well to keep up. Microsoft provides the following recommendations with respect to the following storage architectures:

  • Direct/Locally Attached Storage (DAS): DAS storage architecture featured disks and arrays that are locally attached to the Exchange Server and are commonly supported by Microsoft Exchange 2013 and include Serial ATA (SATA) and SCSI hardware architectures.
  • Storage Area Networks (SANS): SAN architecture is fully supported in Microsoft Exchange 2013 both over Fibre and iSCSI interfaces.
    • Microsoft recommends that you allocate dedicated volumes (spindles) to Exchange Server and not share the same underlying physical disks with other applications on your SAN.  This recommendation is in support of ensuring that you have reliable and predictable storage performance for Exchange and not have to worry about resource contention and bottlenecks that may be common in poorly designed or managed SAN environments.
    • Fibre-channel over Ethernet (FCoE): Microsoft has yet to release any design information pertaining to whether Exchange 2013 supports implementations of FCoE. While technically, FCoE should have no issues supporting the latency requirements and frame sizes of an Exchange 2013 environment, I would recommend that you proceed with caution when implementing any Microsoft technology over a non-supported environment. In the event of a support incident, Microsoft support has the right to declare that your environment is non-supportable due to inconsistencies with their Hardware Compatibility list (HCL).
  • Network Attached Storage (NAS): At this point in time, Exchange Server 2013 does not suppor the use of NAS-based storage devices for either Physical or Virtual implementations of Exchange.

Allocating Storage

Let’s start with the basic storage requirements of Exchange Server 2013. Microsoft indicates that Exchange 2013 requires that you allocate the following amount of storage space to accommodate the following Storage Types:

  • Exchange System Volumes: At least 200 MB of available disk space on the system drive
  • Exchange binaries: At least 30GB of free space on the drive you wish to install the Exchange binaries on. With an additional 500 MB of available disk space for each Unified Messaging (UM) language pack that you plan to install
  • Exchange Database Volumes: Amount of storage required would vary depending on a number of factors including the number of and size of mailboxes in your organization, the mailbox throughput (average number of emails sent/received), high availability features (database copies), as well as email retention policies (how long you need to store emails). These factors will determine an optimal number of mailboxes per database, number of databases and database copies and finally the amount of storage allocated for growth.
  • Exchange Database Log Volumes: You should provision sufficient storage to handle the transaction log generation volume of your organization. Factors that affect the rate of transaction log generation include message throughput (number of sends/receives), size of message payloads and high-availability features such as database copies. If you plan to move mailboxes on a regular basis, or need to accomodate large numbers of mailbox migrations (import/export to Exchange), this will result in a higher number of transaction logs generated. If you implement lagged database copies, then you need to provision additional storage on the transaction log volume for the number of days of lag you have configured. The following requirements exist for log file truncation when lagged copies are configured:
    • The log file must be below the checkpoint for the database.
    • The log file must be older than ReplayLagTime + TruncationLagTime.
    • The log file must have been truncated on the active copy.
  • Message Queue Database: A hard disk that stores the message queue database on with at least 500 MB of free space.

Volume Configuration

Microsoft recommends the following configuration settings for each volume that hosts Exchange-related data:

  • Partitioning: GPT is a newer technology over traditional MBR partitioning formats to accommodate much larger disk sizes (up to 256 TB), While MBR is supported, Microsoft Recommends that you use GPT partitions to deploy Exchange. Partitions should also be  aligned to 1MB.
  • File System: NTFS is the only supported file system type supported by Exchange 2013. Microsoft recommends an optimal allocation unit size of 64KB for both Exchange Database and Log File volumes. NTFS features such as NTFS Compression, Defragmentation and Encrypted File System (EFS) are not supported by Exchange 2013.
  • Windows BitLocker: BitLocker is a form of Drive Encryption supported in newer versions of Microsoft Windows. BitLocker is supported for all Exchange Database and Log File volumes. However, there is limited supportability for Windows BitLocker on Windows Failover Clusters. Link here.
  • SMB 3.0: SMB is a Network File Sharing Protocol over TCP/IP with the latest version available in Windows Server 2012. SMB 3.0 is only supported in limited deployment configurations of Exchange 2013, where fixed virtual hard disks (VHDs) are provisioned via sMB 3.0 only in the Windows Server 2012 Hyper-V or later version. Direct storage of Exchange data is not supported on SMB. Read this article.

Storage Best Practices

The following best practices offer useful guidelines on how to configure your Exchange 2013 Environment

  • Large Disk Sector Sizes: With ever-increasing disk capacities in excess of 3TB, hardware manufacturers have introduced a new physical media format known as Advanced Format that increases physical sector sizes. Recent versions of the Windows Operating System, including Windows Vista, Windows 7 and Windows Servers 2008, 2008 R2 and 2012 with certain patches applied (link) support a form of logical emulation known as 512e which presents a logical sector size of 512k, whereas the physical hardware can actually read or write to a larger sector size, known as atomicity.
  • Replication homogeneity: Microsoft does not support Exchange databases copies that are stored across different disk types. For example, if you store one copy of an Exchange database on a 512-byte sector disk, you should not deploy database copies to volumes on another Exchange server that are configured with a different sector size (commonly 4KB).
  • Storage Infrastructure Redundancy: If you choose externally attached storage such as a SAN solution, Microsoft recommends that you implement multi-pathing or other forms of path redundancy in order to ensure that the Exchange  Server’s access to the storage networks remains resilient to single points of failure in the hardware and interconnecting infrastructure (Ethernet for iSCSI and Fibrechannel for conventional SCSI-based SANs).
  • Drive Fault Tolerance: While RAID is not a requirement for Exchange 2013, Microsoft still recommends RAID deployments especially for Stand-alone Exchange servers. The following RAID configurations are recommended based on the type of storage:
    • OS/System or Pagefile Volume: RAID 1/10 is recommended, with dedicated LUNs being provisioned for the System and Page file volumes.
    • Exchange Database Volume: For standalone (non-replicating) Servers, Microsoft recommends deploying a RAID 5 with a maximum array size of 7 disks and surface scanning enabled. For larger array sizes, Microsoft recommends deploying RAID 6 (5+1) for added redundancy. For high-availability configurations with database replication, redundancy is provided by deploying more than one copy of any single Exchange Database, therefore Microsoft recommends less stringent hardware redundancy requirements. You should have at least 2 or more lagged copies of each database residing on separate servers and if you can deploy three or more database copies, you should have sufficient database redundancy to rely on JBOD (Just a Bunch of Disks) storage. The same recommendations for RAID 5/6 apply for high-availability configurations. In both cases of standalone and high-availability configurations that use slower disk speeds (5400 – 7200 rpm), Microsoft recommends deploying disks in a RAID 1/10 for better performance.
    • Exchange Mailbox Log Volumes: For all implementations, Microsoft supports all RAID types, but recommends RAID 1/10 as a best practice, with JBOD storage only being used if you have at least three or more replica copies of an Exchange Database. If deploying lagged database copies, you should implement JBOD storage only if you have two or more copies of a database.
    • RAID configuration parameters: Furthermore, Microsoft recommends that any RAID array be configured with a block size of 256KB or greater and with storage array cache settings configured for 75% write cache and 25% read cache. Physical disk write caching should be disabled when a UPS is not in use.
  • Database volume size and placement: Microsoft Exchange Server 2013 supports database sizes of up to 16TB, however for optimal database seeding, replication and restore operations, Microsoft recommends that you limit each database size to 200GB and provision the size of each volume to accommodate a 120% of the maximum database size.
  • Transaction Log volume size and placement: Microsoft recommends implementing Database and Log file isolation by deploying your database files and transaction log files in separate volumes. This is actually a good practice, since the performance requirements of databases and transaction logs differ.
  • Basic and Dynamic Disk types: The Windows Operating System supports a form of  disk initialization known as Dynamic Disks, which allows you to configure options such as software-based RAID and dynamic volume sizes. While Dynamic Disks are a supported storage type in Exchange 2013, Microsoft recommends that you deploy Exchange on the default Basic Disk storage.

Conclusion:

In this section, we explored the various storage components supported by Microsoft Exchange Server 2013 and reviewed some deployment best practices for implementing Exchange. There are a number of different types of storage that the various components of Exchange utilize and a well architected storage solution should seek to optimize performance of these various components.

Reference:

Link to Microsoft Technet article on Storage Configuration Options.

Article on MSDN site on Windows support for Advanced Format disk types

Link to Microsoft Technet article on Virtualization Support

PMP Exam Prep – Part 15: Project Procurement Management

The PMBOK addresses Project Procurement Management from the buyer’s perspective, in other words, we play the role of the buyer looking to contract some project work to a seller that is external to our Organization. Buyer and seller are common terms that are used in the exam as opposed to contractor and owner. This is not an exam that focuses on the legal aspects of contracting and procurement and questions typically relate back to processes of Project Procurement Management.

Exam questions frequently put the candidate either in the position of the buyer or the seller. Care should be taken to verify which perspective you need to take when answering a question during the exam. For example, you may be asked to identify whether the buyer or the seller bears the most risk when work is performed via a fixed price contract. The answer is that the seller bears the most risk. This is because in a fixed price contract, the seller agrees to do the stipulated work for a fixed sum of money. This means that if there should be any unanticipated increases in costs to the project, the buyer is not obliged to pay any more money to offset these costs and it falls entirely on the seller to  absorb these costs. The seller therefore bears the most risk.

Project Procurement concepts covered in this section

There are several procurement concepts that we will cover in this section

  • Project Procurement Definition
  • Statement of Work (SOW)
  • Contract Categories
  • Solicitation Process
  • Contract Negotiations
  • Contract Clauses

We will take a look at the types of procurement planning and the issues associated with procurement planning and solicitation planning as well. We will also delve further into solicitation when it comes to source selection; contract administration; contract closeout and the Organizational issues relating to procurement.

Plan Procurements

Procurement Planning is the very first step. A lot of Project Managers do not get involved at this stage. According to the PMBOK, however, the Project Manager is responsible for describing the subcontract requirements in terms of the specification, so the Project Manager is responsible for procurement planning. As the buyer, we need to define what we need the seller to perform for us.

  • Specification: A specification is defined as a precise description of a physical item, procedure or result for the purpose of purchase or the implementation or an item or service. It is important that the Project Manager specify exactly what it is that he or she wants. Additionally, we can also use drawings to complement or supplement the specification. For example, when procurement planning is performed in the construction industry, it is common to define out technical specifications as plans and drawings.
  • Delivery dates: The delivery dates of the product or service must be laid out in the specification as well.
  • Independent Estimate: The independent estimate refers to a cost estimate that the project team obtains from a vendor or external stakeholder that is not directly related to the project delivery.  Independent estimates are commonly used when a buyer is trying to obtain a rough idea of how much money a particular work package might cost, or when the buyer is wants to verify that the quoted prices received by potential sellers seem to be reasonable and accurate.
  • Assistance: It is important to note that the project manager has assistance from his team when coming up with the details that support the procurement exercise. For example, technical specifications are developed with some help from the engineers allocated to the project.

Make-or-buy decisions

Essentially we are trying to determine if we should perform the work ourselves or whether should we simply buy the finished product from another party. This is analysis is not always going to result in an either / or decision. It can go into many degrees. For example, we may choose to do some of the work ourselves and go outside of our project team for the rest.

In the PMBOK, the possible outcomes from a make-or-buy analysis include:

  • Procure virtually all goods or services from a single supplier
  • Procure a significant portion of the goods and services and make the rest
  • Procure a minor portion of the goods and services and make the rest
  • Make all of the good and services

Statement of Requirements / Statement of Work

The project team uses the Statement of Work to communicate their requirements in the event that a decision is made to procure a portion of the deliverables of a project. The Statement of Requirements (SOR) or the Statement of Work (SOW) is a document package that describes the specifications and other details required to tell external parties exactly what is needed to fulfill a procurement need.The SOW is an output from the procurement planning process and this is a fundamental document that you have to be very familiar with for the exams. PMI may sometimes use the acronyms SOW or SOR interchangeably in the exam.

Contract Types

In addition to the Make-or-buy decision, we also need to make a decision as to what type of contract we are going to use in order to procure that product or service. For the exam, you will need to be familiar with the various types of contracts that can be used to structure formal business relationships.

  • Fixed Price Contracts: The principle of shared risk applies to Fixed Price Contracts. In this case, the risk of the contract is shared between both the buyer and the seller.
    • Firm Fixed Price Contract (FFP): This is the most common form of Fixed Price contract and is also known as the ‘lump sum’ contract. In this particular type of contract, the seller bears all of the risk because she agrees to provide all of the goods and services for a fixed price and regardless of the costs incurred. There is one upside to a Firm Fixed Price Contract for the seller. The upside is that when the FFP contract type is used, the seller has the greatest chance to make a profit. There also is one other benefit that both buyer and seller will realize from this type of contract, which is that this type of contract has the least amount of administrative hassle.
    • Fixed Price plus Incentive Contract (FPIC): Here, the seller takes on more risk than the buyer. This is because regardless of the incentives put into the contract, the seller has to deliver the work at a fixed price. If that price is exceeded, the seller is going to incur those additional costs. For any range of costs or pricing mechanisms, the seller has the incentive to earn more fees, but generally as specified in a contract, if a particular level of cost is exceeded, then the seller will not realize any profit from performing the work and every dollar that the seller spends to complete the work will come directly out of his or her own pocket. The Point of Total Assumption occurs when the seller starts to take on all the costs of the contract because the seller is close enough to the ceiling of costs as specified in the contract. The seller is not necessarily losing any money at this point but is assuming 100% of the total costs from that point forward within the contract. The ceiling price as set within the contract can be thought of as being the level where buyer has specified that he will not pay any more money that this level for the work to be performed.
  • Cost Reimbursement Contracts: These are the most unfamiliar types of contracts in the exam, particularly for international candidates.
    • Cost plus Percentage of Cost Contract: This contract type has zero risk to the seller and stipulates that the seller is going to pass all of her costs directly on to the buyer. The profit that the seller is going to make out of this contract will be based on a percentage of the costs incurred. In other words, the higher the costs incurred, the greater the profit for the seller. The obvious motivation for the seller would be to incur as much costs as he possibly can, since his profits will be increased as a factor of those costs. This contract type is illegal in the United States Federal Government. The buyer has all of the risks and the seller has no risks for this type of contract.
    • Cost plus Fixed Fee Contract (CPFF): At the outset of a contract, the costs are going to be estimated by the buyer and the seller for the work prior to the work being performed. Based on these cost estimates, a fee is going to be fixed for the seller. For example, if a job is going to cost $2 million dollars to get the work done, the buyer and seller can agree that 8% of the fee or $160,000 is going to be a reasonable fee. In the event that the job was performed under the agreed to cost of 2 million dollars, the seller is still going to get her fee because the fee is fixed. In fact, if the seller incurs a cost above 2 million dollars, he or she will still get the originally agreed upon fixed fee. The cost may vary, however the fee is going to be fixed. However, there might be a clause in the contract that the fee may have to be re-negotiated if the costs go beyond a pre-determined threshold amount.
    • Cost Plus Incentive Fee Contract (CPIF): A CPIF contract is a risk-sharing contract between the buyer and the seller. All CPIF contracts share one common quality known as the sharing ratio. The sharing ratio can be expressed as 70/30 or 60/40 or 50/50, but is always expressed as buyer share/ seller share. You may see questions in the exam that indicate that you have a CPIF contract and you have a 70/30 sharing ratio and then proceed to ask you who has the 30 percent share. The answer in this case would be the seller, because 30 is the second part of the ratio.
  • Unit Price Contracts: A unit price contract is really a derivative of a fixed price contract. The unit price follows fixed pricing but just on a unit-by-unit basis and not the sum total of a number of units. In this case, we are looking towards a fixed price for a single unit of an item or service. The unit price will be fixed, but it is fixed for each particular unit of goods or service that we want to acquire. For example, we are setting up a new office and we need to buy some desktop computer systems. We can go to suppliers and tell them that we need 1,000 computers. We can also give them the specifications in terms of the specific hardware requirements for each computer. We can tell suppliers that we want a fixed price for a particular quantity of a product or service and we can collect bids and we will generally select the supplier who has the lowest price and conforms to the specifications. This is an example of fixed price. Let us say that we have acquired the 1,000 computers and we realize that we need a few more computers but this time we’re not exactly sure how many computers to obtain. We might just want to buy them two or three at a time. In this case, we would like to get a unit price for each computer. This is an example of unit price.

Comparing Cost Plus and Fixed Price Contracts

In cost plus contracts, all of the allowable fixed costs of the seller will be covered in the contract. When it comes to a fixed price contract, all of the allowable costs may not be covered because the seller might exceed what was allowed under the ceiling price of the FP contract. A tip for candidates to identify incentive contracts such as fixed price incentive or cost plus incentive based contracts is to look for terms such as ‘target costs’; ‘target price’; or ‘sharing ratio’ which will denote that the question is referring to an incentive based contract.

Additional Incentives

There are additional incentives that have not been discussed as yet when it comes to contracts. These incentives need not always be financial. These incentives can be thought of as benefits that the buyer will give to the seller or Organization performing the work if they can complete the work ahead of schedule or below costs.

Solicitation Planning

Solicitation planning comes after procurement planning. At this point, we want to take our SOW and other procurement documents and go to the outside world and start obtaining bids and proposals for our work.

We need to solicit bids and proposals from a number of suppliers and we are interested in putting a procurement package together to help potential sellers or vendors. We have a particular process and we need to structure this process in a standardized way that will be easy to follow and well known for potential sellers.

Contract Origination

There are various accepted ways for an organization to go about looking for potential vendors to fulfill their requirements.

  • Unilateral Contract: A purchase order is a good example of a unilateral contract. One party signs the contract. Typically there is no negotiation required and the contract is of low monetary value. We are looking at standardized solicitation and we are buying commodity items. A good way to look at what might be a unilateral contract is to ask yourself if you can send something via a fax. If you have the ability to send a contract via fax and not expect any reply and just expect work to be done after it is sent, then you can look at this document as a unilateral contract.
  • Bilateral Contract: Bilateral contracts are more conventional and have a tendency of being a lot more involved. There are 3 basic forms of bilateral contracts.
    • Invitation for Bid (IFB): An invitation for bid is a form of contracting which is appropriate for routine items. The primary objective as a buyer is to find the best price. The buyer is able to clearly describe what it is that he or she wants and is able to identify the completed product or service when a potential seller presents it to the buyer. There is typically no negotiation involved in this process and the buyer is usually looking for the lowest price. When you see an IFB, the buyer is not looking for any extra qualities aside from what is described in the initial specification.
    • Request for Quote (RFQ): Requests for Quotes are used for generally low monetary purchases of commodity items. Essentially a certain number of items and exact specifications are known to the buyer and the RFQ is going to be sent out to a few pre-selected and pre-qualified suppliers and the objective is to find the best price.
    • Request for Proposal (RFP): The Request for Proposal is generally used for complex or non-standard items. The monetary value of procurement in this case is going to be higher than in an RFQ or and IFB. What distinguishes and RFP from an RFQ or IFB is that there is going to be some discussion between the buyer and the seller. The buyer will attempt to describe what he wants in the clearest terms that he or she possibly can. The buyer is very interested in not only receiving proposals from vendors but also interested in talking to sellers about his or her needs. The seller might have a better idea of how the work can be performed.

For example, if you are involved in a complex software development project for your Organization, you would be issuing an RFP and not an RFQ or IFB. In this particular case, the buyer wants to meet with different sellers and determine from them the best approach to take in order to write the software, implement it and train the people on how to use it.

Proposal evaluation techniques

At the end of the solicitation process, the buyer would have received responses back from interested sellers and we are now interested in putting these responses through some kind of evaluation criteria. The evaluation criteria can be objective or subjective. There may be metrics to measure the different proposals, or sometimes the buyer will simply want to get a feel for how the contractor will meet the requirements of the contract.

Common evaluation criteria are listed below:

  • Management approach
  • Financial health
  • Contractors understanding of buyers requirements
  • Overall/lifecycle costs of proposed solution
  • Type of contract which should be used

Solicitation

Once a number of responses from respective vendors have been received, it is now time to identify which parties to send the solicitation to. Sometimes, it is hard to pick, it is hard to find the right vendors. It is hard to find out who or where you go looking for vendors. You can go looking for a variety of sources. The project manager is heavily involved in this process. The project manager is not working alone in this task, but is instead working with contracting staff. PMI is looking at this from a centralized contracting perspective.

When it comes to solicitation and source qualification, we’re looking in catalogs; we’re looking in contacts, talking to other suppliers, looking at trade journals. We’re looking at anybody that might be able to meet this particular need. The solicitation is issued with the help of the contacting staff.

  • Source Selection: Once we go through the solicitation, we then go through the source selection where we pick those vendors who responded to our original solicitation itself. There are a variety of issues associated with evaluating these contractors. As we evaluate perspective contractors, the PMBOK talks about a number of different ways to do this.
  • Evaluation criteria: We are going to apply our evaluation criteria to the proposals. We are going to use a weighting system.  A weighting system is a method for quantifying qualitative data to minimize the effect of personal prejudice. For example, imagine a group of three to five people sitting around a table and looking at various proposals from a number of vendors. We are attempting to apply our evaluation criteria to the vendors such that we come up with a  number using a scoring system of some type. We are trying to assign numbers to our various evaluation criteria and we are going to weight our criteria. This means that certain criteria would have more importance to us than other criteria.
  • Screening System: According to PMI, we establish minimum performance requirements for one or more of our evaluation criteria. A common example would be that one of the criteria we have in a project is to require a project manager who has significant project management credentials and inexperience in the field. Using the screening system in our criteria, we can say that the project manager that is offered by the vendor must have a PMP or a master’s degree in a relevant profession. If that project manager does not have a PMP or the relevant master’s degree, then we can rate that particular contractor low or kick them out of the whole process.
  • Team aspect of solicitation: All of the above screening criteria work under the premise that everyone is going to be focused on working together to come up with one solution. Sometimes we don’t have the time to sit around and sort through all of the vendors. Sometimes we delegate the task to one person who can do the job and do it well.

Sole Sourcing

Is there ever a time when we skip to the chase and cut out the entire process and simply hire somebody? There are cases where we know that competition is good and competition keeps prices low but we just know at the outset that we have one supplier in mind and we know that because they are so uniquely qualified that we can’t find anyone else to do the job, so why waste all of the time to look for vendors. There are also times where we can go to one contractor which is known as sole sourcing. Or sole source where we might have the in-house expertise to evaluate the contractors for reasonableness and accuracy.  In other words we have significant expertise and we know if the contractor is going to be good for this job and we really don’t need to have multiple vendors come in and give us prices.

When our project is under extreme pressure for time and the procurement process and the planning involved as well as the other steps requires time that we are is short supply of. There are occasions where we simply don’t have time and there are situations where we know of a contractor and we’ve used him before and he has a great track record and for the sake of the project and the sake of the client we are going to go to that contractor and negotiate a price and move forward. So we are going to sole source you need to know that term for the exam. Sole source means going to a vendor without considering other vendors.

Procurement Negotiations

When we get into a lot of these environments to meet with contractors, it very often involves contract negotiation. We need to be comfortable with the contract negotiation process. One of the issues here is that PMI has changed terminology over time and we need to know a variety of terms for the different issues associated with contract negotiation.

There is a series of steps and you can never afford to jump over a step regardless of how the questions in the exam might try to convince you of otherwise.

  1. Protocol or Rapport Setting: Introductions are made and the atmosphere is set and we are trying to get ourselves organized for discussing the pros and cons of working with one vendor and taking a look at their prices and trying to get an understanding from them whether they truly understand the work that we would like to have them do.
  2. Probing: Probing is where we go in and we are trying to ferret out what the contractor is after and what are we after and what we are able to share with them and that they are actually interested in and what they are going to be able to share with us.
  3. Hard-core Bargaining or Scratch Bargaining: This is where we’re starting to make our concessions and both parties are trying to give and we are trying to come up with a good mutual agreement for a situation that both the buyer and the seller are basically unhappy with in some ways and we are both satisfied that we are basically starting to move forward and that each party has gotten what it thinks it needs to move forward. The buyer has gotten the right price and the seller says yes, I am going to benefit by this particular relationship.
  4. Getting to Yes: PMI is pretty big on the concept of getting to yes they are looking for win-win situations when we get into a contract. And if you have a question in negotiations and it implies that your opponent is trying to crush you like a grape and you decide that you are here to try to get to yes. Trying to achieve a win-win situation, this is what PMI wants you to be thinking
  5. Closure: Closure occurs where we are summing up our positions and most times when you go through a negotiation you’ll often get to closure and find that you didn’t understand what they thought they said that they didn’t understand what they thought you said and you’re back into hard-core bargaining. Again,
  6. Agreement: The important thing about this last stage is that it is documented and the parties sign a document and by their signing indicate that yes, they have complete understanding of this particular relationship

Negotiation Tactics

As you go through negotiation, there are many strategies that can be applied. We will look at a situation where a real-estate broker is trying to sell you a house.

  • Deadline strategy: The broker walks up and says that if you sign this deal by 5 o’clock tonight, I am willing to sell the house for this price. If you come back after this time, then the offer price is no longer valid.
  • Surprise: The broker might mention late in the contract that the house comes with gold-plated fixings in all bathrooms in order to lure you into closing the deal.
  • Limited authority or missing man strategy: This is the most ubiquitous tactic. The broker will tell you that he has to go talk to the owners or some higher power in the universe before he can get back to you.
  • Fair and reasonable: Sometimes you will meet brokers who are honest and trustworthy and when you come together they tell you that they are offering you a fair and reasonable price and that they are trying to come up with an agreement.
  • Reason together: Both parties, in this case, the buyer and the representative for the seller or the sellers themselves would sit down and try to come up with a reasonable price that will result in both parties with a win-win strategy
  • Fait accompli: This refers to a done deal. In this instance, the broker acts surprised, indicating that he was under the impression that the deal was agreed upon is surprised that it wasn’t so.

The End of Negotiations

This refers to the moment in negotiation where we have signed the contract, both parties are in obvious agreement and there are a few things that we want to keep in mind at this stage. Our objectives in reaching the end of contract negotiations are to obtain a fair and reasonable price while still trying to get the contract performed within certain time and performance limitations. We also want to ensure that there is a good relationship between the buyer and seller after the contract is sealed and signed. This makes sense because the buyer and sellers may have a relationship over a long period of time and we want to ensure that the working relationship between both parties is good.

Administer Procurements

This is where the work actually gets done. This is where the Project Manager along with the contracting staff watches performance of the vendor to keep the project moving forward. Our focus shifts from finding and selecting a seller to making sure that the seller is performing the work in accordance with the contract specifications.

Terms of the contract

There are a variety of specifications that would define how the work gets performed in a contract. The most commonly used features in contracts are listed below and these are the things that the project manager and the project team should look out for as they are administering the project.

  • Delivery schedule
  • Handling of changes
  • Warranties
  • Inspections
  • Subcontracts

Clauses

You should be familiar with common clauses found in contracts.

  • Standard Clauses: First and foremost we look at our standard clauses to see how much of the work is covered. If there are gaps, then we should proceed to develop some new clauses ourselves. We take a look at the standard clauses first, because typically, this project will not be the first work that you have done before and therefore you have a lot of information written down to your standard clauses so far. Many organizations also prefer to standardize the standard terms and clauses of their contracts.
  • Change clauses: This is one of the clauses that you can expect to see tested in the exam. This gives us some sense of who initiated the change, where the changes come from, how the changes are  going to be funded, what some of the approval authorities are going to be from. Some of the configuration management issues are going to be addressed in the change control clauses.
  • Pricing Change: This also refers to how we are going to deal with change especially when it comes to the pricing of the contractor.  PMI prefers that we use lump sum or Firm Fixed Price changes. PMI suggests that we use lump sum prices for changes even if we have a cost plus basic contract. This might not have any real practical applications, however, we need to understand the exam from PMI’s perspective.
  • Express warranties: An express warranty is explicitly written out and we have an understanding of exactly what the functions or features a product or service should have.
  • Implied warranty: There are a couple of terms used such as merchantability and fitness of use. For example if we look at a desk that we are sitting at, we will realize that the desk came with an implied warranty. The implied warranty is that the table should be sturdy enough to serve it’s purpose as a desk and that we are going to use this desk is going to hold objects that desks usually hold. For example, if we buy this desk and we put our stationery and notepads on it and the desk collapses, we bought it under the assumption that the desk was going to be strong enough to hold notebooks, which is a reasonable assumption. We don’t need to have an explicit description of that warranty. This desk has collapsed and it obviously isn’t fit for use and we can go and get a replacement for the desk.
  • Doctrine of Waiver: This refers to the fact that if we fail to exercise our contract rights, we lose them. This is a legal doctrine that has a lot of practical applications especially when it comes to the issues of change control. If we do not enforce the change control process, we might not have the change to exercise the change control process.
  • Delays: We need to look at who caused the delays as well as the nature of the interruption and the impact of the delays.
  • Performance Bond: The performance bond secures the performance and fulfillment of the contract for the buyer. In other words, we want to make sure that if we hire a contractor, they are able to do the job and if they cannot do the job, then the bonding company is going to step in and do the necessary work to complete the job.
  • Payment bond: There is guaranteed payment to sub-contractors and laborers by the prime contractor. In many situations, the buyer may pay the prime contractor for the work and the prime contractor may be paying sub-contractors for a former job that the subcontractor had performed for the contractor prior to this contract. We want to ensure that the prime contractor will use our payments to pay the sub-contractor because we want to make sure the sub-contractors and laborers are being paid to do the work and will not walk off due to some issues with the prime contractor.
  • Basic Breach: This is an important issue when it comes to contract administration. Breach of contract says that if it is just a basic breach where we have somehow violated some part of the contract, but this doesn’t mean that the entire contract is invalid.
  • Material Breach: This type of breach is a lot more serious. It means that the breach is so bad that the contract expires at that point in time. Usually, a material breach occurs in a situation where the work to be performed is time sensitive. The buyer needs a particular product or service by a specific date. If he gets the product or service after that date, then he has no more need for that product or service.Therefore, if there is no delivery by the date specified in the contract, the contract is breached.

Summary

  1. Procurement Planning
  2. Types of Procurement
  3. Contract Types
  4. Vendor Selection process
  5. Contract Negotiation process
  6. Contract Clauses

Conclusion

In this section on Project Procurement Management, we identified various types of contracts and common clauses of contracts. We also identified the common sequence of procurement activities, to go from procurement planning to solicitation and vendor selection and finally procurement administration.

PMP Exam Prep – Part 13: Project Communications Management

Historically, most candidates find this section on Project Communications Management to be the easiest to pass. This is the time and place where you can gain a lot of ground in your exam score, particularly if you don’t have a lot of knowledge and experience in other sections on Cost and Risk management.

Project Communications concepts covered in this section

There are several Communications concepts that we will cover in this section

  • Informal vs. Formal Communications.
  • Conflict Resolution
  • How differing and different management styles represent the form of communications.
  • Communications Model
  • Communications Channels
  • Kickoff meeting
  • Barriers to Communication
  • What Role the PM should play in Communications Management

The following Project Management processes are covered in this knowledge area:

  • Identify Stakeholders: We need to be able to assemble our project team with the right mix of skill sets.
  • Plan Communications: We need a systematic and repeatable process that helps stakeholders gain access to key information, both in the right place and at the right time.
  • Distribute Information: This process covers the systems and tools for disseminating information to the appropriate stakeholders, as well as covering issues of confidentiality and security.
  • Manage Stakeholder Expectations: A project manager should always be attuned to the varying expectations among different stakeholder groups and work to ensure that these expectations are being met in order to minimize conflict.
  • Report Performance: We refer to the formal and informal processes of communicating information about project performance to stakeholders.
Exam Hint – Look out for questions in the exam pertaining to teamwork. You should look for the ‘rah-rah’ type answers, in other words the answers that seem most like cheerleading. Also, when it comes to Roles and Responsibilities as relating to Communications in this exam, the Project Manager is one of the most common examples.

Communications Model

The Communications Model is a Basic Model for Communications in which you have a communicator, who is actually conveying something and the recipient, who receives the message.

  • Communicator: The Communicator conveys the message. As I deliver this lecture to you, my message is coming to you off the particular medium of live presentation.
  • Recipient: The Recipient is the person for whom the message is intended. The recipient must accept and understand the message before communication has taken place.

Active Listening

A good presenter should be able to offer a message that his audience will accept and understand occasionally. The reason why we use the term occasionally is because of the concept of Active Listening. You can’t always be Actively Listening. Active Listening is when you are participating even by just sitting there and paying attention in the communications process.

You might have experienced this phenomenon before in your car. You are driving along and you are listening intently for the traffic report on the radio. You are focusing on trying to find out about the traffic conditions on the way to work. The report is coming, you hear the music coming on for your traffic report but all of a sudden, someone almost cuts you off and you have to swerve to avoid him on the road. You suddenly realize that you have missed the traffic report which you have been trying to catch for the past fifteen minutes. Even though you were attempting to actively listen, you got distracted.

On the exam, you have to understand that the concept of Active Listening is that the recipient has to  be an active participant in the communications process.

Tools for Active Listening – Feedback

The person who is getting the message is going to get back to the communicator. We also use para-phrasing as a good form of feedback. As you are listening, you want to go back to the speaker and say, I really want to make sure that I know what you are saying. Here’s what I think you said and I want to know if that was the message that you meant to communicate. There’s the feedback to the communicator that you’ve taken in the information, you’ve digested the information and you’re now trying to make sure that you have in fact gotten the message as intended.

Communications Channels

The topic of Communications Channels is a big deal these days. You have to be able to calculate Communications Channels in the exam based on the following formula:

Number of Channels = n (n-1)/2

Where n is the number of participants in the communications model. For example, if 2 people are talking and we substitute n for 2 in the formula, we can calculate that the number of channels is 1. If we have 4 people, then we are going to have 6 channels. If we have 6 people we are going to have 16 channels. It is very interesting to note that as each additional person is added into our communications loop, the number of channels does not rise in a linear manner; it rises in a geometric manner.

Significance of Communications Channels

With an understanding of communications channels, we start to gain an appreciation of how an organization or project needs to structure itself to maximize communications.

  • We need to think about the division of labor within our project Organization
  • We need to think about our types of communications. If we have 1 or 2 or 3 people in our team, we can resort to very informal communications. When our team gets up to 15 or 20 people, informal communications is less effective in getting the same message across to every member of the team. We need to start thinking about using formal communications as our predominant means of communicating to larger groups.

Hint: If you get brain freeze on the exam you have that moment where you forget the formula, the thing to do is to take the scrap paper that you are given and start drawing little circles, one for each person, and then connecting the circles with lines. You draw one circle for every participant in your project and start connecting all the circles with lines. You simply have to count the number of lines and this will give you the result that you are looking for.

Another notion is regarding the way PMI is presenting some of the communications questions is not to ask how many communications channels there are, but to ask how many more channels will be created if we add another 3 people into the group. You have a team of 4 members, and the team is about to be increased to 10 members, how many more channels will you have to deal with? You will have to perform calculations twice to find out the answer.

Different Types of Communications

Communication does not always have to occur verbally, it can occur in different ways. Numerous studies have shown that most of a message is relayed to the recipient though vocal intonations and facial expressions than the actual content of the message, so the verbal component is actually a very small piece of the message. Communication can be formal, or informal. Communication can also be written or verbal and you need to be able to break all of these down as well.

Communications can occur in a variety of formats, some of which are listed below:

  • Formal written: project charter or management plan
  • Informal written: engineers notes and memos
  • Formal verbal: presentations with PowerPoint
  • Informal verbal:  conversations, run-ins in the hallways

Remember that if it is formal, it is part of the project record. Oral presentation would still be part of the formal project presentation, if people think about it, they are still going to reflect back and acknowledge that it is part of the formal presentation itself, whereas a hallway conversation is generally forgotten over time

Why do we need to know this? You will be asked to give examples of these in the exam and you will most likely encounter examples of the four types of communications.

Communication Requirements

The project plan is really a series of plans, there is the risk plan, the schedule plans the quality plan and there is the communications plan. In order to develop the communications plan we need to understand what the requirements for communications are.

This task is the Project Managers responsibility. Communications requirements are the identified needs of the project stakeholders. We need to define the requirements of all of our stakeholders for a particular project and then put a process into place in order to provide these stakeholders with relevant information. This is typically done with a project management information system (PMIS).

The things we need to look at in assembling our project communications requirements are our project Organization and the stakeholder responsibility relationships, our disciplines, our departments and any specialties involved in the project, by discipline, we are referring to a mechanical engineer, a software programmer, we have a marketing expert. Those will be the kind of disciplines that we refer to.

We need to look at the logistics of the number of individuals who will be involved with the project at each location because we are going to collect information and we are going to distribute information to each stakeholder. We also need to identify any external information needs. For example we may have to be involved with the media in very large projects especially where the Government is involved and when the media is very interested. So it is important for a Project Manager to see outside of the Organization and and identify external stakeholders and manage their requirements accordingly.

Timing of Communications

We want to create the communications plan early on in a project. We want this to occur in the planning phase and not wait until it comes to the implementation. Communications planning is really part of stakeholder management.

The Kickoff Meeting

There are multiple objectives for the kickoff meeting, some of which are listed below:

  • The kick off meeting is going to save us time and not cost us time.
  • The kickoff meeting really builds the team’s identity.
  • The kickoff meeting also allows team members to get to know each other.

Furthermore, the project team needs to have an understanding of the various working relationships and lines of communication. We can also reach a common agreement on the goals of our project. We can identify some problem areas and we are hoping to define the objectives of our project to our stakeholders. As we can see, there are multiple things going on in the kickoff meeting.  Essentially underlying all of these activities is the notion that we are going to build the team for the project as we move forward.

One of the things that the kickoff meeting is not intended to do is discuss any hard technical issues you may encounter into the project. It is not a status meeting of the project, you are not trying to explore alternatives to performing tasks in the project its really a get to know one another and establish a common framework of moving forward with the project

For example, I was involved in a pretty large project for the construction of a hangar and we had about 55 people in the room. I had just finished delivering the project objectives and administrative processes to the group and I asked if there were any questions.  Our construction contractor raised his hand and he started to give me a verbal proposal of the change order on the very first day of the project. I had to tell the contractor that the kickoff meeting was not a place to discuss such things and that we would address his issues offline. During the kickoff meeting, we don’t talk about the content of the work, but how together we’re going to get the work done.

When it comes to the exam, you want to look at the answer which says ‘rah rah’ when it comes to the kickoff meeting. PMI will sometimes try to lure you from this question on the exam by saying that the project is a great deal of time pressure from the customer, the project is time constrained, everybody is feeling the weight and the everybody is so stressed out and has no time available. PMI may ask you to consider foregoing the kickoff meeting since you are in a very simple project with a small team. In all these cases, we cannot do without the Kickoff Meeting.

Barriers to Communications

There are a number of barriers to communications and these things stand in the way of active communications.

  • Ineffective Communications Plan: A poorly written or ineffective communications plan with poorly defined communications requirements can result in a lot of wasted time getting information to people who were not initially informed.
  • Time: Time can be an impediment to open communications. We need to have open communications channels in order to be effective in serving our projects. This occurs especially in environments where team members are in different geographical locations.
  • Technical Jargon: It is common for projects to involve team members from different disciplines. Each discipline has its own jargon or terminology. The people who are not familiar with your profession or industry will not have effective communications.  A lack of a consistent set of commonly used terms in a particular industry can also have a negative impact on communications. For example, the term WBS as used in project management is not consistently used in all Organizations. In fact, there are Organizations that choose to use other terms such as Project Breakdown Structure (PBS). We can really get carried away with our own jargon and start confusing those around us. There is a barrier that the Project Manager always has to look at when dealing with the client
  • Noise: Most people think of noise as background noise, or an audio sound such as the clicking of a pen or the hum of the air-conditioning in the background, when in fact noise is a lot more than that. Noise refers to any external environmental factor that is interfering in your ability to communicate. You may have been in a meeting where you were sitting right under the air vent and it was so cold that you wished that you had brought a sweater into the meeting. For example, there was a huge construction project across the street during a meeting and the construction workers were constantly swinging these huge barrels back and forth and the students stood with their eyes focused outside of the window, watching the construction crews swinging the barrels back and forth. For the exam, you need to know what noise is and be sure to be able to distinguish noise from a detrimental attitude or a barrier to communication.

Communications Barriers and Conflict

Communications barriers increase the level of conflict. If you can’t understand the language or terms that other people are using in a room, you will not be able to communicate with them effectively and that will raise your level of stress. Any person who has worked overseas or been in an overseas environment for any length of time will understand the stress of not being able to speak the language. This occurs regardless of whether we are referring to a technical language or a physical language.

Unresolved conflict is a real detriment, and well structured communications will help to eliminate conflict. Conflict may not be entirely eliminated by good communications, since the nature of projects themselves contribute to conflict.

Stakeholder communications

The Project Manager’s role is the key to all project communications. The Project manager must be skilled with communications with the following parties involved in the project

  • Management (Project and Functional): The Project Manager acts as liaison between management and project team. An effective project manager should act as the go-between for management and the project team and prevent management from bothering the project team and getting in their way. The Project Manager has to be able to communicate effectively to management so that they feel that everything is going well and won’t have to bother the team. PMI has found that when people are uncertain about the status of a particular project, that is when they will bypass the project manager and go straight to the team. If you want your top management to stay out of your hair and not bother your team, then you have to think of the best way of communicating with top management and top management as well. So we have to be able to communicate with management and the customer as well
  • Client: It is very important that the Project Manager act as a representative to the customer. It is not always desirable for  project team members to discuss all aspects of the project with the client due to confidentiality concerns. PMI is looking for a single point of contact between the project team and client. It makes it easier for the customer to give feedback because they know who to go to in order to complement or give complaints. It allows the team to designate and train one person who is skilled in client relations; this becomes very important as the project team grows in size. It also eliminates the confusion when many team members talk about to the customer at the same time. PMI’s perspective is that the Project Manager is responsible for establishing good communications between the client and the team. In the exam, you might see questions where the Project Manager is asked to communicate horizontally, vertically and diagonally.

Time invested in Communications

PMI would like the Project Manager to invest a total of 90% of his time invested in acquiring and communicating information to stakeholders. For the exams, if presented with a range of percentages for how much time the project manager should devote to communications, we should look for the answer with the highest percentage, with the exception of 100%, because this value is not realistic. The project manager needs time to breathe, so to speak.

Skills for effective Communications

A project manager should possess the following skills in order to be an efficient facilitator for communications in the project environment.

  • Set up Networks: The Project Manager needs to be ready to share information with his team and be willing to build the networks that go there. PMI Actually makes references to informal networks as well. Informal networks are commonly known as the grapevine. This is a term that occasionally shows up in the exam as a possible form of informal communications. A grapevine is an idiomatic term, which we use to convey the idea of an informal network.
  • Communications Expeditors: This is someone who makes things happen and is very active in bringing people together for effective communications. This involves initiating relationships between stakeholders in a project and establishing communications links and also making people understand what the formal communications channels are and encouraging good informal communications as well.

Communications Blockers

Blockers impede our communications and refer to anything that kills or inhibits innovative ideas. For example, someone brings up an idea in a meeting and someone else tries to assert that the idea will never work. Common statements uttered throughout meetings take on the tone of “That will never work.”; or “We’ve tried that already.

There are some examples of great communications blockers that have occurred throughout history:

  • ‘I think there is a world market for maybe 5 computers.’ – Tom Watson, Chairman IBM 1943
  • Who the hell wants to hear actors talk’Harry Warner, founder Warner Bros Studios (1927) when asked if silent movie audiences would like to hear actors talk.

As the communications expeditor, the project manager takes on the responsibility of trying to eliminate the occurrence of such communications blockers and encourage more open communications.

Tight Matrix

The tight matrix refers to putting all members of the project team into the same office space if at all possible. Studies have shown that when people are physically working together on a project team, better results will occur.

The tight matrix occurs as one way of ensuring more open communications within the team. A tight matrix is not related to the weak or strong matrices that occur in the Project Human Resources section.  A lot of companies have attempted to place all of the members of a team into the same office, or at least the same office space. This practice is popular in the automotive industry where the designers and fabricators are located right next to one another.

In circumstances where it is not possible to place team members in the same location, such as having project team members collaborating from different countries across the globe, then a virtual environment has to be created where team members still have the ability to interact with one another. Weak or Strong matrices might appear as distracter answers in the exam. Loose Matrix is another term that might come up as a distracter answer. The loose matrix does not exist.

Project War Room

One of the best methods of getting project team members working in close proximity with one another is the use of the Project War Room. This term originates from a war-like setting where we have a central location for the military to collect and analyze information on the tactical situation of battle. You can imagine a room filled with maps and charts where senior military officers huddle around a table to come up with winning strategies. The most important thing about the project war room is that it provides the project team with a sense of team identity. A lot of Organizations are starting to realize that the cubicle office spaces that most office workers work in are not conducive to establishing a strong project team identity.

The war room is a place that the project team members can hang up all of their gantt charts and responsibility matrices and could be a conference room or even a temporary office space designated for the members of the project team to interact in. The virtual project war room can be a web page on a company portal that serves as a common repository of information and provides a venue for people to interact specifically regarding the project.

Effective Meetings

The Project Manager is is responsible for organizing and coordinating a variety of meetings. There are staff meetings; status review meetings; schedule meetings and budget meetings. The project manager has to be able to conduct these meetings in an effective fashion such that people will want to attend the meetings.

The Project Manager can establish a meeting policy. For example, only certain people may be allowed to call a meeting; or meeting notes need to be circulated the day before the meeting; or that an agenda must be prepared and followed. Participation should be encouraged throughout the meeting. Finally, all meetings, regardless of purpose, should always be thought of as having some element of team building, as meetings are an excellent opportunity to build the team.

Minutes should also be prepared and circulated. There should be a formal record of the meeting prepared as a means to follow up for what was initially discussed during the meeting. This is an effective tool to ensure and track that the meeting actually resulted in some forward progress for the project.

Time wasted in Meetings

PMI performed a study to investigate the amount of time that is actually productive during a meeting. The results of their survey indicated that up to 25% of the time spent during a meeting was devoted to non-productive or irrelevant issues.

This can be attributed to the following reasons

  • Poor planning
  • Bad leadership during the meeting
  • Unruly or undisciplined participants

Management Styles

These management styles are situational and are neither better nor worse than one another. PMI wants the project manager to realize that there would be certain situations where is would actually be desirable for the project manager to adopt a particular style.

  • Authoritarian management: You have project team members, stakeholders and functional managers associated with a project and they typically want to know why they are here and what you want them to do. There is nothing wrong with being direct with these people to let them know what their level of participation is on the project and what you would like them to do.
  • Combative management: This can be very useful under certain circumstances where you want to generate conflict. Not all conflict is bad and you can be with a group of people and you think that everything is going okay but what you really need is someone to step in and give some feedback.
  • Conciliatory:  You are basically ready to give in based on what is being said across the table. Anything that can be done to keep the brevity going in a certain situation.
  • Disruptive: This style tends to disrupt unity and cause disorder. Sometimes this is going to be very important in a project situation where we need a very different line of thinking on how to conduct our project. Disruptions all and all can be very important which communicates a style or form which a project manager can use.
  • Ethical: Applying fair even-handed judgment as you work through a project
  • Facilitating: The Project Manager can play an important role in supporting the team with resources and helping to take away barriers or obstacles.
  • Intimidating: The tough guy image where you say that it is your way or the highway. In particular situations, intimidating style can be very useful.
  • Judicial: Generally applying sound judgment
  • Promotional: Someone who is out there trying to cheer on the team, trying to motivate the team with the proper types of actions
  • Secretive: Not open or outgoing in speech; activity or purpose. There are times where things are going on in an Organization where the Project Manager knows that he or she may not want to share with the project team because they cause disharmony or loose morale in some way. So the project manager may not at that particular moment in time care to reveal some of that information and that may be good.
  • Management skills:  Recall the five components of general management that are required by a good Project Manager [PLINC] and covered in the section under Project Integration Management.
    • Problem Solving
    • Leading
    • Influencing
    • Negotiating
    • Communicating

The project manager wants to be acutely aware of these concepts when it comes to using his management skills.

Organization Structure

We looked at this previously in the Project Human Resources section. There are some specifics that we have to look in to when addressing the communications aspects of project management in an Organization. Each Organization structure is going to have some impact or ramification on communications.

  • Projectized Organizations:  Very strong group communications boundaries because everyone is working for the project manager in one established team.
  • Strong matrix: Good, strong team identity. Perhaps the team is collocated, making the team’s communication generally straightforward.
  • Weak matrix and functional organizations: complicated group communications because as team members are spread around the Organization, they generally do not have the team identity that we find in the strong matrix.

There is also less face-to-face interactions causing a higher opportunities for misunderstandings. Remember that in both functional and weak matrix Organizations, the orientation is to the functional Organization and not to the project Organizations.

Summary: Project Communications Management

  1. Communications model
  2. Formal and informal as well as written and verbal communications
  3. Barriers to communications
  4. Kickoff meeting
  5. Documentation

 Conclusion

In this section, we covered topics such as the communications model, the role of a project manager, effective communications; barriers to communications as well as the importance of the Kickoff Meeting.

In the next section, we will cover Project Risk Management.

Ook!