Today I’m excited to make two announcements:

First, my new book, Ahead in the Cloud, became available last week.

Over the last three and a half years as the Global Head of Enterprise Strategy for AWS I’ve had the opportunity to talk to hundreds of the world’s largest companies about the role cloud is and will play in their long-term IT strategies. I’ve learned a lot about what makes cloud hard for large companies (hint: it’s rarely about the technology), and have compiled these learnings across a number of blog posts.

Over the last few months, I’ve combined these posts with some new content from customers, myself, and other AWS leaders (including our CEO and founder, Andy Jassy) into a book called “Ahead in the Cloud: Navigating the Future of Enterprise IT”. You can find it on amazon.com here. (Kindle version should be available on/around April 9th).

I hope that any leader contemplating change management in their organization will find this book useful, and I look forward for your feedback!

Second, I’m moving on to build something new at AWS, and Philip Potloff will take the reins of Enterprise Strategy at AWS.

There are many reasons why I love working at Amazon, but near the top has got to be our appetite to pioneer and invent. Last month I was given a special opportunity and have become the GM of something new for AWS. I’m looking forward to sharing this with you all when the time is right!

Yet another reason I love working at Amazon is all the amazing people I get to learn from every day. Everyone brings a unique superpower that helps our customers innovate faster, makes us more efficient, invents new ideas, and so on. One of the people I’ve learned the most from over the last several years — both as a customer and a colleague — is Philip Potloff. This is why I’m proud to announce that as I move in to my new role, we’ve asked Philip to lead the Enterprise Strategy team for AWS and take it to the next level.

I’m not going far, and I’ll still be around to meet with customers, discuss (and sign, if you want to catch me at an event) my book, and occasionally post on the enterprise blog.

So, please give us feedback on Ahead in the Cloud, and help me congratulate Philip as the new Head of Enterprise Strategy for AWS!

Keep building,
-Stephen
@stephenorban
Read my book

Ahead in the Cloud: My New Book on the Future of Enterprise IT (and My New Role) was originally published in AWS Enterprise Collection on Medium, where people are continuing the conversation by highlighting and responding to this story.

Mark

The Enterprise Strategy blog is moving! was originally published in AWS Enterprise Collection on Medium, where people are continuing the conversation by highlighting and responding to this story.

We’ve probably all heard the DevOps principle “you build it, you run it.” In theory, DevOps makes each team responsible for both the development and operation of its code, giving DevOps teams complete responsibility — and complete visibility and transparency — for the entire value stream, including not just coding, testing, securing, and complying, but even the business results of the code when it is running in production. But IT operations includes much more than the limited “ops” functions we typically fold into a DevOps team. I’m talking about things like ticket management, incident handling, user management and authorization, backups and recovery, network management, security operations, infrastructure procurement and cost optimization, compliance reporting, and much more. In today’s IT organization, where do these responsibilities fall? And how can we improve these operations and perhaps even apply DevOps and Agile principles to them?

This post, the first in a series on how to best think about operations in the cloud, will explore that set of operations functions that is not typically assigned to DevOps teams. We will also talk about how organizations not yet using DevOps can still benefit from streamlined operations when they migrate their applications as-is to the cloud.

Why aren’t these functions typically performed by the DevOps product teams? For one thing, fast feedback is critical to DevOps. You want DevOps teams to have a streamlined, low lead-time, lean pipeline to production. Devoting team capacity to this broader set of operational functions may slow down this pipeline. There are also efficiencies to be gained by sharing these practices across the work of all the DevOps teams. For example, it might not make sense for each team to have its own way of communicating about production incidents, and certain functions like user management and cost optimization require a view across all systems and IT capabilities.

All of this is to say that a portion of IT operations still exists independently of the DevOps teams, performing those “ops” functions that are not in “DevOps” while the DevOps teams focus on that subset of ops functions specifically related to deploying code and responding to code-related incidents (“wearing the pager”).

You might recognize the voice of hard-won experience here. In my role as CIO of USCIS I once made the mistake of not paying enough attention to that portion of ops that lies outside of DevOps. We had a large initiative going on with about 15 agile teams. When they released code into production, they found that they needed to set up a process for handling user problems and questions, production incidents, and monitoring alerts. As the system became more complex, this burden became heavier. In a few cases, business leaders as well as teams working on other systems downstream and upstream complained that they hadn’t been notified of outages that affected them.

One day, the head of our Network Operations Center (NOC) happened to visit the offices of that project. When he saw what they were doing, he was stunned. “Why aren’t you just using our normal incident handling process? We have a situation room at the NOC, escalation procedures, an incident response team, and a runbook for contacting the people who need to be informed or involved in diagnosing issues. We can show you our statistics on how good we are at this and how we’ve been getting better and better at it. Why would you re-invent the wheel?”

Why indeed? Focused as I was on combining ops and dev into each team, the DevOps teams, feeling responsible for managing their system in production, had responded by cobbling together their own incident response processes. They also authorized new users to use the system, tried to resolve network issues that affected their system, and lots more. That degree of ownership was admirable, and it was my fault for not clearly thinking through how the combined effort of the entire organization could be harnessed to provide the best results.

Part of what IT leadership needs to do then — ouch! — is to set up the environment or the context in which DevOps teams can be most successful. This certainly involves cultural change, changes to governance and investment management processes, and in some cases organizational changes, but it also involves integrating the DevOps teams with the rest of the IT organization and its processes.

Some organizations stand up a central Platform and Tools team to provide a common infrastructure on which DevOps teams build and operate. Sometimes this team provides test suites and monitoring capabilities that serve as guardrails to ensure security and compliance. A centralized team might also handle Network Operations Center (NOC) and Security Operations Center (SOC) functions. A Site Reliability Engineering (SRE) group might help DevOps teams optimize and oversee the performance of their code. There are usually teams that provision and support the devices that employees will use. There may be a Tier 1 and Tier 2 support help desk. Netflix even has an Insight Engineering team that tries to make monitoring and logging easily actionable by DevOps teams. Many variations are possible — what they have in common is they provide an ecosystem in which DevOps teams operate.

All of this raises some interesting questions, though. First: Can these other operations functions borrow some of the ideas of DevOps to streamline what they do? Automation, for example. Or testing in production. Or building cross-functional teams with end-to-end responsibilities. Second, can we avoid a “handoff” between the DevOps teams and these other operational teams? Remember, DevOps was created to avoid handoffs between Dev and Ops. But how about that moment when the help desk starts handling Tier 1 and Tier 2 support calls? When end user devices must be provisioned to give employees access to the new system? When the SOC begins monitoring activity on the system?

On the first question, I have some good news. The idea of applying DevOps-influenced best practices to all of operations has taken a large step forward with the release of the latest version of AWS’s Well-Architected framework, which describes best practices for architecting systems in the cloud. It includes a fleshed out Operational Excellence Pillar, which adds details about best practices for operating workloads once they are in the cloud.

Most AWS services can be accessed via APIs, which makes it possible to script their activities. As a result, it is possible to automate a great number of operational processes. Many other processes are amenable to AWS Lambda functions that can be triggered by events that take place in the operational environment.

AWS Managed Services (AMS), which provides ongoing management of the infrastructure for customer applications, has developed and uses these well-operated practices in delivering its services, and has based them on its own experiences helping customers. It has automated a great deal of the operations processes it uses to support its customers, and instituted processes that allow it to continuously learn and improve.

Although taking full advantage of the cloud often requires some refactoring or rewriting of applications, AWS customers have found that even a simple lift-and-shift of existing applications into the cloud can result in cost savings, higher availability, and increased security — simply because being in the cloud allows for better operational processes. AMS has created repeatable, highly automated processes to harness these improvements. By using these automated, best practice operations mechanisms, AMS can help customers lift-and-shift legacy workloads while helping them gain the advantages of best-in-class cloud operations. Customers can then begin refactoring their applications on their own schedule and move them to a DevOps underpinning.

On the second question, how to avoid handoffs, the answer is that enterprises simply must involve experts from across operations domains in the development process. At AWS, we like to say that systems must be designed with operations in mind. This is no different from other DevOps practices — it is already common to talk about designing with security in mind from the start. An interesting technique to consider is creating operations runbooks for things that cannot be automated as part of the system deliverable — artifacts that are developed along with the system and checked into version control along with code, tests, and deployment scripts.

Runbooks can help facilitate compliance. Changes to them can be audited and restricted to certain roles. The existence of the automated scripts and manual runbooks can serve as proof that compliance controls have been implemented.

The upshot is that enterprises need to create the context around DevOps that takes advantage of its short feedback cycles, risk reduction, and compliance and quality controls. As I have learned, IT operations, in its broadest sense, is an important part of this ecosystem. DevOps teams run what they build but do so in the context of other IT operational functions, just as they do in the context of governance processes, compliance audits, and organizational structures.

As I continue this blog series, we will talk about organizational structures for IT, cost optimization in the cloud, security operations, and what it looks like when an integrated IT organization is functioning optimally.

Mark

The Critical Missing Piece of DevOps…And How to Find It was originally published in AWS Enterprise Collection on Medium, where people are continuing the conversation by highlighting and responding to this story.

I remember an incident from my previous CIO role. A number of us were in a meeting discussing the severe problems we were having with the performance of a large contractor. At one point, someone suggested that we start a new RFP (request for proposal) process to replace the contractor. “Too risky,” said one of the more senior executives at the table. “We don’t know what kind of a contractor we’ll wind up with or how good they will be.”

I’ve heard many variations of this line of thought. Essentially, we had a contractor who had a 100% chance of performing poorly, since they already were doing so, yet the perceived risk of working with an unknown was somehow believed to be higher. Similarly, organizations often think that it is risky to move to Agile ways of working or to migrate to the cloud if they haven’t done so before. This is equally strange, since Agile techniques were invented as a way to reduce risk, and the cloud provides many ways to reduce risk compared to onsite implementations.

I have had similar conversations with security experts in the government, and frequently with managers at AWS customers. “Is the cloud secure enough?” people often ask. But this is the wrong question. What they should be asking is, “Where will my security posture be better — in the cloud or in my onsite data center?” I didn’t have to ask myself that question in my role at the Department of Homeland Security (DHS). I knew the answer: the cloud clearly enabled us to build a much more robust security architecture.

Ask anyone in the information security arena if they are happy with their organization’s current onsite security posture:

“No way. Too many people have privileged access; we have too many insecure legacy platforms; we don’t patch often enough; our firewall rules are too complex; production systems aren’t reviewed often enough…” and on and on.

“How about moving to the cloud, then?”

“Well, that would be risky…”

There is a pattern here: we tend to attach too much weight to the risk of the new and too little weight to the risk of the status quo.

In fact, this is simply an instance of a common bias, described in a 1988 article by W. Samuelson and R. J. Zeckhauser, “Status Quo Bias in Decision Making.”[1] Samuelson and Zeckhauser’s experiments showed that people disproportionately decide to stick with the status quo when presented with alternatives. In a 2016 Psychology Today blog, Rob Henderson says, “Status quo bias is a cognitive bias that explains our preference for familiarity. Many of us tend to resist change and prefer the current state of affairs.”[2]

Status quo bias was further explored by Daniel Kahneman, J. L. Knetsch, and R. H. Thaler in their paper “Anomalies: The Endowment Effect, Loss Aversion, and Status Quo Bias.”[3] The authors relate status quo bias to a phenomenon called the endowment effect, the tendency of people to give a higher weighting to things they already have, when making decisions.

What at first seems like fear of the new is perhaps better thought of as an emotional preference for what we already have. The effect is stronger the more choices we are confronted with (think of all the options available in the cloud!) and, interestingly, stronger the longer we have held the object we may be giving up.

For enterprises looking to transform digitally, it is critical to move beyond that bias, to move beyond the fear and the perceived risk of the new. Instead, enterprises should focus on how new ideas in the IT world can help reduce risk — reduce the risks of IT investments, reduce the risk of disruption in their industries, and reduce the risk of security breaches. They can reduce all of these risks through a combination of moving to the cloud, introducing DevOps, and architecting and operating their systems using contemporary best practices (see the AWS Well Architected framework).

There are all sorts of risks in today’s business technology environment — all sorts of things for good managers and leaders to worry about. There is the risk that a large IT investment will not return the business benefits that were intended. There is the risk that a disruptive startup will shake up the industry. The risk that a hacker will steal sensitive customer data. The risk that a competitor will think of a brilliant new idea first. That costs will spiral out of control. That a new technology will make the current infrastructure obsolete. That the government will suddenly change a regulation that deeply affects the business.

I could go on and on. There are so many risks it is a wonder that an enterprise can do anything at all. But that is exactly the point — the biggest risk is never change, but stasis. Unless you are sure that your enterprise is already prepared to meet all of the risks mentioned above, the status quo is a terrible place for you to be, and the risk of the new should seem negligible compared to the urgency of change.

It turns out that many innovative companies have found ways to reduce risks in an environment of digital transformation, and central to most of them is agility — the ability to learn and quickly adapt. Or to put it another way, to systematically and deliberately reduce the cost of change and the cost of learning. How do you reduce the risk of a large IT investment not returning enough benefit? By insisting on results while the investment is being made and pivoting as soon as possible if it is not showing results. How do you reduce the risk of disruption? Stay ahead of it and respond quickly when the industry changes. How do you reduce your security risk? Test often, patch quickly, respond to incidents at lightning speed — these are at least a large part of new security models.

The cloud is the underpinning of these strategic approaches to risk management. It allows enterprises to respond quickly and elastically to changing market conditions. It facilitates agility and innovation. It provides pre-developed services that can be quickly assembled as building blocks. It makes it possible to automate software delivery and create security and compliance guardrails.

In an upcoming series of posts, I will return to these questions of risk and show how the cloud helps companies improve their security postures, ensure that their investments deliver business outcomes, stimulate innovation to stay ahead of disruption, control costs, and avoid obsolescence in many different senses.

In short, I will show how the cloud lets companies dramatically reduce risk.

Mark

[1] W. Samuelson and R. J. Zeckhauser, “Status Quo Bias in Decision Making,” Journal of Risk and Uncertainty 1, no. 1 (1988): 7–59.

[2] Rob Henderson, “How Powerful is Status Quo Bias?” Psychology Today blog, September 29, 2016, https://www.psychologytoday.com/blog/after-service/201609/how-powerful-is-status-quo-bias.

[3] Daniel Kahneman, Jack L. Knetsch, and Richard H. Thaler, “Anomalies: The Endowment Effect, Loss Aversion, and Status Quo Bias,” The journal of Economic Perspectives 5, no. 1 (1991): 193–206.

Reducing Risk in the Cloud by Overcoming the Status Quo Bias was originally published in AWS Enterprise Collection on Medium, where people are continuing the conversation by highlighting and responding to this story.

Getting Started with Training for the Cloud

Enterprise executives contemplating a move to the cloud sometimes worry that their current employees will not be up to the job. After all, the cloud involves a rather different set of practices and technologies, and making the most of the cloud through DevOps, infrastructure as code, automated security and compliance guardrails, and so on requires a very different mindset and different techniques. Looking at the enterprise’s current employees, the executive may conclude that these skills are lacking.

Well, of course they are! Current employees are unlikely to have cloud skills simply because the company has selected and trained them to support the company’s earlier, non-cloud technical needs! Working at this company, they most likely haven’t had a chance to work in the cloud, let alone demonstrate proficiency. But as Stephen Orban says in his blog post “You Already Have the People You Need to Succeed in the Cloud,” those employees already have invaluable knowledge about the company, its products, and its customers. Furthermore, Orban says, “I’ve yet to come across a use case where someone with the appropriate skills and a can-do attitude couldn’t find a place for themselves in the cloud.” The fact is that technology employees tend to be fast, flexible, eager learners.

The real question is how to take advantage of their wealth of knowledge as company insiders by helping them leverage their existing technical skills to learn the critical new ones.

In this post, Sara Snedeker of Amazon Web Services (AWS) Training and Certification describes the many training options available for your employees from AWS. Among these is a new series of free digital training courses that enterprises can use to start building that base of knowledge among their employees.

In a future post I will discuss some strategies that businesses can use to spread ideas and practices once a critical mass of employees has attained proficiency.

— Mark

Educate Your Staff to Get the Most Out of the Cloud

By Sara Snedeker, AWS Training and Certification

There is a growing skills gap in the IT marketplace. The 2017 Global Knowledge IT Skills and Salary Report shows that 68% of IT decision makers reported a gap between their team’s skill levels and the knowledge required to achieve organizational objectives. That’s a 127% increase over the 2016 reported gap.

Organizations looking to migrate to and take advantage of the cloud to meet their business goals need employees with cloud skills on their team.

How to Address the Skills Gap

Given the skills gap, it can be expensive and time consuming to recruit and hire new employees to fill cloud-related roles. In any case, most businesses already have the staff and support they need to transition from traditional IT to the cloud, but they need to help those employees learn to apply the fundamental IT skills and institutional knowledge they already have to new cloud roles. Training can help you build internal buy-in, get your staff speaking the same language, and help your teams accomplish business objectives more efficiently.

Whether you are just getting started with cloud or looking into a cloud-first strategy, educating your staff should be part of your plan in 2018.

An IDC whitepaper, “Train to Accelerate Your Cloud Strategy,” demonstrates that training enables organizations to accelerate cloud adoption, achieve business objectives sooner, and overcome concerns related to cloud adoption. The research shows that comprehensively trained organizations are 80% faster to adopt cloud, 2.7 times more likely to realize that the cloud can help jump-start innovation, 3.8 times more likely to meet cloud ROI requirements, and 4.4 times more likely to overcome operational and performance concerns.

Training can also help you save time and money. Not only will you be able to avoid hiring new staff to fill cloud-related roles, but your employee retention may improve. Research from Global Knowledge shows that training is positively associated with job security and satisfaction. Trained individuals are more likely to report feeling fully satisfied and more secure in their jobs.

Leverage AWS Training to Educate Your Organization

AWS Training can help your organization build cloud skills with a range of training topics, modalities, and class levels.

We recently released over 100 free digital training courses at AWS.training. These free courses are a great place for your employees to start their education. Digital courses are generally ten minutes long and are designed to help individuals build foundational knowledge for dozens of AWS services and solutions. You can encourage your team to take Cloud Practitioner Essentials, for example, for an overview of cloud concepts and AWS services, security, architecture, pricing, and support. You can also search our catalog for digital training on the newest AWS services, such as Introduction to Amazon GuardDuty.

Foundational training can provide value across your organization, not just for IT professionals. The leaders IDC interviewed all agreed that organizations should establish a training program that includes cloud fundamentals training for a wide range of stakeholders:

· Vincent Perfetti, VP, Product Strategy and Customer Engagement at GE Digital recommends, “Train as many people as possible on cloud basics.”

§ Matt P., Deputy Chief Information Security Officer of the CIA suggests, “Everyone should get at least 30 minutes of familiarity training to learn the vocabulary of cloud.”

§ Moritz Onken of Siemens Healthcare suggests, “Train C-level IT leaders and their direct reports. The CIO needs to reinvent his/her team.”

§ Peter Loeffler, Head of Innovation and Industry Affairs at Siemens Building Technologies, believes, “It’s important to have everyone speaking the same language — not just the direct [cloud] team but also their managers and executives.”

If you are looking to go beyond the fundamentals, we also offer classroom training, which provides one to three days of intensive training with an AWS instructor. If you are interested in classroom training, we can work directly with your organization on an education strategy. We evaluate your needs and build a customized learning path so your team can learn the skills they need to accomplish your specific business goals.

Our private onsite training gives your team the opportunity to learn from an instructor who comes directly to your location and who is familiar with your AWS use cases. We also offer virtual classroom training for private engagements. This option gives you more flexibility, reduces travel expenses, and allows you to more easily strengthen teams’ skills across the globe.

Contact the AWS Training team to learn more about private training, or search for a public class near you at AWS.training.

Next Steps

Any cloud initiative should include an education strategy to help guide your organization through the change. One good first step is to identify a leader on your team to spearhead your training efforts.

Ultimately, your goal should be to cultivate a culture of continuous learning so you can develop experts on your team to help you as your business evolves, as new staff joins, and as the industry continues to innovate.

Getting Started with Training for the Cloud was originally published in AWS Enterprise Collection on Medium, where people are continuing the conversation by highlighting and responding to this story.

Large enterprises beginning a transformation journey often find that they already have pockets of transformation that have escaped notice. But why haven’t good practices spread from these isolated corners of the enterprise and transformed the rest of the organization? I think the answer is that transforming the whole of the enterprise requires…well, a large transformation. The enterprise must find a way to make its deep changes scale broadly until they reach critical mass. Pockets of innovation, as a general rule, don’t have the leverage it takes to cause a global change.

In this contributed blog post, Milin Patel, the Principal Architect and Co-Founder of Rearc and formerly the Head of DevOps of Dow Jones, talks about Dow Jones’s move to the cloud and DevOps, and the organizational changes this shift inspired. Fundamental to their transformation strategy was the use of a Cloud Center of Excellence (CCOE) to gain leverage across the enterprise for the change initiative. It is tempting to think of a CCOE as just a team of experts who can be consulted for their knowledge of operating in the cloud. But as Patel points out, a CCOE can be much more than this: it can be the driver of change across the enterprise, the focal point for transformation that is broad as well as deep — the Archimedean lever that moves the world, or at least the enterprise.

Thank you to Milin Patel for providing this how-to guide on how to set up a CCOE that can drive organizational change.

— Mark

Building a Cloud Center of Excellence (CCOE) Team to Transform Your Tech Organization

By Milin Patel

I have been very fortunate to be involved in a few enterprise IT transformations over the past five years. It all started at Dow Jones under the leadership of Stephen Orban, who was CIO at Dow Jones at the time. I was tasked with figuring out a new way to build and run software in the cloud so that Dow Jones could stay relevant and cater to constantly changing customer demand. Stephen is currently the Head of Enterprise Strategy at Amazon Web Services (AWS) and talks in detail about our journey in his upcoming book, Ahead in the Cloud: Best Practices for Navigating the Future of Enterprise IT. I can only add that the three years I was leading Dow Jones’s DevOps transformation were the most fulfilling and rewarding years of my career. So much so, that I have partnered with other colleagues from this journey, including Mahesh Varma and Chad Wintzer, to found Rearc, a business that helps other companies unleash the potential of DevOps and the cloud.

Our success at Dow Jones and Rearc can largely be attributed to making developers the focal point of the transformation journey, and treating them as paying customers. My goal with this blog is to inspire leaders looking to transform their organization with my experiences building and running a successful Center of Excellence (COE) team at Dow Jones. The COE team at Dow Jones enabled the transformation of our software development and operations practices. While this COE story is mostly geared toward cloud and DevOps adoption, the six-step approach I outline can be applied to solve other problems across your organization as well.

From my perspective, the goal of a Center of Excellence team is to take a large, widespread, deep-rooted organizational problem and solve it in a smaller scope with an open-minded approach and then leverage the small wins to scale it across the organization.

Dow Jones is a more than 125-year-old news and business information company, and has been innovative in its ability to deliver news information in new ways. Its flagship product, The Wall Street Journal, has been a subscription-only website (www.wsj.com) since its launch in 1996. By 2013, WSJ was seeing a big shift in consumer behavior around consumption of news and media content. Their readers were looking to consume news on their phones, tablets, and other connected devices on the go, all the time, and they expected Dow Jones to provide a seamless digital experience across devices and platforms. Print-based businesses had been experiencing a slow decline for years, but all of a sudden, even the digital businesses were being challenged by freemium news sites, digital upstarts, and tech giants. In order to meet their readers’ needs, Dow Jones had to deliver new products, features, and experiences at a very fast pace. They had to experiment, learn, and adjust quickly, but they were just too slow.

At the time, Dow Jones was following a waterfall project management approach, which required planning, budgeting, and capital expenditure in advance before the technology could be tested. Additionally, the data center hardware procurement and installation process took anywhere from one to three months.

The specific business problem we identified and set out to solve within the technology department was to accelerate software delivery and promote experimentation. Dow Jones’s current ways of building and running software were not meeting their growing business and customer needs. We had to solve this problem by fundamentally changing their approach to software development and delivery.

Given this problem and some initial experience working with the cloud, Stephen was confident that cloud-based technologies and DevOps practices were necessary ingredients for success. But how do we take a large organization accustomed to working in a specific way and change everything it knows about infrastructure, operations, and software delivery? At the time, there wasn’t much industry knowledge for us to draw from. We figured we had to move away from infrastructure-driven projects with huge capital costs and slow delivery cycles to a nimble, software engineering-driven, cloud-first approach that would allow us to iterate quickly without the fear of failure and financial risk. All of this led to the formation of our Center of Excellence team at Dow Jones (internally referred to as the DevOps team). I am still very honored to have been given the opportunity to be one of the three founding members of our COE team working alongside Tejash Patel, who is now leading several transformation efforts at Guardian Life Insurance.

The COE’s mission statement was to figure out the right tooling and practices that would empower our development teams to deliver awesome digital experiences for our customers with agility and confidence. It was given the autonomy to make the necessary design and process choices rather than being forced to operate within the boundaries of what the organization already knew or was comfortable with.

Public cloud adoption was the most important part of our overall solution to deliver software development agility, which required us to experiment, fail fast, and move on to the next experiment until we found the right answer. The cloud abstracted away the undifferentiated heavy lifting so that we could focus on our internal customers.

The target end state was clear, but the approach on how to get there was built over time. I am hoping that sharing my six-step approach will provide a path for you to consider within your organization.

Step 1: Forming the Team

The COE team should start small. After having done it at Dow Jones, I have realized how important it is to have the right people be part of the founding team. A few important traits to look for in members of the founding team include:

● Experimentation-driven: able to learn from failures and iterate quickly

● Bold: not afraid to challenge the status quo

● Result-oriented: can take an idea from its ideation phase to successful implementation

● Customer focused: appreciates the impact of developer productivity and operational excellence

● Able to influence: can scale his/her skills through others

Our COE was composed of engineers with strong technical skills and diverse backgrounds. Your top-notch tier-1 engineering talent usually has a good amount of trust built within the organization, which makes it easy for them to have a positive influence across the rest of the organization. I personally think internal hires work best for founder members of your COE team, but having a mix of internal talent and new hires or strategic partners can also jump-start your COE efforts. In our case, we needed engineers that understood networks, systems, and software development. While our founding team comprised of internal hires, we subsequently added external hires and recent college graduates.

Step 2: Deliver Some Quick Wins

With the larger vision of an organizational transformation it is necessary to narrow the initial focus to deliver a single, relatively small but important project successfully. In our case, one of our early projects was to migrate out of a data center in Hong Kong.

Within six weeks, we migrated the WSJ Asia data center to AWS’s Tokyo region using a lift-and-shift (directly moving an on-premises application to cloud with minimal changes) approach. It was a perfect start for the COE team — we had to figure out the networking (VPCs, load balancing, WAN acceleration, data replication between our US data centers and Tokyo), machine images (AMIs) in AWS, application performance, traffic distribution, change management, etc. We were able to do this only because we had the autonomy to make all the necessary decisions to make the migration successful.

The success of our first production cloud deployment allowed us to showcase our work to the rest of the organization and get over that initial concern of running a production app in the cloud. There was less fear and uncertainty about operating in the cloud. It opened up a dialogue around what’s possible rather than what’s unknown.

Step 3: Acquire Leadership Support

It is important that the technology leadership delivers a clear message to the rest of the organization about the challenges the organization is facing and what the plan of action is to address those challenges. In our case, Stephen and his leadership team did not miss any opportunity to talk about the COE’s work. While you absolutely need to have a grassroots adoption for a new technology, I strongly believe that a clear vision and message delivered from leadership is equally needed. For us, internal blogs and town hall-style meetings strengthened the message across the entire organization.

Step 4: Build Reusable Patterns and Reference Architectures

Once we gained some experience running the first set of applications in the cloud, it was clear that we needed a somewhat repeatable process to onboard more applications. In talking with multiple application teams, a few patterns started emerging. We were able to build reference architectures and blueprints that were slightly opinionated but mostly non-objectionable to the app teams.

Our COE team did an awesome job with not only building the reference architectures but also building the necessary tooling to automate the provisioning and operations of applications leveraging these reference architectures. It was a carrot for the application teams to get their apps up and running quickly while we got to standardize patterns across the organization and reduce the operational overhead.

Step 5: Engage and Evangelize

Riding on the success of our initial wins and leadership support, the rest of the organization began to engage and be part of the transition. The COE team capitalized on this. We started engaging with the rest of the organization by doing DevOps Days, lunch workshops, training sessions (via a DevOps University program), and showcased case studies of successful cloud projects. Our internal customers (development teams) presented their work at the DevOps Days and workshops, which was a much more powerful message than just external presentations. Bringing architects and developer evangelists from AWS and Chef brought in a good deal of excitement and showcased how serious we were in our transformation efforts.

Step 6: Scale and Reorganize

Once you have a few initial projects delivered successfully using the newer approaches and practices, the rest of the organization should become eager to leverage the services, tools, and expertise of the COE for their specific needs and problems.

You have to carefully plan for this critical last step of scaling the COE function across the rest of the organization. In our case, we were a little late to find out that the COE had become a bottleneck for the rest of the organization to adopt cloud and DevOps practices. Eventually, we built federated teams and DevOps capabilities within each application team to scale out the COE’s function.

Conclusion

Our approach to building and growing a COE team to help jump-start a large-scale, company-wide transformation allowed us to discover what’s possible and to implement new experiences for our customers at a pace never before seen at the company. In 2013, a change to WSJ.com required a developer to submit their changes to QA by 10 a.m. for Tuesday and Thursday build nights. Ten to fifteen engineers got on a conference bridge that lasted for hours and often didn’t succeed. In 2016, we had 100+ deployments throughout the day across multiple services in production and non-production environments. No one missed the build nights anymore, but perhaps most importantly, the number of production incidents decreased significantly, speed of delivery improved significantly, and the confidence level was much higher across all the engineering teams. We were able to launch several new web and mobile products for our customers, improve the user experience and performance of our products, grow our digital subscription base, and enter new markets with confidence.

Using a Cloud Center of Excellence (CCOE) to Transform the Entire Enterprise was originally published in AWS Enterprise Collection on Medium, where people are continuing the conversation by highlighting and responding to this story.

The weekly Excel report comes out and is delivered to your email. As you review it, you see an anomaly in the financial data that you don’t understand, despite the pivot table provided in the report that allows you to drill down to at least some level of detail. You ask your operations analyst what’s going on. To which your analyst responds, “I’m not sure. Let me find out.”

The next day the analyst tells you that the reason for the anomaly is productivity was way down at the manufacturing plant.

“That doesn’t make sense,” you say. “Can you ask HR if sick days are impacting the productivity numbers? Or could it be there was an issue with the time capture application at the plant?”

“It will take a week to get at that data and merge it with the financial data,” your analyst says.

“Can’t you just send me a dump of the data from the ERP and time application, and I’ll work with it myself?”

The analyst responds, “I don’t have access to the data, and it will take a few days to submit the right tickets to get access to it.”

“Well, surely we can get access to the Centers for Disease Control dataset with location specific data to see if flu outbreaks in the local area are impacting productivity numbers at the plant?” you respond sarcastically, your frustration rising. Your analyst looks at you like you have two heads, to which you say, “Uh, never mind.”

If this scenario seems all too familiar to you, your organization has a big data problem. Your first reaction may be this is a business intelligence process and tools challenge that has plagued organizations since time began, not a true big data problem. But without getting into a religious debate about what analytics vs. reporting vs. business intelligence is, stay with me as I show why every organization has a big data problem. And, with artificial intelligence and machine learning capabilities starting to come to fruition, it’s even more important for enterprises to get a better handle on the data they have.

This blog will be the first in a multi-part series covering the issues and approaches, both organizationally and technically, to creating a data driven organization. In this series, I hope to help you see why every organization should revisit their data and analytics strategy to better utilize the advantages big data can bring.

Most of us think of big data problems as one of volume, and we usually associate them with use cases like the Internet of Things (IoT) or the storage of large objects like images. But the truth is every organization has a big data issue that has been masked in a number of different ways. I’m going to use the 4 V’s model to describe big data and hopefully convince you that you do indeed have a big data problem (and don’t worry, there are solutions).

Volume

An architect once asked me why we need something like Hadoop when the average database size in their environment was something like 50 GB. In many cases the entire database for applications was running in memory in the database engine. However, what most organizations don’t realize is that a lot of interesting data is thrown away or is just not accessible.

For example, what about user activity in the application? Is that information readily available? Or the telemetry of the infrastructure hosting the application (including load balancers and switches)? What about where users are interacting with the application? How they are using the application in relation to other applications? What about the older versions of data that are no longer compatible with the current table schemas? Yes, there are application and end user monitoring tools, but rarely are they analyzed in context with the business processes and activities.

The other volume problem is that data is siloed across many different applications and data warehouses. While no one application may be “big,” the entirety of all applications in an enterprise are big. When businesses become focused on outcomes that span functions or units, the need to analyze data from many sources becomes very challenging. Data warehouse technologies can do this to a certain extent, but most are constrained and cannot house it all. In my own past, I’ve owned shared reporting platforms with hundreds of data warehouses, data marts, and operational data stores.

These silos of data pose another problem: access. Each place data is stored has its own access roles, rules, and ceremonies to be adhered to when trying to access the data. This becomes really pronounced when you do your first data science experiment only to see it stall because you can’t get at the data. (By the way, if you have archives of data because of performance issues, that’s another data silo. When you expand the data types to include object or unstructured data, you have big data)

Velocity

Velocity is the speed at which data moves, but I would argue that it is also the speed at which data changes. Spark, AWS Kinesis, and other streaming technologies seem, again, to have little applicability outside of IoT-type use cases and aren’t relevant to enterprise applications. If you buy my argument that data about what is happening with your infrastructure is a business and business application concern, then having the ability to store and process this information is really important. Tools like Splunk or Sumo Logic are great, but how many times did you wish you had metadata other than some cryptic server name? You can use Amazon Kinesis to enhance log data by adding things like application name, business criticality, and owner. Then you can send it on to your log analytics tool of choice.

How about if I were to ask you how much time is required to change an interface at your organization? Or how long it takes for data to propagate from your ERP to your downstream systems? How many of your business users would love to be notified and alerted on key events or algorithms?

By the way, service-oriented architectures (SOAs) and application programming interfaces (APIs) are not sufficient to solve these issues. I recall working with architects in my newly inherited integration platform organization. They were declaring that all new data integrations must only be API based. To which someone asked, how are you going to deliver 100 GB reference data dumps when you need to refresh a data set and post a master data change in SAP? Silence.

Variety

Many enterprises realize there are troves of data that don’t neatly fit into the traditional database storage technologies (e.g., images, sensor data, etc.). However, most enterprises also don’t realize how easily and quickly this information can be acquired and stored in fit-for-purpose solutions like Amazon Simple Storage Service (Amazon S3), which is great for object storage. Or how easy it can be to store relationships like social networks in a graph database, like Amazon Neptune.

But variety doesn’t stop with types of data. There is also variety in how you analyze and consume insights from the data. When I launched an analytics initiative at a prior company we had a tenet to deliver interactive insights to users where they are. In order to meet that objective we quickly realized that no single reporting or visualization solution could fulfill all the requirements. You can only ask Excel to do so much. We were delivering insights processed through algorithms through API’s, within applications using custom visualization widgets using JavaScript frameworks like D3.js, and through business intelligence portals leveraging Tableau and other visualization solutions.

Veracity

Data veracity speaks to the noise, abnormality, accuracy, or usefulness of data. When you start tapping into unstructured or object-based data, there’s going to be noise. Just like in electronic noise, there are filtering, enhancement, and amplification mechanisms you can use to get at the data you want.

One use case that many enterprises are concerned with is the spiraling costs of sending data into proprietary log aggregation, security, or monitoring tools. However, in most cases a vast portion of the log data can be filtered out, as it is not useful. One pattern I’ve seen implemented is instead of sending data directly to specialized log analytics tools, it can be sent into a data lake architecture and then filtered in real time using tools like AWS Apache Spark, allowing users to siphon off only the useful data.

If you’ve struggled with or resonate with any of the points above, it’s time your organization revisits your analytics approach and architecture. I actually don’t like the term “big data” because it misdirects the application and opportunity big data architectures can provide. What it really boils down to is that every enterprise has the opportunity to deploy fit-for-purpose analytics solutions (storage, processing, querying, analysis, presentation, etc.) to meet their existing business and IT challenges.

As I continue this multi-part series, I’ll cover more details on the future of analytics solutions and some ideas for how best to organize around making your company a data driven organization.

Never stop innovating,

Joe

chung@amazon.com
@chunjx
http://aws.amazon.com/enterprise/

Analytics Part 1: Why Every Company Has a Big Data Problem was originally published in AWS Enterprise Collection on Medium, where people are continuing the conversation by highlighting and responding to this story.

When an enterprise migrates into the cloud, it is empowered with many new ways to manage its costs. One important benefit is increased agility which in turn leads to leaner delivery processes with corresponding savings. With agility comes the possibility for rapid feedback-and-adjustment cycles, from which comes improved quality and a lower cost of rework. Managed services — anything from databases and analytics to identity and access management and logging and monitoring — can reduce operational costs and result in a lower Total Cost of Ownership (TCO). And the pay-as-you-go model for higher-level services is likely to reduce costs for most enterprises. These savings come from using the cloud environment in new ways, and have the greatest long-term impact on costs.

Nevertheless, it is the cost of infrastructure that many enterprises first focus on, and that often yields the most easily measurable and visible savings. To maximize those savings, enterprises should make a focused effort and take advantage of the tools AWS makes available to them. In the cloud, an enterprise has a great deal of control over how it provisions and uses its infrastructure, and the choices it makes can greatly affect its costs.

One of our Migration Competency and APN partners, TSO Logic, has contributed the article below, demonstrating how important some of these choices can be. In particular, TSO Logic discusses the importance of rightsizing instances. There are many other levers that can be used to reduce costs for workloads in AWS, but TSO’s article demonstrates their deep expertise in this area.

***

Slash data center costs by 36% — move to the cloud and stop paying for resources you’re not using

By Aaron Rallo, Chief Executive Officer, TSO Logic

You want to start capitalizing on cloud. You’ve targeted some workloads you think are good candidates. You’re ready to get started. So where do you begin? Seems simple enough: you glance down at your spreadsheet showing how much you’re paying to run those workloads on-premises now, and look up how much it would cost to run them in AWS. But something seems off here. Cloud doesn’t look much less expensive. For some workloads, cloud actually costs more. What’s happening here? Have you fallen victim to a bunch of cloud hype?

Not at all. More likely, the numbers you’re looking at aren’t telling you the whole story. For one thing, it is difficult to compare apples-to-apples. You may be assuming that the server platforms in the cloud are basically comparable to what you’re running on-premises. In reality, the platforms you’ll use through AWS are latest-generation hardware that can likely do a better job at a lower cost than what you’re currently running.

But the biggest mistake when comparing numbers on a spreadsheet is assuming that your current resources are provisioned appropriately. In reality, most on-premises workloads are overprovisioned — more than 80 percent according to our new research. (http://tsologic.com/resources/economics-of-cloud-migration-2017/)

Data Reveal that Most Instances Are Over-Provisioned

TSO Logic recently conducted a statistical analysis of nearly 105,000 operating system (OS) instances across North America — one of the largest data sets ever assembled for this type of analysis. The results might surprise you:

· Just 16% OS instances were sized appropriately for their workloads. 84% could run on a smaller footprint and directly porting these to cloud based instance types of the same size would be a signifigant waste.

· However, by right-sizing those instances — porting them to optimally sized AWS resources, based on historical analysis of real-world utilization — they could run in the cloud for just $90,000,000. That’s a savings of more than $55 million annually — a 36% cost reduction.

Where do those savings come from? Let’s dig a little deeper.

Inside the Numbers

Here’s a real-world sample from our analysis. Table 1 shows a Dev environment with three OS instances, each running on Intel dual-core E5 processors running at 2600 MHz. Table 1a shows the details of each of these instances. For example, Instance A, is in use 100% of the time. The servers peak usage is 39.55% and the peak memory is pinned, using all 6,145 provisioned Megabytes.

Each of those instances ties back to a specific real-world server, with a specific annual cost in terms of rack units and power consumption. Those figures, as well as the hardware generation, capabilities, and the way it’s historically been used, figure into the total current operating cost for those instances: $4,633 per year, and the cost of operating the compute only is $4,103 per year.

To “direct match” those resources in the AWS cloud — to provision in AWS exactly what’s deployed on premise — you’d need two m4.large instances and an r4.large instance to satisfy those workload levels. Total cost at AWS: $3,741 a savings of just $361 per year. But do you really need to provision in AWS based on old provisioning assumptions? If you look at their historical real-world utilization, the answer is no.

· For Instance A, the OS is in use 100% of the time, but when its used at peak its only consuming 39.55% of the processor, with the average at 13.8%. Looking closer at the processor type, we can see that it is from 2008. Considering the improvements in modern processors and historical workload levels, the optimal instance size is not an m4.large, but a T2.Xlarge, which costs $970 per year.

· Instance C is currently used 17.3% of the time on-premise and has 4,096 MB of Ram. When it is in use the CPU average is 5.6% and peaks out at 16.8%. This processor is also from 2013. The optimal match for this workload pattern is a t2.medium, costing just $286 per year.

· Note: The “direct match” instance size is less expensive out of the box but it would become even more cost effective when you use RI optimization.

By “rightsizing” each OS instance — migrating from two m4.large and r4.large to two tx.large and an t2.medium — the total cost is now $2,588 per year. That’s a 55% cost savings compared to current workload provisioning levels.

And that’s just the savings from basic right-sizing of resources. It doesn’t include the discounts available through purchasing Spot Instances, which can result in further cost reductions of up to 80%!

Applying the analysis to each of the nearly 105,000 OS instances in this sample will reveal some that are provisioned appropriately, where the economic case for cloud is not as strong. But for many more — 84% — the organization is paying for substantially more resources than those workloads actually need.

Hard Data Drives Smarter Migrations

When crafting a business case for cloud, spreadsheets won’t give you complete answers. You need to model and compare costs over time, not just in a one-time snapshot, because compute patterns, as well as the options including the cost of an instance, software and storage in AWS, are always changing. You need to understand differences in hardware — a later-generation single-core Intel processor in AWS, for example, may deliver better price/performance than an older dual-core processor on-premise. And you need to know where and how you’re overprovisioned as well as what software and storage options are required. Only then can you start making meaningful decisions.

At TSO Logic, we create fine-grained statistical models of all on-premises resources to determine the most cost-effective place to run each workload. Ingesting millions of data points from the environment — including age, generation, software, storage and configuration of all hardware, the OSs they’re running, and each instance’s historical utilization — our platform algorithmically profiles compute patterns. It then uses multiple heuristics including pattern matching to determine the best fit for each workload from thousands of potential cloud options. Using validated information from Intel and AWS, we normalize and compare processing capabilities between various generations of Intel processors and the myriad of hardware, software and storage options in the AWS cloud.

By putting aside paper-and-pencil calculations and moving to algorithmic statistical analysis, you can automatically discover the data points needed to understand your real-world needs. And you can create a much more accurate business case for cloud planning that will deliver bottom-line results.

Ready to find out how much you could be saving based on your environment’s real-world needs? Visit www.tsologic.com or talk to your AWS sales rep.

Rightsizing Infrastructure Can Cut Costs 36% was originally published in AWS Enterprise Collection on Medium, where people are continuing the conversation by highlighting and responding to this story.

Training the Entire Enterprise to Harness the Cloud’s Full Value

The cloud changes not just how enterprises deliver IT capabilities, but how they compete, how they advance their top lines, how they foster innovation, and how they organize themselves for success. It stands to reason, then, that training for the cloud should extend not just to technical employees, but to the entire organization as well. This conclusion is supported by a new study from IDC, “Train to Accelerate Your Cloud Strategy” (October 2017).

Indeed, the IDC study confirms that enterprises that “comprehensively” train their employees are materially better equipped to realize value from the cloud. And “comprehensively” means that they offer at least 8 hours of training for each of four topics (cloud technologies or platforms; methodologies or processes; organizational objectives; and potential use of cloud) to a wide range of stakeholders, both technical and non-technical.

Specifically, IDC reports that “comprehensively” trained organizations are 80% faster to adopt the cloud and 1.9 times as likely to move from limited deployments to deeper adoption. These organizations are also 3.8 times more likely to meet cloud ROI requirements and 4.4 times more likely to overcome operational/performance concerns.

You might legitimately wonder why the speed with which companies can migrate, and how quickly they’re able to overcome concerns, are benefits. The answer, I think, is that because the cloud offers compelling value, the sooner that value is realized the better. This applies to costs, of course, where the time value of money guarantees that savings today are better than savings tomorrow. But it also applies to innovation and competitive positioning, where a revenue increase today is better than a revenue increase tomorrow, and the ability to quickly adjust to an industry under disruption can mean the very survival of the enterprise.

The IDC study reinforces this, by citing a number of case studies with cloud customers.

GE Digital, for example, realized that the need for training went well beyond technical employees. “We trained as many people as possible on cloud basics because the benefits of understanding cloud extend beyond the IT organization,” says Vincent Perfetti, the former VP of Product Strategy and Customer Engagement at GE Digital, who is now with AWS.

Peter Loeffler, Head of Innovation and Industry Affairs at Siemens, adds: “Cloud training shows IT professionals, managers, and executives a new paradigm. Everyone needs to understand what cloud services (and tools) exist.”

And Moritz Onken, Head of Healthcare Insights Data Engine at Merck KGaA, suggests that companies “Train C-level IT leaders and their direct reports” because “without buy-in from upper management, there will be no trust in the solutions (and no money to try them).”

Making the most of the cloud requires new skills and new categories of knowledge. Of course, skilled technical professionals are good at learning — they do it constantly to keep up with their fields. And they can often do it on their own. But since the cloud is all about speed, accelerating the learning process for technical employees is as much of an advantage as any other increase in agility; in fact, when all is said and done, it might end up providing an even larger benefit.

For other stakeholders in the organization, all the way up through the CEO, the changes brought about by the cloud are, perhaps, less obvious but deeper. The agility of the cloud gives them superpowers, so to speak, but they must learn how to use them. Training other parts of the organization adds leverage to the new skills of the technical organization, amplifying and hastening the changes that the technologists can bring about.

AWS offers a host of cloud training programs designed to enrich organizations across a broad spectrum of people and industries. For additional insights on training, I strongly recommend these two posts by Stephen Orban, the Global Head of Enterprise Strategy at AWS.

The cloud is only a game-changer to the extent that we learn how to use it to change the game.

Read the full IDC study.

Training the Entire Enterprise to Harness the Cloud’s Full Value was originally published in AWS Enterprise Collection on Medium, where people are continuing the conversation by highlighting and responding to this story.

Executives are under increasing pressure to deliver Cloud transformation results quickly. Getting fully setup for success should not stop you from getting going, that said, understanding lessons learned from those who have gone before, saves time and money.

As an enterprise strategist for Amazon Web Services, I currently travel around the globe helping the largest companies in the world discover and unlock the power of AWS Cloud. Having lived the journey to cloud and worked with countless customers, executives are always keen to understand what journey lessons they can learn from those who have gone before.

With the benefit of hindsight here are 12 steps to get going when starting, that consistently deliver results : -

Step 1 — Don’t Over-Think It; Assign a Developer and start!

Just start. Focus on developing the skills and then delivering. All the help you’ll need is available. And all the answers to your questions have already been written. Start with one forward thinking Engineer or Developer and have them start using AWS through the Console, get to know the services and spin up an EC2 test instance. From my own journey we started with a very small team of forward leaning Engineers, the early lessons they harvested, compounded and continually informed our journey.

Step 2 — Empower a Single-Threaded Leader

In my experience, it can be fatal if you don’t have the support of a single-threaded executive leader during the transition. This leadership function simply can’t be delegated. The CIO, or, at the very least, a direct report of the CIO has to lead this effort and be visible each and every day to provide direction and remove roadblocks. And ensure inclusive executive alignment and air cover across the leadership spectrum to reinforce the profound benefits of moving to the public cloud across the spectrums of cost, security, and speed of product development speed.

Put another way, the single-threaded leader must be a rallying point for all change curves. They must listen well and bring a “can-do” attitude to the cloud transition. When I was acting in this capacity at Capital One UK, I used the tenet “All of your assumed constraints are debatable” this served as a forcing function so people looked at each perceived problem as an opportunity instead. Finally, the single-threaded leader must be responsible for establishing Step 3, which is crucial.

Step 3 — Create Your 2-Pizza Cloud Business Office

Amazon’s 2-pizza team concept means a team of around 8–10 people. And, in this case, I’m referring to the virtual leadership team, which needs to provide strategic oversight and tactical air cover for engineers and developers as you move to the public cloud. It’s essential that this cloud leadership team takes into account — and addresses — everybody’s fear (of the unknown).

The best Cloud Business Office teams include —

CIO or Direct Report with Single Threaded Ownership
Procurement or Vendor Management
Legal Lead
Chief Information Security Officer
Chief Financial Officer or Direct Report
Head of Infrastructure
Head of Delivery
Engineering or Product Manager of the first Cloud Engineering Team
Risk Leader (needed in most organizations, but especially regulated ones)
Audit Leader (needed in most organizations, but especially regulated ones)

These folks need to follow the Agile cadence established in your organization and meet at least weekly (if not daily), to review progress and remove roadblocks.

Step 4 — Establish Your Tenets (And Be Prepared to Amend Them as You Go)

Tenet — “a principle, belief, or doctrine generally held to be true; especially: one held in common by members of an organization, movement, or profession.” Common Tenets provide a common frame of reference for everyone to understand the ‘how’ questions that can arise. When creating them, seek feedback from a wide source, but strive for a small but powerful list. I’ve written and read a lot of cloud tenets over the past year; here are some of the best and things you should consider when creating yours:-

• Be Clear on your Business Goal — Are you reducing cost? Transforming to digital native? Reducing your app footprint? Or closing your data center? It’s challenging to do all this at the same time, so my advice is to go cloud first for all new, lift and shift the old, and then optimize and eliminate apps. Stephen Orban, the Global Head of Enterprise Strategy at AWS, has written a series of posts on this topic, which are well worth reading.
• Choose a Predominant Public Cloud Partner — This provides focus for your organization to get to an expert level with a predominant platform, avoiding the distractions that come with too many platforms, across people, process, and technology paradigms.
• Agree on Your Security Objectives. I recommend reading these excellent white papers and getting advice on this from AWS ProServe and working backwards from your regulators’ compliance bar, this enables wide adoption as engineers and developers will understand the ‘Why’ things need to be a certain way.
• Remember That the Team You Have Is the Team You Need. Recruiting new takes a very long time, so invest in the people in your organization. Training, hands-on management, and certification will make a significant difference.
• You Build It, You Support It — small 2 pizza teams that own what they build can be transformational for a business. Its one of the mechanisms Amazon uses to scale and innovate.
• Command and Control or Trust, But Verify approaches for your Engineers and Developers. Both have pros and cons, see step 12 for more.

Step 5 — Create Your Questions Parking Lot

The leadership team (everyone) will have lots of questions. Unfortunately, many hours will be wasted trying to answer them without the right folks in the room, and your progress could stall. Create a parking lot for these questions, and be sure to respect every single question on the list as you keep moving forward.

Pro Tip — The very best way to answer a lot of questions quickly is to arrange an executive briefing centre session with AWS. These sessions, which are always fascinating, enlightening, and exciting, can be best held in Seattle or can potentially be arranged for the country where you are based. Speak to your account manager to arrange a session, and we’ll work with you to answer all your questions or ping me.

Step 6 — Create Your 2-Pizza Cloud Engineering Team

Creating your holistic cloud engineering team, which will be hands-on with the AWS cloud, is critical. The word holistic is very important here. This team must comprise a cross-section of multiple skills types, including —

· Infrastructure Engineers, who understand the existing IP addresses, boundary security (firewalls), routing, server build standards, and a whole host of things in between.

· Security Engineers, who will ensure that everything is built and coded to meet your company’s security objectives.

· Application Engineers, who will ensure that the coding logic of the product you’re building gets built.

· Operations Engineers, who will ensure that your ITIL elements can be adapted to benefit from the cloud.

· A Lead Architect, who has deep and broad domain experience. Ideally, this person will also have experience with infrastructure as code. A solid understanding of how to use AWS services and features in an optimal way will also greatly accelerate your cloud journey. This cloud engineering team should work together in one physical group. Remote working, while possible, isn’t optimal. And the team must be fully dedicated to your organization’s cloud journey, side of the desk just won’t work.

Step 7 — Bring in a Partner or AWS ProServe

The cloud engineering team will probably have valid opinions in terms of best approaches and best tools. And it will probably have strong feelings about which practices to keep from your data center and which to discard. To accelerate this process, bring in some experts who have been there and done it before.

Step 8 — Work Backwards From Your Security, Compliance and Availability Objectives

First and foremost, the AWS cloud is secure. Taking time to ensure that the Cloud Engineering Team and the Cloud Business Office understand the AWS Shared Responsibility Model is a crucial priority though. (Illustration below) Then, work with your AWS Solutions Architecture/ProServe resource to ensure that you’re using the Deep Security tools appropriately to meet your security objectives. There are a number of different configuration possibilities available, but my advice is to work backwards from your company’s external regulatory bar (PCIDSS, HIPAA) etc. You should also work with AWS to ensure that you’ve adopted the best practices that meet your compliance and security objectives. Once you agree on these, write them down, publish them, and make sure there’s a direct channel to the leadership team and single-threaded leader so people can cordially challenge them if they want.

Step 9 — Ship Something to Production That Is Important, But Not Critical

You need to get something that’s meaningful into production. When I was doing this at Capital One, getting the first micro service live was the team’s goal. Don’t set a deadline; instead, set a Minimum Viable Product (MVP) and have the Cloud Engineering Team stay focused on the product. Experience has shown that shipping something live in this way can take anywhere from a couple of days to 12 weeks. If it’s taking longer than 12 weeks, one of the earlier steps isn’t working, so have a retrospective and utilize the 5 Why’s to understand why.

Step 10 — Train, Gain Experience, and Certify Your Teams

The key role of the CCoE is to ensure that the people journey for everyone is managed positively and proactively. It’s also crucially important that you put in place the right training and certification programs to enable scaling. I cover this comprehensively here.

Step 11 — Start Migrating — “Plans Are Worthless, But Planning Is Everything” — Dwight D. Eisenhower

Once you have multiple teams you can really start thinking about migrating. And, as teams realize how easy it is to build on AWS, building new on Cloud becomes the default. But what about all the incumbent systems that still require a staggering amount of 24x7x365 maintenance and upgrades? This is a good place for the Cloud Business Office to work with AWS on using the AWS Migration Acceleration Program (MAP) which has been shaped by all the customers who have migrated to AWS. To shape your migration journey, use the 6 R’s, which is a simple looking, yet comprehensive decision guide on how best to migrate your apps. In its simplest guise, using 6 colours of Sticky Notes I have worked with leaders in a day, and have got to an 80% Straw Man Proposal that can help towards generating a Directional Business Case for MAP. The best programmes are continually planning to make the maximum use of Re-Host, some Re-Platforming and a little Re-Architecting to get there quickly alongside a MAP Partner.

Step 12 — Trust, But Verify

Finally, the question that many larger enterprises come back to time and time again is “How do I balance control (especially security) and innovation?” It’s a tough question to definitively answer. At Capital One Cloud Custodian, which allows administrators and users to easily define policy rules for a well-managed cloud infrastructure that’s both secure and cost-optimized worked incredibly well. My good friend and ex-Capital One colleague Kapil Thangavelu talks here on the this great Open Source project for which he is the Product Manager. It was fascinating to hear 3M talk at re:invent 2017 about how they are leveraging the Cloud Custodian tool to help them with their Governance and get their setup just right.

Remember — “All of your assumed constraints are debatable.”

Jonathan Allen
@jonatallen
jnatall@amazon.co.uk
EMEA Enterprise Strategist and Evangelist

12 Steps to Get Started With The Cloud was originally published in AWS Enterprise Collection on Medium, where people are continuing the conversation by highlighting and responding to this story.