Introduction and Background
DevOps, DevOps, DevOps. I personally think DevOps is a requirement in most of the projects, involving software development. Yet, very little amount of people truly understand what DevOps has to offer, or what DevOps itself is. As to what DevOps is, I would leave that part to a future post, however, in this post, I will try to give you an overview of what DevOps has to offer and how you can kickstart your DevOps chains on Microsoft Azure. In this post, we will cover a couple of platform-level services offered by Microsoft and the Azure platform and see how we can build a complete toolchain—or better, how Microsoft Azure can help us build the DevOps pipeline on the services. Our services would include, but won't be limited to,
- Visual Studio Team Services
- Azure App Service
- Azure Application Insights
We will then move ahead and demonstrate how to kickstart an application of any framework or runtime, and how to configure the DevOps pipelines—including but not limited to the continuous integration, delivery, and deployment. The practical part of this post will cover what Azure has to offer for the teams to implement the DevOps, and the theoretical part will cover what DevOps is and why it matters!
I would appreciate if you are aware of DevOps, and what DevOps means to you and whether your teams require DevOps or not.
What is DevOps?
Let's quit the too typical of a definition for DevOps, stating that DevOps is a philosophy and a concept blending the tools and mindset of the software engineers. The problem with this is, where are the operations involved? Or to be precise, what degree of intervention is allowed for a software engineer to make in the operations? Let's redefine the concept, but first, the main concepts and characteristics of DevOps, that make it an important pillar in the modern software engineering are -
- Reduce organization silos.
- Accepting—if not embracing—failure.
- Implement gradual change.
- Leverage tools and automation.
- Measure everything.
I captured the list above from Google's Cloud Platform blog because I wanted to take this from someone who knows what DevOps is, and how to implement DevOps too, on a massive scale. Something similar can be found at Wikipedia, some at Microsoft and so on. The thing is these are the key elements of any DevOps practice. Google has an in-house job of Site Reliability Engineering. What most consider DevOps to be, is done by these SREs.
One more thing, instead of looking at DevOps in a graphical demonstration, how about we take another approach and present the DevOps as what it actually is; after all, we are talking about software engineers and developers, right!?
-
- interface DevOps
- {
-
-
-
- void GrantOwnership();
-
-
-
-
-
-
-
-
-
- void BearFailure(int threshold);
-
-
-
-
-
-
-
-
-
- void ReleaseProdution(int sprintNumber);
-
-
-
-
-
- void ConfigurePipelines();
-
-
-
-
-
- void Watch();
- }
Do they make any sense to you? So now, we visualize the fact that DevOps is like a contract, an interface that your processes have to implement before consuming it. There is no simple way of doing this, as you can see most of the steps are for the processes; building, deployment, testing, and most of these are the cultural changes; that include but are not limited to giving ownership, accepting the failure rather getting frustrated.
Now let's stay on the topic and discuss how Azure provides us with the tools and services required to setup a DevOps environment.
Azure DevOps Project
Microsoft had tools and services, for developers to develop applications and platforms for operations teams to deploy and manage the applications. For developers, tools like TFS, VSTS enable an unmatched productivity, and for operations teams, services and platforms like Azure App Service and Azure Application Insights enable a richer experience to deploy and scale apps, and monitor them as needed with as much of depth and control as needed by the enterprise.
The Azure DevOps Project is a continuum of these two platforms; App Service and VSTS. Since the service is in the preview, we cannot say what other services might join hands (most notably, Azure Container Service), but currently, the blend of these services is what gives us Azure DevOps Project. Previously, the steps needed to create and startup a simple pipeline for DevOps CI/CD on Azure and VSTS would require,
- VSTS Account with Azure Service Endpoint configured.
- Azure App Service setup with proper Deployment Options setup
Then to take more control, you might would also need to create a separate monitoring package, such as Application Insights. Although Azure supports Resource Groups but even that would require you to configure this yourself.
Then, finally, once all of this was done, now you still did not have a dashboard! A dashboard is the one-place store to visualize everything about your product/project/application. You can check the current status of the app, deployment state, current usage of the app, failure to success ratio and much more. Now let's dig into the practical part of this port and deploy a new DevOps pipeline and practically understand what benefits can Azure provide us, in terms of having our DevOps pipeline setup on the cloud.
Be sure to also check out Azure DevOps Project documentation and product details on Azure.
Creating the project
Like every other service, Azure DevOps Project also has to be deployed on the cloud, you can easily create a new resource. Just head over the Azure Portal, and search for "devops", you will be provided with a DevOps Project resource. That is the resource we are currently interested in. The page will show you a complete cycle of the pipeline, the interesting thing in this resource creation is that Azure shows you the pages and fields relevant to the selections you make. We are going to create a basic ASP.NET Core app and run it on a Windows based App Service, but the workflow will provide you with all major runtimes, their frameworks, different deployment models and services to use, such as Kubernetes, Virtual machines, Windows or Linux based App Services and much more.
If you would have started the resource creation process, you will find the following page on the first page; instead of the form for resource creation.
Figure 1 - DevOps project asking for the runtime to choose for the project.
As you can see, this project will be able to configure every kind of runtime and framework for you. Selection in one of these will lead you to supported selections further down the road. This will generate a resource template that will contain all the settings for this project, other options in our track would be,
Figure 2 - DevOps project asking for the framework to choose over the runtime.
One more thing to check here is that this is the place where a basic template project is created for you. That is why, Azure will also ask whether you would want to add a new database connection or not. A database connection can be used for accounts and other storage. We are going with ASP.NET Core, as ASP.NET Core exposes a lot of more platforms to deploy the app on,
Figure 3 - DevOps project asking for the deployment model.
Azure is a huge platform of choices. Now that we have chosen ASP.NET Core, we can see that this platform can be deployed on a huge plethora of services.
- Kubernetes Orchestration for containers, your app will be packaged and shipped as an image on Azure, where orchestration will be taken care of—we will check this in a later module.
- Service Fabric apps are also orchestrated and managed.
- Web App on Windows and Linux are two offerings, your app's structure will be the same but the platform will be different.
- Web App for Containers is a Linux-based platform offering, to run your images in a serverless container platform. You can read my other articles to learn how to deploy an image on a Web App for Containers.
- Virtual machine then gives you an IaaS offering to deploy your apps on Azure.
In this sample, we are going to discuss how to publish the apps on Web App and use Windows as the offering—remember, Linux based App Service has a better performance and good latency too! I am selecting just for the same of simplicity.
Figure 4 - DevOps project asking for the VSTS project as well as the naming for Azure resources.
This is the page where we are shown with the configurations. As seen, we need to have a Visual Studio Team Services account (or we can always create a new one here), and then the last thing to do is to create the platform hosting service as well where we can host the app. Now in this, we have a dev-to-production chain built and setup using Visual Studio Team Services and Azure App Service. Enter these settings and you are done.
DevOps pipeline
A typical DevOps pipeline will contain steps to,
- Build (integration)
- Test and package (delivery)
- Deploy (deployment)
Most of the times, building and testing is done in a single-go, causing the delivery step to blur out. However, these are the steps taken to perform a single operation in a DevOps pipeline. At the boundaries of these steps are the steps taken to,
- Design and plan the sprint
- Code the features and stories
- Monitor and app after deployment
Everything is incorporated in a single cycle, and that starts from the inception of the app, all the way down to the lifecycle of the app, til retirement of the product or launch of the next phase.
| Feature | Visual Studio Team Services | Azure App Service | Azure Application Insights |
| Reduce Silos | VSTS offers IAM and management, to allow other members to contribute and collaborate in a repository. Kanban boards, Scrum backlogs etc. are all visible to the owners, members and the stakeholders. | Azure AD can be used to be configured to allow the users from a domain to collaborate and participate in the deployment and management of a service on the Azure platform. | Azure Application Insights can generate reports that can be accessible from anywhere, even Power BI dashboards, or custom solutions. |
| Accepting Failure | Our failed tests although pose a serious threat, but we can check out how much a build fails and what we need to do about it. | | We can easily visualize the reports for our application, and check out how they are performing against a certain timespan. |
| Implement Gradual Change | You can define features and epics in a way that in one release the change is measurable for improvement or bugs. | It is recommended to submit the changes to a Deployment Slot, or run an A/B test on them so you know whether to keep or ignore the change. | Insights can help you measure the overall improvement or bugs and errors after a certain release. |
| Automation | VSTS enables you to use triggers in each step to autmatically start and keep the pipeline running. | | |
| Monitoring | | App Services supports basic insights with the help of Application Insights. | Application Insights provide a complete solution for monitoring and post-mortem of the apps. |
This table pretty much sums up the features that one can utilize in order to better optimize the rundown of a DevOps cycle. All these three services are created and are provided in a single dashboard for us to manage and visualize what is going on.
Figure 5 - Azure Portal showing the dashboard for the Azure DevOps project.
Now, this demonstrates what DevOps means on Azure. This dashboard gives us access to everything, including the current version control, release version currently running and the endpoint to access the application.
Triggering CI/CD
All that developers now need to know is the repository path, the reason being, we do not need to configure the build or release definitions; as they were created by Azure while we were creating the project. The definition holds configurations for ASP.NET Core building and packaging, and lastly does all the copying of the built artifacts.
Figure 6 - VSTS Build Definition for our project.
This automatically gets triggered and deploys the application to the platform, from where we can easily capture the currently build or release information in the dashboard.
Figure 7 - VSTS Build successful completion.
We can go ahead and use the online Visual Studio editing capabilities and trigger a change in the website, just going to change the title of the website and the front-page elements to show how this works. The change will automatically trigger the commit, inturn triggering the DevOps pipeline. Let's take a look.
Figure 8 - Code being changed to trigger a commit for CI/CD.
Commiting the changes would trigger the CI,
Figure 9 - Commiting the changes.
Now, at the backend, integration is running and will build the the project.
Figure 10 - Build outputs being printed on the screen.
That will now trigger deployment, and will have the application deployed on Microsoft Azure's App Service. This still was not a good practice, because we had deployed the application directly on the production slot. You should consider pushing this to the Deployment Slots and have them tested before exposing that to the wider audience. Read this article of mine on the Microsoft MVP Team's blog to learn how to do this. Lastly, the website gets deployed and the traffic will start to be shown on the dashboard; drilling down is possible.
Figure 11 - Site updated with the DevOps pipeline.
Check the updated dashboard that shows us what is happening in the product,
Figure 12 - Dashboard showing updated information, requests, code status.
So, that was the way a DevOps based project is run, managed and measured.
Monitoring
Lastly, the monitoring of the application happens in the Application Insights, which itself is a complete platform. So, I will show you a dashboard of my personal website that I had hosted on Azure and had some insights to be shown.
The dashboard shows everything from usage, to active users, to requests, to latency all the way down to errors and resource consumption. Take a look at the dashboard,
Figure 13 - My own website's stats being shown as an example of a dashboard.
Using this our operations team can easily check what is going on. When the latency goes up, and when an exception is raised. The dashboard for insights has a lot more to offer, including management of sessions, how much time a user spends, where users are interacting and the geographical information; which is not shown here and is beyond the scope of this article.
This was it for the article. I hope you have learned something out of this. There are many things that we missed because they would increase the overall size of the article and might take us off-the-track from what we had to talk about; DevOps on Azure. I will write another comprehensive article on the orchestration using Kubernetes as well.