ARM concepts in Azure Stack for the WAP Administrator – Packaging and publishing templates in Azure Stack

Hello Readers! This blog is part 5 of the series “ARM concepts in Azure Stack for the WAP Administrator.” In this post we’ll discuss how templates are packaged and published in Azure Stack Technical Preview 1 (TP1), compared to the concepts you may be familiar with today in Windows Azure Pack (WAP).

Note
Some information relates to pre-released product which may be substantially modified before it’s commercially released. Microsoft makes no warranties, express or implied, with respect to the information provided here.

I’m including the table of contents for this series of post so that you’ll find it easier to navigate across the series:

Table of contents

1. Introductory post, and some first information on the Azure Stack POC architecture and ARM’s role
   
2. Cloud Service Delivery
3. Plans, offers, and subscriptions
4. Resource Deployment
5. Packaging and publishing templates on Azure Stack – this post
6. Multi-tier applications with ARM
7. In-guest configuration with ARM, and technologies such as Virtual Machines Extensions, including PowerShell Desired State Configuration (DSC)
8. Troubleshooting IaaS deployments in Azure Stack—this maps to an understanding of how the different Resource Providers (RPs) work together in an Azure Stack installation
9. We will update the post with links to the new posts, as we publish them. With no more delay, let’s get started!
   

   ---

   Understanding templates differences in WAP and in Azure Stack

   When it comes to templates, Windows Azure Pack (WAP) is mainly dealing with Infrastructure as a Service (IaaS), through the use of “Standalone Virtual Machines” and “Virtual Machine Roles”.

   • Standalone virtual machines can be deployed from a sysprepped disk or a Virtual Machine Manager (VMM) template stored in the library. As their name implies, standalone virtual machines cannot be scaled out.
   • Virtual Machine Roles (VM Roles) are more advanced templates that let you to define specific end user questions in the deployment form and what should be executed post-setup. VM Roles represent a single tier that can optionally be scaled out, at or after deployment.

   For other resources supported by WAP (like SQL Server Databases or Web Sites), there are tenant-side APIs you can use, but there is no direct equivalent to VM Roles.

   In Azure Stack, Azure Resource Manager (ARM) templates provide a unified template approach for all services supported by Azure Stack. Today, in Technical Preview 1 (TP1), this means deploying Compute resources (Virtual Machines), Storage, Networking, Web Apps, SQL Server databases and MySQL databases.

   With ARM templates, you can deploy a single multi-tier applications encompassing resources from these different providers, and deploy this template consistently through the portal, PowerShell, CLI and the REST APIs (resource deployment was covered in the previous blog post in this series)

   When it comes to IaaS, there are a few features that you may not find available in ARM templates compared to WAP, and those generally fall under two categories:

   • Features not available because they are not supported in Azure today. With Azure Stack coming from Azure, we want to ensure we do not break this consistency promise with Azure Stack. This includes Generation 2 VM support or VHDX format support for virtual hard disks. There is a “User Voice” forum for Azure Stack, where you can vote for these items and other features that would be important to you.
   • Features not yet available in the current previews, but planned later in the cycle. This include VM Scale Sets for example.

   In the rest of this post, we will focus on the differences in template structure and publishing, between VM Roles in WAP and ARM templates in Azure Stack. So, even though the focus will be mainly around IaaS, remember that the ARM templates can also be used for other workloads too!

   ---

   Comparing the structure of VM Roles in WAP and ARM templates in Azure Stack

   VM Roles concepts are well explained in this blog post from MVP Marc van Eijk, and here is a summary of some key concepts:

   VM Roles are packaged with two files:

   • A mandatory “Resource Definition”, which describes which questions should be asked to tenants when deploying the VM Role gallery item from the gallery, and also details which commands should be run post-setup (including order, error handling, etc.). This resource definition maps 1:1 to the gallery items visible in WAP, as long as they’ve been made available for this specific plan:
   • An optional “Resource Extension”, containing the scripts or files the VM Role may need, when executing the customization commands inside the guest. Some VM Roles may not require this Resource Extension, for example if the resource definition just calls a built-in operating system command line to configure a setting or deploy a role/feature. At the other end of the spectrum, some VM Roles have quite large payloads. In these situations, the resource extension may only store scripts while the application payload may be stored in a data disk that the resource definition also deploys to the Virtual Machine. Finally, it is possible to have two resource definitions calling the same resource extension, if they need the same script/application payloads.

   When a VM Role is finalized, it consists of two files, one for the Resource Definition, and one for the Resource Extension:

   Those files are actually compressed archives, that you can open if you append the “.ZIP” extension. In there, you would see the JSON files, as well as the payload.

   So, how does this compare to Azure Stack?

   In Azure Stack, Azure Resource Manager (ARM) also uses a JSON template, which could conceptually be mapped to the Resource Definition. This template will define which resources are to be deployed, and defines the expected parameters and their optional default values, as well as some variables. The chosen deployment method (PowerShell, CLI, Visual Studio, REST API) will refer to this template to understand what the user should input, as explained in the previous blog post in this series.

   The concept of an IaaS Resource Extension does not directly map to ARM templates. The payloads that would be stored in resource extensions (and subsequently in VMM) with WAP, would generally be hosted in blob storage with Azure Stack. In Azure Stack, the in-guest customization commands would be called by VM extensions, calling scripts hosted in blob storage, or even pulling down bits from blob storage when needed. For example, the preview of the Web Apps Resource Provider does this during installation, pulling installers from blob storage.

   Note: In-guest customizations will also be covered in blog post #7 of this series (coming soon!). There are important aspects to it, leveraging the VM extensions, and technologies like Desired State Configuration (DSC). Those concepts are also important as a WAP administrator today, because it can help you design VM Roles that should be easier to port to Azure Stack later in the cycle.

   Note: Going into deeper details on ARM templates is out of the scope of this blog post, but there are many documentation articles already available on this topic.

   ---

   Publishing and deploying templates in WAP and in Azure Stack

   In Azure Pack, tenants can only deploy what the provider has published and made available to them. VM Roles have to be imported into the WAP admin portal (for the resource definition) and into Virtual Machine Manager (for the resource extension). Tools like the Gallery Resource Import Tool (GRIT) may help with this process, when needed. Once imported and made available through plans by the WAP administrator, the VM Role Gallery Items can then be deployed by tenants.

   In Azure Stack, it is also possible to package templates and customize how they appear in the portal, by publishing them to the Azure Stack Marketplace. An example can be seen with the “Contoso Items” category in the following screenshot:

   Every Marketplace item has an ARM template (for resource provisioning), metadata (like strings, icons), and formatting information (to display the item in the portal). Marketplace items use a format called the Azure Gallery Package (.azpkg extension). Deployment or runtime resources (like code, zip files with software, virtual machines, or disks) shouldn’t be included in the Azure Gallery Package, but would rather be stored in blob storage. The process to publish an item to the marketplace is explained in the documentation, and also in this blog post from MVP Christopher Keyaert. Once again, these templates could be a line of business application composed of virtual machines, websites and databases – and of any resource type supported by Azure Stack in the future.

   Also, in addition to the marketplace approach, tenants can also come with their own template and deploy them directly from the portal, using the “custom” option:

   Tenant templates can then be copied and pasted directly into the user interface from the “edit template” option, or could be retrieved from the Azure Stack GitHub repository. Using this approach, or when using PowerShell, CLI or Visual Studio, tenants are also able to use their own templates, enabling DevOps scenarios in a more seamless manner.

   Note that a tenant can also directly deploy images from the Platform Image Repository (PIR) – which would be very similar to executing a “quick create” to deploy a specific disk with Standalone VMs in WAP.

   It’s even possible for a tenant to upload his own image in a blob location, and use that in their own templates, as shown in this blog post from MVP Daniel Neumann.

   ---

   Conclusion

   We hope this latest post in our series was useful to help you map the concepts from Windows Azure Pack to Azure Stack. We’ve got more to come, including how we can work with multi-tier applications and with in-guest customization in both environments. Stay tuned!