Showing posts with label Guides. Show all posts
Showing posts with label Guides. Show all posts

Tuesday, 14 February 2017

PowerShell DSC Extension in Azure Resource Manager - PowerShell Method

Here are the steps I took to get the PowerShell DSC Extension for Azure Virtual Machines working using the PowerShell publishing method. In another post, I'll go through the JSON template method.

First, create an Azure VM using whichever method you prefer. I used the portal for simplicity. Most of my settings won't affect this, but for completeness, the settings I used for my VM are as follows:

Image: Windows Server 2016 Datacenter
Deployment Model: Resource Manager
Name: MyCoolVM
Disk Type: HDD
User name: Dave
Password: hunter2
Resource Group (New): DSC-Test-Resources
Location: North Europe
Size: A1 Basic
Settings Tab: All defaults except monitoring disabled.
Storage Account Name: dsctestresourcesdisks871

After creation, I opened the VM's network security group and added an inbound rule to allow port 80. I also added a DNS alias to the machine's public IP.

Once the VM is set up, open a text editor and create the following example DSC file:

In the same folder, make a deployment script as follows: 



The first command will publish the DSC configuration into a Zip file in your Azure storage account and return a predictable URL. The resulting blob will not be public.

Mine ended up here, but you can see the format which appears to use the normal blob URI and the powershell-dsc container:
https://dsctestresourcesdisks871.blob.core.windows.net/windows-powershell-dsc/dsc-webserver.ps1.zip

A few seconds after the second command executes, you can see the new extension in your VM in the portal.



The status will move from (unavailable), to Transitioning, to Provisioning succeeded (or failed.)

If you click the extension you can get more details on the process, you can view the entire log from this blade.




And here is the finished web server:




Start to finish, including manual creation of the VM, this process took about 20 minutes.

If you use additional non-standard DSC resources, the PowerShell publish command will include the modules in the uploaded Zip file which is super handy.






References:
Introduction to the Azure Desired State Configuration extension handler

Tuesday, 17 January 2017

Azure Resource Manager (ARM) Templates Getting Started Guide

Preface
I've been doing some research and development work on ARM templates recently and thought I would put together a getting started guide while I was at it.

There are some great online references for ARM template creation:





Getting Started

The basics
ARM templates are JSON formatted files that describe Azure Resources. You can send the file off to Azure in a number of ways and the resources described will be implemented by the Service. ARM resources are deployed in parallel, so even complicated templates can be deployed quickly.

The ARM template is broken down into four sections: Parameters, Variables, Resources and Outputs. Only one section is actually required to have content - resources - the others help to make the templates more dynamic and flexible.

I use Visual Studio Code for creating and editing ARM templates as it has some excellent plugins which provide intellisense when creating resources. Using this means that you can just hit Ctrl+Space to get a hint at the possible resource properties on the fly. To set up VS Code for intellisense, the official guide is here.

You can also download templates from the Azure Portal to edit later. You can download templates from created resources, from the last page of the resource creation process, or even from entire resource groups.

Here is a really basic, valid ARM template to create a Storage Account. It's pretty rigid in that the storage account name is hard-coded, to fix that, parameters can be used to take user input.



Execution
ARM templates can be deployed in a number of ways: PowerShell, X-plat CLI and the Azure Portal are some common ways.

The following PowerShell code will create a Resource Group and deploy the template into it.



Taking input
The parameters section allows the template to take input. This input can be from the user at deployment time, from a parameters answer file or from another template. A basic parameter looks like this:


Storing Variables and Transforming input
The variables section is useful for holding regularly used values or for transforming input. For example in some of my templates, I use a resources prefix which is concatenated with resource names so that related resources can be seen at a glance.

The example includes a variable to apply a tag to a resource. In this example there is only a single resource, but in more complicated templates, the variable can be reused in all of the resources.



Producing Output
It can be useful to produce outputs from a template to get information on the resources that were created. In this example the DNS name assigned to the public IP is returned after deployment. Outputs can also be used in chained templates.


Creating multiple copies of a resource
A really powerful feature of ARM templates is the copy loop. This means you can create multiple, identical copies of specific resources in your template. For example you can create 3 copies of a VM using one code block. When you use a copy loop, you can reference the copyIndex() function in order to generate unique names for resources. In the following example, there is a simple copy loop to generate 3 public IPs. 



Using arrays in loops
Arrays can be combined with the copy loop in order to name resources with a list of predefined or runtime input names. In the example, the array is generated as a parameter and then indexed using the copyIndex() function. The count of the copy property can be set to the length of the array so that all array elements are indexed dynamically.


Running scripts in VMs using the extension resource
A nice part of Azure Resource Manager is the ability to add virtual machine extensions. One such extension is the script extension which allows the Resource Manager service to run scripts inside a virtual machine. No external access is required to the virtual machine since Azure executes the script for you. Scripts need to be hosted on an accessible web server or storage account. If there is any sensitive information in the script you should use storage keys and a key vault to access the script privately from the template. The following example just runs a publicly accessible script in the virtual machine 'myVirtualMachine'.


Linking to other templates for complex solutions
Chaining templates is super useful for creating complex solutions. For instance you can create a template to build an availability group of VMs, then call that template from another template with the copy loop and parameters to build multi-tier applications. 

I have a full implementation of this on my GitHub, the implementation will loop through several templates and create multiple availability groups and VMs. Be careful if you run this as it will spawn a lot of resources.

Here is the 'master' template which calls the other templates in the solution:



And here are all the resources it creates!




Monday, 5 December 2016

PowerShell DSC Getting Started Guide - Part 2, Pull Servers

Part 1 - Pushing a configuration and Credentials
Part 2 - Pull Server Setup
Part 3 - Custom Scripts
Part 4 - Partial Configurations.

Pull Server Configs - SMB / HTTP / HTTPS

A DSC Pull Server can be configured using SMB by creating a fileshare that allows the machine account of the clients to read the share. Simply dropping configurations in the correct format and configuring the LCM will allow clients to pull a configuration from the server. The share should be secured to only allow trusted admins to read and write data.

Alternate methods are HTTP and HTTPS which are identical except for TLS. HTTP configuration is not recommended since all communications are done in the clear and configuration data is sensitive. HTTP also opens the possibility for man in the middle attacks.


Configure HTTPS pull server

An HTTPS pull server can be bootstrapped using a DSC configuration with the xWebService DSC module.

The full MSDN guide can be found here.

The following .ps1 file will create and configure a web server as a DSC pull server on the machine it's run on. This will be using a self signed certificate, so should not be used in production.

Once this has completed, it should write on screen the thumbprint for the new self-signed certificate. This is required to configure the client.




Configure HTTPS Pull Client

To configure the pull client, the LCM on the machine needs to be set up to collect it's configuration from the new pull server. To do this run the following DSC configuration, then apply it with the Set-DSCLocalConfigurationManager command. Make sure to note the certificate thumbprint from the pull server creation script and add it to the client configuration below.



Then run:

  Set-DSCLocalConfigurationManager -ComputerName dscclient.example.com -Path C:\DSC\PullClient -Verbose


Create and deploy a configuration on the Pull Server

Once the client is configured, a configuration for it can be created and added to the pull server. There are 2 methods for adding configurations to a pull server. The configuration .mof files can be named the same as the GUID configured in the LCM, or the configurations can be named friendly names and referenced in the LCM. The friendly names require registration of the pull client with the server. Further information can be found here.

Configurations held on a pull server also require a checksum to be created for them. This is shown a bit further down.

The above LCM configuration uses the GUID method. When it is run it should write to the screen the GUID for the client as configured. Be careful as the script will create a new GUID if run again.

To set up a basic configuration to be pulled to the client, create and run the following configuration:



Once the mof file is created, run the following script on the pull server to move the mof file to the correct location, name it appropriately and create the required checksum file.


It's now set up for the client to pull the config. You can force the client to pull the config and watch the results with the following command:

  Update-DscConfiguration -ComputerName dscclient.example.com -Wait -Verbose 

To run the config after download run:

  Start-DscConfiguration -ComputerName dscclient.example.com -Wait -Verbose -Force -UseExisting

Part 3 - The Custom Script Resource

Additional useful commands:

Remove pending configuration after seeing the error:

Warning "LCM state is changed by non-DSC operations"

Remove-DscConfigurationDocument -CimSession 'ComputerName' -Stage Pending -Force


Saturday, 3 December 2016

PowerShell DSC Getting Started Guide - Part 1, Pushing a configuration and Credentials

Part 1 - Pushing a configuration and Credentials
Part 2 - Pull Server Setup
Part 3 - Custom Scripts
Part 4 - Partial Configurations.

What is PowerShell DSC

PowerShell DSC - Desired State Configuration - is a declarative platform for provisioning Windows (and non Windows) machines with configuration information. Under the covers, there is PowerShell code which enforces the configuration on the machine.

PowerShell DSC can be used to install and configure software and enforce configuration of the Operating System.

PowerShell DSC is idempotent, which means a configuration can be run against a machine over and over and if the machine is already in the described state, then nothing will be changed.

Included in-box are 'modules' which contain 'resources' that allow installation of packages and Windows features, copying of files and folders, ensuring registry entries are set to specific values and more.

A very basic PowerShell DSC configuration, using the file resource looks a lot like a normal PowerShell function. It can also take parameter values like a function:


Running the above configuration creates a .mof file for the 'localhost' computer in the folder C:\DSC\BasicConfig. To push this configuration, execute the following command:

  Start-DscConfiguration -ComputerName 'localhost' -Path C:\DSC\BasicConfig -wait -verbose -force

During execution, DSC will ensure C:\Temp exists. Running the same configuration again will result in no change, since the folder already exists.

Omitting the -wait switch will create a PSJob and run the configuration in the background, omitting -verbose reduces the output from the configuration job and -force just tells DSC to ignore any other jobs in progress and force this one to run.



Modules, Resources and Extending PowerShell DSC

View installed resources and modules:

  Get-DscResource

Finding resources on the PowerShell Gallery can be done directly from PowerShell with:

  Find-Module

Resources can then be installed with:

  Install-Module

Quick syntax information for a resource:

  Get-DSCResource -Name User -Syntax

The PowerShell Gallery hosts a ton of available DSC resources which can be downloaded directly.
PowerShell Gallery

The PowerShell GitHub page has source code for many of the first party DSC modules. Viewing the source for a module can aid debugging a configuration considerably.

PowerShell GitHub



The Local Configuration Manager (LCM)

The LCM runs on all versions of WMF 4 and above, it is responsible for getting and applying the configuration to the machine. The configuration of the LCM can be queried with:

  Get-DscLocalConfigurationManager



Modes: Push / Pull

PowerShell DSC supports two modes of getting a configuration to a client. 

In push mode, the LCM will apply a configuration sent to it using the Start-DscConfiguration command.

In pull mode, the LCM is set up to check a server for it's configuration, download and apply it. 



Push a basic configuration to another machine

DSC Client Config

✔ WinRM needs to be set up and working as a prerequisite to this, to make this easy both machines should be a member of the same domain. 


✔ WMF 5.0 installed.

DSC Push Workstation Config

Create and execute the following .ps1 file



This will create a .mof file for dscclient.example.com and save it in C:\DSC\BasicPush.

Pushing the configuration can be done with the following command:

  Start-DscConfiguration -ComputerName 'dscclient.example.com' -Path C:\DSC\BasicPush -wait -force -verbose



Set up certificates for credential usage

For most non-trivial configurations, it's likely that credentials will be required for access to installation packages or AD for example.

Passing credentials with DSC requires a special encryption certificate to be created, with the private key installed on the remote client and the public key accessible on the workstation where the configurations are created.

Executing a configuration will encrypt the credential with the public key so that the remote machine can decrypt and use it with it's private key. 

It's worth remembering that anyone with access to the private key can decrypt the credential from the stored mof file. Mof files and private keys should be secured appropriately. Anyone with admin level access on the client will have access to the private key in the certificate store and therefore be able to decrypt the password. Any credentials used should have the minimum amount of rights to get the job done.

Credential management appears to be broken between versions 4 and 5 of the WMF. I would therefore advise installing the production release of WMF 5 on Server 2012 R2 and Windows 8.1 to ensure credentials can be encrypted and recovered properly.

To create the certificate, you can use Windows PKI and create a special template. I have detailed this process in another blog post here. 

An interesting alternative, that I haven't tried myself may be to use the xDSCUtils module to bootstrap with self-signed certs. 



Push a configuration with credentials

Prerequisites for the client

✔ Generate the certificate and add it to the local computer personal store on the client ( cert:\LocalComputer\My )

From your push workstation

✔ Run the following PowerShell script to export the certificate public key to the local machine. This will store the key in a local .cer b64 encoded file. The script will also generate 'configdata' for the push command.

  New-DscClient.PS1 -ComputerName 'dscclient.example.com'


✔ Generate the configuration .mof file with the following script



✔ Configure the LCM on the remote machine to use the correct certificate. The LCM configuration information was generated in the mof file using the LocalConfigurationManager section in the DSC-InstallApp.PS1 above. The command below will push the LCM configuration to the remote machine:

  Set-DscLocalConfigurationManager -Path C:\DSC\CredentialPush -verbose

✔ Execute the configuration on the remote machine.

  Start-DscConfiguration -Path C:\DSC\CredentialPush -verbose -Force -Wait

✔ It's worth opening the .mof file in a text editor so that you can see the encrypted credential in the file.



Continued in part 2 - Pull Server Setup

Nutanix CE 2.0 on ESXi AOS Upgrade Hangs

AOS Upgrade on ESXi from 6.5.2 to 6.5.3.6 hangs. Issue I have tried to upgrade my Nutanix CE 2.0 based on ESXi to a newer AOS version for ...