AzureVM is a package for interacting with virtual machines and virtual machine scalesets in Azure. You can deploy, start up, shut down, run scripts, deallocate and delete VMs and scalesets from the R command line. It uses the tools provided by the AzureRMR package to manage VM resources and templates.
The primary repo for this package is at
https://github.com/Azure/AzureVM; please submit issues and PRs there. It
is also mirrored at the Cloudyr org at
https://github.com/cloudyr/AzureVM. You can install the development
version of the package with
devtools::install_github("Azure/AzureVM")
.
Here is a simple example. We create a VM using the default settings, run a shell command, resize the VM, and then delete it.
library(AzureVM)
<- AzureRMR::get_azure_login()$
sub get_subscription("sub_id")
# calling create_vm() from a subscription object will create the VM in its own resource group
# default is an Ubuntu 18.04 VM, size Standard_DS3_v2, login via SSH key
# call sub$list_vm_sizes() to get the sizes available in your region
<- sub$create_vm("myubuntuvm", user_config("myname", "~/.ssh/id_rsa.pub"),
vm location="australiaeast")
# some resources used by the VM
$get_vnet()
vm$get_public_ip_resource()
vm$get_disk("os")
vm
# run a shell script or command remotely (will be PowerShell on a Windows VM)
$run_script("echo hello world! > /tmp/hello.txt")
vm
# ... and stop it
$stop()
vm
# ... and resize it
$resize("Standard_DS4_v2")
vm
# ... and delete it (this can be done asynchronously for a VM in its own group)
$delete() vm
AzureVM comes with a number of predefined configurations, for deploying commonly used VM images. For example, to create an Ubuntu Data Science Virtual Machine accessible via SSH, JupyterHub and RStudio Server:
$create_vm("mydsvm", user_config("myname", "~/.ssh/id_rsa.pub"), config="ubuntu_dsvm",
sublocation="australiaeast")
And to create a Windows Server 2019 VM, accessible via RDP:
$create_vm("mywinvm", user_config("myname", password="Use-strong-passwords!"), config="windows_2019",
sublocation="australiaeast")
The available predefined configurations are ubuntu_18.04
(the default), ubuntu_16.04
, ubuntu_dsvm
,
windows_2019
, windows_2016
,
windows_dsvm
, rhel_7.6
, rhel_8
,
centos_7.5
, centos_7.6
,
debian_8_backports
and debian_9_backports
. You
can combine these with several other arguments to customise the VM
deployment to your needs:
size
: VM size. Use the list_vm_sizes
method for the subscription and resource group classes to see the
available sizes.datadisks
: The data disk sizes/configurations to
attach.ip
: Public IP address. Set this to NULL if you don’t
want the VM to be accessible outside its subnet.vnet
: Virtual network/subnet.nsg
: Network security group. AzureVM will associate the
NSG with the vnet/subnet, not with the VM’s network interface.nic
: Network interface.other_resources
: Optionally, a list of other resources
to deploy.# Windows Server 2016, with a 500GB datadisk attached, not publicly accessible
$create_vm("mywinvm2", user_config("myname", password="Use-strong-passwords!"),
subsize="Standard_DS4_v2", config="windows_2016", datadisks=500, ip=NULL,
location="australiaeast")
# Ubuntu DSVM, GPU-enabled
$create_vm("mydsvm", user_config("myname", "~/.ssh/id_rsa.pub"), size="Standard_NC12s_v2",
subconfig="ubuntu_dsvm",
location="australiaeast")
# Red Hat VM, serving HTTP/HTTPS
$create_vm("myrhvm", user_config("myname", "~/.ssh/id_rsa.pub"), config="rhel_8",
subnsg=nsg_config(list(nsg_rule_allow_http, nsg_rule_allow_https)),
location="australiaeast")
Full customisation is provided by the vm_config
function, which also lets you specify the image to deploy, either from
the marketplace or a disk. (The predefined configurations actually call
vm_config
, with the appropriate arguments for each specific
config.)
## custom VM configuration: Windows 10 Pro 1903 with data disks
## this assumes you have a valid Win10 desktop license
<- user_config("myname", password="Use-strong-passwords!")
user <- image_config(
image publisher="MicrosoftWindowsDesktop",
offer="Windows-10",
sku="19h1-pro"
)<- list(
datadisks datadisk_config(250, type="Premium_LRS"),
datadisk_config(1000, type="Standard_LRS")
)<- nsg_config(
nsg list(nsg_rule_allow_rdp)
)$create_vm("mywin10vm", user,
subconfig=vm_config(
image=image,
keylogin=FALSE,
datadisks=datadisks,
nsg=nsg,
properties=list(licenseType="Windows_Client")
),location="australiaeast"
)
The equivalent to create_vm
for scalesets is the
create_vm_scaleset
method. By default, a new scaleset will
come with a load balancer and autoscaler attached, but its instances
will not be externally accessible.
# default is Ubuntu 18.04 scaleset, size Standard_DS1_v2
$create_vm_scaleset("myubuntuss", user_config("myname", "~/.ssh/id_rsa.pub"), instances=5,
sublocation="australiaeast")
Each predefined VM configuration has a corresponding scaleset
configuration. To specify low-level scaleset options, use the
scaleset_options
function. Here are some sample scaleset
deployments:
# Windows Server 2019
$create_vm_scaleset("mywinss", user_config("myname", password="Use-strong-passwords!"), instances=5,
subconfig="windows_2019_ss",
location="australiaeast")
# RHEL scaleset, serving HTTP/HTTPS
$create_vm_scaleset("myrhelss", user_config("myname", "~/.ssh/id_rsa.pub"), instances=5,
subconfig="rhel_8_ss",
nsg=nsg_config(list(nsg_rule_allow_http, nsg_rule_allow_https)),
location="australiaeast")
# Ubuntu DSVM, GPU-enabled, public instances, no load balancer or autoscaler
$create_vm_scaleset("mydsvmss", user_config("myname", "~/.ssh/id_rsa.pub"), instances=5,
subsize="Standard_NC6", config="ubuntu_dsvm_ss",
options=scaleset_options(public=TRUE),
load_balancer=NULL, autoscaler=NULL,
location="australiaeast")
# Large Debian scaleset (multiple placement groups), using spot VMs (low-priority)
# need to set the instance size to something that supports low-pri
$create_vm_scaleset("mylargess", user_config("myname", "~/.ssh/id_rsa.pub"), instances=10,
subsize="Standard_DS3_v2", config="debian_9_backports_ss",
options=scaleset_options(priority="spot", large_scaleset=TRUE),
location="australiaeast")
Working with scaleset instances can be tedious if you have a large scaleset, since R can only connect to one instance at a time. To solve this problem, AzureVM can leverage the process pool functionality provided by AzureRMR to connect in parallel with the scaleset, leading to significant speedups. The pool is created automatically the first time it is needed, and is deleted at the end of the session.
# this will create a pool of up to 10 processes that talk to the scaleset
$run_script("echo hello world! > /tmp/hello.txt") mylargess
You can control the size of the pool with the global
azure_vm_minpoolsize
and azure_vm_maxpoolsize
options, which have default values 2 and 10 respectively. To turn off
parallel connections, set options(azure_vm_maxpoolsize=0)
.
Note that the pool size is unrelated to the scaleset size; it
only controls how many instances can communicate with AzureVM
simultaneously.
You can also include an existing Azure resource in a deployment, by
supplying an AzureRMR az_resource
object as an argument in
the create_vm
or create_vm_scaleset
call. For
example, here we create a VM and a scaleset that share a single virtual
network/subnet.
## VM and scaleset in the same resource group and virtual network
# first, create the resgroup
<- sub$create_resource_group("rgname", "australiaeast")
rg
# create the master
$create_vm("mastervm", user_config("myname", "~/.ssh/id_rsa.pub"))
rg
# get the vnet resource
<- rg$get_resource(type="Microsoft.Network/virtualNetworks", name="mastervm-vnet")
vnet
# create the scaleset
# since the NSG is associated with the vnet, we don't need to create a new NSG either
$create_vm_scaleset("slavess", user_config("myname", "~/.ssh/id_rsa.pub"),
rginstances=5, vnet=vnet, nsg=NULL)