Azure
Microsoft provides a generous (but limited) annual gift of Azure credits to help support C3.ai DTI-sponsored research.
If you're interesting in using Azure resources for a current C3.ai DTI project, please contact the DTI team at help@c3dti.ai.
Private data
Microsoft Azure infrastructure can support your program of compliant stewardship of data in most scenarios. If your data requires special attention, please let us know when asking for Azure resources.
Azure Portal tips and links for DTI researchers
When you're logged in to the Azure portal, ensure that you've selected the appropriate Account and Directory for your Subscription. You can set these by clicking on your personal identifier in the upper right of the portal:
Azure Portal dashboards are called "Blades." Here are links to the Azure Portal and to some Blades that may help you manage your Azure use:
- Azure Portal
- Education | Courses Blade
- https://portal.azure.com/#blade/Microsoft_Azure_Education/EducationMenuBlade/classrooms
- This page will show your DTI Assigned credit and Consumed credit. Updated daily, not in real time.
- If this page does not show your credit status, you may have been added directly to your research group's Subscription by one of your research group's Azure coordinator. Please check with your research group's Azure coordinator and/or DTI staff at help@c3dti.ai.
- Resource groups Blade
- https://portal.azure.com/#blade/HubsExtension/BrowseResourceGroups
- View and manage your deployed Resources by Resource Group.
- Subscriptions Blade
- https://portal.azure.com/#blade/Microsoft_Azure_Billing/SubscriptionsBlade
- Ensure that all of your Subscriptions are shown to you by deselecting the box "Show only subscriptions selected in the global subscriptions filter"
- After selecting your DTI subscription, you may find the following Blades useful:
- | Activity log
- Browse and filter all Activity under your Subscription
- Note that the default view shows history from only the last six hours; change this selection to see older activity.
- | Access control (IAM)
- Grant fine-grained access to your Azure Subscription and Resources to other team members
- | Resources
- Browse, filter and manage your deployed Azure Resources
- | Usage + quotas
- View your current service instance count usage, your current service instance type quotas, and request quota increases.
- Plan ahead: it may take several days for your quota increase request to be approved.
- | Activity log
- Monitor Blade
- https://portal.azure.com/#blade/Microsoft_Azure_Monitoring/AzureMonitoringBrowseBlade/overview
- Utilities for evaluating your Azure use
- Monitor | Metrics Blade
- https://portal.azure.com/#blade/Microsoft_Azure_Monitoring/AzureMonitoringBrowseBlade/metrics
- Plot resource use over time
Research team users
If you are one of your research group's Azure coordinators, you can grant fine-grained access to your Azure Subscription and Resources to other research group members via the Access control (IAM) options in the Subscriptions Blade for your subscription.
Your research group's Azure coordinator(s) can view DTI Assigned credit and Consumed credit for your group's subscription in the Education | Courses Blade.
Spend management
For C3.ai DTI research groups with an Azure sponsorship, your team's designated Azure manager(s) can see your sponsorship's assigned credit and credit consumed via this link, once you are signed in to your Azure context:
https://portal.azure.com/#blade/Microsoft_Azure_Education/EducationMenuBlade/classrooms
The DTI team can can access (modestly) more-detailed daily reporting by Azure resource category; please contact help@c3dti.ai for assistance with spend management.
Azure training for C3.ai DTI awardees
Details to come; please contact help@c3dti.ai for assistance.
Notes on the Azure model
Azure Virtual Machines
Size and Region
Azure VM offerings are grouped into series, with each series optimized for distinct purposes – e.g., GPU, data protection, hyper-threading, compute, memory, and so on. Virtual Machine series descriptions are available here:
https://azure.microsoft.com/en-us/pricing/details/virtual-machines/series/
To browse more-detailed specifications of Azure Virtual Machines, see this page:
https://azure.microsoft.com/en-us/pricing/details/virtual-machines/linux/
Note that both pricing and VM size availability vary by region.
Disk Roles and Persistence of Data
Read about Data, OS and Temporary disks, including notes regarding data persistence, on this Azure documentation page:
https://docs.microsoft.com/en-us/azure/virtual-machines/managed-disks-overview#disk-roles
cloud-config: VM configuration in a text file
Many standard Azure VM images (including examples running Ubuntu, CentOS, and Windows) run the cloud-init service. This service allows you to submit a yaml-formatted cloud-config text file as part of the VM creation request. This cloud-config file encodes your set of VM configuration instructions, including software package installations. This enables you to define your VM's configuration in a text file, which can be documented, shared, re-used, and even managed under version control. DTI researchers are strongly encouraged to consider using the built-in cloud-init service in Azure VMs to help manage deployments.
To read more about using cloud-config files with Azure VMs, see this documentation:
https://docs.microsoft.com/en-us/azure/virtual-machines/linux/using-cloud-init
This page features many cloud-config file examples:
https://cloudinit.readthedocs.io/en/latest/topics/examples.html
Note: the cloud-init service is not Azure-specific, with support from other cloud providers as well.
Managing Azure resources
There are three distinct ways to manage (create, destroy, inspect, modify) your Azure resources:
1) Azure Web Portal
You can manage Azure resources via the Azure Web portal:
The Web portal provides a point-and-click GUI that offers quick and easy access. Note that other resource management methods (whether used alone or in combination with Web portal use) can provide better reproducibility, flexibility, and higher efficiency, particularly for complex and larger-scale deployments.
2) Command Line
There are several ways to engage Azure via the command line. Documentation for access, installation and use is available here:
https://docs.microsoft.com/cli/azure/
Note that you can install and run the Azure command line tool on a local machine or access it in a Web browser via Azure Cloud Shell:
https://docs.microsoft.com/azure/cloud-shell/
Cloud Shell also offers additional, complementary features; see documentation link immediately above.
3) Azure REST API
You can develop an application to interact with Azure to manage resources on your behalf via HTTP operations with Azure API service endpoints. More information is available in this documentation:
https://docs.microsoft.com/rest/api/azure/
Note that applications require configuration of an authentication token that represents appropriate role-based permissions. See documentation above or contact the DTI team at help@c3dti.ai.
Spend management
Spend reports (updated manually)