Monitor Azure Resources

Illustration

Summary

Monitor Azure Resources monitors the Land Spill resources on your Microsoft Azure® Batch account that are instantiated and used by the Run Cases on Azure tool.

To learn more about the Liquids HCA Tool in general, please see Liquids HCA Tool Frequently Asked Questions.

Usage

Each GeoClaw case submitted by Run Cases on Azure to your Azure Batch account corresponds to one task in the overall landspill-azure-job batch job. The individual tasks in the landspill-azure-job batch job are processed on the computing nodes of the landspill-azure-pool processing pool in your Azure Batch account. Monitor Azure Resources provides you with information about the status of the nodes in the landspill-azure-pool processing pool and about the tasks in the landspill-azure-job batch job. This information is portrayed in a separate task monitor window that runs as a spawned Python process, and which updates on a 30-second interval. Two examples of the task monitor window are shown in the above illustration. The example task monitor window on the left displays an example of active tasks being processed by a pool consisting of four nodes, with all four nodes running. In the example task monitor window on the right, processing is complete. Two donut graphs are displayed in the task monitor window. The top donut graph displays landspill-azure-pool processing pool status, including pool state, allocation status, and, in the donut graph itself, the state of the individual nodes in the pool. Note that the landspill-azure-pool processing pool is dynamic; nodes are instantiated on demand and exist only while there are tasks to process. The bottom donut graph displays landspill-azure-job job task status, including the total number of tasks in the job, and, in the donut graph itself, status of the individual tasks in the job.

You may exit the task monitor window at any time by clicking on the Close button in the upper right corner of the window. Once task processing is complete, you may download your GeoClaw results using the Download Cases from Azure tool.

Monitor Azure Resources is intended to provide a highly interactive but redacted view of the state of actively processing tasks. For a more detailed picture, you may wish to access Azure directly. To access your Azure portal account, visit portal.azure.com.

Monitor Azure Resources uses the following parameters:

  • Working Directory – This must be the same working directory that you used in running the Create GeoClaw Cases and Run Cases on Azure tools.
  • Azure Credential Method – You can either use the encrypted, passcode-protected Azure credential file that you created using the Create Encrypted Azure Credential File tool, or you can manually input your Azure Batch and associated Storage account credentials. When you select “Encrypted file” for this parameter, the following required dependent parameters are exposed:
    • Encrypted Credential File – This parameter is the path to your encrypted, passcode-protected credential file. By default, the tool looks for a credential file named azure_cred.bin  in the scratch folder for the active ArcGIS Pro project.
    • Encrypted Credential File Passcode – This parameter requires you to specify the passcode that you supplied when creating your encrypted Azure credential file. The encrypted Azure credential file can only be unencrypted for use when you supply this passcode. You should apply the same security measures for safeguarding this passcode that you apply to your Windows password.
  • The following required dependent parameters are exposed when you select “Manual input” for the Azure Credential Method parameter:
    • Azure Batch Account Name – Your Azure Batch account name.
    • Azure Batch Account Key – The primary or secondary access key for your Azure Batch account.
    • Azure Batch Account URL – The URL for your Azure Batch account.
    • Azure Storage Account Name – The name of the Azure Storage account associated with your Azure Batch account.
    • Azure Storage Account Key – Key1 or Key2 for the Azure Storage account associated with your Azure Batch account.

In a typical Liquids HCA Tool workflow, Monitor Azure Resources is run after Run Cases on Azure, during the period while Azure is actively processing tasks, and before Download Cases from Azure.

For visual reference on Liquids HCA Tool execution order, see Liquids HCA Tool Process Flow Diagrams.

Syntax

MonitorAzureResources_ (working_dir, cred_type, {cred_file}, {passcode}, {azure_batch_name}, {azure_batch_key}, {azure_batch_URL}, {azure_storage_name}, {azure_storage_key})

Parameter Explanation Data Type
working_dir

Dialog Reference

Specify the working directory storing your release point case folders.

There is no Python reference for this parameter.

Workspace
cred_type

Dialog Reference

Select the Azure authentication method.

There is no Python reference for this parameter.

String

cred_file

(Optional)

Dialog Reference

Select your encrypted, passcode-protected Azure credential file.

There is no Python reference for this parameter.

File

passcode

(Optional)

Dialog Reference

Specify the passcode for your encrypted, passcode-protected Azure credential file.

There is no Python reference for this parameter.

String Hidden

azure_batch_name

(Optional)

Dialog Reference

Specify the name of your Azure Batch account.

There is no Python reference for this parameter.

String Hidden

azure_batch_key

(Optional)

Dialog Reference

Specify your Azure Batch account key.

There is no Python reference for this parameter.

String Hidden

azure_batch_URL

(Optional)

Dialog Reference

Specify your Azure Batch account URL.

There is no Python reference for this parameter.

String Hidden

azure_storage_name

(Optional)

Dialog Reference

Specify your Azure Storage account name.

There is no Python reference for this parameter.

String Hidden

azure_storage_key

(Optional)

Dialog Reference

Specify your Azure Storage account key.

There is no Python reference for this parameter.

String Hidden

Code sample

The following script demonstrates how to use Monitor Azure Resources with encrypted passcode-protected file:

import arcpy
arcpy.ImportToolbox(r”C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\Lib\site-packages\liquidshca\esri\toolboxes\LiquidsHCA.pyt”)
in_dir = r”C:\data”
cred_type = “Encrypted file”
file_path = r”C:\data\azure_cred.bin”
passcode = “XXXXXXX”
batch_name = None
batch_key = None
batch_url = None
storage_name = None
storage_key = None
arcpy.liquidshca.MonitorAzureResources(in_dir, cred_type, file_path, passcode, batch_name, batch_key, batch_url, storage_name, storage_key)

Environments

This tool does not use any geoprocessing environments.

Licensing information

This tool requires a valid Liquids HCA Tool user license or subscription. Please see the Request License and Register License tool help topics for details on obtaining and registering a Gas HCA Tool software license.

Related topics

Tags

Liquids HCA, GeoClaw, Microsoft Azure, Batch account, Storage account, pool, node, job, task.

Credits

Copyright © 2003-2020 by G2 Integrated Solutions, LLC. All Rights Reserved.

geoclaw-azure-launcher:

Copyright © 2019-2020 Pi-Yueh Chuang, Lorena A. Barba, and G2 Integrated Solutions, LLC. All Rights Reserved.

Use limitations

There are no access and use limitations for this item.