Appendices

Prerequisite Packages

White-list the https://packages.microsoft.com/repos/azure-cli/ URL to allow the Quantum Fabric Containers on Azure Solution to download Azure CLI.

Network Settings - Access Azure SQL Database

Communication from the Temenos License servers originates from the following IP address, which must be white-listed in your organization's firewall configuration:

115.113.211.130

White-listing the Temenos IP address enables access to the Azure SQL Database.

Configure NAT Gateway

A network address translation (NAT) gateway is used to enable instances in a private subnet to connect to the internet or to other AWS services, but prevent the internet from initiating a connection with those instances. For more information about NAT, refer to the AWS documentation.

To enable NAT Gateway, set the AZURE_FIREWALL_ENABLED flag in properties file to true.

The Azure Firewall acts as a NAT Gateway. The internet traffic from Kubernetes nodes (VMs) is routed through the NAT Gateway. Deployment of the Azure Firewall increases the overall cost. The cost for Firewall in the east US region is $1.25/hour on Azure. On a monthly basis, the cost of azure firewall deployment costs around 900$. If you do not want to configure the NAT Gateway, disable the firewall deployment by setting the AZURE_FIREWALL_ENABLED flag to false.

Host your domain with Azure DNS

The Domain Name System, or DNS, is responsible to translate (or resolve) a website or service name to its IP address. Azure DNS is a hosting service for DNS domains, providing name resolution by using Microsoft Azure infrastructure. By hosting your domains in Azure, you can manage your DNS records by using the same credentials, APIs, tools, and billing as your other Azure services.

To create your first DNS zone and record by using the Azure Portal, follow these steps:

  1. Create a DNS Zone
    1. Sign into the Azure portal.
    2. On the Hub menu, navigate to Create a resourceNetworking, and then click DNS zone.
      The Create DNS zone page opens.
    3. On the Create DNS zone page, enter the following values, and then click Create:
    4. Setting Value Details
      Name customdomainname.com The name of the DNS zone
      Subscription [Your subscription] Select a subscription to create the DNS zone in.
      Resource group Create new: testresourcegroup Create a resource group. The resource group name must be unique within the subscription you selected.
      Location West US  

      NOTE: The resource group refers to the location of the resource group, and has no impact on the DNS zone. The DNS zone location is always global, and is not shown.

  2. Create a DNS Record
    1. In the Azure portal, navigate to DNS Zones, and then select customdomainname.com DNS zone.
    2. At the top of the DNS zone page, select + Record set to open the Add record set page.
    3. On the Add record set page, enter the following values, and then click OK. In the following example, you are creating a CNAME record.
    4. Setting Value Details
      Name www Name of the record.
      Type CNAME Type of DNS record that you want to create. Acceptable values are A, AAAA, CNAME, MX, NS, SRV, TXT, and PTR. For more information about record types, visit Overview of DNS zones and records.
      TTL 1 Time-to-live of the DNS request.
      TTL unit Hours Measurement of time for TTL value.
      IP Address Alias value This value is the Alias that the DNS record resolves.

  3. Update Name Servers
    After you set up the DNS zone and records, you must configure your domain name to use the Azure DNS name servers. This enables other users on the Internet to find your DNS records. The name servers for your zone are given in the Azure portal:

    The name servers must be configured with the domain name registrar (where you purchased the domain name). Your registrar offers the option to set up the name servers for the domain. For more information, refer to Delegate your domain to Azure DNS.

NOTE: Currently, Azure DNS does not support purchasing of domain names. To purchase domains, you must use a third-party domain name registrar. The domains can then be hosted in Azure DNS to manage the DNS records.

Generate a PFX file from PEM

Different platforms and devices require SSL certificates to be converted to different formats. Use the following command to generate the .pfx file from a .pem file:


$ openssl pkcs12 -inkey private.pem -in public.pem -export -out certificate.pfx -passout
 pass:xxxxxxxx -certfile more.crt

The code snippet uses the following commands and parameters:

  • openssl specifies the command to execute OpenSSL
  • pkcs12 specifies the file utility for PKCS#12 files in OpenSSL
  • -export -out certificate.pfx exports and saves the PFX file as certificate.pfx
  • -inkey privateKey.key uses the privateKey.pem file as the private key to combine with the certificate.
  • -in infile specifies the parameter for the input filename
  • -passout p specifies the output file to pass phrase source; this must be passed as pass:<password>
  • -certfile more.crt specifies any additional certificates you would like to include in the PFX file; this parameter is optional

Extract Logs from your Application

To extract logs from a container, execute the following commands from your terminal:

COMMAND LOG

$ kubectl config current-context

Display the current-context

$ kubectl config use-context <cluster-name>

Set the default context to the cluster-name if the current context does not point to the Quantum Fabric cluster.
$ kubectl get pods Lists all the pods
$ kubectl logs -f <pod-name> To tail logs from a specific pod
$ kubectl logs pod-name > logs.txt To extract the logs of a specific pod to local file

Connect to AKS nodes through Jumpbox

  1. Get the Public IP of the Jumpbox from the Azure Portal.
  2. Connect to the Jumpbox by using the following SSH command:
    $ ssh username@publicip

    NOTE: The default username is devops and password is Devops@12345.

  3. Get the Private IP of the node from the Azure Portal, which is required to sign into Azure Kubernetes node from Jumpbox.
  4. Use the private key of the public key that you provided during the Quantum Fabric installation. Copy the private key to Jumpbox. Ensure that you delete the key during logout.
  5. Sign into AKS node by executing the following command:
    $ ssh -i key.pem username@privateIpOfNode

    NOTE: The default username is azureuser.

Log Analytics

Microsoft Azure provides OMS (Operation Management Suite) Log Analytics solution to view the application logs. If you enable the flag for installation of log analytics solution (AZURE_LOG_ANALYTICS_ENABLED) in the properties file, then the Operations Management Workspace is created in the Azure Resource Group.

Follow these steps to view the application specific logs:

  1. Sign into the Azure portal.
  2. On the home page, select the resource group in which the AKS cluster is created. You can find the resource group (AZURE_RESOURCE_GROUP) in the properties file that was used in the Quantum Fabric installation.
  3. Select Containers solution.
  4. Select Container Monitoring Solution.
  5. Select CONTAINER LOGS.
  6. In the Log Search page, you need to execute the following queries to fetch the logs.
    • ContainerLog | where LogEntrySource == "stdout" | sort by TimeGenerated desc : To fetch all the logs.
    • ContainerLog | where LogEntrySource == "stderr" | sort by TimeGenerated desc : To fetch the error logs.

New Relic Monitoring

New Relic is a monitoring system that helps you understand the performance of the application, dependencies, bottlenecks, and also increases the efficiency and accuracy.

You can use New Relic to accomplish the following tasks:

  • Get a complete overview of the application and the operating environment
  • Manage your application's performance
  • Identify and troubleshoot bottlenecks
  • Analyze data about customers' usage and experience
  • Monitor your technical resources and ecosystem

Monitoring of Quantum Fabric on Azure using New Relic is performed at two levels:

  • Infrastructure Monitoring, which helps you to monitor node-level metrics such as CPU, Memory, Disk Usage, Average load, and I/O metrics.
  • Application Performance Monitoring, which helps you to view application performance trends such as page load times, error rates, slow transactions, and a list of servers running the application in real time. It helps to identify and troubleshoot the issues and metrics of Transactions, Databases, JVM's, and Error analytics of every pod.

Rolling Updates

Microsoft Azure provides rolling updates feature to update the deployment with zero downtime by incrementally updating the pod instances with new pods.

Using the rolling updates feature, you can perform the following actions:

  • Update kubernetes pod configuration.
    • Update the properties in the deployment section.
    • Update the -d parameters.
  • Increase or decrease the pod count.
  • Update the docker image of the container present in the pod.

Kubernetes Dashboard

You can access the Kubernetes dashboard in the Azure portal to view the information related to an application deployed in Azure Kuberntes cluster. Using the kubernetes dashboard, you can also view pod metrics such as CPU Usage, Memory Consumption, as well as Kubernetes objects like deployments, configmaps etc.

Pod Anti-Affinity

Pod anti-affinity is used to handle the creation of pods in worker nodes in Azure Kubernetes cluster for deploying a resilient application in the Kubernetes cluster. Certain rules are defined in a pod configuration which allows pod to be deployed only on a particular node when the required conditions are satisfied. No two pods of same kind will be deployed on a single node. Pod anti-affinity helps in distributing the pods across the cluster nodes and helps in creating resilient applications.

AKS Autoscaling

AKS autoscaling feature helps to scale the service in case of a spike or a drop in application traffic. Incase of a spike in traffic new pods need to be created and the cluster should create new worker nodes to deploy additional pods to serve the incoming traffic. Similarly, when there is a drop in traffic, nodes and pods need to scale down. This ability to automatically scale up or down the number of nodes in the AKS cluster helps in running an efficient and cost-effective cluster.

There are two components involved in auto scaling of AKS cluster.

  • Cluster Autoscaler: Scale the nodes in the cluster based on the pending pods which need to be deployed.
  • Horizontal Pod Autoscaler: Monitors the resource demand of pods. If a service needs more resources, the number of pods is automatically increased to meet the demand. You must give inputs of memory, CPU limit, and the min and max number of pods that can be scaled.
  • You can check the runtime usage of memory and CPU by using the following command (hpa is horizontal pod autoscaler):

    "kubectl get hpa"

    Whenever the usage of either the memory or the CPU exceeds the limit given for that particular pod, the Horizontal Pod Autoscaler is triggered and it starts scaling up the pod.

    As pod anti-affinity is implemented, multiple nodes of same component can’t be scheduled on a node to take care of fault-tolerance of deployed application.

    When the traffic goes down and the memory and CPU usage falls below the threshold values, newly created pods are terminated. This also results in scaling down of newly created nodes to the minimum number of nodes specified by the user.

For the details about user inputs, refer to the Configuration section.

Azure Resource Group Role Based Access Control (RBAC)

The way to control access to resources using RBAC is to create role assignments.
A role assignment is the process of binding a role definition to a user, group, or service principal at a particular scope for the purpose of granting access. You can create Role assignments using the Azure portal, Azure CLI, Azure PowerShell, Azure SDKs, or REST APIs.

The creation of Role Assignments has the following three elements:

  • Security Principal: An object that represents a user, group, or service principal that requests access to Azure resources.
  • Role Definition: A collection of permissions such as read, write, and delete. Roles can be generic, like owner; or specific, like virtual machine reader. you can either use the built-in roles or create custom roles for assigning.
  • Scope: A boundary that the access applies to. It can be specified at multiple levels such as management group, subscription, resource group or resource.

To create and remove role assignments, you must have Microsoft.Authorization/ roleAssignments/ * permission. The Owner or User Access Administrator roles can grant this permission.

NOTE: When planning the access control strategy, it is a best practice to grant users the least privilege to get their work done.

Block IP addresses in the Azure Web Application Firewall

Follow these steps to block IP addresses in the Azure Web Application Firewall:

  1. Log on to the Azure portal (portal.azure.com) using your Microsoft account credentials.
  2. Select the Resource group in which WAF is created.
  3. Select the Web Application Firewall resource.
  4. Select the subnet which has the WAF attached.
  5. Create a new Network Security Group (NSG).
  6. By default, NSG is not created for AppGateway.
  7. Configure inbound rules in the Network Security Group.
  8. Add new rules in the Network Security Group.
  9. Associate the AppGateway subnet with the Network Security Group.

Block IP addresses in the Azure CDN

Follow these steps to block IP addresses in the Azure Content Delivery Network (CDN):

  1. Log on to the Azure portal (portal.azure.com) using your Microsoft account credentials.
  2. Select the Resource group in which CDN is created.
  3. Go to CDN profile from the list of resources available in the Azure Resource Group.
  4. Click Manage from the top navigation bar.
  5. From the HTTP Large list, select Rules Engine.
  6. Configure the new Rule.

Whitelist IP Address in Azure CDN

Automation of rules configuration is not supported by az-cli, therefore we need to access CDN manage portal to configure rules.

Follow these steps to whitelist IP addresses in the Azure Content Delivery Network (CDN):

  1. Log on to the Azure portal (portal.azure.com) using your Microsoft account credentials.
  2. Select the Resource group in which CDN is created.
  3. Go to CDN profile from the list of resources available in the Azure Resource Group.
  4. Click on the Manage from the top navigation bar to access CDN Manage Portal.
  5. Configure the new Rule.
  6. Set Deny Access (403) to Enabled.

    This determines whether all requests are rejected with a 403 Forbidden response or not.

    You can set Deny Access (403) to the values shown in the following table:

    ValueResult
    EnabledCauses all requests that satisfy the matching criteria to be rejected with a 403 Forbidden response.
    DisabledAllow the origin server to determine the type of response that will be returned.

NOTE: It takes some time for the rule to be propagated to the CDN edge nodes. Check the status of rule in CDN Manage Portal.

Configure Email Alerts for ClamAV and OSSEC

VPN Reference Implementation

NOTE: VPN Reference Implementation is documented assuming that the FortiGate Firewall is used on the On-Premise setup. If you are using any other firewall, contact your system administrator for setting up the incoming policies from Quantum Fabric set up on Microsoft Azure Cloud.

A Site-to-Site VPN gateway connection is used to connect your on-premises network to an Azure virtual network over an IPsec or IKE (IKEv1 or IKEv2) VPN tunnel. This type of connection requires an on-premises VPN device that has an externally facing public IP address assigned to it. Follow the steps described here to setup a site-to-site VPN.

Configure Backup and Restore for Azure File Share

An Azure file share is a convenient place for cloud applications to write their logs, metrics, and crash dumps. Logs can be written by the application instances via the File REST API, and developers can access them by mounting the file share on their local machine. To understand how to create a file share on Azure, refer to Create a file share in Azure Files.

Before you back up an Azure file share, you must ensure that it is present in one of the supported Storage Account types.

Set custom VNet and Subnet ranges

Steps to use custom VNet and Subnet ranges for Azure Quantum setup:

For Azure installation, we use the 10.X.X.X/8 VNet and Subnet ranges. In a few cases where Quantum Fabric must connect to the customer backend network which also has IP ranges in the same private address space, you can use a different private address range for Azure Quantum installation. Perform the following steps to use a different private IP address range.

  1. Open the input properties file <Installation Directory>/conf/enterprise.properties.
  2. Find properties related to vnet and subnet ranges starting from the VNET_ADDRESS_SPACE property.
  3. Update all the properties to use the preferred private address range. For example: 192.168.X.X.
  4. Also, update the network.properties file located under <Installation Directory>/conf/ to use address ranges matching the custom private address range.

Features and Supported Regions

Following is the list of features and their supported regions. If the region specified by the user’s input(AZURE_LOCATION property) does not support a specific feature, then that feature is skipped and an alternate option is installed.

  • Regions where WAF v2 is not supported:

    australiacentral,southafricanorth,uaenorth,westcentralus
  • Regions where CDN is supported:

    australiaeast, australiasoutheast, brazilsouth, canadacentral, canadaeast, 
    centralindia, centralus, eastasia, eastus, eastus2, japaneast, japanwest, 
    northcentralus, northeurope, southcentralus, southindia, southeastasia, westeurope, 
    westindia, westus, westcentralus
  • Regions where multiple availability zones are supported for AKS:

    centralus, eastus2, eastus, francecentral, japaneast, northeurope, 
    southeastasia, uksouth, westeurope, westus2
  • Regions where multiple availability zones are supported for Application Gateway:

    centralus,eastus,eastus2,westus2,francecentral, 
    northeurope,uksouth,westeurope,japaneast,southeastasia
  • Regions where ZRS is not supported for Storage account:

    westus2,eastus,eastus2,northeurope,westeurope,southeastasia, 
    francecentral,japaneast,centralus,uksouth,australiaeast,southafricanorth