SolaceDeployer

Demonstrates different deployment strategies.
This project can be used inside a CICD pipeline to execute deployments and undeployments of applications in a broker mesh.

Prerequisites

python3 [>= 3.12]
virtual environment activated
Cloud Console Token with the following permissions:
- event_designer:access
- modeled_event_broker:get:*
- application_domain:get:*
- application:get:*
- ep_environment:get:*
- modeled_event_mesh:get:*
- modeled_event_broker:get:*
- mission_control:access
- services:get
- services:get:self
- services:view
- services:view:self

 python -m venv .venv
 source .venv/bin/activate

By running the following command, all dependencies will be installed/updated according to the content of the pyproject.toml file.

python -m pip install -e .

Strategies for Deployment

There are 2 CICD strategies possible.

Use config-push in Event Portal for all environments, have it triggered by CICD pipeline for TST, ACC and PRD
Use config push in Event Portal for DEV and optional TST, extract the proper configuration from a given application from Event Portal, store it in GitLab, create merge request that triggers deployment to a given environment.

Config-Push triggered by CICD pipeline

Advantages:

all configuration is stored inside Event Portal
all deployments to all environments are reflected in Event Portal.
Uses Cloud API

Disadvantages:

uses Event Management Agent, not yet ready for secure connections.
You need predefined applicationIds to get the list of applicationVersionIds, you need predefined environmentIds for each environment and MEM's, you can use the MEMId to get the specific broker service(s) for an environment and you can use the broker service id(s) to deploy the specific applicationIds onto the broker service

Steps to execute for a deployment:

Get the predefined values for an environment: environmentName, modeledEventMeshName, applicationDomainName(s), applicationName(s) and its applicationVersion(s) to deploy
Get the environmentId of the target environment: GET /api/v2/architecture/environments, extract the environmentId (id) from the response where name === {{environmentName}}
Get the meshId in the target environment: GET /api/v2/architecture/eventMeshes?environmentId={{environmentId}}, extract the meshId (id) from the response where name == {{modeledEventMeshName}}
Get the connected brokerIds: GET /api/v2/architecture/messagingServices?eventMeshId={{MeshId}}, extract the broker ids (data?.id)
For each application domain:
1. Get the applicationDomainId: GET /api/v2/architecture/applicationDomains?name={{applicationDomainName}}, extract the applicationDomainId (id) from the response
2. For each applicationName:
  1. Get the applicationId: GET /api/v2/architecture/applications?applicationDomainId={{applicationDomainId}}, extract the applicationId (id) where name === {{applicationName}}
  2. get the applicationVersionId: GET /api/v2/architecture/applicationVersions?applicationIds={{applicationId}}, extract the applicationVersionId (id) where version === {{applicationVersion}}
  3. For each brokerId from step 4:
    1. Deploy applicationVersionId: POST /api/v2/architecture/runtimeManagement/applicationDeployments with body: { "applicationVersionId": "{{applicationVersionId}}", "action": "deploy", "eventBrokerId": "{{brokerId}}

Pull Config from portal and push to environment with SEMP via CICD pipeline

Advantages:

complete control on provenance

Disadvantages:

Mix of Cloud API, SEMP/V2 API and/or terraform API
convoluted, because it needs a lot of coding in the CICD pipelines. (extract the proper configuration sections for the different queues, get the ACL and client profiles from the DEV/TEST brokers, store the information into GitLab, create a PR to get it approved, once approved, the pipeline can deploy on a given environment.
keep track of terraform state inside the pipeline

Steps to execute for a deployment:

Get the predefined values for a source environment: environment, environmentName, modeledEventMeshName to get the preview
Get the predefined values for a target environment: environment, environmentName, modeledEventMeshName, applicationDomainName, applicationName(s) and its applicationVersion(s) to deploy
Get the list of brokers with url, user, password, ands msgVpnName for the target environment to use for deployment
Get the dev environmentId: GET /api/v2/architecture/environments, extract the environmentId (id) from the response where name === {{environmentName}}
Get the meshId for the environment: GET /api/v2/architecture/eventMeshes?environmentId={{environmentId}}, extract the meshId (id) from the response where name == {{modeledEventMeshName}}
For each application domain:
1. Get the applicationDomainId: GET /api/v2/architecture/applicationDomains?name={{applicationDomainName}}, extract the applicationDomainId (id) from the response
2. For each applicationName:
  1. Get the applicationId: GET /api/v2/architecture/applications?applicationDomainId={{applicationDomainId}}, extract the applicationId (id) where name == {{applicationName}}
  2. get the applicationVersionId: GET /api/v2/architecture/applicationVersions?applicationIds={{applicationId}}, extract the applicationVersionId (id) where version == {{applicationVersion}}
  3. Create a preview applicationVersionId: POST /api/v2/architecture/runtimeManagement/applicationDeployments with body: { "applicationVersionId": "{{applicationVersionId}}", "action": "deploy or undeploy", "eventBrokerId": "{{brokerId}}

This will return a json file containing the new (requested) configuration and the original (existing) configuration for the application version. The new (requested) configuration will be used for deployment and the original (existing) configuration willnbe used for undeployment)

Create the semp config sections:

The configuration can be divided into 3 parts:

solaceAcl section (only 1)
solaceClientUsername or solaceAuthorizationGroup or solaceClientCertificateUsername section (only 1)
one or more solaceQueue sections (1 or more)

These have to be transformed to appropriate patch commands for the SEMP/v2 API or a set of terraform modules.

1. SolaceACL section

The Solace ACL section is translated into 3 separate SEMP/V2 calls for create and one for delete:

Create aclProfile: POST /msgVpns/{msgVpnName}/aclProfiles
Create clientConnectExceptions: POST /msgVpns/{msgVpnName}/aclProfiles/{aclProfileName}/clientConnectExceptions (Not yet supported in the Portal)
Create subscribeTopicExceptions: POST /msgVpns/{msgVpnName}/aclProfiles/{aclProfileName}/subscribeTopicExceptions
Create publishTopicExceptions: POST /msgVpns/{msgVpnName}/aclProfiles/{aclProfileName}/publishTopicExceptions
Delete aclProfile: DELETE /msgVpns/{msgVpnName}/aclProfiles/{aclProfileName}

Create aclProfile:

Get body from response: $..requested[?(@.type=='solaceAcl')].value.aclProfile and add msgVpnName

POST /msgVpns/{msgVpnName}/aclProfiles

Body:

{
    "clientConnectDefaultAction": "allow",
    "publishTopicDefaultAction": "disallow",
    "subscribeTopicDefaultAction": "disallow",
    "aclProfileName": "app-[applicationId]",
    "subscribeShareNameDefaultAction": "allow",
    "msgVpnName": "[msgVpnName]"
}

Create subscribeTopicExceptions:

Determine the aclProfileName with $..requested[?(@.type=='solaceAcl')].value.aclProfile.aclProfileName and use that in the url

For each entry in : $..requested[?(@.type=='solaceAcl')].value.subscribeTopicExceptions

POST /msgVpns/{msgVpnName}/aclProfiles/{aclProfileName}/subscribeTopicExceptions For now only support for SMF syntax Body:

{
    "aclProfileName": [aclProfileName],
    "subscribeTopicException": "[entry]",
    "subscribeTopicExceptionSyntax": "[smf or mqtt]",
    "msgVpnName": "[msgVpnName]"
}

Create publishTopicExceptions:

Determine the aclProfileName with $..requested[?(@.type=='solaceAcl')].value.aclProfile.aclProfileName and use that in the url

For each entry in : $..requested[?(@.type=='solaceAcl')].value.publishTopicExceptions

POST /msgVpns/{msgVpnName}/aclProfiles/{aclProfileName}/publishTopicExceptions For now only support for SMF syntax Body:

{
    "aclProfileName": "[aclProfileName]",
    "publishTopicException": "[entry]",
    "publishTopicExceptionSyntax": "[smf or mqtt]",
    "msgVpnName": "[msgVpnName]"
}

Delete aclProfile

Get aclProfileName from response: $..existing[?(@.type=='solaceAcl')].value.aclProfile.aclProfileName and use that in the url.

DELETE /msgVpns/{msgVpnName}/aclProfiles/{aclProfileName}

2.a. SolaceClientUsername

The Solace ClientUsername section is translated into a single SEMP/v2 call:

Create clientUsername: POST /msgVpns/{msgVpnName}/clientUsernames
Delete clientUsername: DELETE /msgVpns/{msgVpnName}/clientUsernames/{clientUsername}

Create clientUsername:

Get body from response: $..requested[?(@.type=='solaceClientUsername')] and add msgVpnName

POST /msgVpns/{msgVpnName}/clientUsernames

Body:

{
    "password": "[app-user-pwd]",
    "subscriptionManagerEnabled": false,
    "clientUsername": "[app-user]",
    "clientProfileName": "[clientProfileName]",
    "guaranteedEndpointPermissionOverrideEnabled": false,
    "aclProfileName": "app-[applicationId]",
    "enabled": true,
    "msgVpnName": "[msgVpnName]"
  }

Delete clientUsername

Get clientUsername from response: $..existing[?(@.type=='solaceClientUsername')].value.clientUsername and use that in the url.

DELETE /msgVpns/{msgVpnName}/clientUsernames/{clientUsername}

2.b. SolaceAuthorizationGroup

The Solace AuthorizationGroup section is translated into a single SEMP/v2 call:

Create authorizationGroup: POST /msgVpns/{msgVpnName}/authorizationGroups
Delete authorizationGroup: DELETE /msgVpns/{msgVpnName}/authorizationGroups/{authorizationGroup}

Create authorizationGroup:

Get body from response: $..requested[?(@.type=='solaceAuthorizationGroup')] and add msgVpnName

POST /msgVpns/{msgVpnName}/authorizationGroups

Body:

{
    "aclProfileName": "[aclProfileName]",
    "authorizationGroupName": "[authorizationGroup]",
    "clientProfileName": "default",
    "enabled": true,
    "msgVpnName": "[msgVpnName]"
  }

Delete authorizationGroup

Get authorizationGroup from response: $..existing[?(@.type=='solaceAuthorizationGroup')].value.authorizationGroupName and use that in the url.

DELETE /msgVpns/{msgVpnName}/authorizationGroups/{authorizationGroup}

2.c. SolaceClientCertificateUsername

The Solace ClientCertificateUsername section is translated into a single SEMP/v2 call:

Create clientCertificateUsername: POST /msgVpns/{msgVpnName}/clientUsernames
Delete clientCertificateUsername: DELETE /msgVpns/{msgVpnName}/clientUsernames/{clientCertificateUsername}

Create clientCertificateUsername:

Get body from response: $..requested[?(@.type=='solaceClientCertificateUsername')] and add msgVpnName

POST /msgVpns/{msgVpnName}/clientUsernames

Body:

{
    "subscriptionManagerEnabled": false,
    "clientUsername": "[clientCertificateUsername]",
    "clientProfileName": "[clientProfileName]",
    "guaranteedEndpointPermissionOverrideEnabled": false,
    "aclProfileName": "app-[applicationId]",
    "enabled": true,
    "msgVpnName": "[msgVpnName]"
  }

Delete clientCertificateUsername

Get clientCertificateUsername from response: $..existing[?(@.type=='solaceClientCertificateUsername')].value.clientUsername and use that in the url.

DELETE /msgVpns/{msgVpnName}/clientUsernames/{clientCertificateUsername}

3. SolaceQueue section

The Solace Queue section is translated into 3 separate SEMP/V2 calls per queue:

Create Queue: POST /msgVpns/{msgVpnName}/queues
Create Subscription: POST /msgVpns/{msgVpnName}/queues/{queueName}/subscriptions
Delete Queue: DELETE /msgVpns/{msgVpnName}/queues/{queueName}

Get the queue definitions: $..requested[?(@.type=='solaceQueue')].value

For each definition $def:

Create Queue:
Create queue with body: $def.queueConfiguration and add the msgVpnName attribute:

POST /msgVpns/{msgVpnName}/queues

Body:

{
    "msgVpnName": "[msgVpnName]",
    "accessType": "exclusive",
    "ingressEnabled": true,
    "owner": "[clientUsername or clientCertificatUsername or authorizationGroup]",
    "queueName": "[queueName]",
    "egressEnabled": true,
    "permission": "no-access",
    "maxMsgSpoolUsage": 5000
}

Get subscriptions $def.subscriptions
For each entry in $def.subscriptions
Create subscription:

POST /msgVpns/{msgVpnName}/queues/{queueName}/subscriptions

Body:

{
    "msgVpnName": "[msgVpnName]",
    "queueName": "[queueName]",
    "subscriptionTopic": "[entry]"
}

Delete Queue Get existing queueName with $def.queueConfiguration.queueName

DELETE /msgVpns/{msgVpnName}/queues/{queueName}

Testing it out

The file SolaceSampleDomain.json can be loaded into the Event Portal. Make sure there are at least 2 environments present [Dev, Test] Make sure ech environment has at least one broker running Create a Modeled Event mesh for each environment and note their names. Connect at least one Broker to each Modeled Event Mesh. This will use the Cloud EMA for that Modeled Event Mesh (if enabled) Make a note of the 'Manage with Semp' configuration from the brokers in the Event Portal CLuster Manager.

url
username
password
message vpn name

These values are needed in the next section

My test situation:

Environment	MEM	Broker
Dev	memName	brokerDEV
Test	memName	brokerTST

Create config/eventPortal.json

{
  "baseUrl": "https://api.solace.cloud/api/v2",
  "token": "[some token with sufficient permissions]"
}

config/dev.json

{
  "environment": "dev",
  "environmentName": "Dev",
  "memName": "SampleMesh"
}

config/tst.json

{
  "environment": "tst",
  "environmentName": "Test",
  "memName": "SampleMesh",
  "domains":[
    {
      "domainName": "SolaceSampleDomain",
      "applications": [
        {
          "name": "Application_1",
          "version": "0.1.2",
          "user": {
            "name": "app1-tst",
            "type": "solaceClientUsername",
            "password": "app1-tst",
            "clientProfileName", "[some existing client profile name]"
          }
        },
        {
          "name": "Application_2",
          "version": "0.1.0",
          "user": {
            "name": "app2-tst",
            "type": "solaceClientCertificate"
          }
        },
        {
          "name": "Application_3",
          "version": "0.1.0",
          "user": {
            "role": "app3-role",
            "name": "app3-name(will be used as queueu owner)"  
            "type": "solaceAuthorizationGroup"
          }
        },
        {
          "name": "RDP_Application_1",
          "version": "0.1.0"
        }
      ]
    }
  ],
  "brokers": [ {
    "name": "[test broker name]",
    "url": "[Base path  from EP ClusterManager/Manage/SEMP-REST API]",
    "user": "[Username from EP ClusterManager/Manage/SEMP-REST API]",
    "password": "[Password from EP ClusterManager/Manage/SEMP-REST API]",
    "msgVpnName": "[Message VPN Name from EP ClusterManager/Manage/SEMP-REST API]"
  }]
}

If you are using client profile templates, make sure these exist on the broker to which you are deploying. Also DO NOT SPECIFY A USER WHEN CONFIGURING AN RDP APPLICATION!!

Make sure the Cloud EMA's are running for each environment:

If these don't run you cannot use configPush Execute a config-push on the EventPortal for both applications on the dev broker

Now you can test both strategies for deployment on tst:

ConfigPush

Deploy version to Test

runAction --mode configPush --action=deploy --target=tst --appl "[{\"Domainname\":[\"app_1\",\"app_2\",\"app_3\",\"app_4\"]}]"

Check on the Test broker if acl, clientUsernames and Queues are created

Undeploy version from Test

runAction --mode configPush --action=undeploy --target=tst --appl "[{\"Domainname\":[\"app_1\",\"app_2\",\"app_3\",\"app_4\"]}]"

Check if acls, clientUsernames and queues are removed

Semp Deployment

Deploy version to Test

runAction --mode semp --action=deploy --target=tst --appl "[{\"Domainname\":[\"app_1\",\"app_2\",\"app_3\",\"app_4\"]}]"

Check on the Test broker if acl, clientUsernames and Queues are created

Undeploy version from Test

runAction --mode semp --action=undeploy --target=tst --appl "[{\"Domainname\":[\"app_1\",\"app_2\",\"app_3\",\"app_4\"]}]"

Check if acls, clientUsernames and queues are removed

Save deployment of version for Test

runAction --mode semp --action=save --target=tst --appl "[{\"Domainname\":[\"app_1\",\"app_2\",\"app_3\",\"app_4\"]}]"

Check if there are folders created in store with this format: store/[app_domain]/[application]/[version]/preview-[state].json

Test connectivity Dev environment

Send message to Application_1 on Dev

sdkperf_amqp -cip amqps://mr-connection-[dev serviceId].messaging.solace.cloud -cu app2-dev@[dev msgvpn] -cp 'app2-dev' -ptl 'topic://#P2P/QUE/Event_2MessageQueue' -msa 100 -mn 5 -mr 1

Receive messages from Application_1 on dev

sdkperf_java  -cip tcps://mr-connection-[dev serviceId].messaging.solace.cloud -cu app1-dev@[dev msgvpn] -cp app1-dev -sql 'Event_2MessageQueue' -md

Send message to Application_2 on Dev

sdkperf_amqp -cip amqps://mr-connection-[dev serviceId].messaging.solace.cloud -cu app1-dev@s[dev msgvpn] -cp 'app1-dev' -ptl 'topic://#P2P/QUE/Event_1MessageQueue' -msa 100 -mn 5 -mr 1

Receive messages from Application_2 on dev

sdkperf_java  -cip tcps://mr-connection-[dev serviceId].messaging.solace.cloud -cu app2-dev@[dev msgvpn] -cp app2-dev -sql 'Event_1MessageQueue' -md

Test connectivity Test environment

Send message to Application_1 on tst

sdkperf_amqp -cip amqps://mr-connection-[tst serviceId].messaging.solace.cloud -cu app2-tst@[tst msgvpn] -cp 'app2-tst' -ptl 'topic://#P2P/QUE/Event_2MessageQueue' -msa 100 -mn 5 -mr 1

Receive messages from Application_1 on tst

sdkperf_java  -cip tcps://mr-connection-[tst serviceId].messaging.solace.cloud -cu app1-tst@[tst msgvpn] -cp app1-tst -sql 'Event_2MessageQueue' -md

Send message to Application_2 on tst

sdkperf_amqp -cip amqps://mr-connection-[tst serviceId].messaging.solace.cloud -cu app1-tst@s[tst msgvpn] -cp 'app1-tst' -ptl 'topic://#P2P/QUE/Event_1MessageQueue' -msa 100 -mn 5 -mr 1

Receive messages from Application_2 on tst

sdkperf_java  -cip tcps://mr-connection-[tst serviceId].messaging.solace.cloud -cu app2-tst@[tst msgvpn] -cp app2-tst -sql 'Event_1MessageQueue' -md

Name		Name	Last commit message	Last commit date
config		config
deployer		deployer
.gitignore		.gitignore
LICENSE		LICENSE
README.md		README.md
SolaceSampleDomain.json		SolaceSampleDomain.json
pyproject.toml		pyproject.toml

Navigation Menu

Repository files navigation

SolaceDeployer

Prerequisites

Strategies for Deployment

Config-Push triggered by CICD pipeline

Steps to execute for a deployment:

Pull Config from portal and push to environment with SEMP via CICD pipeline

Steps to execute for a deployment:

1. SolaceACL section

2.a. SolaceClientUsername

2.b. SolaceAuthorizationGroup

2.c. SolaceClientCertificateUsername

3. SolaceQueue section

Testing it out

ConfigPush

Semp Deployment

Test connectivity Dev environment

Test connectivity Test environment

About

Releases

Packages

Used by

Contributors

Languages

Latest commit

History

Folders and files

Repository files navigation

SolaceDeployer

Prerequisites

Strategies for Deployment

Config-Push triggered by CICD pipeline

Steps to execute for a deployment:

Pull Config from portal and push to environment with SEMP via CICD pipeline

Steps to execute for a deployment:

1. SolaceACL section

2.a. SolaceClientUsername

2.b. SolaceAuthorizationGroup

2.c. SolaceClientCertificateUsername

3. SolaceQueue section

Testing it out

ConfigPush

Semp Deployment

Test connectivity Dev environment

Test connectivity Test environment

About

Resources

Stars

Watchers

Forks

Releases

Packages

Used by

Contributors

Languages