Double curly braces replacement

Using Django Templates & Frontend
1

I hope I’m in the right place… I develop python scripts for an application that was written in Django. Not getting into the why, but I’m trying to take a string variable that has various {{ something.parameter }} and have Django, Shell Plus, what ever it is that normally handles replacement of double curly braces to do a replacement on that string. I know I can write code in python to do specifically coded variable replacements, but that would defeat one of the purposes.

More details: The string variable is any script that someone needs to run on a remote server. For my purposes it is Powershell scripts. These are loaded into a string (one by one) and then created on a remote server via Paramiko then executed. Normally the Out of the Box application (Cloudbolt, written using Django) handles this and also handles the {{ something.parameter }} replacement. But it uses pyvmomi to copy and execute scripts on servers. I’m trying to move away from that to Paramiko. When the Python script runs ( that Cloudbolt is running ?using Shell Plus? ?using Django? ?using what ever?) it will do the substitution/replacement of the {{ something.parameter }} in that Python script, but when that script gets and assigns string variable with the other Powershell script data it doesn’t act on it (replace the {{ }}. I need to somehow make the {{ something.parameter }} replacement happen on it as well during the python script execution.

2

Just tossing out an idea here - the Django template engine is a module in Django with a defined and documented class and interface.

My guess is you should be able to import that module, and use it to render your templates using a defined context, having it return a string.

3

Being the curious type, I chased down exactly what needs to be done to make this work.

For using parts of Django outside a compete “Django runtime”, the information can actually be found in the settings docs.

Of particular value is the paragraph on Calling django.setup()

This, combined with a couple of answers on StackOverflow, and I found that this sequence of commands works:

import django
from django.conf import settings
from django.template import Engine, Context

templates = [
  {'BACKEND':'django.template.backends.django.DjangoTemplates'}
]
settings.configure(TEMPLATES=templates)
django.setup()

string_template = Engine().from_string('{{a}} {{b}}')
context = Context({'a':'Hello', 'b':'World!'})
string_template.render(context)
4

Hey Ken, I’ve been working on this on and off most of the day but can’t seem to get it to work. Your example above does work. However the piece that I’m missing is how to use the existing environment as the context. I’m testing in Shell Plus and here is some of the stuff I’ve tried with the output.

. shell_plus

Shell Plus Model Imports

from accounts.models import CBPermission, ExternalUserAttributeMapping, Group, GroupRoleMembership, GroupType, PasswordHistory, Role, UserProfile
from behavior_mapping.models import CustomFieldMapping, PreconfigurationMapping, ResourcePoolMapping, SequencedItem
from bookmarks.models import Bookmark
from cbadmin.models import StaticPage
from cbhooks.models import CloudBoltHook, CopyFileAction, EmailHook, FlowHook, HookInput, HookMapping, HookPoint, HookPointAction, OrchestrationHook, RecurringActionJob, RemoteScriptHook, ResourceAction, RunHookInputMapping, ServerAction, TerraformPlanHook, TerraformStateFile, TriggerPoint, WebHook
from connectors.ansible.models import AnsibleConf, AnsibleGroup, AnsiblePlaybook, AnsibleVariable
from connectors.chef.models import ChefConf, ChefCookbook, ChefNode, ChefRole, CommunityCookbook
from connectors.models import ConnectorConf, Feature, FeatureMap
from connectors.puppet.models import PuppetClass, PuppetConf, PuppetNode, PuppetReport
from connectors.puppet_ent.models import PEConf, PEGroup, PENode, PEReport
from containerorchestrators.kuberneteshandler.models import Kubernetes, KubernetesObject
from containerorchestrators.models import ContainerOrchestrator, ContainerOrchestratorTechnology, ContainerResource
from costs.models import ApplicationRate, BillingLineItem, BillingSummary, CustomFieldRate, DiskTypeMultiplier, LicenseRate, OSBuildRate
from cscv.models import ActionCITTest, CITConf, CITTest
from django.contrib.admin.models import LogEntry
from django.contrib.auth.models import Permission, User
from django.contrib.contenttypes.models import ContentType
from django.contrib.sessions.models import Session
from emailtemplates.models import EmailTemplate
from extensions.models import UIExtension, UIExtensionComponent
from externalcontent.models import Application, OSBuild, OSBuildAttribute, OSFamily, OSVersion, VendorApplication
from files.models import ConfigurationFile, FileInput, FileInputMapping
from health_check.models import TestModel
from history.models import EnvironmentHistory, GlobalAllocationTotals, GroupProfileTotals, HistoryModel, LicensePoolHistory, OrderHistory, ResourceHandlerHistory, ResourceHistory, ServerAggregatedTotals, ServerHistory
from infrastructure.models import ControlValue, CustomField, DataCenter, Disk, DiskStorage, DiskType, Environment, FieldDependency, Namespace, Preconfiguration, ResourcePool, ResourcePoolValueSet, ScheduledTime, Server, ServerExpireParameters, ServerNetworkCard, ServerSnapshot, ServerStats, ServerStatsSample
from ipam.models import IPAM, IPAMNetwork, IPAMTechnology
from jobengine.models import JobEngineWorker
from jobs.models import CFVChangeParameters, DeleteSnapshotsParameters, ExternalJob, FunctionalTestParameters, HookParameters, InstallApplicationsParameters, Job, JobParameters, ManageNICsParameters, NetworkActionParameters, ProgressMessage, RecurringJob, RunFlowParameters, SyncSvrsFromPEsParameters, SyncUsersFromLdapParameters, SyncVMParameters, TriggerActionParameters, TriggerParameters, UninstallApplicationsParameters
from licenses.models import License, LicensePool
from loadbalancers.models import LoadBalancerAppliance
from networks.models import F5LoadBalancer, HAProxy, LoadBalancer, LoadBalancerTechnology, NetscalerLoadBalancer
from orchestrationengines.hpoo.models import HPOO
from orchestrationengines.models import OrchestrationEngine, OrchestrationFlow, OrchestrationFlowParameters, OrchestrationTechnology
from orchestrationengines.vco.models import VCO
from orders.models import ActionJobOrderItem, BlueprintItemArguments, BlueprintOrderItem, CustomFieldValue, DecomServerOrderItem, InstallPodOrderItem, Order, OrderItem, PreconfigurationValueSet, ProvisionNetworkOrderItem, ProvisionServerOrderItem, ServerModOrderItem
from portals.models import PortalConfig
from provisionengines.cobbler.models import CobblerProfile, CobblerServer
from provisionengines.hpsa.models import HPSACore, HPSAOSBuildAttribute, SoftwarePolicy
from provisionengines.models import ProvisionEngine, ProvisionTechnology
from provisionengines.razor.models import RazorRepository, RazorServer
from quota.models import Quota, ServerQuotaSet
from reportengines.jasper.models import JasperReportingEngine
from reportengines.models import ReportingEngine
from resourcehandlers.acropolis.models import AcropolisDisk, AcropolisImage, AcropolisResourceHandler, AcropolisServerInfo
from resourcehandlers.alibaba.models import AlibabaImage, AlibabaResourceHandler, AlibabaServerInfo, AlibabaVSwitch
from resourcehandlers.aws.models import AWSBillingLineItem, AWSHandler, AmazonMachineImage, AwsVpcSubnet, EBSDisk, EC2ServerInfo
from resourcehandlers.azure.models import AzureHandler, AzureImage, AzureServerInfo, AzureSubnet
from resourcehandlers.azure_arm.models import ARMResourceGroup, ARMStorageAccount, AzureARMAvailableImage, AzureARMDisk, AzureARMExtension, AzureARMHandler, AzureARMImage, AzureARMImageLocation, AzureARMNodeSize, AzureARMServerInfo, AzureARMServerNetworkCard, AzureARMSubnet
from resourcehandlers.azure_stack.models import AzureStackHandler
from resourcehandlers.centurylink.models import CTLNetwork, CTLOSBuildAttribute, CTLResourceHandler, CTLServerInfo
from resourcehandlers.dellblade.models import DellBladeHandler
from resourcehandlers.dimensiondata.models import DimensionDataNetworkDomainVlan, DimensionDataOSBuildAttribute, DimensionDataResourceHandler, DimensionDataServerInfo
from resourcehandlers.gce.models import GCEHandler, GCENetwork, GCEServerNetworkCard, GCESubnetwork
from resourcehandlers.gcp.models import GCPHandler, GCPNetwork, GCPProject, GCPServerInfo, GCPServerNetworkCard, GCPSubnetwork
from resourcehandlers.helion.models import HelionHandler
from resourcehandlers.hpblade.models import HPBladeOnboardAdminHandler, VConnectNetwork
from resourcehandlers.hyperv.models import HyperVImage, HyperVResourceHandler
from resourcehandlers.ipmi.models import IPMIResourceHandler
from resourcehandlers.libcloudhandler.models import LibcloudImage, LibcloudServerInfo
from resourcehandlers.manualpowerserver.models import ManualPowerResourceHandler
from resourcehandlers.models import ResourceHandler, ResourceLimitItem, ResourceNetwork, ResourceTechnology
from resourcehandlers.oci.models import OCIImage, OCIResourceHandler, OCIResourceNetwork, OCIServerInfo
from resourcehandlers.openstack.models import OpenStackHandler
from resourcehandlers.oracle.models import OracleNetwork, OracleOSBuildAttribute, OracleResourceHandler, OracleServerInfo
from resourcehandlers.qemu.models import QemuOSBuildAttribute, QemuResourceHandler
from resourcehandlers.rhev.models import RhevNetwork, RhevOSBuildAttribute, RhevResourceHandler
from resourcehandlers.slayer.models import SlayerNetwork, SlayerOSBuildAttribute, SlayerResourceHandler
from resourcehandlers.terremark.models import TerremarkHandler, TerremarkNetwork, VAppTemplate
from resourcehandlers.vcloud_director.models import VCDBaseImage, VCDDisk, VCDHandler, VCDNetwork, VCDServerInfo, VCDStorageProfile
from resourcehandlers.vmware.models import VmwareDatastore, VmwareDisk, VmwareNetwork, VmwareServerInfo, VsphereOSBuildAttribute, VsphereResourceHandler
from resourcehandlers.vmware.nsx.models import NSXEdge, NSXEdgeConfiguration, NSXScope
from resourcehandlers.xen.models import XenNetwork, XenOSBuildAttribute, XenResourceHandler
from resources.models import Resource, ResourceType, SoftwareDefinedNetwork, SoftwareDefinedNetworkAppliance
from reversion.models import Revision, Version
from servicecatalog.models import BlueprintServiceItem, CopyFileActionServiceItem, InstallPodServiceItem, LoadBalancerServiceItem, NetworkServiceItem, ProvisionServerServiceItem, RunCloudBoltHookServiceItem, RunEmailHookServiceItem, RunFlowHookServiceItem, RunRemoteScriptHookServiceItem, RunTerraformPlanHookServiceItem, RunWebHookServiceItem, ServiceBlueprint, ServiceBlueprintGroupPermissions, ServiceItem, TearDownServiceItem
from taggit.models import Tag, TaggedItem
from tags.models import AutoCorrectTagValueMapping, CloudBoltTag, TaggableAttribute, TaggedItem
from utilities.models import CBReleaseInfo, ConnectionInfo, DBLock, GlobalPreferences, LDAPMapping, LDAPMappingGroup, LDAPUtility, PKIUtility, RADIUSUtility, RootCertificate

Shell Plus Django Imports

from django.core.cache import cache
from django.conf import settings
from django.db import transaction
from django.db.models import Avg, Case, Count, F, Max, Min, Prefetch, Q, Sum, When
from django.utils import timezone
from django.urls import reverse
Python 3.6.4 (default, Apr 30 2018, 16:20:11)
Type ‘copyright’, ‘credits’ or ‘license’ for more information
IPython 6.2.1 – An enhanced Interactive Python. Type ‘?’ for help.

In [1]: server = Server.objects.get(id=23946)

In [2]: test = " {{ server.id }} "

In [3]: from django.template import engines

In [4]: engine_list = engines.all()

In [5]: engine_list
Out[5]: [<django.template.backends.django.DjangoTemplates at 0x7fcd826b74e0>]

In [6]: engine = engine_list[0]

In [7]: t = engine.from_string(test)

In [8]: t
Out[8]: <django.template.backends.django.Template at 0x7fcd7f105f98>

In [9]: t.render()
Out[9]: ’ ’

I think I am missing a fundamental piece of the pie here. Normally the application replaces the {{ }} when it loads scripts so it has to have used some sort of context like your example. Wether that is it using some sort of environment it sets up or some other thing. I was looking for a solution where it would still use that same thing instead of specifically coding the context again. At that point I could just do a normal python string.replace().

Thank you very much for spending time on this Ken.

5

Wanted to mention that most of what I tried, along with what you provided, was from https://stackoverflow.com/questions/2167269/load-template-from-a-string-instead-of-from-a-file

6

I’ve found a couple different examples like the one you’ve found - the problem with most of them is that they are old and out of date. (Originally asked in 2010!)

Notice in my example that I’m passing the context containing the variable data to the render function. You will need to do the same. You’ll need to know what objects a particular template is looking for, and then supply them to render.

7

Hey THANKS again, I was thinking about this a little more and realized I may not have to pass ever single thing individually into Context. I tried passing the existing obj and that worked! It still has me hardcoding things that already exist in the current runspace but only at a higher level (just each obj).

In [34]: test = " {{ server.id }}, {{ server.group }}, {{ server.status }}, etc"

In [35]: context = Context({‘server’:server})

In [36]: st = Engine().from_string(test)

In [37]: st.render(context)
Out[37]: ’ 23946, Development, ACTIVE, etc’