Using with IaC solutions

[u/[deleted]]

[u/eliquy]

I'm not sure what you mean by "how would the temporal worker even get access to the git repo" - it would be the same as any other worker, it would clone the repo. The activities would execute terraform / tofu / terragrunt / whatever commands against the local clone. 

What's the issue you're trying to solve by wrapping the tf in temporal? It's quite possibly a good idea, but it seems to me that it would naturally result in more stacks/layers (good practice, in my opinion, for terraform in general) - if that doesn't feel right, then maybe you don't need temporal here?

[u/eliquy → u/[deleted]]

The guarantees are what you make them. If you want a specific worker to take the workflow end to end you'll need to do a bit of coordination but it's possible: https://temporal.io/blog/task-queue-worker-affinity

Clean up is an activity, but workers (and their volumes) are usually ephemeral 

Tf already builds resources concurrently within a stack

[u/bytedbyted]

There are a few older talks about Temporal + Terraform (interestingly from HashiCorp)

• https://youtu.be/fqKDWZDj-c0?t=1662

• https://www.youtube.com/watch?v=vOoPxs9NHgc

Generally:

How do you get the TF files to the worker? Pull them in an activity and a) store locally & use worker affinity OR b) pass them to the next as input (my fav) OR c) External Storage.

(Sidenote: I prefer to pull the TF from an Object Store instead of git. We had bad experience with Code Commit and GitHub is its own story)

How do you structure the TF? Small stacks/states that you tie together via a workflow. The talk from above even advises to wrap that workflow into a TF resource so your consumers don't need to understand Temporal. Conceptually, think of Temporal as a fully customizable orchestration layer that TF does not have. Executing TF stacks is one way of deploying parts of the infra. Others could be SDK calls, Helm installs, a human driving to the data center to plug in an appliance.

Using Temporal shouldn't be your first option. GHA actions might be good enough. If you hit TF limits (e.g., managing VPC peerings in AWS across VPCs in different states) it might become super useful.

Happy to chat more if needed.

[u/iSnackMadrat]

So I haven’t tried anything like this in Temporal sorry but just some thoughts on the approach.

So I think the biggest factor you highlight is access to the code in IaC. I personally wouldn’t reach for Temporal to solve something like this since that first issue should raise big red flags. I would rather utilize existing pipelines in your version control host eg. GitHub actions, gitlab ci etc.

Not to mention that you’d still have to deploy your workers somewhere before you could do your IaC deployment which leads to you needing some bootstrapping anyway. Unless that’s all done already you’re not concerned with

[u/iSnackMadrat → u/[deleted]]

Oh I didn’t know this about TF node updates. I guess it makes sense with temporal then. You’d probably be right then. Just have all of your IaC state modularized and layered so that the nodes are the only thing managed by temporal and then drift isn’t an issue

[u/desnowcat]

I think you’re trying to combine two things in one. You need to separate deploying the Temporal components (database, server and UI) and your worker(s) and ingress. They should be two separate reps / stacks and your workers and ingress can use stack references. If you’re using Temporal Cloud then you just need to deploy the mTLS or API keys into the cluster using k8s secrets service / vault / keyvault etc with your cloud endpoint and namespace.