Add support for remote state blocks

[hubbcaps]

Currently TF Local doesn't handle remote state blocks, and remote state blocks don't inherit endpoint overrides from the main backend block in terraform.

This code addition adds support for iterating over all remote state blocks, basically in the same exact manner as we grab the main backend block, and injects the endpoints needed to the configuration overrides.

I'm gonna be real, I have zero idea on how to even go about setting up pytest locally for this, let alone writing the actual unit tests, so I apologize for that part lacking. This is working well for me locally currently though, it overrides and adds in the blocks perfectly and I no longer need to specify environment variable overrides for the necessary endpoints.

[hubbcaps]

Found a bug with Workspaces on this if they exist in the data blocks, not getting properly scooped up, trying to add some code to handle that case but it keeps choking out when the workspace is interpreted from the current workspace. Will continue to work on it.

[bentsku]

Thanks a lot, again, for adding this!

I've taken the opportunity to update and restructure the code a bit to make use of the "s3 backend generation" in both remote_state and backend, and added some tests.

I've also fixed the issue mentioned with #74 as I was refactoring the generation of the s3 backend, might as well. Also added a test for the fix.

Would you mind trying it out and see if it still works for you?

Found a bug with Workspaces on this if they exist in the data blocks, not getting properly scooped up, trying to add some code to handle that case but it keeps choking out when the workspace is interpreted from the current workspace. Will continue to work on it.

Regarding the Workspace issue, I can also have a look if you'd like, if you think that's fully blocking this PR. We could also tackle it in a follow-up PR if you'd like, and I'd only release a new version once we fix the workspace issue?

In any case, if you'd like me to also have a look, would you have a minimal reproducible sampel so I could reproduce the issue and work on it? Thanks!

[hubbcaps]

Absolutely, if you'd like to take a stab at it I'd love some help, my attempts so far have been a bit heavy handed imo.

Here are two example remote state blocks, one whos workspace is interpreted, and one who's workspace is hardcoded:

data "terraform_remote_state" "terraform_infra" {
  backend   = "s3"
  workspace = terraform.workspace

  config = {
    bucket               = "our-org-tfstate"
    workspace_key_prefix = "terraform-infrastructure/place"
    key                  = "terraform.tfstate"
    region               = "us-west-2"
    assume_role = {
      role_arn = "arn:aws:iam::000000000000:role/state-bucket-role"
    }
  }
}


data "terraform_remote_state" "build_infra" {
  backend   = "s3"
  workspace = "build"

  config = {
    bucket               = "our-org-tfstate"
    workspace_key_prefix = "terraform-infrastructure"
    key                  = "terraform.tfstate"
    region               = "us-west-2"
    assume_role = {
      role_arn = "arn:aws:iam::000000000000:role/state-bucket-role"
    }
  }
}

Essentially the only thing missing is grabbing that additional top level key for workspace and shoving it in if it exists

[bentsku]

I've updated the code to now handle the workspace issue, both when interpreted and hardcoded. I've also added some more tests for workspace in this context.

This is a great addition, thanks a lot for pushing for it, it makes it all more transparent and reduce the amount of "if" and workarounds.

Adding the documentation to remote_state_data for context: https://developer.hashicorp.com/terraform/language/state/remote-state-data

Thanks again! Once merged, I'll release a new 0.23.0 version and write here on the PR once it is available publicly.

[bentsku]

The new version has been published: https://pypi.org/project/terraform-local/0.23.0/