[{"data":1,"prerenderedAt":369},["ShallowReactive",2],{"\u002Fen\u002Fpost\u002F2021\u002F03\u002Fterraform-design-best-practices":3},{"id":4,"title":5,"author":6,"body":7,"createdAt":355,"description":356,"extension":357,"meta":358,"navigation":359,"path":360,"seo":361,"slug":362,"stem":363,"tags":364,"__hash__":368},"posts\u002Fposts\u002F2021\u002F03\u002Fterraform-design-best-practices.md","Terraform Design Best Practices",null,{"type":8,"value":9,"toc":341},"minimark",[10,15,19,22,45,48,60,69,75,80,88,93,98,101,105,109,118,132,135,139,154,157,160,177,180,183,194,201,209,213,216,219,225,229,248,287,291,299,302,319,327],[11,12,14],"h2",{"id":13},"intro","Intro",[16,17,18],"p",{},"As someone who believes in empowering people and distributing power in order to\nachieve higher outcomes I always felt that the best existing best-practices\nproposals don't touch some key aspects (IMHO) on code evolution and business\nstructures.",[16,20,21],{},"Therefore, this document shall compose on the previous ones and extend them with\nsome self-service Ops and micro-services spice to the mix.",[16,23,24,25,32,33,38,39,44],{},"On ",[26,27,31],"a",{"href":28,"rel":29},"https:\u002F\u002Fwww.terraform-best-practices.com",[30],"nofollow","Terraform best practices"," great insights on how to write code inside a\nmodule is provided, e.g. ",[26,34,37],{"href":35,"rel":36},"https:\u002F\u002Fwww.terraform-best-practices.com\u002Fnaming",[30],"naming conventions",", ",[26,40,43],{"href":41,"rel":42},"https:\u002F\u002Fwww.terraform-best-practices.com\u002Fcode-structure#getting-started-with-structuring-of-terraform-configurations",[30],"Terraform file naming",".",[16,46,47],{},"We can't leave Terragrunt epic blog post unmentioned:",[49,50,51],"ul",{},[52,53,54,59],"li",{},[26,55,58],{"href":56,"rel":57},"https:\u002F\u002Fblog.gruntwork.io\u002F5-lessons-learned-from-writing-over-300-000-lines-of-infrastructure-code-36ba7fadeac1",[30],"5 Lessons Learned From Writing Over 300,000 Lines of Infrastructure Code",";",[16,61,62,63,68],{},"As well as the ",[26,64,67],{"href":65,"rel":66},"https:\u002F\u002Fterragrunt.gruntwork.io\u002Fdocs\u002Fgetting-started\u002Fquick-start\u002F#promote-immutable-versioned-terraform-modules-across-environments",[30],"Terragrunt documentation pointing"," \"one of the most important\nlessons\" is that:",[70,71,72],"blockquote",{},[16,73,74],{},"large modules should be considered harmful. That is, it is a Bad Idea to define\nall of your environments (dev, stage, prod, etc), or even a large amount of\ninfrastructure (servers, databases, load balancers, DNS, etc), in a single\nTerraform module. Large modules are slow, insecure, hard to update, hard to code\nreview, hard to test, and brittle (i.e., you have all your eggs in one basket).",[76,77,79],"h3",{"id":78},"bad-idea-capitalized","\"Bad Idea\" capitalized!",[16,81,82,83,44],{},"Which is totally true, as this \"Bad Idea\" usually coming from a lack of care\ntowards Terraform code design tend to be harmful in the long run, with a\ntendency towards making the implementation a ",[26,84,87],{"href":85,"rel":86},"https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FBig%5Fball%5Fof%5Fmud",[30],"big ball of mud",[70,89,90],{},[16,91,92],{},"A Big Ball of Mud is a haphazardly structured, sprawling, sloppy,\nduct-tape-and-baling-wire, spaghetti-code jungle. These systems show\nunmistakable signs of unregulated growth, and repeated, expedient repair.\nInformation is shared promiscuously among distant elements of the system,\noften to the point where nearly all the important information becomes global\nor duplicated.",[70,94,95],{},[16,96,97],{},"The overall structure of the system may never have been well defined.",[16,99,100],{},"Oftentimes, Terraform code implementation fluctuate towards mono-repositories\n(a.k.a. monorepos) containing all the specification in a single place. In order\nto tame the chaos, the Terraform state needs to be at least sub-divided into\nlogical sections.",[11,102,104],{"id":103},"design","Design",[76,106,108],{"id":107},"shallow-tree-of-shared-resources","Shallow \"tree\" of shared resources",[16,110,111,112,117],{},"Following the ",[26,113,116],{"href":114,"rel":115},"https:\u002F\u002Fwww.terraform-best-practices.com\u002Fcode-structure#common-recommendations-for-structuring-code",[30],"recommendations for structuring code"," one of the proposals is to\nkeep a shallow \"tree\" of resources and modules. This tree produces a small and\nclear distribution of Terraform code.",[16,119,120,121,126,127,131],{},"Why a shallow \"tree\" of resources? It helps achieving a short amount of\nresources and modules that result in a small ",[26,122,125],{"href":123,"rel":124},"https:\u002F\u002Fwww.terraform.io\u002Fdocs\u002Flanguage\u002Fstate\u002Fremote.html",[30],"remote state"," file. With a small\nremote state we speed-up the development process and reduce waste (",[128,129,130],"em",{},"Muda"," in the\nToyota 3M model), as the shallow tree enables faster executions of Terraform\n(less data to sync and compare).",[16,133,134],{},"The granularity level will be defined for each specific case (no silver bullet)\nbalancing the smallest and most feasible composition possible.",[76,136,138],{"id":137},"product-areas-aka-business-capabilities-structure-and-ownership","Product areas (a.k.a. Business capabilities) structure and ownership",[16,140,141,142,147,148,153],{},"Ideally, the ",[26,143,146],{"href":144,"rel":145},"https:\u002F\u002Fwww.terraform-best-practices.com\u002Fkey-concepts#composition",[30],"composition level"," would be organized around Product Areas (either\nsquads\u002Fcrews or guilds) with a fallback to shared technologies (e.g. vpc,\ndatabases). Therefore, Terraform compositions are designed around what Martin\nFowler ",[26,149,152],{"href":150,"rel":151},"https:\u002F\u002Fyoutu.be\u002FwgdBVIX9ifA?t=388",[30],"calls \"Business capabilities\""," in micro-services terminology, ideally the\nTerraform composition will follow the organizational structure so that each team\n\"owns\" (in both senses: ownership and freedom) its own state.",[16,155,156],{},"The main goal here is to structure the Terraform code as a reflection of the\norganization so that is fosters self-service Ops. If the Infrastructure as Code\nis mature enough to the point of having well-described Terraform modules,\neveryone should be empowered to define these modules by setting the parameters\naccording to their needs, without centralizing power on a Operations team.",[16,158,159],{},"The resource composition must gravitate towards the following (ordered by\npriority from higher to lower):",[161,162,163,174],"ol",{},[52,164,165,166],{},"Product Areas (ownership) directory structure:",[161,167,168,171],{},[52,169,170],{},"squad\u002Fcrew OR guild;",[52,172,173],{},"product.",[52,175,176],{},"Shared resources, around technologies.",[16,178,179],{},"Looking on the structure from bottom-up it starts from the product and then\nattributes the product to a crew through the directory tree.",[16,181,182],{},"e.g.:",[184,185,191],"pre",{"className":186,"code":188,"language":189,"meta":190},[187],"language-text","# Squad or Crew\nred-team\n└── payment     # Product (i.e. micro-service) name\n    └── main.tf # Any resource used by the payment product\n\n# Guild (organized around technology)\nback-end\n└─ monolith    # Shared application in terms of ownership\n   └── main.tf # Cloud resources used by the monolith\n","text","",[192,193,188],"code",{"__ignoreMap":190},[16,195,196,197,200],{},"On the example above, we can't ignore that ",[192,198,199],{},"monolith"," is a product with shared\nownership among back-end developers and therefore it is organized to follow the\nbusiness structure.",[16,202,203,204,208],{},"The structure is inspired ",[26,205,207],{"href":65,"rel":206},[30],"on Terragrunt's best-practices"," to some extend.\nHowever, it distinct from Terragrunt proposal in the way resources are divided,\nrather than organizing resources exclusively around technologies.",[76,210,212],{"id":211},"shared-resources-organized-around-technologies","Shared resources, organized around technologies",[16,214,215],{},"Oftentimes in organizations we will face shared resources among products, there\nis no way around reality. e.g. a shared VPC or SQL database.",[16,217,218],{},"However, these situations should be the exception and not the norm. Dealt\nsimilar to the organization of Terraform compositions around\nguilds\u002Ftechnologies.",[184,220,223],{"className":221,"code":222,"language":189,"meta":190},[187],"platform # as in Platform Engineering\n└── vpc\n    └── main.tf\n\nback-end\n└── database\n    └── main.tf\n",[192,224,222],{"__ignoreMap":190},[76,226,228],{"id":227},"files-inside-the-composition","Files inside the composition?",[16,230,231,232,236,237,38,240,243,244,247],{},"Ideally the files in the sub-directory (which specify the composition) are going\nto partially ",[26,233,235],{"href":41,"rel":234},[30],"follow this spec"," and include ",[192,238,239],{},"data.tf",[192,241,242],{},"terraform.tf"," and\n",[192,245,246],{},"providers.tf"," on top of that.",[49,249,250,257,263,269,275,281],{},[52,251,252,256],{},[253,254,255],"strong",{},"main.tf:"," contains locals, module and resource definitions;",[52,258,259,262],{},[253,260,261],{},"variables.tf:"," contains declarations of variables (i.e. inputs\u002Fparameters)\nused in main.tf;",[52,264,265,268],{},[253,266,267],{},"data.tf:"," contains data-resources for input data used in main.tf;",[52,270,271,274],{},[253,272,273],{},"outputs.tf:"," contains outputs from the resources created in main.tf;",[52,276,277,280],{},[253,278,279],{},"providers.tf:"," contains provider and provider's versions definitions;",[52,282,283,286],{},[253,284,285],{},"terraform.tf:"," contains the terraform back-end (e.g. remote state)\ndefinition;",[76,288,290],{"id":289},"what-about-terraform-modules","What about Terraform modules?",[16,292,293,298],{},[26,294,297],{"href":295,"rel":296},"https:\u002F\u002Fwww.terraform.io\u002Fdocs\u002Flanguage\u002Fmodules\u002Findex.html",[30],"Terraform modules"," are containers for multiple resources that are used together\nto achieve a shared goal. Modules can be used to create lightweight\nabstractions, facilitating reusability and distribution of Terraform code.",[16,300,301],{},"Therefore, we assume that the following are anti-patterns that make Terraform\nmodules' reusability difficult:",[49,303,304,307,310,313,316],{},[52,305,306],{},"Configuration of Terraform Providers inside a module;",[52,308,309],{},"Implementation of Business logic and\u002For hard-coded parameters in a\nmodule;",[52,311,312],{},"Default values are specified in optional variables instead of\nhard-coding;",[52,314,315],{},"Modules should be self-contained and provide a clear contract.\nDependencies (pre-existing resources) must be specified through required\nvariables.",[52,317,318],{},"Modules must serve to a singular purpose. Multiple purpose must be\nachieved through composability of modules and not by \"monolithic\" modules.",[16,320,321,322,44],{},"Modules are abstractions that should be used to reduce the amount of code\nduplication, implementing the ",[26,323,326],{"href":324,"rel":325},"https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FDon%27t%5Frepeat%5Fyourself",[30],"DRY (don't repeat yourself) principle",[16,328,329,330,335,336,44],{},"On top of that, modules are an important factor to reduce the parity among\nenvironments, which helps to better address the ",[26,331,334],{"href":332,"rel":333},"https:\u002F\u002F12factor.net\u002F",[30],"Twelve-Factor App model"," in\nregards to ",[26,337,340],{"href":338,"rel":339},"https:\u002F\u002F12factor.net\u002Fdev-prod-parity",[30],"Factor X (ten)",{"title":190,"searchDepth":342,"depth":342,"links":343},2,[344,348],{"id":13,"depth":342,"text":14,"children":345},[346],{"id":78,"depth":347,"text":79},3,{"id":103,"depth":342,"text":104,"children":349},[350,351,352,353,354],{"id":107,"depth":347,"text":108},{"id":137,"depth":347,"text":138},{"id":211,"depth":347,"text":212},{"id":227,"depth":347,"text":228},{"id":289,"depth":347,"text":290},"2021-03-31T00:00:00+02:00","Composing on the existing Terraform best-practices documents to empower developers and distribute the power of managing Infrastructure. In doing so, some self-service Ops and micro-services architecture were added to the mix.","md",{},true,"\u002Fposts\u002F2021\u002F03\u002Fterraform-design-best-practices",{"title":5,"description":356},"terraform-design-best-practices","posts\u002F2021\u002F03\u002Fterraform-design-best-practices",[365,366,367],"infrastructure-as-code","cloud","terraform","IqLbKyCdWg0VIZ70HZpIxE1-m-hZNH2LRJmifbapGDU",1778441744084]