Troubleshooting AWS CDK: part 1 - Nested Stacks

5 min read, last updated on 2020-12-22

After familiarizing ourselves with theoretical concepts related to our new tool, it’s time to test it properly in the wild.

Testing…

Let’s create our environment for 20 users, as we initially wanted:

$ cdk deploy                         \
        -c numberOfParticipants=20   \
        CommonWorkshopResources      \
        ParticipantResources

Oops, something went wrong!

Reason: Too many resources in the stack.

Well, that’s was an expected-unexpected situation. Long story short, there is a non-adjustable by default limitation of how many resources you can declare in a single AWS CloudFormation equal to 200. You can probably overcome that after a lengthy discussion with AWS Support, never tried, though, because so many resources in a single stack clearly show a problem in a different place.

Nevertheless, such a class of problems (visualized in a better way than one that I will present in summary) is the signal of a trait that we need to deal with.

To be honest, that’s the biggest issue that I currently have with the AWS CDK that I cannot unsee: an additional layer of abstraction over AWS CloudFormation forces me to go up and down the stack when troubleshooting. As it is a trait of this solution, we embrace it with its consequences, which can disqualify the whole approach.

Solutions

So instead of explaining ourselves in front of the AWS Support, we can do two things:

1. We can prepare separate stacks via constructs and execute looping on a higher level.
2. We can employ Nested Stacks functionality here.

1st solution is relatively easy to implement, especially if you are familiar with the constructs (if not, please read my article about them here). It’s also a recommended way for AWS CDK in most cases to leave it as homework for the reader. Well, if there is a limit on the AWS CloudFormation side, it’s clear that we will hit it on the AWS CDK side too.1st solution is relatively easy to implement, especially if you are familiar with the constructs (if not, please read my article about them here). It’s also a recommended way for AWS CDK in most cases to leave it as homework for the reader.

Instead, I would like to present the 2nd approach. How to use nested stacks in this case?

Instead of a cdk.Construct, you need to introduce a NestedStack that will hold both ParticipantVPC and Participant constructs. First, we need to install the new dependency, as it resides in the @aws-cdk/aws-cloudformation module:

# Replace the version depending on your needs:

$ npm install --save @aws-cdk/aws-cloudformation@1.33.0

Then we need to introduce nested resource in the following way:

import * as cdk from '@aws-cdk/core';

import { NestedStack } from '@aws-cdk/aws-cloudformation';

import { ParticipantVpc } from '...';
import { Participant, ParticipantProps } from '...';

// Pay attention to the base class here:

export class ParticipantStack extends NestedStack {

  constructor(scope: cdk.Construct,
              id: string,
              props: ParticipantProps) {
    super(scope, id);

    new ParticipantVpc(
      this,
      `ParticipantVpcForUser${props.zeroPadded}`
    );

    new Participant(
      this,
      `ParticipantUser${props.zeroPadded}`,
      props
    );
  }

}

import * as cdk from '@aws-cdk/core';

import { ParticipantStack } from '...';
import { ParticipantProps } from '...';

const DEFAULT_NUMBER_OF_PARTICIPANTS: number = 1;

export class ParticipantsStack extends cdk.Stack {

  constructor(scope: cdk.Construct,
              id: string,
              props: ParticipantProps) {
    super(scope, id);

    const numberOfParticipants =
      parseInt(
        this.node.tryGetContext('numberOfParticipants') ||
          DEFAULT_NUMBER_OF_PARTICIPANTS,
        10
      );

    const prefixSize =
      numberOfParticipants.toString().length;

    for(let i = 1; i < numberOfParticipants + 1; ++i) {
      props.zeroPadded =
        i.toString().padStart(prefixSize, '0');

      new ParticipantStack(
        this,
        `ParticipantStack${props.zeroPadded}`,
        props
      );
    }
  }

}

Looks simple enough, so why is this not a recommended approach?

If you have worked with AWS CloudFormation, you are perfectly aware of the downsides of Nested Stacks. Let’s reiterate that for those who are not aware of it yet.

Nested Stacks are like typical AWS resources inside the parent stack. It means that they are managed as a whole via the parent stack. This can be both a benefit and a nightmare - depending on the context. They introduce very tight coupling and strict lifecycle management mechanism to the table.

In this case, this is an ideal solution and perfect for our use case - those stacks shouldn’t be updated separately and logically should be owned by the same parent.

Are we there yet?

Almost. We will hit one more issue that is not related to AWS CloudFormation nor AWS CDK.

As we are creating so many VPCs at once and doing this on a new account, we need to raise the limits - as you are allowed to create only 5 VPCs per region, per account by default.

Outside of VPC limit, we need to raise the limit for the following elements:

• Elastic IPs (required for NAT Gateways), under the name Number of EIPs - VPC EIPs.
• Number of IGWs available per region, under the name Internet gateways per Region.

So we need to raise 3 limits in total, luckily we can do it in a pretty easy way (and 2 out of 3 will be applied automatically). For this, I would like to advertise the AWS Service Quotas service page:

That place finally can help you with history and limit management from one single place (I don’t get why it arrived late in the process - in June 2019, better late than ever tho 😉).

Now our stack for 20 participants can finally move forward.

Summary

So far, so good! This time it runs and will create some stacks correctly. However, we will hit here a second problem. 😭

This one is more problematic because it is not deterministic! But we will tackle the new issue in the next article.