Kubernetes Advocate · follow
published in · 10 min read · June 8, 2022
--
If you are new to cloud and container concepts, the following pages will give you more background information:
- Why use AWS?
- Why use containers?
- Why use infrastructure as code?
- EC2 or AWS Fargate?
Amazon Web Services (AWS)Since 2006, AWS has provided IT infrastructure services, now commonly referred to as cloud computing. AWS has the broadest and deepest set of mature, reliable cloud services to support your applications. Deploying applications with AWS has several key benefits:
AWS allows you to avoid large upfront investments in the infrastructure to run your applications. Instead, you only pay for what you use and use the tools available to do more with less and avoid wasting money.
With AWS, you also benefit from enormous economies of scale. Because AWS serves millions of active customers each month, overhead can be optimized more effectively, reducing costs passed on to customers at lower prices. As AWS has grown, it has cut prices more than 60 times.
AWS managed services enable you to build your business faster and better with fewer engineering resources. Instead of reinventing the wheel for every application you develop, you can leverage AWS tools to remove the "undifferentiated heavy lifting" and allow you to focus on specific parts of your business that make it unique.
Not only does this help you go from idea to implementation faster, it also allows your engineering organization to be more responsive to market needs.
AWS already powers many of the largest applications in the world, includingEpic Game's popular online game Fortnite, with over 125 million players generating 2 petabytes of data per month.
The capacity and distribution of the AWS cloud can help companies like Epic Games succeed at all stages, from small services to large-scale global operations. Fortnite experienced incredibly rapid growth and needed to scale out to 24 AWS-run Availability Zones around the world. Also, they have a 10-fold difference between peak and low peaks in each region. AWS gives Epic Games the capacity they need when they need it, while allowing them to scale down and stop paying when they don't.
RoustaboutContainers have become a popular open source standard for developing, packaging, and running applications at scale. There are several key benefits to using Docker:
Docker containers provide you with a reliable way to collect your application components and package them into a build artifact. This is important because modern applications are often composed of various parts, not only code, but also dependencies, binaries or system libraries.
With Docker, you can write adocumentDescribe the application. For example:
Slave node: 9 as build
working directory /srv
Add package.json.
run npm installFrom node: 9-slim
Copy --from=build /srv .
Add to . .
Exposure 3000
cmd["node", "index.js"]
This file is a script that describes how to set up a development environment to fetch Node.js dependencies from NPM, install them (also compiling any binary dependencies), and then package a slim container image with the final build product for delivery to any machine. Needed to run the container.
Because Docker containers allow an application to carry all its dependencies with it, you can take your container anywhere you want and have your application run reliably. Whether it's a local development laptop, a local data center, or a cloud provider, containers will run there.
Docker containers increase efficiency by providing a lightweight, efficient isolation model. Unlike heavier virtual machines, you can run many small docker containers on a single machine. It's not uncommon to populate an EC2 instance with 10-20 Docker containers.
This helps you use the cloud resources you paid for more efficiently. Rather than spending money on a large EC2 instance and only getting 10-20% utilization out of the instance, pack many application containers onto the instance and get 70-80% utilization.
Infrastructure as code is the process of configuring and managing cloud resources by writing human-readable and machine-consumable template files. For AWS cloud development, the built-in choice for infrastructure as code isAWS CloudFormation.
Using AWS CloudFormation, you write a description of a resource to be created on your AWS account, and then ask AWS CloudFormation to bring that description to life. For example, the following YAML template fragment describes the AWS ECS service resource to be created:
Serve:
Type: 'AWS::ECS::Service'
DependsOn: 'Service Discovery Service'
characteristic:
Service Name: "Application"
Cluster: 'production'
Deployment configuration:
Maximum percentage: 200
Minimum health percentage: 75
Expected count: 5
Task Definition: !Ref 'TaskDefinition'
Service registry:
- RegistryArn: !GetAtt ServiceDiscoveryService.Arn
Container port: 3000
Container name: 'myapp'AWS CloudFormation takes this template and then takes care of creating, updating, and deleting resources on your AWS account as described in the template. If you add a new resource to the file, CloudFormation will create that resource on your account. If you update a resource, CloudFormation will update or replace any existing matching resources. If you delete a resource from a template, it will be cleaned up and deleted from your AWS account.
Infrastructure as code brings many benefits:
- Visibility: Infrastructure as code templates can very clearly reference resources in your account and their settings. You don't have to navigate to the web console to check parameters.
- Stability: If you accidentally change the wrong setting or delete the wrong resource in the web console, you can break everything. Infrastructure-as-code helps with this, especially when combined with version control such as Git.
- Scalability: With infrastructure as code, you can write it once and reuse it many times. This means that a well-written template can be used as the basis for multiple services in multiple regions around the world, making it easier to scale out.
- Security: Infrastructure as code again gives you a unified template to deploy your architecture. If you create a well-secured schema, you can reuse it multiple times and know that every deployed version follows the same settings.
- Transactional: CloudFormation not only creates resources on your AWS account, but also waits for them to stabilize as they start. It verifies that the configuration was successful, and if something goes wrong, it can gracefully roll back the infrastructure to a past known good state.
There are two main models for how to run containers on AWS:
- EC2 (deploy and manage clusters of EC2 instances running containers)
- AWS Fargate (runs containers directly, without any EC2 instances)
Both are perfectly valid techniques for operating containers in a scalable and reliable manner. Which one you choose mainly depends on what factors you want to optimize for.
With the introduction of EC2, type billing is based on the cost of the underlying EC2 instance. This allows you to optimize prices using billing models such as Spot Instances (pay a low price for an instance) or Reserved Instances (get a flat discount for using an instance for a specific period). However, it's your responsibility to make sure you pack your containers densely onto your instances to get the most out of them, otherwise, you'll be wasting money.
Billing with the AWS Fargate launch type is based on how many CPU cores and GB of memory your tasks require per second. You only pay for the capacity your tasks use, and you no longer pay for unused EC2 capacity.
High workload, optimized for price
Large workloads, optimized for low overhead
Small workload, occasional burst
small workload
batch workload
Below are all common architectural patterns that apply to the vast majority of applications. Some are designed for external, public-facing services, while others are designed for internal, private services:
public service, public network
Public-facing services are one of the most common architectural patterns for deploying containers on AWS. It's great for:
- A static HTML website, possibly hosted by NGINX or Apache
- A dynamically generated web application, possibly served by a Node.js process
- An API service intended for public access
- A public-facing endpoint designed to receive push notifications, possibly from Amazon SNS (Simple Notification Service)
- Edge services that require outbound connections to other services on the Internet
At a high level, the architecture looks like this:
Everything is deployed in an Amazon Virtual Private Cloud (VPC), which has a subnet exposed to the internet. An internet gateway is attached to allow resources launched in the VPC to accept connections from the internet and initiate connections to the internet. Inside a VPC, each resource has its public IP address. Some resources are included:
- A public-facing load balancer that accepts inbound connections on specific ports
- One or more EC2 instances hosting application containers are configured to accept inbound connections from the load balancer on specific ports (optionally from any source).
Deployed in a self-managed EC2 cluster
Use these templates: Launch a custom EC2 cluster in a public VPC with an internet gatewayemissiondownloadAdd external public ALB entryemissiondownloadDeploy a public EC2 serviceemissiondownload
Deploy in AWS Fargate
Use these templates: Launch an AWS Fargate cluster in a public VPC with an internet gatewayemissiondownloadAdd external public ALB entryemissiondownloadDeploy a public Fargate serviceemissiondownload
Sometimes you want to create a public-facing service, but you want tighter control over the service's network. This pattern applies to many of the same use casespublic service, but it is especially used in the following cases:
- A public-facing service that requires an extra layer of security hardening, even without a public IP address to which an attacker can send requests directly.
- A service that requires massive horizontal scaling while not being limited by the number of IP addresses available.
- A service that initiates outbound connections, but for the public, you want those connections to originate from a specific and limited set of IP addresses that can be whitelisted.
At a high level, the architecture looks like this:
Everything is deployed in an Amazon Virtual Private Cloud (VPC) with two subnets:
- Public subnet: There is an attached Internet gateway that allows resources launched in this subnet to accept connections from the Internet and initiate connections to the Internet. Resources in this subnet have public IP addresses.
- Private subnet: for internal resources. Instances in this subnet cannot directly access the Internet, only private IP addresses inside the VPC, which cannot be directly accessed by the public.
The public-facing subnet hosts some resources:
- Public-facing load balancer: Accepts inbound connections on specific ports and forwards acceptable traffic to resources within the private subnet.
- NAT Gateway: A bridge that allows resources within a private subnet to initiate outbound communication to the Internet while disallowing inbound connections.
Private subnets are used to run your application containers. EC2 instances hosting containers do not have public IP addresses, only private IP addresses inside the VPC. Therefore, if your application initiates an outbound connection, the connection will be routed through the NAT gateway in the public subnet. Also, no traffic can reach your container directly. Instead, all inbound connections must go to the load balancer, which will choose whether to pass inbound connections to protected containers within the private VPC subnet.
Deployed in a self-managed EC2 cluster
Use these templates: Launch a custom EC2 cluster in a private VPC with a NAT gatewayemissiondownloadAdd external public ALB entryemissiondownloadDeploy a public-facing private network EC2 serviceemissiondownload
Deploy in AWS Fargate
Use these templates: Launch an AWS Fargate cluster in a private VPC with a NAT gatewayemissiondownloadAdd external public ALB entryemissiondownloadDeploy Fargate services for public and private networksemissiondownload
Check the CloudFormation output of the ALB ingress template for the service's public-facing URL. You can add your custom hostname using Route53 to create a CNAME for this address.
Autoscaling is important to ensure your services stay online during unexpected increases in traffic. In EC2 and AWS Fargate, one way to ensure that services scale automatically is to scale up and down the number of replicas of application containers running in the cluster.
Auto scaling works like this:
The following template automatically sets up CloudWatch alarms and autoscaling policies and attaches them to ECS services.
Add autoscaling to your service
Use these templates: Scale services up and down based on CPU usageemissiondownloadScale services up and down based on memory usageemissiondownload
These templates try to keep their respective resources (CPU or memory) above 20% but below 70%. By default, if the CPU or memory is 70% - 85%, they add one more container; if it's between 85% - 95%, they add two containers; if it's over 95%, they add three containers. If the CPU goes below 20%, they stop the container. You can adjust the exact threshold by downloading the template and adjusting the values in it.
️Follow usLinkedIn,Twitter,Facebook, andinstagram
If this post was helpful to you, please hit the clap 👏 button below a few times to show your support! ⬇
FAQs
How do ECS and Fargate work together? ›
AWS Fargate is a technology that you can use with Amazon ECS to run containers without having to manage servers or clusters of Amazon EC2 instances. With AWS Fargate, you no longer have to provision, configure, or scale clusters of virtual machines to run containers.
How many tasks can be run in ECS Fargate? ›You can now launch up to 500 concurrent ECS tasks and EKS pods running on Fargate On-Demand and 500 concurrent ECS tasks running on Fargate Spot, up from 100 and 250 respectively.
Which AWS service enables AWS users to run containers on Amazon ECS? ›Outposts is a fully managed service that extends AWS infrastructure, AWS services, APIs, and tools to virtually any connected site. With Amazon ECS on Outposts, you can manage containers on-premises with the same ease as you manage your containers in the cloud.
What is the maximum size of ECS Fargate? ›Short description. By default, Fargate tasks that are launched with platform version 1.40 include a task storage size of 20 GiB as a single ephemeral volume. If you need more than 20 GiB of storage, you can configure more storage using two options: Use Fargate's ephemeral storage option for storage up to 200 GiB.
Can we run multiple containers within a single task in ECS? ›You can use an Amazon ECS task definition to specify multiple containers. All the containers that you specify are deployed along the same compute capacity. Don't use this feature to add multiple application containers to the same task definition because this prevents copies of each application scaling separately.
What is the difference between ECS and ECS Fargate? ›If you need auto-scaling or run containers in a serverless environment, then Fargate is the right choice. But, ECS is better if you need more flexibility or are on a budget. Overall, both services are excellent choices for running containers in AWS.
What are the limitations of Fargate? ›The default nofile and nproc soft limit is 1024 and the hard limit is 65535 for Fargate Pods.
Can a Fargate task have multiple containers? ›In Fargate, when you launch multiple containers as part of a single task, they can also communicate with each other over the local loopback interface. Fargate uses a special container networking mode called awsvpc, which gives all the containers in a task a shared elastic network interface to use for communication.
How many containers can be defined in a task definition in ECS? ›The task definition is a text file, in JSON format, that describes one or more containers, up to a maximum of ten, that form your application.
How can I monitor high CPU utilization for Amazon ECS tasks on Fargate? ›- View Amazon CloudWatch metrics in either the Amazon ECS console or CloudWatch console.
- Enable the awslogs log driver.
- Add the required log configuration parameters to your task definition.
- View your logs in the CloudWatch console.
What is the maximum percent in ECS deployment? ›
The default value for the maximum percent is 200 percent. If a service is using the blue/green ( CODE_DEPLOY ) deployment type and tasks that use the EC2 launch type, the minimum healthy percent and maximum percent values are set to the default values.
Does Amazon ECS handle container orchestration? ›Amazon ECS is a fully managed container orchestration service that helps you to more efficiently deploy, manage, and scale containerized applications.
Which AWS service allows you to run Docker containers? ›Amazon Elastic Container Service (Amazon ECS) is the AWS service you use to run Docker applications on a scalable cluster.
Which AWS service is used to run containerized applications on AWS? ›Amazon Elastic Container Service (Amazon ECS) is a fully managed container orchestration service that provides the most secure, reliable and scalable way to run containerized applications.
Can you use Fargate without ECS? ›Can I use Fargate without ECS? AWS Fargate is a container serverless compute engine that integrates with two platforms: Amazon Elastic Container Service (ECS) and Amazon Elastic Kubernetes Service (EKS). So, you can use Fargate without ECS – assuming that you will be using it with EKS.
What is the difference between ECS Fargate and Lambda container? ›Fargate also supports various runtime environments and you have more control over compute capacity allocation when you register your containers with ECS. With Lambda, you are limited to 250MB deployment package size when uploaded to S3 or 10GB when using container images, and 10GB available disk space.
Why use ECS on EC2 instead of Fargate? ›If you want to run a machine learning model, EC2 is the right choice. AWS EC2 has the following advantages over Fargate: It is relatively cheaper for running fully utilized clusters, especially if your task consumes a set or predictable amount of vCPU and memory.
How many containers can you run on a host machine? ›Using this simple calculation, we can estimate that we can run about 1,000 containers on a single host with 10GB of available disk space.
How many Docker containers are allowed to run on the same infrastructure? ›43. How many containers can run per host? There can be as many containers as you wish per host. Docker does not put any restrictions on it.
Can we run more than one process in a container? ›It's ok to have multiple processes, but to get the most benefit out of Docker, avoid one container being responsible for multiple aspects of your overall application. You can connect multiple containers using user-defined networks and shared volumes.
What are the advantages of Fargate over ECS? ›
Fargate does away with the need for needless server costs and over-provisioning. No other jobs or pods share CPU, memory, storage, or network resources as individual ECS tasks or EKS pods run in their exclusive kernel runtime environment. This ensures workload separation and better security for each job or pod.
Why choose Kubernetes over ECS? ›Kubernetes can handle requirements with complete flexibility. With Kubernetes, you have complete and granular control over how your workload can scale. This allows you to avoid vendor lock-in with ECS or any other container services when you need to make the switch to a more powerful platform.
Why ECS is better than Lambda? ›Choose ECS when dealing with longer-running jobs, as it avoids the Lambda timeout limit above. You need to schedule jobs. ECS provides a service scheduler for long running tasks and applications, along with the ability to run tasks manually.
Does Fargate have a timeout? ›I thought one reason to use AWS Fargate over Lambda is because it has not its limits, like max runtime (and others). However, today I discovered that the parameter stopTimeout which is by default set to 30s while its maximum is 120s .
Why use Fargate instead of Lambda? ›Lambda allows you to customize IAM roles for each function or service, while Fargate customizes each container and pod. Fargate tasks run in an isolated computing environment wherein CPU or memory is not shared with other tasks. Similarly, Lambda functions run in a dedicated execution environment.
Do you need VPC for Fargate? ›To run Fargate tasks in a private subnet without internet access, use VPC endpoints. VPC endpoints allow you to run Fargate tasks without granting the tasks access to the internet. The required endpoints are accessed over a private IP address.
How many containers could be run in the same pod? ›Pods in a Kubernetes cluster are used in two main ways: Pods that run a single container. The "one-container-per-Pod" model is the most common Kubernetes use case; in this case, you can think of a Pod as a wrapper around a single container; Kubernetes manages Pods rather than managing the containers directly.
How many Docker containers can I run on one machine? ›Runs Eight Containers per Host. The median company that adopts Docker runs eight containers simultaneously on each host, a figure that has climbed steadily over the years.
Does Fargate need a cluster? ›AWS Fargate is a technology that you can use with Amazon ECS to run containers without having to manage servers or clusters of Amazon EC2 instances. With Fargate, you no longer have to provision, configure, or scale clusters of virtual machines to run containers.
How many applications can run in a container? ›Container-based application design encourages certain principles. One of these principles is that there should just be one process running in a container. That is to say, a Docker container should have just one program running inside it.
What is the difference between a task and a container? ›
A Task Definition is a collection of 1 or more container configurations. Some Tasks may need only one container, while other Tasks may need 2 or more potentially linked containers running concurrently.
How many Docker containers can I run on an EC2 instance? ›Docker containers improve efficiency by providing a lightweight, efficient isolation model. Unlike a heavier virtual machine, you can run many small docker containers on a single machine. It isn't uncommon to fill an EC2 instance with 10–20 Docker containers.
What is the most important metric to monitor in ECS? ›Monitoring ECS on EC2
Some of the most important metrics to monitor include: CPU and memory usage: You should monitor the CPU and memory usage of your EC2 instances, as well as the individual containers running on them.
- Check your application logs to see if your tasks are performing CPU-intensive operations. The logs are exported to various destinations, depending on your logging driver. ...
- Check for an increase in incoming traffic to your Amazon ECS service by viewing the CloudWatch metrics of your load balancer.
Domino measures CPU usage as a percentage of single core utilization. Therefore a machine with multiple cores can regularly utilize greater than 100% of a single processing core's maximum frequency. For instance a m4. xlarge EC2 machine in AWS has 4 CPU cores and would show a maximum utilization of 400%.
How many containers can run in ECS? ›You can now launch up to 5,000 instances per cluster, an increase from 2,000. The limit increase enables customers to scale further and improve manageability of their clusters. The new limit applies in all regions. The higher limit is reflected in your account automatically and you do not have to take any action.
How many tasks can be run in ECS cluster? ›Amazon Elastic Container Service (Amazon ECS) today increased the default service quotas for ECS tasks per service and services per cluster. You can now launch up to 5,000 tasks per service and 5,000 services per cluster, an increase from 2,000 each.
How do I speed up my Fargate? ›- Over allocate the CPU.
- Reduce the deregistration delay.
- Set the health check threshold to 2 and interval to 5 seconds. don't forget to account for a health check grace period if your app needs it.
EKS enables a greater degree of portability and reduces lock-in risks, as compared to ECS. Because it is proprietary, ECS has no equivalent in other public clouds. EKS is based on Kubernetes, which is open source and can run in any public cloud or on-premises location.
Is Amazon ECS the same as Docker? ›Amazon ECS uses Docker images in task definitions to launch containers. Docker is a technology that provides the tools for you to build, run, test, and deploy distributed applications in containers. Docker provides a walkthrough on deploying containers on Amazon ECS.
Which of the two deployment types are supported by ECS? ›
An Amazon ECS deployment type determines the deployment strategy that your service uses. There are three deployment types: rolling update, blue/green, and external. You can view information about the service deployment type on the service details page, or by using the describe-services API.
Which service allows you to run serverless containers on Amazon ECS? ›AWS Fargate is a serverless, pay-as-you-go compute engine that lets you focus on building applications without managing servers. AWS Fargate is compatible with both Amazon Elastic Container Service (ECS) and Amazon Elastic Kubernetes Service (EKS).
Which AWS service is used for container orchestration? ›Amazon Elastic Container Service (ECS)
Easily deploy, manage, and scale containerized applications with AWS' fully managed container orchestration service.
- Create the Docker image.
- Create an ECR registry.
- Tag the image.
- Give the Docker CLI permission to access your Amazon account.
- Upload your docker image to ECR.
- Create a Fargate Cluster for ECS to use for the deployment of your container.
- Create an ECS Task.
- Run the ECS Task!
Amazon ECS supports Fargate technology and customers canchoose AWS Fargate to launch their containers without having to provision or manage Amazon EC2 instances. AWS Fargate is the easiest way to launch and run containers on AWS.
Which AWS service is a compute engine for Amazon ECS that allows you to run containers links to an external site without having to manage servers or clusters? ›AWS Fargate is a compute engine for Amazon ECS that allows you to run containers without having to manage servers or clusters. With AWS Fargate, you no longer have to provision, configure, and scale clusters of VMs to run containers.
How many Docker containers can be run on a single machine? ›Using this simple calculation, we can estimate that we can run about 1,000 containers on a single host with 10GB of available disk space.
Can you run multiple containers at once? ›Yes. You can run multiple containers on one server, and it is not restricted to the number of CPUs. Your command creates and starts exactly 1 container that has access to at most 16 CPUs (and in the 2nd example only precisely CPUs 0-15). What you want to do is execute your command N times for N containers.
What is the maximum number of container you can run per machine? ›If you take the standard 32768 value then that's at most 16384 container processes.
How many containers can be run per host? ›The number of containers per host depends on the design ratio of the host and the workload ratio of the containers. Both ratios are Throughput/Capacity ratios. In the old days, this was called E/B for execution/bandwidth. Execution was cpu and banwidth was I/o.
What is the average number of containers per host? ›
The typical organization that uses a container orchestrator runs 11.5 containers per host, as compared to about 6.5 containers per host in unorchestrated environments.
Can one EC2 instance have multiple containers? ›1 Answer. Yes, it is possible. When you configure a container, you specify memory and CPU usage. That configuration is later used by ECS to schedule or pack an EC2 with Docker containers.
Can a container have multiple Microservices? ›While it is possible to run multiple microservices in the same container, it is not recommended. To manage each microservice independently, each one should have its own Docker container, which can be orchestrated using Docker Compose or Kubernetes. To achieve this, a Dockerfile must be created for each microservice.
What is the maximum number of EC2 instances per AWS account? ›By default, AWS has a limit of 20 instances per region. This includes all instances set up on your AWS account. To increase EC2 limits, request a higher limit by providing information about the new limit and regions where it should be applied.
What is the limitation of Fargate? ›The default nofile and nproc soft limit is 1024 and the hard limit is 65535 for Fargate Pods.
What happens if a source has two containers? ›Note: A work can sometimes have two containers. A container can be nested in another container. When there is more than one container, include as much information as possible about each container to help the reader understand and locate the original source.
Is it good to have multiple containers in a pod? ›The primary reason that Pods can have multiple containers is to support helper applications that assist a primary application. Typical examples of helper applications are data pullers, data pushers, and proxies. Helper and primary applications often need to communicate with each other.