AWS Development Environment Building Guide | Explaining everything from service selection to security and costs

Amazon Web Services (AWS)Building a development environment on the cloud enables flexible development beyond the constraints of a local environment. In this article, we will summarize the features and selection points of major services such as AWS Cloud9, Amazon EC2, and the Code series, and explain configuration examples for different purposes, security design, and cost optimization concepts.

We will introduce practical steps for developing efficiently and safely on AWS, from cases where individual developers are setting up a learning and testing environment to configurations that assume corporate team development and CI/CD implementation.

AWS development environment building basics

Benefits of building a development environment on AWS

AWSThe biggest advantage of building a development environment on a distributed environment is that it provides flexibility and reproducibility that is difficult to achieve in a local environment. Since resources can be expanded only when needed, you can quickly respond to situations where you need to temporarily increase computing resources, such as for high-load testing or parallel builds. Once the work is finished, you can simply stop the resources, eliminating the need for fixed costs.

In team development, the ability to standardize the environment is an advantage. With AWS, you can quickly recreate a unified development environment using AMIs, container images, CloudFormation, Terraform, and more. This reduces issues caused by differences in each developer's environment and avoids problems like "it works on my PC." It also streamlines the onboarding of new members, leading to increased productivity for the entire team.

Furthermore, you can utilize the cloud's unique security features as standard, such as access management with IAM, and auditing and threat detection with CloudTrail and GuardDuty. This clarifies access control, making it easier for organizations to operate a safe development environment.

Differences from the local environment

Compared to a local environment, the development environment on AWS has the advantage of being extremely similar. Development and testing can be done under conditions similar to the production environment, including the OS, middleware, and network conditions, reducing issues caused by differences in dependencies.

It's also easy to integrate with CI/CD pipelines. Pushing to a repository triggers automatic build and test execution, making it easy to implement workflows such as creating a preview environment for each PR. Processing that would be heavy locally can be executed smoothly by offloading it to the cloud.

The boundaries of networks and permissions are also clear. Communication routes can be defined using VPCs, subnets, and security groups, and are not dependent on the state of individual devices. This reduces the risk of vulnerabilities in PCs taken outside the company and the risk of important data being accessed from personal devices.

Points to consider when building

When building a development environment on AWS, be sure to organize the following points in advance.

■ Clarifying the purpose

The services you choose and how you think about costs will vary depending on whether you are looking for a simple environment for learning, a collaborative development environment for team use, or a testing environment with an eye toward production.

■ Number of users and their roles

The required access control differs between personal use and team use. IAM design is required according to roles, such as developer, reviewer, and operations manager.

■Security requirements

Incorporate authentication methods (MFA/SSO), network isolation (VPC/subnet/security group), secret information management (Parameter Store/Secrets Manager) from the beginning. Deciding on log recording and retention period will also make future audits and troubleshooting easier.

■ Cost management

Set a budget based on the expected instances, usage time periods, and number of units. By standardizing tagging, you can manage costs separately for each team or project. By notifying thresholds with AWS Budgets, you can prevent unexpected cost inflation.

■ Operability and reproducibility

By using IaC with CloudFormation and Terraform, we can ensure the reproducibility of the environment and change management. It is also important to have a system that reduces the hassle of daily operations, such as automating startup and shutdown schedules and regularly deleting unnecessary resources.

■ Preparation for future expansion

Minimizing dependencies will make it easier to change the configuration later as the number of developers increases, the service configuration expands, and CI/CD becomes more advanced.

AWS development environment configuration patterns

With AWS, you can choose from several configuration patterns depending on the scale of development and purpose. Here we will organize four representative patterns and explain in what cases they are suitable.

For personal development (AWS Cloud9/Amazon EC2 only)

If the purpose is personal development or learning/testing, a simple configuration is sufficient.

The easiest way is to use a browser development environment using AWS Cloud9. Because the IDE is on the cloud, there is no need to set up a local environment; you can start working immediately after logging in. Also, because the development environment and dependencies are unified on AWS, reproducibility of behavior is increased.

If you want more flexibility, you can use Amazon EC2 to prepare your own dedicated server. You can freely configure the OS, middleware, and packages, and it is suitable for cases where you want to use a specific version of a language runtime or framework. It is suitable for small-scale development such as learning, PoC, and personal projects.

For team development (VPC + IAM + Code series)

When multiple people are working together on a collaborative project, it is necessary to build an environment that takes into account security, permission management, and standardization. The basic configuration for this is a team development environment centered around VPC + IAM + Code Series.

• Define network boundaries with VPC and subnets to create a development environment isolated from external networks.

• IAM roles are used to subdivide permissions and clarify the access scope for developers, reviewers, and operations personnel.

• Automate CI/CD with CodeCommit, CodeBuild, and CodePipeline to unify the flow from code review to deployment

Because the environment is standardized, it is easy to add members and onboard new employees, and it is compatible with security controls. This is the most common pattern for development environments in companies.

Container development environment (ECS/EKS/Docker on EC2)

If application development is premised on containers, a container development environment using ECS ​​or EKS can be built. The advantage is that the Docker-based environment can be unified, minimizing the differences between development, testing, and production.

An example configuration is as follows:

• Create a Docker image locally or on Cloud9

• Push the image to Amazon ECR

• Launch a task on Amazon ECS (Fargate/EC2) and verify

• Incorporating CodeBuild/CodePipeline into CI/CD to achieve automated builds and deployments

If you want to use Kubernetes, choose Amazon EKS. It is suitable for cases requiring microservices and complex orchestration, and for product development that assumes the adoption of Kubernetes.

Serverless development environment (Lambda + API Gateway + DynamoDB)

A serverless configuration is ideal if you want to minimize infrastructure management or if you are developing a small-scale web API/event-driven app.

Typical combinations are:

• AWS Lambda: Code execution platform (no server management required)

• Amazon API Gateway: Flexible REST/HTTP API construction

• Amazon DynamoDB: A Fully Managed NoSQL Database

Only necessary processes are executed on an event-based basis, and no costs are incurred when the processes are not in use, making it possible to maintain a development environment inexpensively.In addition, by codifying the environment definition using CloudFormation or SAM and combining it with CI/CD, you can easily recreate a testing environment that is similar to the production environment.

Key points for selecting major services

When building a development environment on AWS, the usability and operational costs of the environment will change depending on which services you use as the core. Here, we will summarize the features of the major services and the cases in which you should choose them.

AWS Cloud9 (Instant development in the browser)

AWS Cloud9 is a fully managed development environment that lets you use an IDE in your browser. There's no need to set up a local environment; you can start working immediately after logging in. It comes with built-in features like a terminal, code completion, a debugger, and Git integration, and also provides smooth integration with EC2 and AWS Lambda.

When is Cloud9 a good fit?

• You don't want to mess up your local environment or spend time preparing it.

• Eliminate differences in environments and share a unified development experience across teams

• I want to learn, test, and develop a small prototype in a short period of time.

Another advantage is that you can work using only a browser, so you can develop comfortably on a single laptop without relying on the performance of your device.

Amazon EC2 (configuration with emphasis on flexibility)

Amazon EC2 offers the flexibility to freely configure the OS and middleware, making it ideal for projects that require specific runtime versions or proprietary libraries, or for cases where a complex development toolchain is used.

Main benefits

• Configuration from the OS level allows for compatibility with niche environmental requirements

• Machine performance (CPU, memory, storage) can be adjusted as needed

• You can create an AMI to create a template environment.

Since the operational effort required, such as configuration management, updates, and patch application, is greater than Cloud9, it is more realistic to choose it for projects where operations can be easily stabilized.

Code Series (CodeCommit/CodeBuild/CodePipeline)

The AWS Code Series is a development platform that can consistently automate everything from repository management to builds and deployments. It is a set of essential services for team development who want to introduce CI/CD.

Each role

• CodeCommit: AWS-native Git repository

• CodeBuild: Automated build and test execution platform

• CodePipeline: Automating the build-to-deploy workflow

Although it can connect to external services such as GitHub and GitLab, if you prioritize integration with AWS services, Code Series is the easiest to use. In team development, you can standardize the review, build, and deployment process, preventing operations from becoming dependent on one person.

Integration with Amazon S3, AWS CloudFormation, and AWS Systems Manager Parameter Store

In order to manage a development environment in a way that can withstand production, it is also important to integrate with peripheral services.

■Amazon S3

It can be used to store logs, build artifacts, and backups. It is scalable and low-cost, and can be used in any phase from development to production.

■AWS CloudFormation

This is the most important service when you want to standardize your environment construction with IaC and ensure reproducibility. You can create templates for Cloud9 environments, EC2, ECS, IAM, etc., and eliminate environmental differences.

■AWS Systems Manager Parameter Store

This service allows you to safely store API keys, DB connection information, and various parameters. It allows you to avoid embedding environment variables directly in your code, improving security and operational efficiency. If you need stricter management, use Secrets Manager in conjunction with this service.

By combining these, you can centralize environment creation, configuration management, and confidential information management, significantly reducing operational burden.

Security Design Fundamentals

When building a development environment on AWS, it is essential to incorporate security design from the early stages. Risk management in development environments tends to be laxer than in production environments, but insufficient permission and log management can lead to information leaks and operational errors.

Separation of IAM users and roles

To properly manage developer permissions, it is important to separate "users" and "roles" and operate with least privilege. By not directly granting permissions to individual IAM users, but instead allowing them to work by switching to the required role, the scope of possible operational errors can be minimized.

• IAM user: Login only (MFA required)

• IAM roles: Separation by work content (development, testing, management, etc.)

• Policy: Created according to the principle of least privilege, and explicitly excluding unnecessary APIs

In team development, it is important to separate roles according to their functions, such as developer, reviewer, operator, etc., and to keep permissions separate.

VPC/subnet design (isolation of development environment)

You also need a network design to protect your development environment from external networks. In AWS, you can logically separate environments using VPCs and subnets.

• Build a separate VPC for development and production

• Use NAT Gateway as needed to control internet connectivity

• Minimize the scope of communication permissions with security groups

• Administrator access is stricter with IP restrictions and SSO integration

A clearly defined network perimeter ensures that internal data and applications under development are safe.

Secret information management (AWS Systems Manager Parameter Store/AWS Secrets Manager)

Embedding API keys, DB connection information, etc. directly in code or environment variables increases the risk of information being leaked. AWS offers the following services to safely manage confidential information.

• Parameter Store: Securely store configuration values ​​and parameters

• Secrets Manager: Encrypt and manage more sensitive information such as passwords and API keys

Both can be accessed through IAM, allowing you to clearly manage who has access to what. Safely referencing these services from your applications and build processes allows you to achieve both security and convenience.

Access control and auditing (AWS CloudTrail/Amazon GuardDuty)

Since unintended operations and configuration errors are likely to occur in a development environment, logging and anomaly detection mechanisms are essential.

■CloudTrail

All API operations are logged and you can track who did what and when, which is useful for troubleshooting and audits.

■GuardDuty

This service automatically detects suspicious access and behavior, allowing you to quickly identify issues such as authentication information leaks and abnormal API operations.

By enabling these, you can reduce "invisible risks" even in your development environment and take action before they lead to security incidents.

Cost optimization thinking

When operating a development environment on AWS, you are charged only for what you need, when you need it, but if you use it improperly, unnecessary costs can easily pile up. Because development environments tend to be left in use or forgotten to be shut down, we incorporate cost optimization mechanisms from the early stages.

Main cost structure in the development environment

The following are some of the most cost-intensive aspects of an AWS development environment:

• EC2 instance startup time

  • If you leave it running even when you are not using it, you will incur fixed costs.

• Storage (EBS) capacity

  • EBS charges apply even when the instance is stopped. Be careful not to leave too many snapshots.

• ECR・S3 storage amount

  • Costs will increase as unused images and logs increase.

• NAT Gateway

  • In team development environments, NAT usage increases, and costs fluctuate greatly depending on communication volume.

• CI/CD build count (CodeBuild)

  • Projects with more PR builds and automated tests will have longer build times.

The development environment is subject to greater fluctuations than the production environment, and it is difficult to notice unintended increases, so a system for visualizing usage is essential.

Resource shutdown and schedule automation (AWS Lambda/Amazon EventBridge)

The most effective way to reduce development environment costs is to turn it off when not in use.

By combining EventBridge and Lambda, you can automatically shut down resources at night or on holidays and start them only when needed.

Example:

• Automatically start EC2 at 9:00 on weekdays

• Automatically stop EC2 at 20:00 on weekdays

• Detecting and removing unused EIPs

• Automatically delete ECR images that have not been accessed for a certain period of time

Since rules can be established without relying on human judgment, it becomes easier for the entire team to implement them thoroughly.

Savings Plans/Spot Instances

If you have an instance that you plan to use for a certain period of time, you can reduce costs by using Savings Plans. These plans are suitable for testing and CI environments that are expected to be used for a long period of time.

Additionally, for parallel builds and temporary processes, Spot Instances can significantly reduce costs. They are suitable for processes that can be restarted even if they are stopped, such as builds and container tests.

Cost monitoring (AWS Cost Explorer/AWS Budgets)

In order to reduce costs, it is important to make costs visible and set limits.

■Cost Explorer

You can check usage by service and by day and identify the cause of increased costs. In a development environment, it is basic to link "who is using what" with tags.

AWS Budgets

You can set a budget in advance and receive notifications when the threshold is about to be exceeded. This is effective for managing budgets on a team or project basis.

Standardizing tags (Project/Env/Owner, etc.) will make cost analysis and budget allocation smoother and make it easier to understand waste across the entire environment.

Practical configuration example

We will introduce examples of configurations that are often adopted as actual development environments. We have organized four representative patterns to accommodate a wide range of cases, from individual development to team development and container operations.

CI/CD configuration using AWS Cloud9 and CodePipeline

For learning purposes or small projects, a simple configuration centered around Cloud9 is the easiest to use and keeps implementation costs low.

Configuration image:

1. Developed with Cloud9 (Unified IDE and Terminal)

2. Push to CodeCommit

3. CodeBuild runs automated build tests

4. CodePipeline deploys to staging

Cloud9 is integrated with AWS, so you can use the AWS CLI directly from the terminal, and permission management can be unified with IAM. Since the development environment and CI/CD environment can be completed within AWS, it can be used for a wide range of purposes, from learning and PoC to small-scale services before production operation.

Docker development environment using Amazon EC2 + ECR + ECS

In container-based development, a configuration that combines EC2, ECR, and ECS is common. By converting applications into Docker, you can use the same image throughout the development, testing, and production lifecycle, minimizing environmental differences.

Configuration flow:

1. Developers create Docker images locally or in Cloud9

2. Push the image to Amazon ECR

3. Run a task on Amazon ECS (Fargate/EC2) and check operation.

4. Incorporating CodeBuild/CodePipeline into the CI/CD pipeline for automation

ECS is easy to scale and manage deployments, and Fargate eliminates the need for server management. It is suitable for applications that consist of multiple containers or for companies that use container-based designs for production operations.

Separation of development, staging, and production

In corporate development, it is common to design environments that are clearly separated into three: development, staging, and production. By isolating environments for each AWS account and VPC, rather than simply separating directories and branches, you can reduce the risk of operational errors or missing settings.

Typical design examples:

• Account Separation (Organizations)

  • dev (development)

  • stg (verification)

  • prod (production)

• Separation of privileges for each environment (IAM roles)

• Environment-specific parameter management (Parameter Store/Secrets Manager)

• Configure deployment stages for each environment in CodePipeline

The flow of deploying to production after verification in a staging environment and approval is standardized, ensuring both quality and security.

Example of permission and access design for team development

When developing with multiple people, it is essential to design a system that clearly separates permissions and access. The basic principle is to design with IAM roles at the core, and to clarify who can do what in which environment.

Example:

• Developer role (dev-developer)

  • Access to EC2/ECS/Lambda in the development environment

  • Build/push possible

  • No changes to production environment

• Reviewer role (dev-reviewer)

  • Source code review possible

  • Allow checking the staging environment

  • Production environment operations are not permitted

• Operational role (prod-operator)

  • Production environment monitoring and configuration changes possible

  • Minimal development environment operations

By separating roles in this way, you can prevent operational errors and configuration mistakes, and maintain a safe development process.

Tips for avoiding trouble and operation

When operating a development environment on AWS, you will face common problems such as insufficient permissions, connection errors, increased costs, etc. These problems are often caused not only by configuration issues but also by insufficient operational rules and management methods, so it is important to incorporate them into your system at an early stage.

Checkpoints for insufficient permissions and connection errors

Problems with permissions and connections are likely to occur due to improper IAM settings or network design. When a problem occurs, try to isolate the cause from the following perspectives:

• The IAM role has the required policies.

  • In particular, there are many cases of permission leaks for related services such as ECR, ECS, S3, and CloudWatch.

• Security group inbound/outbound settings

  • Development environments can be unintentionally restrictive, blocking necessary communication.

• VPC, route table, and NAT Gateway route settings

  • If a process that requires internet access is failing, the NAT settings are likely to be the cause.

• Check operation history with CloudTrail

  • Track which actions were blocked through logs and compare them with policies and network settings.

By implementing a system for basic checks, you can reduce the time it takes to troubleshoot.

Operational rules to prevent cost inflation (daily and weekly)

The main causes of increased costs in development environments are "forgetting to shut down" and "remaining unnecessary resources." By establishing rules for daily and weekly check points, unnecessary costs can be reduced.

Items to check daily

• Are there any unnecessary instances such as EC2 or RDS that are running?

• Is there an abnormal increase in the number of builds or build times on the day?

• Has the network traffic increased due to the setting change?

Items to check weekly

• Are there any unnecessary ECR image snapshots remaining?

• Are there any logs or unnecessary files accumulating in S3?

• Check the increase or decrease by service with Cost Explorer

• Are there any unused IAM users or roles?

If you want to automate these processes, you can further reduce the operational burden by using AWS Lambda or Config Rules.

Guaranteeing configuration management and reproducibility with IaC

To ensure stable operation of a development environment, it is important to manage configuration as code. By using CloudFormation or Terraform, you can automate the environment construction procedure and eliminate environmental differences.

Benefits of Introducing IaC

• It is clear who made changes to the environment and when

• The validity of configuration changes can be confirmed through PR reviews.

• Multiple environments (dev/stg/prod) can be reproduced using the same template

• Reduces the risk of incorrect settings due to incorrect operation

Especially in team development, change management using IaC is directly linked to governance. Traditional operations that rely on manual operations and screenshots tend to break down as the number of environments increases, so we recommend migrating to IaC as soon as possible.

My Feelings, Then and Now

When setting up a development environment on AWS, it's important to select services that meet your objectives and incorporate a design that takes operations into consideration from the start. For individual development, a minimal configuration using Cloud9 or EC2 is sufficient, but for team development, permission management with IAM, environment isolation with VPC, and CI/CD automation with Code Series are essential. Even when adopting containers or serverless, combining services such as ECR, ECS, and Lambda allows for stable operation from development to deployment.

On the other hand, development environments are prone to cost growth if left unattended, and configuration errors are also common. By incorporating automatic resource shutdowns, budget alerts, and configuration management using IaC, you can prevent unnecessary spending and disruption to the environment, and maintain an environment that is easy to operate over the long term.

The basic principle for successfully operating a development environment on AWS is to start small with what is needed and gradually expand while keeping security, cost, and reproducibility in mind.