How to migrate resources between AWS accounts? How to migrate between AWS accounts and how to configure them

There are many situations where you might need to migrate resources between AWS accounts, such as for organizational restructuring, security enhancement, and environmental isolation. Because the migration methods and constraints differ for each service, such as S3, EC2, and RDS, not understanding the appropriate procedures can lead to data loss or operational issues.

This article explains migration methods, tools, and configuration patterns for each major service to ensure a safe and efficient migration between AWS accounts. It also summarizes practical tips to help you plan your first migration between different accounts without any hesitation.

Typical cases where migration between AWS is required

The need to migrate resources between AWS accounts can arise due to organizational or business changes, security requirement revisions, etc. We will explain four typical cases that often occur in practice.

Account consolidation due to organizational restructuring or subsidiary mergers

When multiple AWS accounts are operated within a group of companies, the accounts may need to be consolidated due to organizational restructuring or subsidiary mergers. Having multiple accounts in disarray can complicate cost management and security controls, and increase operational burdens.

Typical scenario

• A parent company wants to incorporate the systems of its subsidiaries and manage them under a unified policy.

• I want to centrally manage accounts using AWS Organizations after integration

• I want to consolidate billing across multiple accounts and optimize costs.

In such cases, it is necessary to safely migrate existing resources while simultaneously proceeding with post-integration operational design.

Change security policies and review authority management

Due to changes in security requirements or company rules, situations may arise that cannot be managed using existing accounts.

Migration Background Example

• I want to make the AWS environment I built provisionally in the past conform to security standards.

• I want to eliminate the situation where anyone has administrator privileges

• To ensure separation of privileges, we want to separate important systems into new accounts.

We carefully design IAM policies and cross-account access before migrating resources to cleanly renovate your operations.

Separation of development/testing environment from production environment

If development/testing and production are operated within the same account, there are the following risks.

• The development team accidentally operates on production resources

• Test and production data are mixed, creating audit issues

• Resource complexity makes cost management difficult

To avoid these risks, it is common to migrate the development and testing environments to separate accounts and physically separate them from the production environment. This simplifies permission management and prevents problems and human error.

Account transfer due to business sale or change of contractor

Although it is a rare case, other factors include changes in business structure, such as selling a business or changing outsourcing destinations.

• Example 1: Business Sale

  • Migrate AWS resources related to the business being sold to accounts controlled by the buyer.

• Example 2: Change of contractor

  • The AWS environment is transferred when switching from an external vendor that has outsourced system operations to another contractor or in-house operations.

Data integrity and security are particularly important. To ensure transparency during the migration process, strict records must be kept in the work log and permissions must be managed.

Points to consider when migrating between AWS

Here we will explain the points you should check during the planning stage.

The complexity of migration, not just copying

AWS migration is not just a matter of copying files and data. Each AWS service has its own migration procedures and constraints, and ignoring them can lead to data loss or service outages.

• Procedures and restrictions vary for each service

  • For example, S3 can be migrated using cross-account copy, but RDS requires snapshot sharing.

  • You can use AMI copies for EC2, but you will need to reconfigure Elastic IP addresses and security groups.

• The importance of understanding dependencies

  • If EC2, RDS, ECS, etc. are linked in a complex manner, the services will not function properly if the migration order is incorrect.

  • It is essential to create a system configuration diagram and dependency list in advance and reflect them in your migration plan.

IAM Permissions and Security Management

For AWS account-to-account migration, you must design appropriate permissions between the source and destination accounts.

• Permission design basics

  • Implementing secure cross-account access using IAM roles

  • Based on the principle of least privilege, only necessary operations are permitted

  • Separately manage temporary permissions for migration and permanent permissions after operation

• Configuring Cross-Account Access

  • Some AWS services, such as S3 and DMS, can safely share resources by using the cross-account feature.

  • It is important to properly configure resource policies and trust policies and clarify access routes.

• Key Points

  • Misconfiguration of IAM can stop the entire migration process, so we recommend testing it with a PoC (Proof of Concept).

Network design and connection methods

When transferring large amounts of data between AWS, you need to carefully design your network configuration and security measures.

• Utilizing VPC Peering and Transit Gateway

  • VPC Peering is simple and low-cost for connecting a small number of VPCs

  • Transit Gateway is suitable for integrating multiple accounts and multiple VPCs.

• Security measures during data transfer

  • Data must be encrypted using TLS

  • Use SSE-KMS encryption for S3-to-S3 copies

  • If you use Direct Connect, consider using a VPN for dual protection.

• Warnings

  • If the network bandwidth is insufficient, the migration will take a significant amount of time. Estimate the amount of data to be transferred in advance and increase the bandwidth as necessary.

Overall flow of migration between AWS accounts

Migration between AWS accounts should be carried out in a planned, step-by-step manner. This section explains the process from initial preparation to switching over to operation.

Inventory of current assets and development of a transition plan

A migration project starts with identifying the current situation. Identify all AWS resources to be migrated and organize their dependencies.

• Items to check during inventory

  • List of major resources such as EC2, RDS, and S3

  • Network, IAM, and security groups used by each resource

  • Connections between systems, whether or not there is external collaboration

  • Regions and service restrictions in use

Based on these, we determine priorities, risks, and transition order, and clarify the overall schedule and scope of responsibility.

Risk confirmation through PoC

Since it is risky to immediately migrate to production, we will conduct a PoC.

• Migrate a small sample of resources to validate the procedure

• Get proactive insight into data transfer rates, downtime and costs

• Check whether there are any problems with IAM or network settings

By improving the migration plan based on the knowledge gained from the PoC, we can prevent problems in later processes.

Production migration schedule and downtime design

When migrating to production, it is important to design a schedule that minimizes the impact on business operations.

• Key Points

  • Agree upfront on how much downtime is acceptable

  • Plan for migration work to be performed outside of business hours or on holidays

  • Reduce risk with blue/green deployments and phased cutovers

  • Define your rollback procedures in advance

This will allow for quick recovery in the unlikely event of a failure.

Migration execution and testing

The production migration is a carefully planned process, with different procedures for each service followed and testing at each step.

• Migration checklist

  • Is the data transfer completed successfully?

  • Are IAM and security settings appropriate?

  • Are the resources operating as expected after migration?

Switching to production without sufficient verification increases the risk of serious problems occurring after migration.

Post-migration monitoring and operation switching

Once the migration work is complete, we will switch over to the new AWS account.

• Reconstructing the monitoring infrastructure with CloudWatch and AWS Config

• Recheck IAM policies and log settings

• Safely delete resources you no longer need in the source account

• Update the operation manual and share it with the relevant parties.

Neglecting this phase can lead to security holes and operational problems after the migration, so it's important to strengthen monitoring for several weeks after the migration is complete to ensure stable operation.

Migration methods and tools for each major service

When migrating between AWS accounts, the optimal migration method and tools vary for each service, so you need to develop an individual strategy. We will explain the migration methods for the most commonly used major services.

Amazon S3

Amazon S3 provides a cross-account copy function, which allows for a relatively smooth migration. However, the procedure differs depending on the data volume and bucket policy, so you need to plan it in advance.

• Cross-account copy

  • Configure S3 cross-account access and copy directly between buckets

  • The bucket policy must explicitly allow the destination account.

• S3 Batch Operations

  • Effective for efficiently copying millions of objects or more

  • You can create a manifest file to automate bulk migrations.

• AWS CLI usage example

  • Small data can be copied using the CLI.

  • Example: aws s3 sync s3://source-bucket s3://target-bucket --source-region ap-northeast-1 --region ap-northeast-1

• Key Points

  • It is recommended to use SSE-KMS encryption to ensure security in transit.

Amazon EC2

The basic method for migrating Amazon EC2 instances is to use AMIs, but you will need to reconfigure things like Elastic IPs and security groups.

• AMI sharing/copying

  • Share the AMI with the destination account and launch a new EC2 instance from that AMI

  • When copying, make sure the encryption key (KMS) settings are also used in the destination account.

• Leveraging AWS MGN

  • You can migrate running EC2 instances with real-time replication.

  • Ideal for minimizing downtime

• Handling Elastic IPs

  • Elastic IPs cannot be transferred between accounts and must be newly allocated.

  • It's important to plan for DNS configuration changes

Amazon RDS

Amazon RDS uses snapshots and database migration tools to migrate data. Choosing the right migration method is key to minimizing business impact.

• Snapshot Share

  • Share the snapshot to the destination account and restore it to create a new RDS.

  • Simple and reliable, but with downtime

• AWS Database Migration Service (DMS)

  • Replication of running DBs is possible in near real time

  • Effective when you want to minimize downtime

Container resources such as Amazon ECS/Amazon EKS

The container environment focuses on migrating to ECR (Elastic Container Registry).

• Amazon ECR image migration

  • Use docker pull to retrieve data from the source ECR and use docker push to register it to the destination ECR

  • Automation is also possible using AWS CLI scripts and CodePipeline

• Amazon ECS/Amazon EKS settings

  • Export the ECS task definition and EKS manifest in JSON/YAML format and apply them to the destination account.

IAM and configuration migration

IAM and other configuration information cannot be automatically copied, so they must be rebuilt. This is one of the most time-consuming aspects of migrating between AWS accounts.

• Manual rebuild

  • Create a new IAM user, role, and policy.

  • Set up from scratch by referring to the configuration of the source account

• Utilizing IaC (Infrastructure as Code)

  • Code using CloudFormation or Terraform and reconstruct at the destination

  • High reproducibility makes it easy to prevent mistakes even with complex settings

• Key Points

  • IAM should be designed with least privileges in mind, and be careful not to leave any unnecessary permissions after migration.

Configuration pattern examples and design concepts

When migrating between AWS accounts, you need to decide which configuration to use depending on your operational continuity and security requirements during the migration period. We will introduce three typical patterns and explain the characteristics and design points of each.

Migrate incrementally with cross-account access

The most common method is a staged migration using cross-account access, where resources are temporarily shared between the source and destination accounts and migration is performed incrementally.

• Features

  • You can run both accounts in parallel and migrate gradually.

  • The verification environment can be migrated first, and the production environment can be switched over at the final stage.

  • Minimize downtime

• Design Points

  • S3 and RDS use cross-account functionality to safely share data

  • Use IAM roles to design least privilege access between accounts

  • Clarify the schedule as costs will be doubled during the transition period

• Suitable cases

  • If you want to migrate in stages while verifying, or if you want to minimize production downtime

Migration configuration using a temporary shared VPC

This configuration involves temporarily creating a shared VPC, and both the source and destination accounts use resources on the same VPC.

• Features

  • Minimize network connection and security group configuration

  • Data transfer can be performed within the same VPC, improving speed and stability.

  • After migration is complete, you can delete the shared VPC, simplifying post-processing.

• Design Points

  • Prepare a dedicated account to manage Shared VPC

  • Pre-define subnet and route table designs

  • After the migration is complete, unnecessary permissions are promptly deleted to ensure security.

• Suitable cases

  • Suitable for systems involving large-volume data transfers and complex network connections

Full cutover migration

This is a one-time switchover from your old account to your new account. The transition period is set short, and all resources are migrated at once.

• Features

  • The transition period is short, and costs due to dual operation are unlikely to occur.

  • Simple design and easy management

  • On the other hand, if a problem occurs, the impact will be large.

• Design Points

  • Verify procedures in advance with a PoC to accurately estimate downtime

  • Have a rollback plan

  • Conduct final dress rehearsals in a staging environment before the production release

• Suitable cases

  • Small environments or those with a few hours of downtime available

The optimal migration configuration varies depending on the system scale, migration period, and tolerable downtime. In some cases, a "hybrid" approach is adopted, where a gradual migration using cross-account access is performed in the initial stage, and then a complete switchover is performed in the final stage.

Common migration mistakes and workarounds

Here we will explain common mistakes that occur in practice and how to avoid them.

Migration failed due to insufficient permissions

If IAM roles and policies are not properly configured between the source and destination accounts, the migration process may be interrupted. If you use the cross-account sharing features of S3 or RDS, you must also consider bucket policies and KMS key permissions.

• Failure example

  • When sharing an AMI, KMS key permissions are not granted and it cannot be started.

  • Data replication cannot be started with DMS

  • Access Denied error occurs when copying to S3

• Avoidance

  • Test permission settings in advance during the PoC stage

  • Based on least privilege, but temporarily increasing privileges as needed

  • Check the operation using the official AWS "IAM policy simulator"

Overlooked regional restrictions and unsupported services

Some AWS services have differences in availability and functionality depending on the region. There are cases where the same services are not available in the destination region, so it is essential to check in advance.

• Failure example

  • Select a region where RDS snapshots cannot be shared

  • Some instance types are not available in ECS or EKS

  • Using region-restricted features such as Aurora Global Database

• Avoidance

  • Check the list of AWS regional services in advance

  • Check whether the features you are using are available in the destination region

  • If there are constraints, verify alternative configurations with PoC

Insufficient estimates of transfer costs and time

When transferring large amounts of data between AWS, if you underestimate the time and cost involved, your migration plan will fall apart.

• Failure example

  • Plan to transfer tens of TB of S3 data was delayed due to insufficient bandwidth

  • Migration via VPN connection without Direct Connect takes several weeks

  • Forgetting to include transfer fees in the estimate resulted in over-budget

• Avoidance

  • Calculate data volume and bandwidth to estimate travel time

  • For large volumes, use physical transfer services such as AWS Snowball

  • Pre-calculate transfer fees for better cost monitoring

Missed settings and security holes that occur after migration

Even if the migration itself is complete, there are cases where failures or security risks occur due to improper configuration after the migration.

• Failure example

  • IAM policies are insufficient and production operations cannot begin

  • Failure detection is delayed due to failure to reconfigure CloudWatch alarms and monitoring settings.

  • S3 bucket is open to the public

• Avoidance

  • Verify your settings with the "Configuration Checklist" after migration

  • Automate security audits with AWS Config and Security Hub

  • Strengthen monitoring for a certain period after migration is complete to detect problems early

Both of these issues arise due to insufficient preparation and verification. By using PoCs and checklists to identify risks from the planning stage, you can prevent many problems and ensure a safe and smooth migration between AWS.

My Feelings, Then and Now

Migrating between AWS accounts is not a simple copy job; it's a project that requires planning and verification. Understanding the constraints of each service and verifying the procedure with a PoC can significantly reduce problems during migration. Also, while taking advantage of phased migration and cross-account access, review your monitoring and security settings after migration to ensure stable cloud operations.