AWS AI from a Comprehensive Perspective: Choosing and Designing Amazon Bedrock, Amazon SageMaker, and AI APIs

The term "AWS AI" is not officially defined as a service name. However, in information gathering and evaluation contexts, it is often used to refer to all AWS AI-related services, including generative AI, machine learning, translation, and speech recognition. If you start researching with this vague premise, you may only see the names of individual services such as Bedrock and SageMaker, and end up not knowing "which one to choose."

What's important isn't the sheer number or novelty of features, but rather determining which layer to consider in light of your company's operations, data handling, and operational structure. The choice will vary depending on whether you want to use generative AI, develop machine learning models, or integrate existing AI functions into your business.

This article re-examines AWS AI not as a list of services, but as a tool for design decision-making. After organizing the overall picture, we will explain how to differentiate between Amazon Bedrock, Amazon SageMaker, and various AI APIs, focusing on points where decision-making often differs.

What is AWS AI?

The term "AWS AI" is not defined as an official service name of Amazon Web Services (AWS). It is not a name that refers to a specific product or feature, but rather a convenient way of referring to all artificial intelligence-related services offered by AWS.

When you break down the contents of AWS AI, it can be broadly divided into generative AI, machine learning infrastructure, and existing AI services such as translation, speech recognition, and image analysis. Their purposes and uses differ, and they cannot be compared using the same criteria. Nevertheless, because interest in generative AI has been growing in recent years, there are cases where AWS AI as a whole is perceived as generative AI, or as a specific generative AI service.

With this understanding, the focus of consideration will be unclear, and decisions will be prone to wavering. The options to consider will differ depending on whether you want to integrate generative AI into your business, develop and operate machine learning models in-house, or use existing AI functions as part of your business.

An overview of AWS AI

Trying to grasp the vast number of AWS AI services simply by looking at a list can actually obscure the overall picture. The key is not to focus on individual service names, but to first clarify which layer of AI you are considering. AWS's AI-related services can be easily understood by dividing them into three layers with different roles and assumptions.

Layer 1: Ready-to-use AI APIs (translation, voice, image, etc.)

This layer is where functions such as translation, speech recognition, transcription, and image analysis are utilized as fully functional AI via APIs. There is no need to train or tune the model, allowing for rapid integration into existing operations and systems.

While it offers low development overhead and quick integration into business operations, the model's internal logic and decision-making criteria are essentially a black box. It's not suitable for fine-tuning accuracy and behavior, or for incorporating company-specific decision-making criteria.

This layer is an option when the goal is not to develop AI, but rather to "use AI" itself.

Layer 2: Generative AI Platform (Amazon Bedrock)

This layer is the foundation for integrating generative AI into business processes and applications. Its key feature is the ability to handle multiple large-scale language models (LLMs) via APIs, allowing for the use of generative AI without being dependent on a specific model.

Amazon Bedrock is best positioned not as a mechanism for training models from scratch, but rather as an intermediate layer for securely integrating generative AI into business workflows and existing systems. Even when using tools like RAG to reference internal company data, the focus is on designing the data connection method and usage scope, rather than retraining the model itself.

At this layer, data handling and permission design—specifically, which data is passed to the AI ​​and who can access it—become crucial points in deciding whether to implement it.

Layer 3: Machine Learning Platform (Amazon SageMaker)

This layer encompasses the entire process from the development, training, deployment, and operation of machine learning models. It includes designing processes such as algorithm selection, preparation of training data, model evaluation, and post-deployment improvements.

While Amazon SageMaker offers high flexibility, it only becomes effective when combined with sufficient data, specialized personnel, and a continuous operational system. Unlike generative AI, which is ready to use immediately, it's an option based on the premise of "building and nurturing" AI in-house.

This layer is suitable for situations requiring differentiation through proprietary models or advanced control, and a medium- to long-term organizational structure is essential before deciding to implement it.

How do you choose between Bedrock, SageMaker, and AI API?

After gaining an overview of AWS AI, the next challenge is deciding which option to choose. This decision can be confusing if you start by comparing the features of each service. In practice, it's easier to make a decision by considering three points: intended use, data handling, and operational structure.

Categorize by purpose (business support or product integration)

When the goal is to streamline internal operations or conduct a proof-of-concept (PoC), speed of implementation and ease of trial are crucial. For uses such as meeting minute summarization, internal FAQs, and document search, it is practical to integrate generative AI into existing operations using AI APIs or Bedrock.

On the other hand, when integrating AI functionality into external services or products, a design that prioritizes stable operation and scalability is required. In this case, you would either use Bedrock as the core or, depending on the requirements, consider developing your own models using SageMaker.

Differentiate based on how the data is handled (don't include it / reference it / use it for training).

The options vary greatly depending on how much data you want the AI ​​to handle. If you only need to use it for tasks like text generation and summarization without providing data, then standard use of AI APIs or generative AI will suffice.

If you want to use internal company documents or business data, a more practical approach is to design it to "reference" rather than "train" the system. In this case, you would use Bedrock while controlling the scope of data reference, similar to how RAG works.

On the other hand, if you want to train or retrain the model itself using business data, the design will be based on SageMaker. This option should be considered when the goal is differentiation in the medium to long term rather than short-term efficiency improvements.

Divide by organizational structure (who will operate it)

If the IT department or planning department is leading the project, it's more practical to choose an option that doesn't increase the operational burden. Configurations that shift infrastructure and model management to AWS, such as AI API or Bedrock, are easier to manage.

If engineers can be continuously involved, then using SageMaker and operating with the assumption of model improvement becomes a viable option. However, how to handle operational costs and the risk of reliance on specific individuals becomes a key factor in the decision.

If you lack sufficient internal resources, you can consider relying on external partners. Even in that case, clarifying which layers of the process you will handle internally and which you will outsource will help prevent confusion in later stages.

Design points where decisions differ when implementing AWS AI

The success of implementing AWS AI depends more on the design assumptions than on the service selection itself. The approach to permissions, cost considerations, and the prior business process restructuring are points that are difficult to correct later.

Authority and guardrails (who can give what to the AI)

The difference between personal and organizational use lies not in whether it's usable, but in who can handle which data and under what conditions. Input content that might not be a problem when tested personally will be handled differently in organizational use from an information management perspective.

It's necessary to define in advance the scope of data that can be passed to the AI, as well as the users and uses for which it can be used. Additionally, points to consider during the design phase include the extent to which input and output logs should be retained, and what information should be available for auditing.

While opt-out settings and log management may seem like configuration tasks, it's actually easier to make decisions if you treat them as a matter of defining operational rules.

How costs are perceived (the pitfalls of "pay-as-you-go").

AWS AI is primarily based on a pay-as-you-go model, but it can be difficult to estimate costs based solely on usage. This is especially true for generative AI, where costs fluctuate depending on the number of inferences and the content of the input and output, making it difficult to make accurate predictions in advance.

It's not uncommon for usage that wasn't a problem during the proof-of-concept (PoC) phase to suddenly increase in production. This is because the number of calls and processing volume change as the number of users increases and the system becomes more integrated into business operations.

Therefore, instead of considering the PoC and the production environment under the same assumptions, organizing the plan with the understanding that the cost structure will change with each operational phase will reduce discrepancies later on.

Prerequisites before AI (data and business process organization)

If the expected results are not achieved after implementing AI, the cause is not necessarily the AI's performance. In fact, in many cases, the implementation proceeds with unclear business processes and data assumptions.

If the target tasks, data to be used, and how the results will be evaluated are not clearly defined, it becomes difficult to evaluate the AI's output. In this state, it becomes easy to waver in your decision-making, regardless of which service you choose.

Before choosing an AI solution, it's helpful to first articulate your workflow and data location to make comparing options easier. Treating AI not as a magic box, but as a system that operates based on preconditions, will lead to more realistic decisions.

What's a realistic way to start considering AWS AI?

Because AWS AI offers so many options, trying to find the "optimal solution" from the start can easily lead to a dead end in decision-making. In practice, it's easier to proceed by establishing assumptions and limiting the methods of testing before entering the technology selection process. Here, we summarize the key considerations to keep in mind when starting your evaluation.

The three things you should decide first

The first things to clarify are: "What will it be used for?", "What will it handle?", and "Who will see it?".

If the intended use of the generated data remains unclear, it becomes difficult to judge its quality. The required role will differ depending on whether you want to assist with a part of the process or change the entire process.

It's crucial to define early on which data will be handled. The design and options will vary depending on whether the system includes confidential information or if only publicly available information is sufficient.

Clarifying who is responsible will make the process easier to move forward. If the decision-making body remains ambiguous, it is easy to get stuck on how to handle the results of the proof of concept (PoC).

Thinking about starting small

In a Proof of Concept (PoC), it's easier to judge whether it's usable in actual business operations rather than immediately pursuing accuracy. This involves checking not only the correctness of the output, but also how the user perceives it and which processes are streamlined.

Ease of input, ease of handling results, and how to deal with unexpected outputs are all factors that cannot be ignored when considering a real-world scenario. Although difficult to quantify, they directly impact the burden once the system is operational.

Rather than viewing a Proof of Concept (PoC) as a place to improve the final product, it's better to view it as a place to gather information to decide whether to proceed or explore other options. This approach will make it easier to move forward with your considerations.

Summary

AWS AI is not a solution that can be achieved by simply choosing a specific service. It consists of multiple layers, such as generative AI, machine learning infrastructure, and AI APIs, each with different prerequisites and roles. First, it is important to understand the overall picture and determine which layer your company should consider.

The choice between Bedrock, SageMaker, and AI APIs isn't based on functional superiority, but rather on the intended use, data handling, and operational structure. The practical options naturally narrow down based on assumptions such as whether it's for business support or product integration, the extent of data handling, and who will be responsible for operation.

Furthermore, design decisions such as permission settings, cost projections, and the organization of operations and data are difficult to modify later. By articulating these assumptions before being swayed by the service name, you can reduce uncertainty during the evaluation process.

With AWS AI, the results depend less on "what you use" and more on "how you organize and make decisions." Understanding the overall picture and the criteria for decision-making before proceeding, while it may seem like a roundabout approach, is the most practical way to go about it.