Introduction
Constructing a voice-enabled ordering system that works throughout cellular apps, web sites, and voice interfaces (an omnichannel method) presents actual challenges. You should course of bidirectional audio streams, keep dialog context throughout a number of turns, combine backend providers with out tight coupling, and scale to deal with peak site visitors.
On this publish, we’ll present you easy methods to construct a whole omnichannel ordering system utilizing Amazon Bedrock AgentCore, an agentic platform, to construct, deploy, and function extremely efficient AI brokers securely at scale utilizing any framework and basis mannequin and Amazon Nova 2 Sonic. You’ll deploy infrastructure that handles authentication, processes orders, and gives location-based suggestions. The system makes use of managed providers that scale routinely, lowering the operational overhead of constructing voice AI functions. By the top, you’ll have a working system that processes voice orders throughout a number of buyer touchpoints. The AI orchestration layer connects to a pattern backend structure with pattern menu information, supplying you with a head begin whereas implementing a venture of this nature. This venture was divided into modules supplying you with flexibility if you’re trying to reuse parts for integrating along with your present backend APIs.
On this publish, you’ll discover ways to:
- Deploy a multi-channel Voice AI ordering infrastructure utilizing AWS Cloud Growth Package (AWS CDK)
- Implement an agent utilizing Strands with Amazon Nova 2 Sonic for real-time speech processing hosted on AgentCore Runtime
- Join your AI agent to backend providers utilizing Mannequin Context Protocol (MCP) via AgentCore Gateway
- Take a look at your system with lifelike ordering situations together with route-based pickup suggestions
Amazon Nova 2 Sonic is a speech-to-speech basis mannequin obtainable via Amazon Bedrock that you need to use for real-time voice interactions. When mixed with Amazon Bedrock AgentCore, you get pure voice ordering throughout all buyer touchpoints.
Resolution overview
This answer structure separates your frontend, AI agent, and backend providers into distinct parts. This separation permits you to develop and scale every part independently. The MCP is an open customary for connecting AI functions to exterior information sources, instruments, and workflows. It gives standardized communication between your agent and backend providers.
The answer will deploy:
- Amazon Cognito – Handles consumer authentication and gives short-term AWS credentials for safe API entry. You’ll be able to change this to your IDP of your alternative so long as it’s OAuth 2.0 compliant.
- Amazon Bedrock AgentCore Runtime – Hosts your AI agent with microVM isolation. Every consumer session runs in an remoted digital machine, which retains your buyer periods safe and performant even underneath excessive load. It prevents one buyer’s session from affecting one other’s efficiency or accessing their information.
- Amazon Bedrock AgentCore Gateway – Offers a safe means for builders to construct, deploy, uncover, and connect with instruments at scale, which gives standardized communication between the agent and what you are promoting logic with out tight coupling, so to modify backends or add new instruments with out rewriting integration code
- Amazon API Gateway – Exposes your backend providers via Representational State Switch (REST) endpoints with AWS Identification and Entry Administration (IAM) based mostly authorization
- AWS Lambda – Executes what you are promoting logic for menu retrieval, order processing, and placement providers
- Amazon DynamoDB – Shops buyer profiles, orders, menu objects, and buying carts with single-digit millisecond latency
- AWS Location Providers – Offers location-based options for pickup suggestions
- AWS Amplify – Hosts the frontend software
Structure diagram
The next diagram represents the answer structure, which incorporates three key sections:
Part A: Backend infrastructure
This part deploys a pattern restaurant structure as backend providers utilizing infrastructure as code. It provisions information storage for buyer data, orders, menus, carts, and areas. It additionally units up location-based providers for deal with dealing with and mapping, Lambda capabilities for enterprise logic, an API layer for exterior entry, and consumer authentication and authorization providers. Assets are deployed within the acceptable dependency sequence.
Part B: AgentCore Gateway
This part deploys the AgentCore Gateway infrastructure. It provisions the mandatory IAM service permissions, creates the AgentCore Gateway service, and configures API integration to reveal backend endpoints as agent-accessible instruments.
Part C: AgentCore Runtime and ECR picture
This part deploys the AgentCore Runtime setting. It provisions Amazon ECR for container storage, Amazon S3 for supply uploads, AWS CodeBuild for construct automation, and required IAM permissions. The AgentCore Runtime service is configured with WebSocket protocol.
Part D: AWS Amplify
This part deploys the frontend software utilizing AWS Amplify. It provisions the Amplify internet hosting service with deployment configuration and generates the mandatory frontend configuration from backend outputs. The constructed net software is deployed and turns into accessible through the Amplify URL upon completion.
Person request movement:
- The consumer accesses the net software hosted on AWS Amplify from their browser or cellular gadget.
- The consumer authenticates with Amazon Cognito utilizing their username and password and receives JWT tokens (Entry Token and ID Token).
- The frontend exchanges the ID Token with the Cognito Identification Pool for short-term AWS credentials (Entry Key, Secret Key, Session Token).
- The frontend opens a SigV4-signed WebSocket connection to AgentCore Runtime and sends the Entry Token as the primary message for identification verification.
- The agent hosted in AgentCore Runtime validates the Entry Token by calling the Cognito GetUser API and extracts the client’s verified identify, e-mail, and customerId.
- AgentCore Runtime initializes the Nova 2 Sonic mannequin on Amazon Bedrock and builds a personalised system immediate with the verified buyer context.
- AgentCore Runtime connects to AgentCore Gateway as an MCP shopper utilizing SigV4 authentication and discovers the obtainable instruments.
- The consumer speaks their order. The agent processes the voice enter via Nova 2 Sonic and invokes instruments asynchronously via the AgentCore Gateway utilizing MCP.
- AgentCore Gateway exposes the backend REST APIs as MCP instruments, in order that the agent can uncover and invoke them by identify. When the agent calls a software, AgentCore Gateway forwards the request as a REST API name to the API Gateway, which routes it to the suitable Lambda operate. Lambda capabilities question DynamoDB tables and AWS Location Providers.
- Nova 2 Sonic generates a contextual voice response incorporating the software outcomes and streams it again to the consumer over the WebSocket connection.
Stipulations
Earlier than you start, confirm you have got the next in place:
- An AWS account
- Basis mannequin (FM) entry in Amazon Bedrock for Amazon Nova 2 Sonic in the identical AWS Area the place you’ll deploy this answer
- Node.js 20.x or later (required for AWS CDK deployment)
- Python 3.13 or later (required for agent runtime and deployment scripts)
- AWS Command Line Interface (AWS CLI) 2.x configured with credentials
- AWS CDK CLI 2.x: npm set up -g aws-cdk (required for infrastructure deployment)
- CDK bootstrapped in your goal account/area: npx cdk bootstrap
- Boto3 1.38.0 or later (required for bedrock-agentcore-control service assist). Set up utilizing python3 -m pip set up –upgrade boto3 botocore –break-system-packages
- Further Python packages: python3 -m pip set up email-validator pyyaml –break-system-packages
- The accompanying code downloaded from the aws-samples GitHub repo
Deploy answer assets utilizing AWS CDK
Clone the GitHub repository and navigate into the venture listing.
git clone https://github.com/aws-samples/sample-omnichannel-ordering-with-amazon-bedrock-agentcore-and-nova-sonic
cd sample-omnichannel-ordering-with-amazon-bedrock-agentcore-and-nova-sonic
Run the deployment script. Each parameters are required. The e-mail deal with will obtain a brief password for the preliminary Cognito check consumer.
./deploy-all.sh –user-email –user-name “”
The script first runs preflight checks to validate that Node.js, Python, AWS CLI, CDK, credentials, CDK bootstrap, and Bedrock Nova 2 Sonic mannequin entry are all in place. If any test fails, it would report what’s lacking and supply to auto-install what it might probably.
After preflight passes, the script runs 5 steps. Steps 1 via 3 are totally automated. Step 4 (Artificial Information) will immediate you for a location corresponding to a metropolis, zip code, or deal with to make use of as the middle level for looking out close by eating places, a meals sort to seek for (e.g., pizza, burgers, espresso store, sandwich, tacos), whether or not to reuse the identical deal with because the buyer dwelling, and a affirmation earlier than writing the generated information into DynamoDB. Step 5 (Password Setup) will immediate you to optionally change the short-term Cognito password that was emailed to you. For those who select sure, you’ll enter the short-term password from the e-mail and set a brand new everlasting password that meets the Cognito coverage (8+ characters, uppercase, lowercase, digit, image).
After completion, the script outputs the front-end URL (e.g., https://primary..amplifyapp.com) that you’ll use to entry the applying.
Understanding serverless information administration
API Gateway creates a REST API that connects your frontend to backend providers with eight IAM-authenticated endpoints and Lambda integration.
Your backend makes use of 5 DynamoDB tables supporting the whole ordering workflow. The Prospects Desk shops profiles (identify, e-mail, telephone, loyalty tier, factors) for customized suggestions. The Orders Desk shops order historical past with location information and makes use of a World Secondary Index to question by location for figuring out common objects. The Menu Desk shops location-specific objects with pricing and availability that varies by restaurant. The Carts Desk shops short-term buying carts with 24-hour TTL for computerized cleanup. The Areas Desk shops restaurant information (coordinates, hours, tax charges) for order calculations and suggestions. DynamoDB on-demand capability scales routinely with site visitors.
Understanding location-based providers
Location Providers gives location-based options that assist clients discover handy pickup areas. The system deploys three assets: a Place Index (Esri) for geocoding and deal with search, a Route Calculator (Esri) for calculating driving routes and detour instances, and a Map (VectorEsriNavigation model) for interactive visualization optimized for driving.
Lambda capabilities present three capabilities: Nearest Location Search finds the closest eating places sorted by distance utilizing GPS coordinates and the haversine system. Route-Primarily based Search identifies eating places inside a specified detour time (default 10 minutes) utilizing precise driving instances slightly than straight-line distances. Handle Geocoding converts road addresses to coordinates when GPS isn’t obtainable.
These options allow context-aware suggestions like “I discovered a location 2 minutes out of your route” or “Your normal location is 5 miles away.”
Understanding voice AI processing with Amazon Bedrock AgentCore
Your AI agent processes voice interactions via Amazon Bedrock AgentCore. Every consumer session runs in an remoted microVM, which retains buyer periods safe and performant even underneath excessive load. It prevents one buyer’s session from affecting one other’s efficiency or accessing their information. AgentCore gives computerized scaling, built-in monitoring, and WebSocket assist for real-time voice.
The agent makes use of the Strands framework to outline system prompts, instruments, and dialog movement. Nova 2 Sonic gives:
- Speech recognition throughout accents with background noise tolerance
- Speech response adaptation to consumer tone and sentiment
- Bidirectional streaming with low latency response instances
- Asynchronous software calling that fetches information in parallel with out blocking dialog
- Interruption dealing with for pure turn-taking
- Context consciousness throughout a number of dialog turns
The voice processing movement: Audio streams from the frontend (16 kHz PCM) through WebSocket to AgentCore Runtime. Nova 2 Sonic transcribes speech, the agent determines intent and selects instruments, invokes them asynchronously through MCP, and the AgentCore Gateway interprets MCP calls to REST API calls. Lambda capabilities execute enterprise logic and return outcomes, which the agent incorporates into its response. Nova 2 Sonic generates voice output that streams again to the frontend.
This structure minimizes latency for conversational ordering.
Person authentication
The answer makes use of Amazon Cognito consumer swimming pools and identification swimming pools for safe, role-based entry management. Person swimming pools handle authentication and teams. Identification swimming pools present short-term AWS credentials linked to IAM roles. Customers log in with their username and password to the Cognito Person Pool, receiving JSON Net Token (JWT) tokens (Entry Token and ID Token). The frontend exchanges the ID Token with the Cognito Identification Pool for short-term AWS credentials (Entry Key, Secret Key, Session Token). These credentials signal the WebSocket connection to AgentCore Runtime and API Gateway requests utilizing Signature Model 4 (SigV4). This structure ensures that solely authenticated customers can entry the applying and ordering APIs.
WebSocket connection movement
The next sequence diagram illustrates how the authentication credentials from the earlier part set up a direct browser-to-AgentCore connection. Utilizing the short-term AWS credentials, the frontend opens a SigV4-signed WebSocket connection to AgentCore Runtime and sends the Entry Token for identification verification. The browser then streams 16kHz PCM audio and receives voice responses, transcriptions, and gear invocation notifications over the identical connection. This avoids the necessity for a server-side proxy.
Voice interplay and dynamic ordering
The next sequence diagram illustrates the movement of a buyer’s order question, demonstrating how pure language requests are processed to ship synchronized responses:
The diagram exhibits a buyer question (“I wish to order”) which is dealt with via asynchronous software calling. The agent invokes a number of instruments in parallel (GetCustomerProfile, GetPreviousOrders, GetMenu) via the AgentCore Gateway, which interprets them into API Gateway REST calls. Lambda capabilities question DynamoDB and return the outcomes again via the gateway. Nova 2 Sonic then generates a contextual response incorporating all of the software outcomes, creating a personalised buyer expertise all through the dialog.
Ordering walkthrough
Open the frontend URL in your browser and register with the AppUser credentials. After you authenticate, select the microphone button to start out a voice dialog with the ordering agent. The agent greets you by identify, will get your location from the browser, and pulls up your earlier orders within the background. You’ll be able to communicate naturally. Ask to repeat a previous order, browse the menu, discover close by pickup areas alongside your route, or construct a brand new order from scratch. The agent responds with voice in actual time, handles menu questions, provides objects to your cart, and confirms your order with a complete and estimated pickup time. Your entire dialog occurs hands-free over a single WebSocket connection. The agent calls backend instruments asynchronously, so there aren’t any pauses whereas information is being fetched. The next video demonstrates a whole ordering session from greeting to order affirmation.
Clear up
For those who resolve to discontinue utilizing the answer, you possibly can comply with these steps to take away it and its related assets:
Delete the stacks:
./cleanup-all.sh
Conclusion
On this publish, we confirmed you easy methods to construct an omnichannel ordering system utilizing Amazon Cognito for authentication, Amazon Bedrock AgentCore for agent internet hosting, API Gateway for information communication, DynamoDB for storage, and Location Providers for route optimization. The three-layer structure separates frontend, agent, and backend parts for impartial growth and scaling. The system helps menu administration, cart performance, loyalty applications, order processing, and location-based providers via MCP integration. Amazon Nova 2 Sonic gives voice interactions with low latency, asynchronous software calling, and interruption dealing with. Parallel software calling reduces wait instances, voice recognition works throughout accents, customized suggestions use order historical past, and route-optimized pickup areas assist clients discover handy stops. The pay-per-use pricing mannequin and automatic scaling management prices as utilization grows, whereas with MCP integration, you possibly can adapt the answer by including new Lambda capabilities with out modifying agent code. To get began, go to the answer repository on GitHub and customise the answer on your ordering platforms. For extra data, see the Amazon Bedrock documentation and Introducing Amazon Nova 2 Sonic.
Further assets
To study extra about Amazon Bedrock AgentCore, Amazon Nova Sonic, and extra options, discuss with the next assets:
Concerning the authors
Sergio Barraza
Sergio is a Senior Technical Account Supervisor at AWS, serving to clients design and optimize cloud options. With greater than 25 years in software program growth, he guides clients via AWS providers adoption. Outdoors work, Sergio is a multi-instrument musician taking part in guitar, piano, and drums, and he additionally practices Wing Chun Kung Fu.
Salman Ahmed
Salman is a Senior Technical Account Supervisor at AWS. He focuses on guiding clients via the design, implementation, and assist of AWS options. Combining his networking experience with a drive to discover new applied sciences, he helps organizations efficiently navigate their cloud journey. Outdoors of labor, he enjoys pictures, touring, and watching his favourite sports activities groups.
Ravi Kumar
Ravi is a Senior Technical Account Supervisor in AWS Enterprise Assist who helps clients within the journey and hospitality business to streamline their cloud operations on AWS. He’s a results-driven IT skilled with over 20 years of expertise. Ravi is keen about generative AI and actively explores its functions in cloud computing. Outdoors of labor, Ravi enjoys inventive actions like portray. He additionally likes taking part in cricket and touring to new locations.
