Gemma 4 is Google DeepMind's open-weight model family, released under Apache 2.0 and available on Amazon Bedrock as of 2026-06-15 (source: AWS official announcement). The family includes three instruction-tuned variants: Gemma 4 31B (dense, strongest reasoning and coding in the series), Gemma 4 26B-A4B (MoE, roughly 4B-class cost with 25B-class knowledge), and Gemma 4 E2B (smallest, built for low-latency and multimodal classification). You access them through Bedrock's bedrock-mantle endpoint, which is compatible with the OpenAI SDK. Pick 31B if you mainly do reasoning or coding and want the strongest single model in the series; go with 26B-A4B if cost matters and you need high throughput; choose E2B if you're sensitive to latency or doing multimodal classification.
As of launch day, all three models are available in four AWS Regions: US East (N. Virginia, Ohio), US West (Oregon), and Europe (Frankfurt). Check the Bedrock model catalog for the latest region list (source: AWS official announcement, 2026-06-15).
Model Comparison at a Glance
| Model | google.gemma-4-31b | google.gemma-4-26b-a4b | google.gemma-4-e2b |
|---|---|---|---|
| Architecture | Dense | Mixture-of-Experts | Dense (PLE) |
| Total / Active params | 30.7B | 25.2B / 3.8B active | 5.1B / 2.3B effective |
| Context window | 256K tokens | 256K tokens | 128K tokens |
| Input modalities | Text, image | Text, image | Text, image |
| Reasoning mode | Yes | Yes | Yes (recommend high effort) |
| Function calling | Native | Native | Native |
| Service tiers | Standard, Priority, Flex | Standard, Priority, Flex | Standard, Priority, Flex |
How to choose: 31B for reasoning or coding-heavy tasks; 26B-A4B for cost-sensitive, high-throughput workloads; E2B for latency-sensitive or lightweight multimodal classification (source: AWS official announcement, 2026-06-15).
Quick Start
Prerequisites
You need an AWS account, and your IAM principal must have the AmazonBedrockMantleInferenceAccess policy attached. This policy grants permission to call inference operations on the bedrock-mantle endpoint (source: AWS official announcement). Access is through the bedrock-mantle endpoint at:
https://bedrock-mantle.{region}.api.aws/openai/v1
Calling with the OpenAI SDK
Call Gemma 4 31B through the OpenAI SDK. Use short-term API keys—they expire automatically (up to 12 hours) and inherit permissions from the IAM role that created them. In production, manage credentials through AWS Secrets Manager or Parameter Store (source: AWS official announcement, 2026-06-15).
from openai import OpenAI
client = OpenAI(
api_key="<your-short-term-bedrock-api-key>",
base_url="https://bedrock-mantle.us-east-1.api.aws/openai/v1",
)
response = client.chat.completions.create(
model="google.gemma-4-31b",
messages=[
{"role": "user", "content": "Explain the benefits of mixture-of-experts architectures for production inference."}
],
max_tokens=512,
)
print(response.choices[0].message.content)
Migrating an existing app only requires updating the base URL and model ID—no SDK or message structure changes needed.
Console Playground
If you'd rather skip the code, test directly in the Bedrock console: Test playgrounds → Chat/Text → select Google → pick a Gemma 4 model.
Key Features
Reasoning Mode
Gemma 4 outputs its thought process before the final answer. Enable it through the Responses API's reasoning parameter; effort supports low, medium, and high. The thinking is returned as a separate reasoning output item, not mixed into the final answer (source: AWS official announcement, 2026-06-15):
response = client.responses.create(
model="google.gemma-4-31b",
input="If a train leaves at 3pm at 60 km/h and another leaves an hour later at 90 km/h from the same station, when does the second catch up?",
reasoning={"effort": "high"},
)
print(response.output_text)
for item in response.output:
if item.type == "reasoning":
for block in item.content:
print(block.text)
In multi-turn conversations, only send the previous turn's final answer (output_text) back as history—don't include reasoning items. AWS notes that replaying prior reasoning degrades response quality (source: AWS official announcement, 2026-06-15).
For the E2B variant, set reasoning_effort to high. This model tends to do extensive reasoning by default, and high effort keeps that thinking in a dedicated channel so it doesn't leak into the final answer.
Function Calling
Gemma 4 supports native function calling. The flow: define tools → send request → receive tool_call → execute function → return results. The bedrock-mantle endpoint is compatible with OpenAI's tool calling interface. See the AWS article for a full example.
Multimodal (Image Input)
All variants support text + image input. The API accepts base64 data URLs or S3 URIs (arbitrary HTTPS URLs are not supported). For best results, place image content before the text in your prompt (source: AWS official announcement, 2026-06-15):
import base64
with open("chart.png", "rb") as image_file:
image_b64 = base64.b64encode(image_file.read()).decode("utf-8")
data_url = f"data:image/png;base64,{image_b64}"
response = client.chat.completions.create(
model="google.gemma-4-31b",
messages=[{
"role": "user",
"content": [
{"type": "image_url", "image_url": {"url": data_url}},
{"type": "text", "text": "Describe the trend shown in this chart."}
]
}],
)
Streaming
Set stream=True to enable SSE streaming, useful for chat and agent use cases:
stream = client.chat.completions.create(
model="google.gemma-4-31b",
messages=[{"role": "user", "content": "Write a short poem."}],
stream=True,
)
for chunk in stream:
delta = chunk.choices[0].delta.content
if delta:
print(delta, end="", flush=True)
Recommended Sampling Parameters
On the bedrock-mantle endpoint, sampling is controlled by temperature and top_p. AWS recommends temperature=1.0 and top_p=0.95, which work well for both reasoning and non-reasoning modes (source: AWS official announcement, 2026-06-15).
Selection Guide
| Workload Profile | Recommended Variant | Why |
|---|---|---|
| Complex reasoning, code generation, single-model inference | Gemma 4 31B | Largest dense variant, 256K context, strong reasoning and coding |
| High throughput, cost-sensitive, needs broad knowledge | Gemma 4 26B-A4B | MoE design keeps cost and latency near a 4B model while retaining 25B-class knowledge |
| Low latency, on-device-style, lightweight multimodal classification | Gemma 4 E2B | Smallest, fastest variant, ideal for cost or speed-sensitive scenarios |
If different requests in the same application have different needs, you can switch variants per request—all three share the same API surface (system prompt, tool calling, image input, reasoning mode).
Production Notes
Service Tiers
Bedrock offers three on-demand tiers, switched via the service_tier parameter per request (source: AWS official announcement, 2026-06-15):
| Tier | Best For |
|---|---|
| Standard | Everyday AI tasks, standard pay-per-token pricing |
| Priority | Customer-facing, real-time agent workloads sensitive to latency (up to 25% better OTPS latency over Standard, source: AWS official announcement, 2026-06-15) |
| Flex | Model evaluation, content summarization, agentic background tasks (discounted pricing, higher latency) |
Rate Limiting
The bedrock-mantle endpoint rate-limits by token quota (input tokens/minute, output tokens/minute), not RPM. Open models like Gemma don't have published per-account token quotas yet—throughput is governed by internal service capacity. For HTTP 429 (token quota exceeded) or 503 (regional capacity pressure), use exponential backoff with jitter (source: AWS official announcement, 2026-06-15).
The OpenAI SDK's max_retries parameter handles retries automatically:
client = OpenAI(
api_key="<your-short-term-bedrock-api-key>",
base_url="https://bedrock-mantle.us-east-1.api.aws/openai/v1",
max_retries=6,
)
Traffic Ramping
Avoid sudden large spikes in request rate. AWS recommends gradual scaling: start at your target rate; if you hit 503s, reduce by 50% and hold steady for 15 minutes, then increase by 50% and hold for another 15 minutes. Repeat until you reach your target volume (source: AWS official announcement, 2026-06-15). See AWS's Scaling on-demand inference documentation for details.
Implicit Prompt Caching
Gemma 4 on Bedrock has implicit prompt caching enabled automatically—no code changes or markers needed. Cache hits aren't guaranteed on every request, but they're more likely with stable prompt prefixes: multi-turn agents, RAG applications, and long-context analysis are typical use cases (system prompts, tool definitions, source documents reused across requests). Place static content at the front of your prompt and dynamic content at the end to maximize cache hit probability. Cached input tokens don't count toward your input token quota (source: AWS official announcement, 2026-06-15).
Cleanup
On-demand inference only charges when you call it—there's no infrastructure to tear down. Short-term API keys expire automatically (up to 12 hours); to revoke early, delete the key in the Bedrock console. After testing the Priority tier, remove the service_tier parameter to switch back to Standard. See Amazon Bedrock pricing for current rates (source: AWS official announcement, 2026-06-15).
FAQ
What's the difference between running Gemma 4 on Bedrock vs. Google AI Studio?
Gemma 4 is Apache 2.0 licensed and runs on multiple platforms. On Bedrock, inference runs entirely on AWS infrastructure, with AWS as the data processor. Your prompts and completions are not used to train any model and are not shared with third parties (source: AWS official announcement, 2026-06-15).
Do I need Amazon Bedrock to run Gemma 4?
No. Gemma 4 is open-weight under Apache 2.0—you can download the weights from Hugging Face and deploy on your own infrastructure. Running on Bedrock means you don't manage inference infrastructure, pay per token, and automatically get Standard/Priority/Flex tiers within AWS's security boundary.
How much does it cost?
Pricing is per token and varies by model and service tier. Check the AWS Bedrock pricing page for current rates.
What function calling formats are supported?
Gemma 4's function calling is compatible with the OpenAI SDK's tool calling interface. Define a tools array → model returns tool_call → execute the function → return tool results. See the code example above for the full flow.
Does it support quantization formats like GPTQ or AWQ?
The AWS announcement doesn't mention quantization deployment options on Bedrock. For self-deployment, community quantization tools are available.
Is fine-tuning supported?
AWS hasn't announced fine-tuning support for Gemma 4 on Bedrock. If you need to fine-tune on proprietary data, you can do so outside Bedrock and deploy the fine-tuned weights on your own infrastructure.