Amazon S3 FAQs

The total volume of data and number of objects you can store in Amazon S3 are unlimited. Individual Amazon S3 objects can range in size from a minimum of 0 bytes to a maximum of 50 TB. The largest object that can be uploaded in a single PUT is 5 GB. For objects larger than 100 MB, customers should consider using the multipart upload capability.

A vector bucket is purpose-built for storing and querying vectors. Within a vector bucket, you do not use the S3 object APIs, but rather dedicated vector APIs to write vector data and query it based on semantic meaning and similarity. You can control access to your vector data with the existing access control mechanisms in Amazon S3, including bucket and IAM policies. All writes to a vector bucket are strongly consistent, which means that you can immediately access the most recently added vectors. As you write, update, and delete vectors over time, S3 vector buckets automatically optimize the vector data stored in them to achieve the optimal price-performance, even as the data sets scale and evolve.

A bucket is a container for objects and tables stored in Amazon S3, and you can store any number of objects in a bucket. General purpose buckets are the original S3 bucket type, and a single general purpose bucket can contain objects stored across all storage classes except S3 Express One Zone. They are recommended for most use cases and access patterns. S3 directory buckets only allow objects stored in the S3 Express One Zone storage class, which provides faster data processing within a single Availability Zone. They are recommended for low-latency use cases. Each S3 directory bucket can support up to 2 million transactions per second (TPS), independent of the number of directories within the bucket. S3 table buckets are purpose-built for storing tabular data in S3 such as daily purchase transactions, streaming sensor data, or ad impressions. When using a table bucket, your data is stored as an Iceberg table in S3, and then you can interact with that data using analytics capabilities such as row-level transactions, queryable table snapshots, and more, all managed by S3. Additionally, table buckets perform continual table maintenance to automatically optimize query efficiency over time, even as the data lake scales and evolves. S3 vector buckets are purpose-built for storing and querying vectors. Within a vector bucket, you use dedicated vector APIs to write vector data and query it based on semantic meaning and similarity. You can control access to your vector data using the existing access control mechanisms in Amazon S3, including bucket and IAM policies. As you write, update, and delete vectors over time, S3 vector buckets automatically optimize the vector data stored in them to achieve the optimal price-performance, even as the data sets scale and evolve.

You specify an AWS Region when you create your Amazon S3 general purpose bucket. For S3 Standard, S3 Standard-IA, S3 Intelligent-Tiering, S3 Glacier Instant Retrieval, S3 Glacier Flexible Retrieval, and S3 Glacier Deep Archive storage classes, your objects are automatically stored across multiple devices spanning a minimum of three Availability Zones (AZs). AZs are physically separated by a meaningful distance, many kilometers, from any other AZ, although all are within 100 km (60 miles) of each other. Objects stored in the S3 One Zone-IA storage class are stored redundantly within a single Availability Zone in the AWS Region you select.  You specify a single Availability Zone or AWS Local Zone when you create your directory bucket. Objects in directory buckets are stored redundantly within a single Availability Zone or single Local Zone. When using Local Zones, your objects stay in the Local Zone unless you transfer them to an AWS Region. For S3 on Outposts, your data is stored in your Outpost on-premises environment, unless you manually choose to transfer it to an AWS Region. Refer to AWS regional services list for details of Amazon S3 service availability by AWS Region.

You should use S3 in AWS Local Zones if you have data and applications that need to run in a specific geographic locations to address data residency and compliance requirements. For example, some regulations require data must be stored in a particular country, for regulatory, contractual, or information security reasons common in public sector, healthcare, oil and gas, and other highly-regulated industries.

There are no set up charges or commitments to begin using Amazon S3. At the end of the month, you will automatically be charged for that month’s usage. You can view your charges for the current billing period at any time by logging into your Amazon Web Services account, and selecting the 'Billing Dashboard' associated with your console profile. With the AWS Free Usage Tier*, you can get started with Amazon S3 for free in all Regions except the AWS GovCloud Regions. Upon sign up, new AWS customers receive 5 GB of Amazon S3 Standard storage, 20,000 Get Requests, 2,000 Put Requests, and 100 GB of data transfer out (to internet, other AWS Regions, or Amazon CloudFront) each month for one year. Unused monthly usage will not roll over to the next month. Amazon S3 charges you for the following types of usage. Note that the calculations below assume there is no AWS Free Tier in place.

Starting July 15, 2025, new AWS customers will receive up to $200 in AWS Free Tier credits, which can be applied towards eligible AWS services, including Amazon S3. At account sign-up, you can choose between a free plan and a paid plan. The free plan will be available for 6 months after account creation. If you upgrade to a paid plan, any remaining Free Tier credit balance will automatically apply to your AWS bills. All Free Tier credits must be used within 12 months of your account creation date. To learn more about the AWS Free Tier program, refer to AWS Free Tier website and AWS Free Tier documentation.

Amazon S3 Tables deliver S3 storage that is specifically optimized for analytics workloads, improving query performance while also reducing costs. You can access advanced Iceberg analytics capabilities and query data using familiar AWS services like Amazon Athena, Redshift, and EMR through the S3 Tables integration with Amazon SageMaker lakehouse architecture. Additionally, you can use Iceberg REST compatible third-party applications like Apache Spark, Apache Flink, Trino, DuckDB, and PyIceberg, to read and write data into S3 Tables. You can use table buckets to store tabular data such as daily purchase transactions, streaming sensor data, or ad impressions as an Iceberg table in Amazon S3, and then interact with that data using analytics capabilities such as row-level transactions, queryable table snapshots, and more, all managed by Amazon S3. Table buckets perform continual table maintenance to automatically optimize query efficiency over time, even as the data lake scales and evolves.

You should use S3 Tables for a simple, performant, and cost-effective way to store tabular data in Amazon S3. S3 Tables give you the ability to organize your structured data into tables, and then to query that data using standard SQL statements, with virtually no setup. Additionally, S3 Tables deliver the same durability, availability, scalability, and performance characteristics as S3 itself, and automatically optimize your storage to maximize query performance and to minimize cost. With the Intelligent-Tiering storage class, S3 Tables automatically optimizes costs based on access patterns, without performance impact or operational overhead.

S3 Tables provide purpose-built S3 storage for storing structured data in the Apache Iceberg format. Within a table bucket, you can create tables as first-class resources directly in S3. These tables can be secured with table-level permissions defined in either identity- or resource-based policies and are accessible by applications or tooling that supports the Apache Iceberg standard. When you create a table in your table bucket, S3 maintains the metadata necessary to make that data queryable by your applications. Table buckets include an Iceberg REST Catalog endpoint that can be used by any Iceberg-compatible query engines discover, access, and update Iceberg metadata for tables in your table bucket. This allows for multiple clients to safely read and write data to your tables. Over time, S3 automatically optimizes the underlying data by rewriting, or "compacting” your objects. Compaction optimizes your data on S3 to improve query performance. Additionally, snapshot expiration and unreferenced file removal optimize storage cost as the data in your tables age. Read the user guide to learn more.

You can get started with S3 Tables in just a few simple steps without having to stand up any infrastructure outside of S3. First, create a table bucket in the S3 console. As part of creating your first table bucket through the console, the integration with AWS Analytics services happens automatically, which enables S3 to automatically populate all table buckets and tables in your account and Region in the AWS Glue Data Catalog. After this, S3 Tables is now accessible to AWS query engines such as Amazon Athena, EMR, and Redshift. Next, you can click to create a table using Amazon Athena from the S3 console. Once in Athena, you can quickly start populating new tables and querying them.

Alternatively, you can access S3 Tables using the Iceberg REST Catalog endpoint through the AWS Glue Data Catalog, which enables you to discover your entire data estate, including all table resources. You can also connect directly to an individual table bucket endpoint to discover all S3 Tables resources within that bucket. This enables you to use S3 Tables with any application or query engine that supports the Apache Iceberg REST Catalog specification.

You can create a table in your table bucket using the S3 console or using the CreateTable API or CLI operation. To create a table using the console, navigate to your table bucket in the S3 console and choose "Create table with Athena." You'll be prompted to either create a new namespace or select an existing one. After selecting a namespace, you'll be taken to the Athena query editor, where you can use the pre-populated sample SQL statement to create a new table. Once you run the query, your table is created and appears in both Athena and the S3 console. To delete a table, you can use the DeleteTable API or CLI operation. Alternatively, you can use your query engine to delete a table. When you do this, your table will no longer be accessible by your query engine.

S3 Tables deliver up to 10x higher transactions per second (TPS) compared to storing Iceberg tables in general purpose Amazon S3 buckets. S3 Tables automatically perform compaction on the underlying data to continually optimize your tables for optimal query performance. Depending on your workload and query patterns, you can also choose from advanced compaction strategies such as sort and z-order compaction to further optimize your tables. Sort compaction organizes data based on specified columns to improve query performance for filtered operations, while z-order compaction optimizes data organization across multiple dimensions, making it ideal when you need to query data across multiple columns simultaneously.

Table buckets give you the ability to apply resource policies to the entire bucket, or to individual tables. Table bucket policies can be applied using the PutTablePolicy and PutTableBucketPolicy APIs. Table-level policies allow you to manage permissions to tables in your table buckets based on the logical table that it is associated with, without having to understand the physical location of individual data files. S3 Tables support tags for attribute-based access control (ABAC), enabling you to scale access permissions and grant access to tables based on their tags in your IAM policies, AWS Organizations policies, and S3 Tables resource policies. This simplifies access management by reducing the need for frequent policy updates as your data lake grows. Additionally, S3 Block Public Access is always applied to your table buckets.

Table buckets support the Apache Iceberg table format with Parquet, Avro, or ORC data.

Table buckets offer three maintenance operations: compaction, snapshot management, and unreferenced file removal. Compaction periodically combines smaller objects into fewer, larger objects to improve query performance.

You can also choose from advanced compaction strategies such as sort and z-order to further optimize performance based on your query patterns.

Yes, S3 Tables support AWS CloudTrail. You can set up CloudTrail data and management events for your table buckets, similar to how you would with a general purpose S3 bucket. CloudTrail logs for your table buckets include information about both table and data-level requests, as well as automatic maintenance operations performed by S3 Tables on your tables.

With S3 Tables, you pay for storage, requests, and an object monitoring fee per object stored in table buckets. There are also additional fees for table maintenance and replication. To see pricing details, read the S3 pricing page.

Compaction combines smaller objects into fewer, larger objects to improve Iceberg query performance. By default, S3 Tables uses binpack compaction strategy. You can also use advanced compaction strategies such as sort and z-order to further optimize performance. The compacted files are written as the most recent snapshot of your table. Amazon S3 compacts tables based on a target file size optimal for your data access pattern, or a value you specify, with a default target file size of 512MB. You can change the target file size from 64MB to 512MB using the PutTableMaintenanceConfiguration API.

Snapshot management expires and removes table snapshots as per the snapshot retention configuration. Snapshot management determines the number of active snapshots for your tables based on the MinimumSnapshots (1 by default) and MaximumSnapshotAge (120 hours by default). When a snapshot expires, Amazon S3 creates delete markers for the data and metadata files uniquely referenced by that snapshot and marks these files as noncurrent. These noncurrent files are deleted after the number of days specified by the NoncurrentDays property in your unreferenced file removal policy. You can change the default values for snapshot using the PutTableMaintenanceConfiguration API. Snapshot management does not support retention values you configure on the Iceberg metadata.json file, including branch or tag-based retention. Snapshot management for S3 Tables is disabled when you configure a branch or tag-based retention policy or configure a retention policy on the metadata.json file that is longer than the values configured through the PutTableMaintenanceConfiguration API. 

S3 Tables replication enables automatic, asynchronous replication of Apache Iceberg tables across AWS Regions and accounts. S3 Tables replication creates and maintains read-only replicas of your tables, including all table data, metadata, and snapshot history, helping you reduce query latency for geographically distributed teams. When you enable replication, S3 Tables automatically creates read-only replica tables in your destination table buckets, backfills them with the latest state of the source table, and continuously monitors for new updates to keep replicas in sync. The replication process maintains the order of snapshots and preserves parent- child relationships in the snapshot history. You can configure replication at the table bucket level to replicate all tables, or at the individual table level for selective replication.

You can get started with S3 Vectors in four simple steps, without having to set up any infrastructure outside of Amazon S3. First, create a vector bucket in a specific AWS Region through the CreateVectorBucket API or in the S3 Console. Second, to organize your vector data in a vector bucket, you create a vector index with the CreateIndex API or in the S3 Console. When you create a vector index, you specify the distance metric (Cosine or Euclidean) and the number of dimensions a vector should have (up to 4092). For the most accurate results, select the distance metric recommended by your embedding model. Third, add vector data to a vector index with the PutVectors API. You can optionally attach metadata as key value pairs to each vector to filter queries. Fourth, perform a similarity query using the QueryVectors API, specifying the vector to search for and the number of the most similar results to return.

You can create a vector index using the S3 Console or the CreateIndex API. During index creation, you specify the vector bucket, index, distance metric, dimensions, and optionally a list of metadata fields that you want to exclude from filtering during similarity queries. For example, if you want to store data associated with vectors purely for reference, you can specify these as non-filterable metadata fields. Upon creation, each index is assigned a unique Amazon Resource Name (ARN). Subsequently when you make a write or query request, you direct it to a vector index within a vector bucket.

You can add vectors to a vector index using the PutVectors API. Each vector consists of a key, which uniquely identifies each vector in a vector index (e.g. you can programmatically generate a UUID). To maximize write throughput, it is recommended that you insert vectors in large batches, up to the maximum request size. Additionally, you can attach metadata (for example, year, author, genre, and location) as key value pairs to each vector. When you include metadata, by default all fields can be used as filters in a similarity query unless specified as non-filterable metadata at the time of vector index creation. To generate new vector embeddings of your unstructured data, you can use Amazon Bedrock’s InvokeModel API, specifying the model ID of the embedding model you want to use.

You can use the GetVectors API to look up and return vectors and associated metadata by the vector key.

You can run a similarity query with the QueryVectors API, specifying the query vector, the number of relevant results to return (the top k nearest neighbors), and the index ARN. When generating the query vector, you should use the same embedding model that was used to generate the initial vectors stored in the vector index. For example, if you use Amazon Titan Text Embeddings v2 in Amazon Bedrock to generate embeddings of your documents, it is recommended that you use the same model to convert a question to a vector. Additionally, you can use metadata filters in a query, to search vectors that match the filter. When you run the similarity query, by default the vector keys are returned. You can optionally include the distance and metadata in the response.

S3 Vectors offers highly durable and available vector storage. Data written to S3 Vectors is stored on S3, which is designed for 11 9s of data durability. S3 Vectors is designed to deliver 99.99% availability with an availability SLA of 99.9%.

S3 Vectors delivers sub-second query latency times. It uses the elastic throughput of Amazon S3 to handle searches across millions of vectors and is ideal for infrequent query workloads.

For performing similarity queries for your vector embeddings, several factors can affect average recall, including the embedding model, size of the vector dataset (number of vectors and dimensions), and the distribution of queries. S3 Vectors delivers over 90% average recall for most datasets. Average recall measures the quality of query results—90% means the response contains 90% of the ground truth closest vectors, that are stored in the index, to the query vector. However, because actual performance may vary depending on your specific use case, we recommend conducting your own tests with representative data and queries to validate that S3 vector indexes meet your recall requirements.

You can see a list of vectors in a vector index with the ListVectors API, which returns up to 1,000 vectors at a time with an indicator if the response is truncated. The response includes the last modified date, vector key, vector data, and metadata. You can also use the ListVectors API to easily export vector data from a specified vector index. The ListVectors operation is strongly consistent. So, after a write you can immediately list vectors with any changes reflected.

With S3 Vectors, you pay for storage and any applicable write and read requests (e.g., inserting vectors and performing query operations on vectors in a vector index). To see pricing details, see the S3 pricing page.

Yes. While creating a Bedrock Knowledge Base through the Bedrock Console or API, you can configure an existing S3 vector index as your vector store to save on vector storage costs for RAG use cases. If you prefer to let Bedrock create and manage the vector index for you, use the Quick Create workflow in the Bedrock console. Additionally, you can configure a new S3 vector index as your vector store for RAG workflows in Amazon SageMaker Unified Studio.

Yes. There are two ways you can use S3 Vectors with Amazon OpenSearch Service. First, S3 customers can export all vectors from an S3 vector index to OpenSearch Serverless as a new serverless collection using either the S3 or OpenSearch console. If you build natively on S3 Vectors, you will benefit from being able to use OpenSearch Serverless selectively for workloads with real-time query needs. Second, if you are a managed OpenSearch customer, you can now choose S3 Vectors as your engine for vector data that can be queried with sub-second latency. OpenSearch will then automatically use S3 Vectors as the underlying engine for vectors and you can update and search your vector data using the OpenSearch APIs. You gain the cost benefits of S3 Vectors, with no changes to your applications.

S3 Files is a shared file system that connects any AWS compute directly with your data in Amazon S3. It provides fast, direct access to all of your S3 data as files with full file system semantics and low-latency performance, without your data ever leaving S3. That means file-based applications, agents, and teams can now access and work with S3 data as a file system using the tools they already depend on. You no longer need to duplicate your data or cycle it between object storage and file system storage. Now, file-based tools and applications across your organization can work with your S3 data directly from any compute instance, container, and function using the tools your teams and agents already depend on.

With S3 Files, Amazon S3 is the first and only cloud object store that provides fully-featured, high-performance file system access to your data. S3 Files gives you the performance and simplicity of a file system with the scalability, durability, and cost-effectiveness of S3. There are no data silos, no synchronization complexities, and no tradeoffs. File and object storage, together in one place without compromise.

You should use S3 Files when your file-based applications, AI agents, and tools need to work directly with data stored in S3. S3 Files eliminates the need to copy data between storage systems or build custom integrations. Your existing Python libraries, ML frameworks, CLI utilities, and shell scripts work with your S3 data using standard file operations, with no code changes required. You can mount your S3 bucket across multiple compute resources (EC2, EKS, ECS, Lambda, Fargate, and Batch) for distributed applications where teams, agents, and workloads collaborate on shared datasets in real time.

S3 Files works like a traditional high-performance file system that can be accessed by any Linux-based compute resource, but its view of files and folders reflects what's in your S3 bucket. S3 Files is built using Amazon EFS, which intelligently loads your active working set onto high-performance storage. This delivers low latencies for frequently accessed data while keeping costs proportional to what you're actively using. When you read files, S3 Files lazily loads portions of file metadata and contents onto high-performance storage. Data that doesn't meet your configured file size threshold is read directly from S3, with no file system storage involved. When you write data, your writes are sent to the highly durable high-performance storage and then synced back to S3 to keep your bucket consistent. Data that hasn't been accessed within a configurable window (1 to 365 days, defaulting to 30) automatically expires from this storage, so you pay only for what you're actively using while your authoritative data always remains in S3.

S3 Files intelligently loads your active working set onto high-performance storage, delivering sub-millisecond latencies for frequently accessed data. Each file system delivers up to 5 GiB/s of write throughput performance, up to 250K read IOPS performance per file system, and 50K of write IOPS performance. The maximum per-client read throughput is 3 GiB/s, with multiple terabytes per second of aggregate read throughput.

When accessing files that aren't cached in your file system, the file system needs to first retrieve data from your bucket, which has latencies in the tens of milliseconds. Data stored in the file system is read with as low as sub-millisecond latencies. Writes are staged on the file system with single-digit millisecond latencies.

You can create an S3 file system from the S3 console, AWS CLI, or S3 API, and then mount it to your EC2 instances using standard mount commands. If you’re using a Linux distribution other than Amazon Linux, install the amazon-efs-utils on your instance before mounting your bucket.

For EKS clusters, install the Container Storage Interface (CSI) driver add-on efs-csi-driver and then use the Kubernetes CLI (kubectl) to mount the bucket. For ECS instances, add your S3 file system using Task definitions in the ECS Console or ECS API, and then attach the task definition to your cluster. For Lambda functions, select your S3 file system in the console or add it as a file system to your function configuration APIs. See documentation for a step-by-step guide.

S3 Files supports all storage classes in S3 general purpose buckets. All objects are visible as files within the file system, regardless of their underlying S3 storage class.

If you attempt to open a file that’s backed by an asynchronous storage class (S3 Glacier Flexible Retrieval and S3 Glacier Deep Archive), you will receive an I/O error. To access the file, you must first restore it with an S3 API (see documentation).

S3 Files does not support data in S3 Tables, S3 Vectors, or directory buckets.

S3 Files supports POSIX permissions on files and directories. When accessing files, the file system checks the UID and GID of your client against a file’s permissions. These permissions are stored as object metadata in your S3 bucket using the same format as other AWS file services.

Data is always encrypted in transit and at rest. Data is encrypted in transit between your compute resources and your file system using TLS 1.3. By default, S3 encrypts any data residing in your file system with S3-managed keys (SSE-S3), or optionally, when creating your file system, you can specify a KMS key to use (SSE-KMS).

File systems and S3 handle update operations differently. File systems support operations like partial file updates and append operations, while S3 is designed for full object updates. File systems also support directory renames, whereas renaming a prefix on S3 requires renaming each object that contains the prefix.

File updates made to data in your S3 file system are temporarily staged in the file system before being synchronized to S3. By doing so, the file system intelligently aggregates updates (like multiple updates to a single file done in short succession) to reduce S3 API calls.

S3 file systems are automatically synchronized with your S3 bucket to optimize cost and performance. Initially, your file system starts empty, importing object metadata on-demand as you access files and directories. For example, when you list a directory in your file system, metadata for the directory is imported from the S3 bucket. After a directory is accessed for the first time and its associated metadata is imported, the directory will continue to stay synchronized with S3 as you make updates in the S3 bucket. To optimize performance, the file system automatically pre-loads data for files less than 128KiB when a directory is first accessed. Data for larger files is read directly from S3.

You can also configure fine-grained rules for how data is imported from the S3 bucket to the file system using the PutSynchronizationConfiguration API or the S3 Console. For example, you can configure a rule to automatically store data in the file system after a file is read from a specific S3 prefix.

By default, the file system automatically exports all new files and file updates as a new object or a new version of an existing object within minutes. If you delete a file, the corresponding object in your bucket will have a delete marker as the current version of the object.

S3 file systems provide NFS close-to-open consistency, meaning that when a client closes a file, future opens of that file by any client will see the latest version of the file content. All compute resources reading from and writing to the same mounted file system see consistent data in real time. This is the standard consistency model that most file-based applications expect.

S3 Files implements a consistency model where the S3 bucket serves as the authoritative source of truth. S3 API operations maintain strong read-after-write consistency, file system operations provide close-to-open consistency, and the synchronization between these two systems is eventually consistent.

S3 Files is designed to provide 99.999999999% durability (11 nines) for your authoritative dataset in your S3 bucket. Additionally, data in your file system is stored redundantly across a minimum of 3 Availability Zones (AZ) by default, providing built-in resilience against widespread disaster including the permanent loss of an entire data center.

You can monitor S3 Files using Amazon CloudWatch metrics. CloudWatch provides file system metrics such as file system storage usage and number of client connections. CloudWatch also provides metrics for monitoring updates between your file system and bucket, including the number of files pending export. Additionally, you can use AWS CloudTrail to log management events in your file system. For example, events such as creating a file system or creating a mount target generate entries in CloudTrail. See our documentation for more details.

Use S3 Files when you already have data in S3 and you need file-based applications, AI agents, or teams to work with it directly using standard file operations. S3 Files is the right choice when your primary goal is to bring file access to your existing S3 data without duplicating it or managing a separate storage system. Your data stays in S3 as the authoritative source, and you get file system access on top of it.

Use Amazon EFS when you need a fully managed, cloud-native shared file system as your primary storage for workloads like shared home directories, content management, application configuration, and development environments where your data originates in and is primarily accessed through the file system.

Use Amazon FSx if you have existing file-based applications that run on network-attached file storage. FSx is purpose-built to support those workloads by offering fully managed deployments of popular file systems (NetApp ONTAP, Windows File Server, OpenZFS, or Lustre) with the same features, capabilities, and performance you're used to.

By default, your object data and object metadata stay within the single Local Zone (includes Dedicated Local Zones) where you put the object. Bucket management and telemetry data, including bucket names, capacity metrics, CloudTrail logs, CloudWatch metrics, customer managed keys from AWS Key Management Service (KMS), and Identity and Access Management (IAM) policies, are stored back in the parent AWS Region. Optionally, other bucket management features, like S3 Batch Operations, store management metadata with bucket name and object name in the parent AWS Region.