Optimize Your Amazon S3 Costs

removed a few images that are irrelevant and off topic

←Older revision		Revision as of 17:08, 25 June 2016
Line 5:		Line 5:
	== Steps ==		== Steps ==
	=== Getting a broad understanding of S3 ===		=== Getting a broad understanding of S3 ===
−	#Understand your S3 use case. S3 can be used for many goals.[[Image:Optimize Your Amazon S3 Costs Step 1.jpg|center]]	+	#Understand your S3 use case. S3 can be used for many goals.
	#* As a place to store files for live serving on websites, particularly image files.<ref>[http://www.hongkiat.com/blog/amazon-s3-the-beginners-guide/#Amazon_S3_as_Image_Hosting Amazon S3 as Image Hosting]</ref>		#* As a place to store files for live serving on websites, particularly image files.<ref>[http://www.hongkiat.com/blog/amazon-s3-the-beginners-guide/#Amazon_S3_as_Image_Hosting Amazon S3 as Image Hosting]</ref>
	#* As a place for data that you consume from or generate in your applications: By logging all data in S3, you make your data independent of the EC2 instances. This allows for a more flexible EC2 instance architecture. For instance, you can temporarily spin up EC2 spot instances to read data from S3, transform it, write the transformed data back to S3, and then terminate themselves.		#* As a place for data that you consume from or generate in your applications: By logging all data in S3, you make your data independent of the EC2 instances. This allows for a more flexible EC2 instance architecture. For instance, you can temporarily spin up EC2 spot instances to read data from S3, transform it, write the transformed data back to S3, and then terminate themselves.
	#* As a place to store file dumps of structured data stores such as SQL databases, as well as EBS snapshots.		#* As a place to store file dumps of structured data stores such as SQL databases, as well as EBS snapshots.
	#* As a place to store executables, scripts, and configurations necessary to launch new instances with your applications (or update your applications on existing instances).		#* As a place to store executables, scripts, and configurations necessary to launch new instances with your applications (or update your applications on existing instances).
−	#Understand the main way S3 affects costs. Numbers below are for standard storage, caveats associated with other forms of storage are discussed later.<ref name=pricing/>[[Image:Optimize Your Amazon S3 Costs Step 2.jpg|center]]	+	#Understand the main way S3 affects costs. Numbers below are for standard storage, caveats associated with other forms of storage are discussed later.<ref name=pricing/>
	#* '''Storage costs''': The cost is measured in storage space multiplied by time. You do not pay upfront for an allocated amount of storage space. Rather, every time you use more storage, you pay extra for that extra storage for the amount of time you use it. Costs can therefore fluctuate over time as the amount of data you've stored changed. As of August 2015, the costs range from 3 cents per GB-month in all US regions to 4.08 cents per GB-month in Sao Paulo.<ref name=pricing/>		#* '''Storage costs''': The cost is measured in storage space multiplied by time. You do not pay upfront for an allocated amount of storage space. Rather, every time you use more storage, you pay extra for that extra storage for the amount of time you use it. Costs can therefore fluctuate over time as the amount of data you've stored changed. As of August 2015, the costs range from 3 cents per GB-month in all US regions to 4.08 cents per GB-month in Sao Paulo.<ref name=pricing/>
	#* '''Request pricing''': The cost for PUT, COPY, POST, or LIST Requests ranges from $0.005 per 1000 requests in US regions to $0.007 per 1000 requests in Sao Paulo.<ref name=pricing/> The cost for GET and all other requests is an order of magnitude smaller, ranging from $0.004 per 10000 requests in all US regions to $0.0056 per 10000 requests. Note, however, that most of the cost associated with a GET request is captured in the data transfer costs (if the request is made from outside the region).		#* '''Request pricing''': The cost for PUT, COPY, POST, or LIST Requests ranges from $0.005 per 1000 requests in US regions to $0.007 per 1000 requests in Sao Paulo.<ref name=pricing/> The cost for GET and all other requests is an order of magnitude smaller, ranging from $0.004 per 10000 requests in all US regions to $0.0056 per 10000 requests. Note, however, that most of the cost associated with a GET request is captured in the data transfer costs (if the request is made from outside the region).
	#* '''Data transfer costs''': Costs are zero within the same AWS region (both S3 -> S3 and S3 -> EC2 instances), about 2 cents per GB for data transfer across regions, and about 9 cents per GB for data transfer to outside AWS.		#* '''Data transfer costs''': Costs are zero within the same AWS region (both S3 -> S3 and S3 -> EC2 instances), about 2 cents per GB for data transfer across regions, and about 9 cents per GB for data transfer to outside AWS.
−	#Understand the pros and cons of dealing with S3 files.[[Image:Optimize Your Amazon S3 Costs Step 3.jpg|center]]	+	#Understand the pros and cons of dealing with S3 files.
	#* You can use the AWS Command Line Interface,<ref>[https://aws.amazon.com/cli/ AWS Command Line Interface]</ref> the more antiquated s3cmd, or other methods to read and write files in S3. You can also do these operations within your code in all programming languages with the appropriate libraries and extensions.		#* You can use the AWS Command Line Interface,<ref>[https://aws.amazon.com/cli/ AWS Command Line Interface]</ref> the more antiquated s3cmd, or other methods to read and write files in S3. You can also do these operations within your code in all programming languages with the appropriate libraries and extensions.
	#* Despite appearing superficially like a filesystem, S3 is not really one. It is a key-value store. It takes a little effort to get used to the precise differences, but one of the implications is that it requires a bit more gymnastics (and running time) to get a global picture of the amount of data used in a bucket or in subfolders of that bucket. Another implication is that it is not possible to perform operations like appending data to a file: all operations are atomic.		#* Despite appearing superficially like a filesystem, S3 is not really one. It is a key-value store. It takes a little effort to get used to the precise differences, but one of the implications is that it requires a bit more gymnastics (and running time) to get a global picture of the amount of data used in a bucket or in subfolders of that bucket. Another implication is that it is not possible to perform operations like appending data to a file: all operations are atomic.