Currently browsing rfc2616
Fri, 22 May 2009 21:39 UTC
Another php|tek come and gone. I’m saddened by leaving, but, as usually happens, I’m reinvigorated and reenergized to go back to work. It seems I need these events to get together with other developers to raise me up out of periods of burn-out. I’m sure the same goes for others.
I gave my talk this morning on “Making the Most of HTTP In Your Apps.” There was so much material to cover and so little time to cover it in, so I hope people found little nuggets of value in it for their own applications. I’ll be revising it in the future to show more actual code examples.
For your reference, I’ve provided the slides below for you to enjoy and download. Here’s the abstract of what the talk is about, and please don’t forget to rate my talk:
“200, 404, 302. Is it a lock combination? A phone number? No, they’re HTTP status codes! As we develop Web applications, we encounter these status codes and others, and often we make decisions about which ones to return without giving much thought to their meaning or context. It’s time to take a deeper look at HTTP. Knowing the methods, headers, and status codes, what they mean, and how to use them can help you develop richer Internet applications. Join Ben Ramsey as he takes you on a journey through RFC 2616 to discover some of the gems of HTTP.”
2 Comments »
Permalink
Tags: http, phptek, rest, rfc2616, tek09
Mon, 16 Feb 2009 21:02 UTC
Last year, I wrote about the 100 Continue HTTP status code and the usage of the Expect header with the 100-continue expect value. However, I made a few erroneous statements, and a reader recently corrected me on them. So, I’m writing now to correct those statements so that I’m not misleading anymore readers.
The incorrect statements I made:
The service can then respond either “yes” (100 Continue) or “no” (417 Expectation Failed) with an explanation of why not.
Following that logic, I continued with:
Let’s say your service limits uploads to 100MB. In that case, you would return a 417 Expectation Failed including a reason for the failure in the body of the response.
This is incorrect behavior, and Glenn Maynard corrected it in his comments with:
This is incorrect. 417 means that an Expect you sent is unsupported; for example, if you send “Expect: baconâ€, and the server doesn’t support that, then it responds with 417. Responding with 417 because the request was going to be too large badly violates the spec.
The purpose of 100-continue is to give the server a chance to give an error to the client before it sends the request body, rather than after. If doing that required that all of those detailed 4xx/5xx errors were turned into an opaque 417 and useless, plain language errors in the body, it would be useless.
Glenn is correct. What I missed in Section 14.20 of RFC 2616 was this: “The server MUST respond with a 417 (Expectation Failed) status if any of the expectations cannot be met or, if there are other problems with the request, some other 4xx status.” Furthermore, Section 8.2.3 states: “Upon receiving a request which includes an Expect request-header field with the ‘100-continue’ expectation, an origin server MUST either respond with 100 (Continue) status and continue to read from the input stream, or respond with a final status code.” (Emphasis added.)
My faulty logic occurred because I was using the “look-before-you-leap” (LBYL) concept presented in RESTful Web Services, in which Ruby and Richardson propose that a request including the Expect header can be made to determine whether the server will fulfill the request, given the headers present. They state, “if the answer is no, the server sends a status code of 417 (‘Expectation Failed’). The answer might be no because [...] 500 MB is just too big” (pg. 250). According to RFC 2616, this is wrong. If the server does not support the 100-continue expectation, then, and only then, should it send back the 417 Expectation Failed response. Otherwise, if, for example, the Content-Length is too big for the server to fulfill the response, then it would respond with a “final status code,” such as 413 Request Entity Too Large.
You may still use this approach to perform LBYL requests, but be sure to send back a proper 4xx status code, rather than a 417.
Furthermore, for another fallacy of mine, if you read the comments on the 100 Continue post, I also stated “be aware that the Content-Length header cannot be trusted.” Glenn also corrected me on this. I had assumed that I could spoof the Content-Length header, and that if the server only checked the Content-Length header to determine the size of the request entity, then it could be fooled, but this is not the case. Web servers use the Content-Length header to determine how much data to read from the entity body. If your Content-Length header is 50 MB and the entity body contains 100 MB, the server will read only the first 50 MB of the body and stop.
2 Comments »
Permalink
Tags: http, rest, rfc2616, standards
Wed, 28 Jan 2009 4:27 UTC
The last post in my HTTP status code series was just over six months ago. I’m sorry for taking so long to revive the series, but I’m back with a discussion about the 4xx status codes. I hope you enjoy it!
The 4xx series of HTTP status codes is intended for client errors. That is, the server determines that there is an error occurring as a result of bad or missing data in the request from the client. The 404 Not Found status code is the most recognizable of these, and it’s the server’s way of telling the client that the requested URI is not available on the server. This is not the server’s fault, but, rather, it is the client’s, so the server responds saying, “Sorry, pal! I can’t find what you’re looking for.”
In an earlier post, I mentioned the 100 Continue status code. I mention it again here because the use of an Expect: 100-continue request header in a “look-before-you-leap” (LBYL) request can be useful to let you know of potential problems with your request before you receive any of these status code responses from the server.
Let’s use the same request from the 100 Continue post as an example:
POST /content/videos HTTP/1.1
Host: media.example.org
Content-Type: video/mp4
Content-Length: 105910000
Authorization: Basic bWFkZTp5b3VfbG9vaw==
...
Note that this request does not contain the Expect header, and it is assumed that the body of this request contains the full binary content of the video being posted.
For the sake of example, let’s assume that the server doesn’t allow POST requests to the /content/videos URI. If this is the case, then the server should return a 405 Method Not Allowed status code in response to this request. This is a condition that we could have checked for with the LBYL request prior to sending the full 105,910,000 bytes of the video, thus saving bandwidth, client resources, and server resources.
HTTP/1.1 405 Method Not Allowed
Date: Wed, 28 Jan 2009 03:04:25 GMT
Allow: GET,HEAD
In addition, to the 405 Method Not Allowed status, according to RFC 2616, the server must included an Allow header listing the valid methods for the requested resource. Sending the Allow header in response to HEAD requests (and all requests for that matter) can also help inform clients of acceptable methods.
If the server does allow POST requests at this URI but we fail to send the Authorization header with the request, then the server should respond with a 401 Unauthorized status, telling the client to resend the request with proper authorization. The server must include a WWW-Authenticate header in this response to indicate the expected authorization scheme. If the request, however, does include the Authorization header, as illustrated in the example, but the authentication fails for whatever reason, then the server should respond with a 403 Forbidden status and a description with the reason for the failure in the response body.
If the request doesn’t include the Content-Length header, then you might want the server to respond with a 411 Length Required status. Requiring the length and comparing it to the length of the actual request body is a good way to determine whether there was any loss of data. However, if the integrity of data is a concern, you might want to require clients to include the Content-MD5 header in the request. Since there is no status code to indicate Content-MD5 as a requirement, simply return a 400 Bad Request status stating your requirement of the Content-MD5 header in the response body.
The 400 Bad Request status code literally means that the “request could not be understood by the server due to malformed syntax,” but it is a generic enough response that it can be used for a variety of other client errors not covered by the other 4xx series status codes. For example, I have used 400 Bad Request as a way to indicate data validation failures in POST and PUT requests.
If the full request reaches the server, but the server determines that the 105,910,000 bytes of the video is beyond the acceptable limit—maybe we want to limit users to 50MB uploads—then respond with 413 Request Entity Too Large because the request body is larger than the server is willing to process. Again, the client could avoid this by using a LBYL request.
Likewise, if the server doesn’t want to support the video/mp4 media type, then it should respond with a 415 Unsupported Media Type status code. A LBYL request would have told us this, as well.
I’ve already discussed 416 Requested Range Not Satisfiable in a post about range requests and 417 Expectation Failed in a post about the 100 Continue status code.
Finally, we are all familiar with the 404 Not Found status code. I mentioned it earlier in this post. The 404 Not Found status, however, doesn’t tell the client whether the condition is temporary or permanent. It’s a pretty generic “I can’t find it” message. If you know that the resource is gone permanently, you may want to return a 410 Gone status code, instead, as a way of letting smart clients (like search engines) know that they should remove all links to the resource. If the resource has moved permanently, though, I recommend using a 301 Moved Permanently status, which will also tell these smart clients to update their links. How you choose to inform clients of the fate of your resources is up to you, but it’s in your best interest to tell them whether the data has moved or is gone and is no longer available.
I also mentioned the 404 Not Found and 410 Gone statuses when discussing the 201 Created and 202 Accepted status codes.
1 Comment »
Permalink
Tags: http, rest, rfc2616, standards
Sun, 20 Jul 2008 22:58 UTC
I’ve been rather uninspired and, therefore, uninterested in blogging lately, which is why I’ve neglected to continue my series on HTTP status codes. However, while trying to figure out a quick and easy way to delete tons of received direct messages from my Twitter account, I stumbled across one of my HTTP pet peeves coupled with the odd use of a 3xx status code, which inspired me to launch into a short rant followed by a discussion on the redirection series of status codes.
First, the pet peeve: I cannot stand it when someone uses a GET request to modify information. I don’t care how secure it is because you’re checking my session and login details, sending a GET request to /direct_messages/destroy/30264861 should not do anything other that retrieve the resource located there. If using GET, the user agent thinks it’s a safe action and will not know that it should warn me that the action I’m about to take could modify information. CSRF anyone?
Come on, people! GET is a safe method. Client-side developers (among others) would do well to learn this.
Consequently, there is no resource at that location, and that also peeves me. Instead, I get a weird 302 Moved Temporarily HTTP redirect that also includes a Status: 302 Found header. WTF? Come on! It’s 2008. RFC 2616 has been out for nine years now. Is there anyone using Twitter on a user agent old enough that it only works for HTTP 1.0? (I’ll wager that I’ll get responses from people about how some mobile browsers only understand HTTP 1.0.)
This leads me to the second thing I want to discuss: HTTP redirections.
Historically (in HTTP 1.0), HTTP had two status codes for redirection: 301 Moved Permanently and 302 Moved Temporarily. However, 302 became misused and abused by user agents that didn’t follow the spec, which stated: “If the 302 status code is received in response to a request using the POST method, the user agent must not automatically redirect the request unless it can be confirmed by the user, since this might change the conditions under which the request was issued.” Instead, user agents would often simply use a GET request on the value of the Location header, redirecting the user without warning.
Take a moment now to think about this. Why would the user agent send a GET request for the value of the Location header without asking the user if it’s okay? Because the user agent thinks it’s safe to do so. Remember my earlier rant?
Nevertheless, the user agents weren’t supposed to be behaving this way, but developers used 302 Moved Temporarily as the common method for redirecting after POST actions, and following the spec would make for a cumbersome user experience, especially if I had to approve every single redirect that occurs after I POST something. (This actually happens on mobile browsers quite frequently, since the mobile browser—on my phone, at least (a RAZR)—properly implements the expected behavior, thus making a crappy experience for me.)
So, to rectify the situation, HTTP 1.1 introduced 303 See Other and then later 307 Temporary Redirect (changing the meaning of 302 to Found).
The 303 See Other status code unambiguously states that the value of the Location header “SHOULD be retrieved using a GET method” and that the “new URI is not a substitute reference for the originally requested resource.” That is, if the user agent originally used POST to send data, then the server can tell the user agent to safely redirect to another resource using GET, and the new resource should not be expected to act on the data sent in the original POST request.
Likewise, the 307 Temporary Redirect status code unambiguously states that if it is “received in response to a request other than GET or HEAD, the user agent MUST NOT automatically redirect the request unless it can be confirmed by the user, since this might change the conditions under which the request was issued.” So, if I POST a form to a server that sends back a 307 Temporary Redirect, then the user agent should ask me whether I want to proceed. If I say “yes,” then the user agent will proceed to POST the data to the URI identified by the Location header.
So, a couple rules of thumb: 302 Found is still the de-facto redirection method, though it may not mean what you want it to mean or behave properly on all user agents. If you want to process a POST request and then safely redirect the user agent using GET, use 303 See Other. If you want to redirect a POST request (that is, send the posted data to a different location), use 307 Temporary Redirect. If the original request was GET or HEAD, and you simply want to redirect the user agent, then either 302 Found or 307 Temporary Redirect are proper, though I prefer the 307.
To explicitly tell user agents (and particularly search engines and proxies) that something has moved permanently and that they should update their links, use 301 Moved Permanently .
6 Comments »
Permalink
Tags: http, rest, rfc2616, standards
Wed, 7 May 2008 2:08 UTC
The 200 range of HTTP status codes represents successful requests. I’ve already covered 201 Created and 202 Accepted and 206 Partial Content. Today, I’ll wrap up my discussion of the 200 range by talking about 204 No Content and 205 Reset Content. The 200 OK response is probably the status with which most are familiar, and I’ll discuss it later when covering the HTTP verbs.
The 204 No Content response is useful in a web service when you want to return a success message but do not want to return a message in the body or do not have a body to return. In my personal experience, I use this when DELETE requests are sent to an Atom web service. If the resource is successfully deleted, the service returns a 204 No Content status message. This tells the client that the deletion was successful, and that’s really all the client needs to know. There’s nothing to return because it was deleted.
However, even outside of the web services realm, the 204 No Content status actually means something to a user agent (browser). The HTTP specification (RFC 2616) states the following about the 204 No Content status code:
If the client is a user agent, it SHOULD NOT change its document view from that which caused the request to be sent. This response is primarily intended to allow input for actions to take place without causing a change to the user agent’s active document view, although any new or updated metainformation SHOULD be applied to the document currently in the user agent’s active view.
This means that, if I have an HTML form and submit it, then, if the server returns a 204 status code, the browser should not refresh the form or take me to another page. The document view does not change, and I remain at the form. All of the data I entered remains unchanged. All browsers I tested support this, but, in practice, it is not very useful since there’s no indication to a user that anything happened on the server side.
However, consider the use of 204 No Content in an Ajax application. If the Ajax library used simply looks for a success message of 204 in response to the requests it sends, then it can present a success message to the user without changing any of the UI elements, as this response intends. This is also useful because the service doesn’t need to send back any data; it needs only tell the client that the request was successful.
On the other hand, with a 205 Reset Content response, the intent is to tell the client to clear the content from the form or to refresh the UI. That is, I could fill out a form, click the submit button, and the form I am currently working on would refresh to its default values if the client receives a 205 response. According to the HTTP spec, a 205 response means:
The server has fulfilled the request and the user agent SHOULD reset the document view which caused the request to be sent. This response is primarily intended to allow input for actions to take place via user input, followed by a clearing of the form in which the input is given so that the user can easily initiate another input action.
The problem is that I have not found a browser yet that supports this behavior. Browsers either treat a 205 like it’s a 204, or they simply treat it as a 200. This does not mean, though, that Ajax applications cannot properly respect a 205 response. In an Ajax application, if the client receives a 205 response (instead of a 204 response), then the UI elements should change back to their default state.
In RESTful Web Services, Richardson and Ruby make this distinction between 204 and 205:
In data entry terms: 204 is good for making a series of edits to a single record; 205 is good for entering a series of records in succession.
In my simple browser test to see what browsers supported 204 and 205 on forms (without using any Ajax), here’s what I found:
Firefox 2.0.0.14
- 204 - supported
- 205 - treated like 204
Safari 3.1.1
- 204 - supported
- 205 - treated like 200
Opera 9.27
- 204 - supported
- 205 - treated like 200
Internet Explorer 7 (7.0.5730.13)
- 204 - supported
- 205 - treated like 200
Internet Explorer 8 beta 1 (8.0.6001.17184)
- 204 - supported
- 205 - treated like 200
Firefox 3 beta 5
- 204 - supported
- 205 - treated like 204
Since I know I’ll be asked this question, to send back a 204 or 205 response with PHP, see the PHP manual on the use of the header() function. 
5 Comments »
Permalink
Tags: http, rest, rfc2616, standards
Mon, 5 May 2008 2:17 UTC
Akki commented on my 100 Continue post, asking:
I was wondering if there was a response status to allow a large file to be “served” in parts and in just one response?
While I’ve never done this myself, I did some research to see how it might be done, and I’ve come up with a solution that I wish I could call clever, innovative, and many other words that make me sound like I came up with a smart solution, but the reality of it is that HTTP was made to support this behavior, so I’ll just tell you how to do it with standard HTTP.
First of all, let’s create a scenario. Say I want to request a very large file from a service, but before I do so, I want to find out exactly how large it is and whether the service supports “range requests.” To do this, I make a HEAD request to the large resource I want to retrieve. The HEAD method is a standard HTTP method that acts as if I’ve made a GET request, but it returns only the headers and not the body. This allows me to find out some information about the resource without actually taking the time or using the bandwidth to download it. For example, I can read the Content-Length header and determine the size of the resource.
Another header of great importance is the Accept-Ranges header. If it’s present, it may contain a value of “bytes.” If so, I know that I can make a GET request and request a byte range to retrieve only those bytes. This is the first important key component:
If you are building a service that accepts byte range requests, let clients know by providing the Accept-Ranges header in response to GET and HEAD requests.
So, now, I know I can make a range request to the service, which I do by sending a standard GET request including a Range header that specifies the range of bytes I’m requesting, like so: Range: bytes=0-999. The service should then respond with with a 206 Partial Content status code, a Content-Range header, and the requested range of bytes. This is the second key component:
If a client makes a request of your service with a Range header, return a 206 Partial Content response containing a Content-Range header and the requested range of bytes for the resource in the body. The Content-Length value should be the length of what is actually returned and not the full length of the resource.
From then on, it’s a matter of the client continuing to make requests until it has retrieved all of the bytes it wishes to get from the resource. If the client makes a range request that is out of bounds—that is, none of the range values overlap the extent of the resource—the service should respond with a 416 Requested Range Not Satisfiable status.
I should note that the service cannot force the client to make a range request. It is entirely up to the client to make such requests, so the service should honor all standard GET requests that do not contain a Range header by sending back the full representation of the requested resource. If your service allows range requests, it is to your benefit to tell the client with an Accept-Ranges header. If you want to explicitly tell the client that you do not allow range requests, send a value of “none” back with the Accept-Ranges header.
Since I learn best by looking at real-life examples, I’ll provide one for you. Here’s a real example making range requests with Flickr. I’ve removed some of the headers for simplicity and clarity.
HEAD /2390/2253727548_a413c88ab3_s.jpg HTTP/1.1
Host: farm3.static.flickr.com
HTTP/1.0 200 OK
Date: Mon, 05 May 2008 00:33:14 GMT
Server: Apache/2.0.52 (Red Hat)
Accept-Ranges: bytes
Content-Length: 3980
Content-Type: image/jpeg
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
GET /2390/2253727548_a413c88ab3_s.jpg HTTP/1.1
Host: farm3.static.flickr.com
Range: bytes=0-999
HTTP/1.0 206 Partial Content
Date: Mon, 05 May 2008 00:36:57 GMT
Server: Apache/2.0.52 (Red Hat)
Accept-Ranges: bytes
Content-Length: 1000
Content-Range: bytes 0-999/3980
Content-Type: image/jpeg
{binary data}
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
GET /2390/2253727548_a413c88ab3_s.jpg HTTP/1.1
Host: farm3.static.flickr.com
Range: bytes=1000-
HTTP/1.0 206 Partial Content
Date: Mon, 05 May 2008 00:37:54 GMT
Server: Apache/2.0.52 (Red Hat)
Accept-Ranges: bytes
Content-Length: 2980
Content-Range: bytes 1000-3979/3980
Content-Type: image/jpeg
{binary data}
4 Comments »
Permalink
Tags: http, rest, rfc2616, standards
Fri, 25 Apr 2008 18:48 UTC
Continuing my series on HTTP status codes, I’d like to talk today about the use of the 201 Created and 202 Accepted codes.
I should make a point of clarification first, though. When I’m discussing the use of status codes, I have in mind Web services applications with perhaps rich clients that interface with them and not necessarily standard, browser-based Web apps. Browsers should be able to handle these status codes, but they are not necessarily the richest clients to use as an interface to your Web service, so they may not be able to handle the responses as intended.
Moving along…
Recently, while designing an application that uses the Atom Publishing Protocol, I found myself using the 201 Created status code as a response to successful POST requests for creation of new content in the service. In the body of the response, I would return a full representation of the newly created resource, and I would include a Location header with a URI that points to the location of the newly created resource. (Instead of returning a full representation of the created resource, you may return a list of URIs pointing to locations for the resource(s) created.)
In general, this is a good practice, though most of us find ourselves returning the standard 200 OK code. (Using 200 OK is still acceptable, though I’ll have more to say on that in a future post.) However, I ran into a problem when I realized that the resource itself does not yet exist, nor might it ever exist; it all depends on our business rules, which require various processes to occur before we “publish” the content. So, the 201 Created response is misleading to the client because nothing has been created yet, and the representation of the resource is inaccurate since the resource does not yet exist. The resource is really in a pending state.
Enter the 202 Accepted status code.
I decided to use the 202 Accepted status code as the more proper response to these types of requests. 202 Accepted means that the service successfully accepted the request and there are, as of yet, no problems with it (i.e. no immediate data validation problems), but it can’t create the resource until it does further processing. This response does not promise that the resource will be created, though. So, it’s perfect for pending requests, since a pending request could be rejected while it’s being processed.
The HTTP spec says that a 202 Accepted response SHOULD include an indication of the request’s current status. To this end, for our service, I have it return an Atom Entry Document with content that describes what’s going on, i.e. the request succeeded but requires further processing before it can be published. It may also be beneficial to clients to include a Location header with a URI that can be used later to check on the status of the request.
When the client checks the status URI later, if the item is still pending then the status URI might return a 200 OK response with an entity body describing this. If the resource has been created, then perhaps the status URI would return a 201 Created response with a Location header pointing to the location of the new resource. If the item was not created for some reason, then perhaps the status URI would return a 410 Gone response. In this case, you should include an entity body explaining why the resource is gone, i.e. “We were unable to create this resource due to processing errors.” A 404 Not Found response would also be acceptable, but the 410 Gone response implies permanence; the requested resource is gone for good.
The status URI described here is a temporary URI used only for checking on the status of the request. It would eventually be unavailable after the resource has been created or rejected, but the time frame for this is up to you.
3 Comments »
Permalink
Tags: http, rest, rfc2616, standards
Tue, 22 Apr 2008 20:26 UTC
UPDATE: This post contains some false information. Particularly, you should not send back a 417 Expectation Failed response as a way of telling the client that the server could not fulfill the response because of data presented in any of the request headers other than the Expect header. I have corrected my statements here in a newer post, and I apologize if I caused any confusion.
As a self-described HTTP status code pedant and an architect of RESTful Web services, I’ve spent a great deal of time researching and thinking about the proper use of HTTP status codes in Web applications. It occurs to me that many people use status codes in their applications in ways that I find improper, so I’ve decided to start up a series of blog posts to talk about select status codes and how they might appropriately fit within your Web applications, specifically in a RESTful manner. Consider these “Ben’s HTTP Status Codes Best Practices.”
I won’t be covering these status codes in any particular order. However, the first code up is 100 Continue merely because I’ve been thinking about it lately.
The 100 Continue status code is an informational response that tells the client application it can continue sending the request. In short, the client can make an abbreviated request of the service to check whether or not it should continue making the full request. The service can then respond either “yes” (100 Continue) or “no” (417 Expectation Failed) with an explanation of why not. In RESTful Web Services, Richardson and Ruby refer to this as a “look-before-you-leap” (LBYL) request.
This is helpful in situations where the client might want to post or put a very large file to the service but wants to ensure that the service will accept it before actually sending the file. Consider the following request:
POST /content/videos HTTP/1.1
Host: media.example.org
Content-Type: video/mp4
Content-Length: 105910000
Authorization: Basic bWFkZTp5b3VfbG9vaw==
Expect: 100-continue
In this case, the client is attempting to post an MPEG-4 video that is 101MB. The client sends this request to your service without the video in the body, and it includes the Expect: 100-continue header. Let’s say your service limits uploads to 100MB. In that case, you would return a 417 Expectation Failed including a reason for the failure in the body of the response.
By now, you can probably see how this can be extended to apply to any of the other headers. For example, a client can try a POST or a PUT request first just to see if its authorization succeeds. If it does, your service would return 100, letting the client know it may proceed with the request. Likewise, the client might want to determine whether the service will accept a video/mp4 file before sending it. The list goes on.
It should be noted that the service shouldn’t require the client to use a LBYL request, but it can be helpful to a client so that it doesn’t attempt to send something that would fail at the service anyway.
I’m considering implementing these kinds of responses in a service I’m building, so I’ll let you know how it goes.
13 Comments »
Permalink
Tags: http, rest, rfc2616, standards
