Respect for HTTP 304: Positive Feedback Loop?
The FeedLounge crew are considering rewarding feeds that publish feeds that are capable of sending HTTP 304 responses. Here’s how this is a win for most everyone:
- I, as a FeedLounge user, get fresher feed data. This is where the rubber really meets the road!
- Content publishers who send 304 get fresher visits from FeedLounge users. Don’t think that other Web aggregators won’t mimic this idea as they expand, either.
- Aggregators and 304-aware publishers both get huge bandwidth savings from not having to send data back and forth. [See Anne van Kesteren's discussion of 304 for a funny personification of the exchange between aggregator and server.]
- Content publishers who aren’t 304-aware will have positive incentive to get that way. [They already have bandwidth-savings as an incentive, but the blogosphere is notorious for getting secondary effects like this to get CTOs to reconsider problems they're presently ignoring.]
Kudos for the idea, Alex and Scott. I hope Bloglines and the rest of the folks in this space will adopt this approach.
Posted March 23rd, 2006 in FeedLounge.
I saw this on the FL blog this morning, too, and thought it was a brilliant idea. Good use of positive feedback.
March 23rd, 2006 at 14:04Fast feed refresh and “conditional GET” support
When a feed has not changed since the last fetch, a server can answer the request with a “304 HTTP code”. Not all servers have this feature enabled, and the result is bandwidth loss.
March 24th, 2006 at 03:27Here’s a good idea from Scott (Feedlounge). Server…
However ironically, it seems that http://gfmorris.org/feed/rss2/ is not 304 friendly
March 24th, 2006 at 19:32Holy hell!? Is this a characteristic of RSS2, or have I borked a server setting?
March 24th, 2006 at 19:33[...] After I lauded software that publishes HTTP 304-aware feeds, Scott tells me that IJSM.org’s feeds are no longer 304-aware. UHOH! [...]
March 25th, 2006 at 20:40Bloglines crawls every feed every 30 minutes. Why would they ever want to delay that for feeds that don’t support 304s?
March 26th, 2006 at 22:50Paul: It may not ever make sense for Bloglines to do that; if they consider the 30-minute guarantee to be a key component of their service, that’s their choice; there’ll just be a resource penalty for doing so.
Where it might make sense for them, though is in situations where a large feed [say, 30kB] is being loaded every half-hour when the feed averages an update once-a-week. In such a case, you’re looking at 48 polls/day * 7 days/week * 30 kB / pull = 9.84375 MB. A penny here, a penny there …
March 26th, 2006 at 23:05Bandwidtth is only getting cheaper, the harder part is writing your crawler and backend infrastructure to handle millions of sites, every 30 minutes
March 26th, 2006 at 23:47While it is true that bandwidth is generally getting cheaper, at some point, the laws of commodity pricing seem to indicate that it will bottom out.
March 27th, 2006 at 06:06I noticed this independently and found your site via the trac.wordpress.org. I tracked down what I thought was
March 27th, 2006 at 22:11the problem, and a hack to fix it for now. I don’t think my fix has security implications since the results are only being used for string comparison. Here’s my writeup: http://www.emilsit.net/blog/archives/wordpress-etag-bug/
[...] Update: Someone else noticed and was able to file a bug. Their comments led me to realize it was a more recent change, part of the 2.0.2 upgrade, that probably caused the problem. Where are the regression tests? [...]
April 3rd, 2006 at 12:07