Yahoo! Pipes and its Shortcomings

For the past couple of days, I have been fooling around with Yahoo! Pipes. It's a tool and an interface for aggregating and/or manipulating RSS feeds. It's an interesting experiment and has been receiving some hype, most notably from Tim O'Reilly who called it a "milestone in the history of the internet." So how useful/powerful is it? My assessment is: right now, not very. There are several problems that are keeping it from being a useful (or sometimes even usable) tool at present.

The first problem is it is SLOW. When manipulating the first one or two modules, things run fine, even to the point of web 2.0 AJAXy nirvana. The scrolling is snappy, the debugger (which is a well-thought out touch) and DHTML animations look nice, and the interface is responsive. However, after that things start crawling. And after 10 or so modules, it slows to a why-am-I-even-bothering speed (and I'm on a pretty damn fast MIT connection right now with a capable PC).

The second problem are the all-too-frequent site outages. A recent blog post by the development team humorously noted: "Our apologies for the unscheduled downtime this evening. As a reminder, todays (sic) post was sponsored by the letters “b” “e” “t” and “a”."

The third problem is the support of only RSS inputs. But judging from the Pipes developers' posts, however, this problem looks like it will be overcome in the future.

So what can one do right now with Pipes? The canonical example has been putting the RSS feed of the New York Times through a Flickr filter, the result being that you can see the NYT stories in photo form. Which brings us to problem number four: your results are only as good as your data. While a lot of these photos are applicable to current events, others are tangential and some are just irrelevant. Kind of makes one think that a well-implemented semantic web isn't such a bad idea, after all.

Another thing one can do with Pipes is what I did with my first pipe, Ray's Deals. It simply takes several RSS feeds from bargain electronics sites and aggregates them into one feed. I then made the feed (or, more precisely, the titles of the items of the feed) searchable. The utility of this Pipe is that Ray now only needs to subscribe to the RSS feed from this pipe instead of several RSS feeds. (Sidenote: I actually added way more modules than necessary here: instead putting each URL in a separate Fetch module, I could have just had multiple entries in a single Fetch module... live and learn, I guess.)

That's about all the useful things one can do with Pipes at present -- my subsequent, more adventurous, Pipe experiments were fraught with frustration and, ultimately, failure. This was in part due to Pipes problem #5: the limited amount of operators that Pipes provides. There are currently no functions for performing the most basic of operations, such as generating random numbers (see the failure of my Pipe Random Jokes (Incomplete)) or extracting specific information from a feed. There are some workarounds for things like selecting a particular item from a feed, as my second hackish pipe, Selection Experiment 1, demonstrates. To select a certain oldest item, I sorted the feed by Date and then cut off the feed after a certain number of items (6). I then reversed the feed and cut off all but the first item, thus producing the selected item. Wow, that's a lot of work. Not to mention the limitations of some of the other operators, such as Sort not taking in a key value to sort by, etc. To be fair, it's still a beta product and the selection and power of modules will no doubt grow with time.

There are other concerns too, some of which are voiced in the responses to Tim O'Reilly's post. Among these are IP concerns -- is your mashup your property, or the feed providers' partial (or full) property? Also, there is the question of accessibility: will Pipes be a place that democratizes mashups, or is it just for geeky developers (who might be better off coding their own specialized apps anyways)?

All in all, Yahoo! Pipes looks like a tool that has some potential. It has a nice interface (when it's not super slow), the concept is fresh, and it's relatively easy to use (again, when it works). Its current shortcomings, however, make it little more than a novelty item. Once these get addressed, I'll be willing to give it a second try.