How does Datasift compare to Yahoo! Pipes?


How does Datasift compare to Yahoo! Pipes?

I’ve just begun Alpha testing DataSift, and its not yet clear to me what I can do with DataSift that I’m not already doing with Yahoo! Pipes.

There is no clear feed-in / feed-out paradigm, since DataSift taps services internally:

http://datasift.com/a/platform/source-data

To do anything in DataSift involves using its own Curated Stream Definition Language (CSDL):

http://support.datasift.com/help/kb/csdl

For feed-out, DataSift more resembles Tarpipe in terms of pricing:

http://datasift.net/pricing

http://tarpipe.com/plans

= = =

Update: March 2012

Having in some ways become disenchanted with the performance limitations of Yahoo Pipes, I’ve been delving deeper into DataSift.

My number one frustration was why I could not just get a straight feed output directly from DataSift.  I did figure out how to connect the DataSift REST API JSON output to Yahoo Pipes.

However, there seems to be a compatibility problem with Yahoo Pipes “polling” versus DataSift “stream”, and I don’t fully understand the dynamics yet.

They are actually complementary services with many things able to do in one but not the other.  The two together double the possibilities.