Gillmor Gang 09.24.09
Brett Slatkin of PubSubHubbub talks realtime RSS with David Recordon of Facebook, Robert Scoble of Rackspace and Kevin Marks of BT. Recorded live Thursday, September 24, 2009.
Transcript below, courtesy SimulScribe:
Mr. STEVE GILLMOR: Hi. This is Steve Gillmor and welcome to The Gillmor Gang. We’ve got, I hope, an interesting show today because we’re going to be talking as usual about whether RSS is dead. And in order to do that, we’ve convinced Brett Slatkin, the co-author or co-progenitor of PubSubHubbub to tell us what’s the status of all things speeding up RSS and getting into the real time world. And before I say, welcome to you, Brett. We also have with us Kevin Marks.
Mr. KEVIN MARKS: Hi, there. Good to see you.
Mr. GILLMOR: David Recordon of Facebook fame.
Mr. DAVID RECORDON: Hey, guys.
Mr. GILLMOR: And the infamous and I think we’re going to call Scoble the telligible Robert Scoble as oppose to the intelligible or unintelligible Kevin Marks from last week. So to the transcript writers, I’d like to introduce you to each of these four people starting with Scoble. Can you say your name, please?
Mr. ROBERT SCOBLE: Robert Scoble.
Mr. GILLMOR: OK. That’s what Robert Scoble sounds like. David Recordon.
Mr. RECORDON: David Recordon.
Mr. GILLMOR: OK. And that’s what David Recordon sounds like. Kevin Marks.
Mr. MARKS: I’m Kevin Marks.
Mr. GILLMOR: That’s what state your name…
Mr. SCOBLE: He was the 14th close to powerful man in British blogging history or something, I read.
Mr. MARKS: Or something. Yes.
Mr. RECORDON: Recording live from the BBC.
Mr. MARKS: Recording to the telecelegraph(ph), yes.
Mr. RECORDON: There you go.
Mr. GILLMOR: And again, and thanks so much for joining us, Brett Slatkin. Welcome.
Mr. BRETT SLATKIN: Yeah. Brett Slatkin. So thanks.
Mr. GILLMOR: So what’s the status?
Mr. SLATKIN: So the status of speeding RSS or trying to combine?
Mr. GILLMOR: Yeah. Hopefully, we’ll get a little more granular, but…
Mr. SLATKIN: Yeah, things are going well. I think we’ve seen a renewed interest in the idea of a decentralized web of information flowing between companies without anyone being in control of anything and so it’s - that idea sort of didn’t catch on and faster access is a big part of that. So PubSubHubbub is enabled for over a hundred million feeds right now which is really cool. And more subscribers are coming online pretty quickly and so it’s been really great to see people getting interested and actually writing crane doing things.
Mr. GILLMOR: All right. So what is PubSubHubbub and by the way, we have yet another misspelling of it on our - on our lower third. So…
Mr. SLATKIN: Yeah. So…
Mr. GILLMOR: In the tradition.
Mr. SLATKIN: Yeah, sure. So I call it Hubbub for short. It’s easier to say. So Hubbub is a protocol that defines a way of subscribing to and publishing feed content in a way that is pushed to subscribers. So to be clear, the current model of our assess is you as a publisher, publish content and every so often, subscribers have to pull you and say hey, do you have a new content, do you have a new content? Over and over, they keep calling you. So PubSubHubbub turns this around, lets the subscriber register their interest in a feed with a hub and then what happens is every time the publisher has new information, they tell the hub and the hub goes and tells all the different subscribers. So instead of having to ask is there a new content, the hub comes to you and says yes, there is a new content and here it is. So it greatly simplifies subscribing, it makes it more efficient, and it lowers the latency of feeds from, you know, one minute, five minutes is going to be the best you can do before I can do it in about one second. So it’s really big improvement.
Mr. GILLMOR: David Recordon, what’s your opinion about this technology and its usefulness or value?
Mr. RECORDON: Oh, I mean, I think it’s been pretty clear over the past year that as people are doing more interesting things online and sharing more what other people today know, being able to do that in the more real-time fashion is really important. And so PubSubHubbub is (unintelligible) a technology, which helps make that happen and I think it’s something that is going to become a really important piece of infrastructure for the web being able to go and sort of shifted. This is an analogy that Brad Fitzpatrick used, who’s one of the other authors of PubSubHubbub. It’s the sort of like on a road trip kid in the backseat problem. They’re constantly like asking you are we there yet, are we there yet, are we there yet, are we there yet? And PubSubHubbub will logically sort of shift that away from somebody constantly asking you if it’s time, if there’s new content or you instead being able to say no, I’ll tell you when we’re there, in which has a bunch of benefits from an infrastructure perspective as well.
Mr. GILLMOR: Well, what happens when we do get there? Then what? We’re screwed, right?
Mr. RECORDON: What do you mean? Why are we screwed?
Mr. GILLMOR: Well, are we there yet? What is the there that we’re supposed to get to?
Mr. RECORDON: So, the better analogy was the car enough. So you have somebody in the backseat asking (unintelligible)…
Mr. GILLMOR: You’re breaking up in terms of audio. I don’t know what’s going on. Is it Skype?
Mr. RECORDON: Maybe. I’m too excited.
Mr. GILLMOR: They were just throwing another lawsuit out.
(Soundbite of laughter)
Mr. GILLMOR: Go ahead.
Mr. RECORDON: For content, right now, when you’re growing and touching feeds, when you’re trying to see if someone has created new content, you predominantly have to go and ask. Maybe you ask every 15 minutes, maybe you ask every hour, maybe you ask every five minutes. With the real-time technology is it lets you change that. So instead of you’re having to ask if something is updated, you’re able to say, I care about this piece of content. Please tell me when there’s something new.
Mr. SLATKIN: And if I may add, it’s not just tell me if there’s something new, but tell me what is new also. That’s kind of - the distinction here is, you know, it says Hubbub will come along and it’ll tell you not only are you there, but like, here’s what you’re waiting for, what you’re interested in. You know, it’s delivered right to your door, so…
Mr. MARKS: And that’s the distinction between Hubbub and the other notification protocols we’ve had like SUP and so on, right?
Mr. SLATKIN: Yeah, that’s exactly right. It’s pushed, so a lot of these other mechanisms lower the latency of distribution, as David explains and telling people OK, here’s the URL exchange, but they don’t actually deliver a new content and so I call the other way, you know, kind of a Ping pull where it’s still pulling, but there’s kind of a ping first before you have to go fetch the content. And what’s changed now is that it’s actually content push like real push. Data is pushed to the subscribers so they don’t actually ever have to ask for it at all.
Mr. GILLMOR: Is that clear, Kevin?
Mr. MARKS: Yeah, that makes sense to me. I think there’s been a whole series of technologies trying to solve this problem over the years from changes at XML to Ping servers to FeedMesh to SUP, to sort of unify all the updates from one site type things like (unintelligible). I think the key difference here is that this is actually sending the Fat ping, the full content of the update through in the notification. So that makes this protocol a little more complicated in that the server has to do a bit more. On the other hand, it should make the work for the clients at the other end easier because they don’t have to do the – their path through the polling fetch update. The thing is most of the feed as this moment are already doing the polling fetch update thing, so it ends up being more work for them, but hopefully this would be less work for other kinds of clients down the street. Is that a fair summation?
Mr. SLATKIN: Yeah, I think so. I think also, it’s important to remember that the goal here is not to solve the present used cases. Feed reading on the web, it works very well right. That’s not - it’s not necessarily broken. It’s just slow, right? So what we’re trying to do is speed it up and then enable new apps to be built and then have a very fast indication, you know, and messaging protocol. And so this, you know, this idea of push is going to become more relevant as the traffic goes up more and more and as new applications come online from startups and other kind of - and other companies building, you know, new full technology on top of push that wasn’t really possible with their feed crawling the pipeline and all other stuff.
Mr. GILLMOR: Robert Scoble, jump in anytime.
Mr. SCOBLE: I’m just talking on the chat room with Bob Wyman and other people - and Mike Taylor, who’s CTO at (unintelligible) and they seemed to be saying that PubSubHubbub is doing this right. I’m just wondering, is it fast enough to create a real Twitter clone, you know, with hundreds of thousands or millions of people talking real time not in real real time, but in actual real time. And I’m wondering if it could be built - it is to build a system like that.
Mr. SLATKIN: That’s a great question. And Mike and Bob are both big XMPP guys and I’ve had a chance to talk to both of them about PubSubHubbub and how it relates to the Java messaging protocol and Java PubSub, which they both have done a lot with. So can you create an IM-style client or something, you know, that’s as fast as IM client? Absolutely. In fact, as part of our demo for the Real-Time Crunch Up with this (unintelligible), me and Bratt actually built a chat client. So we had a WordPress log on one side and a blogger on the other and we were chatting on, you know, chitchat or whatever and ended up accidentally moving over to the blogs and actually talking blog to blog. In my room, (unintelligible) I’m a blogger because it was so fast. It was - you know, it was basically the same difference. So we’ve actually accidentally sort of IMing through RSS and Atom and PubSubHubbub. So yeah, it’ll work. It’s slowly seeing up to have actually real-time conversations.
Mr. SCOBLE: Could you build an infrastructure like Amazon’s S3 or Rackspace’s Cloud to do this kind of real-time work or do you have to have dedicated servers to do it?
Mr. SLATKIN: Scoble had set me up here, man. We actually ran that demo on EasyTube. So yes, absolutely. And there are hub servers that are in development to run on premise or in the Cloud. Right now, Google runs there on app engine with the reference of limitation of the hub, but you’re going to have more and more people running their own hubs how they want in their data center and that infrastructure could be used to build that kind of experience on - yeah, whatever Cloud service or hosting service you want to use.
Mr. GILLMOR: Some people say that the PubSubHubbub architecture requires, you know, a big player in order to be able to implement and to be able to sustain. That if you have a 100 million users flowing through this stream, that’s going to require some real cloud on the engineering side. Is that accurate?
Mr. SLATKIN: Well, that’s - yeah, I think that that’s a fair concern and I think that was a fair concern maybe four or five years ago. And what’s changed since then is something called epoll. And epoll is an addition to Linux Kernel that’s been around in experimentation for probably a decade and it allows Linux Kernel to scale to, you know, hundreds of thousands of connections on a single machine. And I know that a lot of the best performing web servers out there like Twisted and most likely Tornado to - they used epoll they core in order to manage their connections and that’s what allows a single machine to scale to, you know, to very large amounts of traffic on the modern Linux Kernel. So I think that the tools are there, there are open source for anyone to use, there are similar tools in the Microsoft toolbox. I think, you know, it’s out there for people to use. Java app servers can also do it. So I think that people should take a look at epoll and modern - kind of modern networking substrates and look at the connection limits again and realize that it’s - things are new and different.
Mr. GILLMOR: Kevin, you used to be at Google and you might have one answer then. What’s you’re answer now?
Mr. MARKS: Well, I think I’ve got different answer to different places. The sample hub they built for PubSubHubbub runs on app engines, which means that you can deploy yourself on to your app engine in the sense of Google. I think someone else is building it - has built or is building one in PHP (unintelligible) they get mixed up there.
Mr. SLATKIN: Yeah, there’s Pádraic Brady, who’s a Zend Framework, is in the progress of writing a full implementation in PHP that people are going to use. And then there are also - there’s one in Air Lang (ph) and there’s another one in Python that are making progress.
Mr. MARKS: Right. So, well, you mean - explain Python as opposed to Python on app engine.
Mr. SLATKIN: Oh, yeah. Sorry. Correct, yeah.
Mr. MARKS: Yeah, OK. So I think, you know, part of the point is, it is possible to ligate the things across from hub to hub to hub. So you’re going to have a hub that’s subscribes to another hub and they can flow through. It doesn’t require centralization and so you should be able to do some kind of pen out with that, but I’m not sure how much of that has been built yet. I think most of the things they built so far has been on the main thing server that Brett is running at Google. Is that fair or…
Mr. SLATKIN: Yeah, I think a lot of the Google feeds are running on the Google Hub and I know that there are some - like for now, I know some big European publishers that are in the works and I think they’re going to be running their own hub. So I think that - I’m encouraging people as much as possible, use their own hub. And there are companies like Super Feeder who are running kind of hub as a service for publishers that want to have more control of the stats and that kind of thing. So I think that, you know, it’s still early days in the world of hubs and - but I’m happy to see two big ones. One in Super Feeder. I’m really happy to see one that’s independent and not connected to Google in any way and I’m especially happy to see people like Pádraic writing code that’s open source in PHP on the lap stock that anyone can run on there, its Slicehost there, Rackspace Host there, EasyTube.
Mr. GILLMOR: David Recordon, you used to be independent and now you’re at Facebook. Is Facebook going to join this parade or what do you think is going to happen as far as having some other big players to kind of take the owner (unintelligible) of the large Google penetration around PubSubHubbub?
Mr. RECORDON: I think this is something that’s come up with a lot of the different technologies Google has been creating recently. It’s like (unintelligible) greatest technologies. They’re doing a lot of the right things having building a community around them, but I think they do get that stigma of being just from Google and really benefiting Google. In terms of whether or not we’re going to adopt this specific technology, I mean, we’re always looking at - and trying to learn more about new technologies as they come out and how they could really help create better products, but we don’t have anything to talk about right now, specifically about this technology.
Mr. GILLMOR: So you don’t have any blogs to worry about speeding up.
Mr. RECORDON: I mean, I think Facebook has always had and has really, from the beginning, for a lot of services that have gone real time, has had real-time features inside of our site in terms of things like new speed being able to go and see it in real time, we definitely do fetch a lot of feeds as well.
Mr. MARKS: And of course, you have FriendFeed now, which does support PubSubHubbub.
Mr. RECORDON: Yeah, FriendFeed sort of uses that combination. It was brought up in the chat a little bit earlier, being able to use both something like PubSubHubbub and fetching feeds together. These two technologies do really get good coverage when you’re looking at all the different feeds that you need to interact with.
Mr. SCOBLE: Brett, Jessie stated and I’m interested in this, too. Have you reached out to Twitter in any way and are they - have they told you why or why not they would adopt PubSubHubbub?
Mr. SLATKIN: Yes, Jessie started to throw in their development like newsgroup or (unintelligible) that was asking for Hubbub support and they basically said that they’re happy with the APIs they have implemented today, which is their – they kind of on the streaming API. I understand that they limited resources. I’ve been there, so - I’m there right now, so I know how they feel. I think that, you know, they’re - I know that the guys in their team and I believe Alex Payne, their API Lead , they know about PubSubHubbub, they’re aware of the technical challenges of running XMPP or running a web server and supporting their own hubs. So I think that they’re definitely considering it and I don’t really know what beyond that.
Mr. SCOBLE: Could you tell me if Twitter was starting today? Would they build on PubSubHubbub or would they choose their own system to solve maybe some technical challenge that they see?
Mr. SLATKIN: Yeah, that’s a good question. I think that, you know, RSS and Atom are kind of the universal language for kind of getting across time-based data, right? So feeds of information. And the problem is that the distribution of that information didn’t have well-defined infrastructure before. So you know, if PubSubHubbub existed before Twitter builders stream API or FriendFeed builders stream API, I would like to think that they would consider using Hubbub instead because it could be a common API that all services could provide. So kind of what I’d really like to see - and we have a Wiki page on our - on the PubSubHubbub Wiki about Hubbub and Google and it basically goes in product by product and saying how we support it. And I would really love to see all those products supporting it because it’s just a common API. So as a developer, it’s a lot harder to write one off code for all these different sites. I’d like to just write my code once and just have it work for everyone and I think that Hubbub is a great way to achieve that.
Mr. GILLMOR: Is there any programming mistakes you’re seen people do when they’re first implementing PubSubHubbub that you would like to tell them not do it?
Mr. SLATKIN: Yeah, that’s a good question. Don’t be slow. That’s the first one. We’re…
Mr. GILLMOR: What do you mean by that?
Mr. SLATKIN: Like, if you were a subscriber and it takes you 20 seconds to receive a fat Ping, you know, don’t subscribe to that much information. I think that people are overwhelmed by the stream and they don’t realize how much data we have for them. You know, we have a hundred million feeds out there. They’re changing very quickly. If you would sign up for subscription to something, you’re going to get a lot of content. So I think that people need to remember that, you know, they should make their code as fast as possible and make their processing code as fast as possible because the stream is going to get faster and faster and that’s kind of why something like Hubbub is important because we need to build a really high-quality infrastructure so that as the stream gets faster and faster, we can actually deal with it.
Mr. GILLMOR: Are there any tricks that you’ve found to dealing with that or failing properly? Let’s say your server gets slow for some reason, is there something you can do in your code to fail properly so that you don’t cause other people problems?
Mr. SLATKIN: Yeah, that’s a good question. So I mean, the system was supposed to be as resilient as possible against any kind of failures at any part. But you know, hubs should retry delivery, so it’s the best effort - attempt to be reliable messaging and so it will constantly try to redeliver a message to you up to some number of times over some time period, you know, under a word of minutes. And so you have a chance to make sure that you don’t drop any messages on the floor, which is important. And as time goes on, that will get more and more reliable and I know that friends in Super Feeder, one of their features is the kind of the liability guarantees that you can get from using them. So they’ll say hey, I’ll always deliver the message to you no matter what happens.
Mr. GILLMOR: David, you’re - you’re going to jump in, David?
Mr. RECORDON: Yeah. I was going to say that these are some of the design decisions that went into the protocols, make it really easy for publishers and easy for subscribers and make it - so the hubs have a lot of the heavy lifting to do and that’s really just because when you think about how this ecosystem (ph) gets deployed, you have far more subscribers and mainly subscribers running on less expensive hardware and not as technically adept as you do for the hubs. The hubs can be really large with pieces of infrastructure designed to do this that know how to do it right.
Mr. GILLMOR: What’s the economic incentive for doing it right in this climate? You know, Facebook has got a business model that is, you know, moving into the black. Twitter, you know, they just got another $100 million of investment and being valued, according to the Wall Street Journal publications, at a billion dollars. You know, there’s a lot of money that appears to be orbiting this situation. Yet, what we’re talking about here is a sort of ad hoc, you know, hacker-lead, you know, hub system that’s being constructed. I mean, where is the - where are the economics of this?
Mr. RECORDON: I think a lot of the value they get with this real-time technology is being able to more quickly and more reliably distribute content that’s created within your site whether that’s a blog or a service like the Facebook.
Mr. GILLMOR: Kevin, what do you think? Is BT in this for some reason? You know, I mean, JP Rangaswami is assembling a team of geniuses to sort of rework the telecom model and the infrastructure. How are they going to play in this pond?
Mr. MARKS: I think, you know, for me, this is where the activity stream idea comes. We’ve mentioned that briefly before. The activity stream is a way of standardizing these kinds of notifications that we’re generating in Twitter, in Facebook, in MySpace and other places like that and coming up with common terminology so that we can move from one system to another. And for me, they - there’s a nice part of that in the telephony world as well because a lot of what you do with telephony is you make calls that have time stuff as you send SMS, you get voicemails. So there’s a nice overlap there between the existing stuff we have and the activity stream models that we have in these new social systems and part of that is making sure that we can fit these two ideas together. As we move stuff from - you know, telephony is real time. It requires – you know, by the (unintelligible), requires you to be talking to the person at the same time, but we have ways of extending that out to these sort of history things as well. So I think there’s a nice connection there between the stuff that used to have to be real-time, but now has history as well and these - and PubSubHubbub activity seems to have this model of here’s the stuff that’s coming now, but you can go back and fetch the feed and get the history of it as well. So the first time you sign up, you tend to fetch the feed and then subscribe to the hub so you get the last, you know, 20 or so messages there and then the new ones will come in real time. So I think there’s - it is an all or nothing. All of these work both ways and the model that we build up over the last few years with feeds is fairly resilient and robust, had communication information one site to another and the activity stream model is just trying to codify a few more specs around that so that we can do things like show when you’re posting photographs of a person or sending a link to something else, you know, and codify some of these practices have built into Twitter, into Facebook, into FriendFeed.
Mr. GILLMOR: David, you know, Facebook paid 50 million for that capability. Why would you or why would Facebook want to have that subsumed by an open activity stream system? I mean, what did you spend all that money for?
Mr. RECORDON: I think you’re actually looking at the wrong way, Steve. I mean, the value…
Mr. GILLMOR: I hope so. That’s the idea.
Mr. RECORDON: (Laughing) The value these of services is - I mean, Facebook is always at it and as we say it, being able to connect and share with the people that you know, with your friends, and being able to go and use activity streams as a way to understand the type of content that’s being created, as we might be pulling it in or pushing it out somewhere else, and then being able to do that in a real-time fashion. It really goes back to that core goal of helping people connect and share.
Mr. GILLMOR: OK. So what did you 50 million for?
Mr. RECORDON: I mean, it’s only my first, like, few weeks here.
Mr. GILLMOR: Yeah, yeah. You’re just following orders, in other words.
Mr. RECORDON: (Laughing) Oh, I don’t think…
Mr. GILLMOR: Seriously, I mean, where’s the - what’s the economic rationale for spending this money? I can think of a number of reasons like, you know, that the one billion valuation of Twitter is actually a stalking horse for the - maybe eight billion valuation of Facebook as opposed to four billion, which is what the current market numbers reflect. I mean, there’s something going on here that is being - this is about Twitter and Facebook with Google on some level, you know, sort of swarming around them coming at this - you know, at the value proposition of the activity stream and the filtering that’s going to on around it. I mean, this is big business, evidently. If the Wall Street Journal sends an alert out, I mean, usually the alert is the General Motors is, you know, in bankruptcy and today’s alert was that Twitter is now worth a billion dollars. I mean, what’s going on here?
Mr. RECORDON: Well, I think, I mean, one of the most valuable features of Facebook has been the stream ever since we launched it. It’s a great way to know what your friends are doing and see a lot of that. Earlier this year, we updated the feeds, become real time, and really show you everything that’s going on along with - and the highlights, we’re trying to pull out some more of that. So I agree with you in the sense that being able to see what people that you know are doing is really important and something that a lot of people do every single day. I don’t think I’m the right person to talk about exactly why we purchased FriendFeed or not.
Mr. GILLMOR: But you can talk to the fundamental question because as a technologist, you’re interested in scale, in reaching large numbers of people with, you know, the capability of being able to use that as a way of generating, you know, economic value. However, that’s perceived at the other end of the value chain. You know, this stuff is clearly moving from, you know, Kevin’s and your, you know, standards body or open standards work. It’s moving into some very big players. I mean, the people who are in the investment round with Twitter is a - it’s a mutual fund. I mean, since when the mutual funds get involved in a business that hasn’t had a dime of revenue? And the same thing for Insight, I believe it is, which is a private equity firm. I mean, there’s something going on here that is substantially different than what we’ve seen today. Go ahead.
Mr. SLATKIN: Yeah. So Steve, I think - to take us back and kind of give David a chance to breath, I think that the…
Mr. GILLMOR: Well, good try, but…
Mr. SLATKIN: (Laughing) It’s important to remember that open standards aren’t at odds with economic incentives. If anything, open standards are an economic incentive. So the ability for Facebook and Twitter and Google and Flickr and on and on, all these services, Windows Live, for them, all inter-operates. It’s really best for everyone the same way that the internet itself through interoperability was really good for everyone and good for all the companies that participated. So I think that the economics here are easy integration plus engineering effort put in together, access to more information from all different sources, on and on and on, you know, and then doing it in a way that’s more efficient and scalable so that going forward, we don’t have to keep throwing a lot of money at the problem in terms of engineering effort and machines and whatnot. So I think that it’s important to think about what we’re enabling here, you know, the new used cases, the new apps will be built on top of Twitter, the new apps will be built on top of the Facebook screen that we haven’t really considered before that will render, you know, a tremendous amount of value for these companies and hopefully get those BC’s, some of the dollars back.
Mr. GILLMOR: Well, it’s not the BC’s that are jumping in, that was my point.
Mr. SLATKIN: Yeah, or your, you know, Four One K. Whatever happens to be on the, you know, aiming the balance.
Mr. GILLMOR: All right. So, you’re sounding off a lot like Dave Winer(ph).
Mr. SLATKIN: Which part that sounds like?
Mr. GILLMOR: The part about, you know, standing up for the little guy. I mean that’s what I find fascinating about PubSubHubbub and what Dave is doing with RSS club is that they’re not all that different in terms of their goals as well as the technology. You’ve got a chart that you put together which explores the differences and similarities between not only those two particles, I guess, but others like Sub and so on. Can you sort of give us a sort of an overview of what you’re trying to accomplish with that?
Mr. SLATKIN: Yeah that’s, thanks. So, I think that, I agree that the goal here is, you know, decentralization in the sense that we can all speak the same language and interoperate and that no one is controlled of the data stream completely. And if you think about the internet itself, it was founded on this principal of openness and open protocols and you know, HEP and HTML and TCP even all these protocols were open for people to look at and implement and improve over time. And that’s been really valuable to creating Internet, making it the way it is. So, you know, decentralization is at the core of what the Internet is and why it’s valuable. So one of the other things that to keep in mind though along with decentralization is scalability. So, you look at the IP protocol they talk about Vint Cerf and how he helped build it, you know, back in the 60’s essentially. TCP is the same way. So, the point is that there are these protocols that have been built over the last, you know, 40 years that’s still hold up today. And now that we’re thinking about real-time at Internet scale so that, you know, maybe my mom will take advantage of real-time and be on how she already does on Facebook. You know, we need to build a foundation that’s solid enough that it will keep kind of scaling. So my document which is of the PubSubHubbub Wiki called Comparing Protocols gets into why polling and ping-polling aren’t going to be good enough and why we really need to use fat pinging which is another term for push to make this work. And PubSubHubbub is one fat pinging protocol, XMPP whichever(ph), pubsub is another one and they’re both have their trade offs. And so, it’s really important to understand the technical details of what we’re setting ourselves up for. But I think that a simple kind of tag line is, you know, fat pinging, do it for the children, you know. I think that – I think that we need to build infrastructure that 40 years from now people will look at and say, OK, you know that, that’s going to withstand the test of time.
Mr. GILLMOR: Well, speaking of children, Robert, congratulations.
Mr. SCOBLE: Hey, thanks. I’ll go get little Ryan and bring him up here a little later.
Mr. GILLMOR: Excellent. So, how’s the chat room going? Do you see anything there?
Mr. SCOBLE: It’s going interesting. We are arguing about whether what Twitter’s rules are on re-broadcasting of the firehost feed and it’s clear that Twitter doesn’t allow you to rebroadcast the entire feed but FriendFeed figure that way around that, right? They are allowed to rebroadcast anybody who signed in to FriendFeed from Twitter. So if you got everybody on Twitter to sign up for FriendFeed, you would see the full firehost feed being displayed too but of course not everybody from Twitter is going to sign up for FriendFeed, so it’s a subset at best.
Mr. RECORDON: Let’s go up a level one(ph).
Mr. SCOBLE: That’s part of the point. Yeah, I’m sorry.
Mr. RECORDON: I mean, I was just going to say like there are five or six of us who are chatting live in real-time using video. Well then, another you know like 20 people are going in also in real-time chatting and we’re able to just play back and fourth. I mean, see if going back to your question about sort of like what’s the economic incentive. Like this is just completely changing how people communicate and like that’s what’s really cool about everything that’s happening right now.
Mr. SCOBLE: I think the economics are going to be interesting because like the guys who run big sites like CNN and stuff are really starting to figure out that they’re gifting a lot of free advertising to Twitter by saying on CNN, you know, go to twitter.com/CNN. That’s a gift of the advertising dollars to another brand and they would love to have their own little Twitter where they could say go to CNN.com/twitter and join our Twitter-thing and that way they keep a better touch point with their customer. And if they build a system that integrates in the Twitter and some real - fundamental in real way which as what it looks like this technology will let us do then also you’re going to see a lot of economic activity like that.
Mr. GILLMOR: You are saying something Kevin.
Mr. MARKS: So, I was saying the point is not – is that this isn’t trying to build the firehost, this is trying build the (unintelligible) thing way only get the updates that you want to the piece that you’re subscribing to. You’re not trying to get the entire output of every site and then run stuff over it, you’re trying to just say, I’m interested in this subset of it, I’m following these people, I’m following the streams, and I want those to flow through to me rapidly while than having to build the infrastructure to read the entire flow from every site and then fill that stuff out. There’s a different kinds of problems and the point of this is – is…
Mr.GILLMOR: Well, it’s a philosophical. It’s a philosophical transition as well. I mean, you know, the advantage of our assess was phenomenal, you know, in its inception because it solves the problem of, you know, the guy on the commercial the famous commercial where he sits back and says OK, I just finished the Internet, he read everything. There’s no way to read everything. So, you have to start to apply, you know, an individual Fallow architecture to that. And then what Twitter brought in was the ability to be able to, you know, gang up overlapping Fallow Fields the sort of three degrees of separation that comes from not only following the people that you’re interested in but the people that they’re interested. One of the algorithms in FriendFeed has that capability of essentially bringing to your attention the comments by people that I’m following like for example.
Mr. SPARKS: Yeah.
Mr. GILLMOR: I see a lot of that. That’s, you know, Robert has talked about this extensively for a number of years. So, you start to develop this social graph view into the information space which is, you know, I think extremely powerful and it was – is really representative of what this transition is ultimately going to fuel which is a far more efficient and, you know, human powered environment.
Mr. SCOBLE: The chat room just raised up a good point. What happens, let’s say you have a 100, 000 restaurants all using PubSubHubbub and they’re using a decentralized Twitter-like system to communicate, is there any way to do a track so I can go through all 100, 000 restaurants and look for keywords and have keywords bring back information or do I have to follow all 100, 000 restaurants and then put then put them into a database of some kind and then do might search that way?
Mr. SLATKIN: So, I think – yeah. I think, I can – I like to address all these together because I think you guys are asking the same question in a bunch of different way. So, Kevin’s point was that with technologies like Hubbub that let’s you kind of focus in on the topics that you’re interested in. And then Steve you are saying well Friendfeed, you know, Twitter will take you or social graph of your friends and it’ll maybe expand that out one or two more separations and then it’ll let you kind of follow information that you know, so your focus is a little broader. And then Robert, what you’re talking about is having you know basically looking at the whole firehost in some kind of geographical location. So…
Mr. SCOBLE: Well, not geographical. I want to try a key word.
Mr. SLATKIN: Sure.
Mr. SCOBLE: I want to see everybody on these 100, 000 restaurants who mentions the word sushi for instance.
Mr. SLATKIN: Right, totally. So,what I’m saying is that we’re kind of defining the spectrum of cost where for focus engagement it’s very, with something like PubSubHubbub it will be very low cost for you to keep track of focus engagement or even raising the focus like Steve explained to, you know, to grow a little bit. But it will be more expensive to track the fire host and this is true with keeping up with the Twitter feed also. Not everybody can do it. You need a, you know, special contract in order to get it. There’s a lot of load coming through and same thing goes for stuff like the 64 (ph) Adam stream. Not everybody can handle it. So, I think that people – people will handle it and there will be companies that want to do to kind of track-like functionality but they’re going to need to spend a lot of resources in able to handle that stream. And that stream is going to be large no matter what the underlying protocol is and it’s going to keep getting larger and larger and larger. So, what PubSubHubbub let’s you do is kind of pull out more focus pieces of that stream in efficient way. But you can still get the aggregated view and something that I’ve heard people are working on in multiple places is something like the Twitter trending topics for the whole Internet. So basically, I would subscribe to every single Hubbub feed on the planet and then watch the data as it goes by and do track on it or do whatever kind of analysis you want to do and there’s nothing stopping you from doing that and, you know, it will work efficiently at scale.
Mr. GILLMOR: But how does that get built out, you know, sort of federated track model? I don’t think we’ve actually heard anybody define how that might work. There were some early talk about that in the identical days but I don’t see how. You know, all roads are eventually are going to lead to whether it’s in memory or not. There’s going to be some sort of database of information that has been drawn upon. How do you get there?
Mr. SLATKIN: Yeah. So, I think that you’re asking about processing pipelines. So, you know, we can – I think that Hubbub will get you the information, XMPP can get you the information. You know, there are bunch of proprietary solutions like TIBCO(ph) that will get you data really fast. Analyzing a data pipeline is a lot more complicated and there are some companies out there. I think one of them is called SQLstream that do – and this is kind of the final frontier database design. NSQL is doing real-time analysis, real-time queries on data as it moves. It’s been a research topic for 10, 15 years as I understand it. It’s slowly getting more and more actual kind of investment and then build out. But you know, this is a great economic opportunity for entrepreneurs out there to build federated real-time systems, whatever they maybe. A used case that’s really interesting I think is take that 100, 000 restaurant example and now, you know, let’s say that someone could get the federated fire host across all of these different sites and publishing systems and vendors and whatever. And let’s say this, you know, takes a thousands Hubs and a 100, 000 or 10, 000 different hosting providers. Now, I get a federated fire host. If I want to provide a good news or experience maybe I want, you know, real time over related those reviews depending on my current location. That’s a hard problem to solve, no matter what you’re using to transport, right? XNBP, PubSubHubbub or just plain RSS. It’s difficult problem to solve. So, I think there will be some really pull apps that people are going to build to handle the information overload and this problem kind of transcends the deep technical foundation. And if anything having a strong technical foundation will unable more data to flow and cost more we need to filter it. So, I think that – yeah, this is going to be really interesting to see what people build to fix that.
Mr. GILLMOR: Kevin.
Mr. SCOBLE: Well, Twitter is certainly adding on new Metadata to try to help with filtering. You know, they’re turning on any day now re-twitting which will work like liking in FriendFeed which we know how that helps you filter, right? We can go through FriendFeed and say show us all the items about or with the word Obama in the title that have 15 likes and that have been like first lane by Steve Gillmor, you know. That’s really powerful filtering that we don’t have on Twitter yet. They’re turning on location API’s as well that we’re going to be able to search, you know, instead of just saying show me the Hudson River plane crush, we can say show me only Twits within 25 miles of the Hudson River plane crush, that’s really powerful filtering that will be available to us in the future. It’ll be interesting to see how PubSubHubbub systems deal with that.
Mr. SLATKIN: So, you can do that. The thing is you can do that today with certain feeds available over PubSubHubbub and just RSS and Adam. There is GUR(ph) assess which is an extension to the feed protocols that defines the location tag. There is the viaLink(ph), define the Adam spec(ph) that lot of people use to point back in the original, so like a re-Twit(ph). So, I think that the - all the pieces you just mentioned, although Twitter is going to get the user experience right. Definitely, I think that the foundation as a developer is there to build that in a decentralized way.
Mr. SCOBLE: I’m not even sure that they’re going to get the user experience right. They’re going to…
Mr. SLATKIN: Yeah, I’m saying -
Mr. SCOBLE: They’re going to piss people off of their new re-Twit because you can’t comment in it. And, you know, we have yet to see what it looks and how it works.
Mr. SLATKIN: Well, you know, people are using Twitter a lot. And so, you know, they must like something about it. And I’m saying that the user experience probably part of that so I would imagine that they will be able to service it first similar to how FriendFeed kind of had (unintelligible)…
Mr. GILLMOR: Whether they service it or one of their third parties.
Mr. SLATKIN: Yes.
Mr. GILLMOR: So, this is it. I mean, that’s historically that’s been truer than Twitter. They have needed to do it as part of the reason that they’ve gotten so successful. David, why did you go to Facebook instead of Twitter?
Mr. RECORDON: I never even thought about joining Twitter while I was there. I mean, I joined Facebook because of all the really interesting problems that are being solved here, and sort of where my interest oblige as I spent the past few years working on technology around social networks, and where else would I go.
Mr. GILLMOR: Right. So I mean, some of these interesting problems you’re talking about are probably around the area of UI and how you manage these kinds of opportunities. We’ve been talking about a large scale, the hundred thousand user or the hundred million dollars, you know. But, just the ability – I mean, this has been talked about before on the show today. You know, the fact that we’re able to use this reasonably and in some cases completely low cost technologies, and be able to wire up something that allows us as individuals to be able to create this, you know, overlapping follow clouds that we can leverage for having real time conversations. I mean, we all happen to be in the valley on this call, but it’s really no different when somebody comes in from London or anywhere on the planet. So the ability to be able to level – to leverage this at a micro community level, I think, is the real element(ph) in the room here. And I think what I’m hearing is that the infrastructure is now available to do this in many different ways which is going to drive an economic kind of acceleration where the cost has driven down and the value is distributed more widely to people in general. So I’m wondering if that’s what your vision and understanding is of what Facebook is as an opportunity to do, and if that’s why you’re there.
Mr. RECORDON: I mean, I completely agree with you to certainly. Personally about that of what these tools and what these technologies ultimately enable is people being able to communicate with each other better, and I mean, going to the example that I gave like 10 minutes ago about just what we’re doing right now of being able to go and how video streaming between five or six different people, how to build real time conversation from a chat room which I have out behind my site window being able to see all that interact back and forth. This is a really interesting and exciting time for to really be able to help and influence out internet continues to develop and move forward.
Mr. GILLMOR: Speaking of exciting times. We’re about to have the debut of RSS. So who is that person?
Mr. SCOBLE: This is Ryan Soroush Scoble, RSS and Rocky is here too.
Mr. GILLMOR: So Rocky, hold the baby.
(Soundbite of laughter)
Mr. SCOBLE: Rocky is a softie. You didn’t know that did you? Anyways, lots of fun in my lives.
Mr. GILLMOR: So did, what did you reach for as your social interface at his birth, Robert?
Mr. SCOBLE: Well, I used a bunch of different things. I used FriendFeed. I used Twitter to get – because most of my audience is still on Twitter. I used Flickr to get photos out. I used mvideo by the way. And what else did I use?
Mr. GILLMOR: Did you use an iPhone or a flip phone?
Mr. SCOBLE: iPhone.
Mr. GILLMOR: A flip phone, OK.
Mr. SCOBLE: iPhone because, you know, when you’re – well, I was using a 5D as well. So I had a 5D and an iPhone. The 5D is for the high quality video and high quality pictures for my own family album. The iPhone let me get it out – the word get out to the world while it was happening so.
Mr. GILLMOR: I think you mentioned that the sequela(ph) is at the hospital you’re in.
Mr. SCOBLE: Yeah.
Mr. GILLMOR: Did they were decked out with a Wi-Fi?
Mr. SCOBLE: They were but it was like a Wi-Fi in China. They had a lot of sites blocked. In fact, a lot of the video sites were blocked, but Flickr wasn’t so…
Mr. GILLMOR: Why they’re blocked? Do you think?
Mr. SCOBLE: Because I think – I think, they were trying to protect their bandwidth from people who are downloading porn in the rooms and stuff like that.
(Soundbite of laughter)
Mr. SCOBLE: Now, a lot of the media sites were block and a lot of the porn sites were blocked. So, I think, they were using a standard package that just let people block a bunch of different. It was a like a net nanny kind of site, you know. It’s the site block.
Mr. GILLMOR: Speaking of a strange spamming. I got a direct message the other day from somebody. I think it was yesterday. Who clearly I had no idea who that person was and I never – they’re trying to sell me something. Has anybody else seen that problem?
Mr. SCOBLE: I haven’t, but I saw a lot of talking about it.
Mr. MARKS: And other sites had lots of…
Mr. GILLMOR: Yeah, and what’s going on there? It sounds like that somebody has figured out how to be able to hack into the direct message network and talks to people who haven’t reciprocally followed them.
Mr. MARKS: No. I thought it was a fishing attack, so that people you are following are sending messages under their name. That what was going on there, but…
Mr. GILLMOR: So I did follow this person even though I didn’t recognize the name.
Mr. MARKS: Right. And the message was actually sent by them. They click on a link and that link sent the message out.
Mr. GILLMOR: No. I understand that. I still think…
Mr. MARKS: That’s a good example of what when you want to have a distributed system that’s decentralized so that not everyone is affected by the same cross sites rifting attack at the same time. So I think, you know, I think Dave was right saying that our RSS no fair well because not every, you know, not every front ad will be other goes…
(Soundbite of laughter)
Mr. MARKS: Not every, you know, not every front ad will be other goes can - not every one to make the same mistake, you know. People want things differently and they have, you know, a different security of limitations and…
Mr. GILLMOR: So in other words, we want to have a hundred million companies all losing money instead of having.
Mr. MARKS: We don’t really know. They are private company. But you know, we want to have – we want to have people participating in a stable wealth and see what follows from that. And I think that requires decentralization and good infrastructure.
Mr. GILLMOR: Well, I’m not so sure. I think decentralization is over sold. It’s kind of like a unicorn. It’s like, you know, their exploits are well known but no one’s ever succeed in capturing one.
Mr. RECORDON: Oh, I think, if you look at the…
Mr. GILLMOR: Yeah, go head Kev.
Mr. MARKS: Nonsense. The way…
Mr. GILLMOR: Oh by the way, the (unintelligible) is open, Kevin.
(Soundbite of laughter)
Mr. MARKS: Inside Job did last week show.
Mr. GILLMOR: Yeah.
Mr. MARKS: Did decentralize web, you know, is what we built on? That’s the stuff that works. Is this the protocols that, you know, the web is in agreement. That’s the praise pulled down and put together which push it very well, which is we agreed to use the same protocols to exchange stuff. Therefore, you can interact right with people not only without having a business relationship with them, without even knowing they exist because you’re using the same protocol. And that is the priorities and that’s why it scales out to vast dimensions.
Mr. RECORDON: It’s very easy to join. Yeah.
Mr. MARKS: Yeah.
Mr. GILLMOR: I understand what the logic. I mean on a simple level, the entire web is completely decentralized. It’s about multiple computers connected to each other. So I understand that, but the economics of it and the use of the word decentralize as an alternative to a commercial venture. I think is in sophistry (ph). I don’t think it exists. I think, everything is a commercial venture and everything is decentralized. So what is the point here?
Mr. MARKS: The point is it’s not – it will change to a commercial venture. Not saying the combi commercial ventures that building these protocols. Obviously, they can be. The point is saying, you don’t have to first get permission from someone to use it, that’s the difference, right?
Mr. GILLMOR: Let’s follow that logic for a second. What permission? Who do I have to get permission to do anything on the web? When I’m on Gmail, I have to get permission from Google or else my sign in does not work and I can’t get access to all of my data. It has been trap there for the last 48 hours.
Mr. SLATKIN: I think that you need to – I think that Kevin is talking about at a server level. So if I want to run an SNTP server as my own mail host and send email through Gmail account. Gmail will take that email and deliver it to its recipient. So I don’t have to ask them for permission. I just need to speak to protocol and behave – behave correctly.
Mr. GILLMOR: Right and I would suggest that the reason that that is possible is not necessarily because of some virtuous, you know, religion regarding decentralization as responsible. It happens because it’s good business for Gmail to do that.
Mr. SLATKIN: Yes.
Mr. GILLMOR: OK. So…
Mr. SLATKIN: These principles go hand and hand, I think, that decentralization is good for business. Yeah.
Mr. GILLMOR: Right. So you know, it’s not either or proposition.
Mr. SLATKIN: Not at all.
Mr. GILLMOR: And sometimes, portrayed as that politically and that’s my objection to the use of decentralization as a religion.
Mr. SLATKIN: Yeah, I think that’s really a good point ‘cause I think that, yeah, they usually – they put it on to each other it’s, you know. But I think that – I think that it’s more just a question of, you know, letting anyone come to the pool and swim there whatever. It’s not at odds with the corporate plan or making money. It’s actually a way to gain network effects on products, so that they grow larger and so HTML had amazing network effects and got every company involve and it’s growing, growing, growing, growing. So that was – that’s why it took off and it was good commercially for every one because they kind of ride away for long.
Mr. GILLMOR: We’ve lost, David. I’m not sure whether or not.
Mr. SLATKIN: I think he had to go.
Mr. GILLMOR: It’s purposeful, really.
Mr. MARKS: He said he had to go, yeah.
Mr. GILLMOR: OK, thanks. Well, then, let’s start to wean this down. Brett is there any kind of misconception that you feel has been push forward about PubSubHubbub or conversely. Is there anything that you would like to say about to some of the other protocols and strategies that are out here regarding this, I mean. This is an opportunity I think for you to express something about just the general area, not just your work in it.
Mr. SLATKIN: Yeah. I think that, well, so specifically to have – I think that (unintelligible).
Mr. GILLMOR: Let’s start again. You’re kind of like – try it again please.
Mr. SLATKIN: Hanging. There you go. I’m back in business.
Mr. GILLMOR: Go ahead.
Mr. SLATKIN: Sorry about that.
Mr. GILLMOR: You said you were at Twitter. Are they trying to choke your bandwidth?
Mr. SLATKIN: I think so and I don’t why.
Mr. GILLMOR: OK. Say it again.
Mr. SLATKIN: I was saying that there’s a misconception that Hubbub is controlled by Google and that’s not true. I think that people need to come and participate in open specifications like activity streams and like PubSubHubbub with open mailing list and the open process for getting new stuff added to the specs. So that’s a kind of misconception for me specifically. Generally, I think that people don’t understand that light pinging isn’t going to work. I think that as a developer, when you take a naïve approach and you think about that for 30 seconds. You like, oh yeah, light pinging that should work. But if you actually sit down and think through it and think about scalability and remember what you learn about TCP back in your, you know, classes or not. You realized that it just won’t work. It’s just not going to fly overtime. And you know, we need to – we kind of need to do to strike the right balance between complexity and performance. And I think that so – I think that light pinging a lot of time, it seems really simple but it’s just doesn’t do you good enough job. And so I encouraged people to look at some of the things that I wrote. And there’s a long history of the stuff in just protocol design in general for the last 40 years that talks about push and why it’s good. And you know, Scott (ph) had read it and understand that, you know, we need to do better or else we’re not going to be able to keep growing the stream effectively.
Mr. GILLMOR: So I’m hearing two things. One is that you need to have a rigorous continued influx of new blog. So that this is in proceed as a special Google project or any kind of (unintelligible) strategy. But at the same time, you need to be able to take advantage of heavy lifting that can probably only be done by those kinds of companies.
Mr. SLATKIN: Yeah, I think that – in second part. It’s not heavy lifting. That’s my point. It’s the difference between, you know, lifting your pinky finger and your index finger. It’s really not that much more complicated but initially you might say, Oh, that’s more complex. But it really – it really is actually very simple and anybody can do it. You can do it on your $5 a month share costing environment with PHP and my sequel. You can even do it on free hosts. So I think that – I think that anybody can do it. So that’s kind of not the problem. The problem is that at the surface when people first look it, you know, Apple and Orange, they think their, you know, they think that something like fat pinging is actually more complicated and I guess what Hubbub tries to get across is that it doesn’t actually have to be. So people who are turned off by Java(ph) PupSub or any kind of other push protocol in the past should look at, you know, Hubbub and other, you know, new protocols and say, well, does this make it easier for me? And I think it does. I think that it’s easy and efficient. And as for the new blog, yeah, we have a lot of new blog coming in and more new blog is welcome. I want everyone to participate. I get so much great feedback, you know, like me and Brad both get a lot of great feedback from the whole community of people from different countries and languages and backgrounds and program and languages, and it’s just really cool. This community that’s been growing and I love when people call polls in the spec, I love when people challenge, they never try to present about why it’s necessary…
MR. GILLMOR: Give me an example of a whole that we spoke and how you fixed it.
Mr. SLATKIN: Well, the best example I think is what Bret Taylor did at FriendFeed. He said, you know what, I’m not satisfied with the security model of how hubs delivered to subscribers. He wanted a higher level of kind of trusts that a message received from the hub was authentic. And so, he suggested a way to do it and we went and implemented it and we went and talked around with other people on the thread, on a bunch of threads on the mailing lists, and you know, we kind of reach the consensus about the way it should work, and then the 0.2 version of the spec added that suggestion to the spec. So you know, this is the idea originated with Bret Taylor. And you know, while he was a FriendFeed independently and then ended up in this spec that we’re all working out. So, and it’s a great addition to the spec and absolutely necessary and…
MR. GILLMOR: So, what is the spec that you’re all working on and how are we going to be assured that there’s a reasonable level of interoperability among the important players?
Mr. SLATKIN: Great, great question. So you know, the spec is linked off to the PubSubHubbub website. So, if you just seach for PubSubHubbub or pubsubhubbub.googlecode.com. It’s misspelled at the bottom there, it has 2 b’s. But if you go search for that…
MR. GILLMOR: You weren’t listening, I said that it’s (unintelligible).
Mr. SLATKIN: I know, yeah. And then…
MR. GILLMOR: Real time error.
Mr. SLATKIN: Real time errors. So if you look at there, it’s linked up from there. The mailing list is also linked up from there, and as for interoperability, we have a test that was written by Jeff Lindsay of Webhooks fame who particularly for people to verify the behavior of their hub. And that needs more work too, but the idea is there will be Compliance Suite that you can run against your code to verify that it works properly just like this Compliance Suite for any well defined spec.
MR. GILLMOR: Now, do you think Kevin that this is going to get hooked in with some of the activities for yourself?
Mr. MARKS: Actually, it did. Yeah.
Mr. GILLMOR: Okay, can you explain how?
Mr. MARKS: Well, activity stream is a feed. It’s an item feed that contains actions, you know, some bit of extra mark up to expressed up in more detail. So it plugs in imperfectly just take the feeding process radio system. So, that there’s no, you know, there’s no reason that would not happen until…
Mr. SLATKIN: Yeah, it works today.
Mr. MARKS: Yeah, one thing I’d like to ask Brett is have you looked at hooking up, decide what key stuff to this? I know you’re imposing any (unintelligible) by me…
Mr. SLATKIN: Yeah, yeah. So, and Bob Wyman mentioned on the public thread what would be required to do that and yeah, I mean I think that it would be great to hook that up. I think, that there’s a whole (unintelligible) filled products that we need to put this on and that’s definitely one of them, one of more interesting ones. I think that making it to that content can be federated in access by everyone and real time is really important or else it cannot seem shallow and so that’s something we need to do definitely. But, I don’t know when that’s going to be in time in terms of timeframe.
Mr. GILLMOR: What about an interaction with Wave?
Mr. SLATKIN: Yes, so that’s a good question, and I get that question pretty often. So Wave and Hubbub both do push in terms of content, so it’s good for the children as I was saying. Wave does it in a way that’s more complicated because they need more richness to express the amazing user experience that they have. Hubbub just feeds pushing around. But there’s no reason why you couldn’t use Hubbub as a transport for Wave data or vice versa, so…
Mr. GILLMOR: That’s a replacement for XMPP or…
Mr. MARKS: I’m a bit skeptical. Yeah, well…
MR. GILLMOR: Or on one side?
Mr. ARKS: But there’s a difference right? Because…
Mr. SLATKIN: Absolutely there’s a difference, yeah.
Mr. MARKS: Wave is trying to have your (unintelligible) to be edit the same document so he was sending operation transforms where it actually edits to the same thing.
Mr. GILLMOR: Right.
Mr. MARKS: So, you need to have that document you’re reconstructing somewhere. So yeah, you could feel into pieces of pushing through PubSubHubbub and put it back together again, but you still need the little pieces to flow in. So, I’m not sure of that. And this is part of the problem that, you know. Wave is saying, okay, our structure model is (unintelligible) should have seen the same document. It’s not his flow event is coming from one place to another. So that’s why it doesn’t mix very well with these systems. And then…
Mr. SLATKIN: Well, so I think that at a user experience level, what you’re saying, make sense. But I think that in terms of the actual like implementation in the protocol, you know, Wave basically is just Java PubSub to move, you know, transforms the document between all the people who are participating. And what I’m saying is that you could do that same transport with something like Hubbub, it would be possible. Now, I think that XMPP is better in that case because it’s even faster than Hubbub it can be and has a much organized scaling property. So, that’s why Wave has gone in the direction they have, I think. But, I’m saying that there isn’t an impedance mismatch between the two protocols. You could have them together.
Mr. GILLMOR: OK.
Mr. SLATKIN: Whereas, you know, you couldn’t hook together and changes XML file and Wave very easily. That would be a lot more complicated, you know, the thing falling approach doesn’t, you know, doesn’t mesh well with push. If you want to get data in real time, you want to push it. But, that’s not what I’m just trying to say.
Mr. GILLMOR: OK. Robert any last question? OK.
Mr. SCOBLE: Not really.
Mr. GILLMOR: OK. This is Steve Gillmor. This has been the Gillmor Gang. I want to thank Brett Slatkin, I want to thank Kevin Marks, I want to thank David Recordon, and I want to thank Robert Scoble and I especially want to thank Rackspace for sponsoring the show and all their fine work. And NewTek and their incredible TriCaster which is helping us print this to you. So, we’ll see you again next time and there will be a next time. Bye-bye.
Tags: pubsubhubbub, R.I.P. RSS, realtime
















September 30th, 2009 at 3:14 am
Where are the women in your gang?
September 30th, 2009 at 9:24 am
Still no mp3 link, still no listen from me. I have far better things to do than watch video podcasts, anywhere, not least on PC.
October 2nd, 2009 at 11:03 am
Thanks. The old is new again. Before folks start thinking Push and Pub/Sub is new or groundbreaking, it has been around since TCP and probably before that. Here is good paper the summarizes Pub/Sub over the years and its many forms. http://www.irisa.fr/asap/intranet/the-many-faces-of-publish-subscribe.pdf/
Over the years, we have many forms of this. IMAP and NNTP come to mind. They work so well, nobody even thinks about them anymore. Efforts like Marimba’s Castanet have come and gone. Newer stuff like Mesh also have a Pub/Sub model at Internet scale. The lowest common denominator is TCP, which was designed for this type of thing from the start. Open an socket, make a request (or not) and read 1 byte or many. You define your own protocol ontop of a bidirectional stream.
Things that have made this more difficult over the years is open sockets where expensive (over 1000 or so), and http started as a single request/response/close model - hence the typical model of client polling at intervals. ePoll is interesting, I will have to learn more about how that works to make open sockets “cheaper”.
Moreover, pure push is a white lie. In todays world where clients are behind firewalls and NATs, pure push doesn’t work. The workaround for this has been the client maintains an open TCP connection to a server in the cloud (ala GotoMyPC and others) and can read replies as they come in. Even with this model, there is still reconnection and network timeout issues that force a client to reconnect and “request” again. So everything still starts with an open “request”. You can do pure-push to public IP listeners and probably IPV6 clients, but those are not typical cases yet.
I think the more interesting work here is not Push as such (that is well-known art), but things like: type systems and dynamic types, how anonymous types can work, client Projections (i.e. Select), type and service discovery, standards, and remoting client Expression Trees (e.g. LINQ) and remoting client code blocks (For example, a client should be able to serialize a Linq expression to an Event Service to do their own Search -the Twitter Track problem). How to sort Expression trees to allow efficient server side processing and stop a search as early as possible. How can developers get publisher type systems into there IDEs in an easy way for developing and compile type checks?
May also want to see some other work in this area that is very interesting:
Reactive Framework (Rx) with Erik Meijer - Reverse IEnumerable
http://channel9.msdn.com/shows/Going+Deep/E2E-Erik-Meijer-and-Wes-Dyer-Reactive-Framework-Rx-Under-the-Hood-2-of-2/#Page=3
October 2nd, 2009 at 11:08 am
UI request. IMHO, I would not add “Scribe” in-line, but would add a link to another page to keep your Posts cleaner. It makes it too difficult to scroll down to find the next topic. Also, I think it is unneeded detail in the general usage, unless you want to drill down on something after watching. tia.