TOC 
DraftB. Fitzpatrick
 B. Slatkin
 Google, Inc
 M. Atkins
 Six Apart Ltd.
 J. Genestoux
 Notifixious Inc.
 June 20, 2013


PubSubHubbub Core 0.4 -- Working Draft

Abstract

An open, simple, web-scale and decentralized pubsub protocol. Anybody can play.

As opposed to more developed (and more complex) pubsub specs like Jabber Publish-Subscribe (Millard, P., Saint-Andre, P., and R. Meijer, “Publish-Subscribe,” .) [XEP‑0060] this spec's base profile (the barrier-to-entry to speak it) is dead simple. The fancy bits required for high-volume publishers and subscribers are optional. The base profile is HTTP-based, as opposed to XMPP (see more on this below).

To dramatically simplify this spec in several places where we had to choose between supporting A or B, we took it upon ourselves to say "only A", rather than making it an implementation decision.

We offer this spec in hopes that it fills a need or at least advances the state of the discussion in the pubsub space. Polling sucks. We think a decentralized pubsub layer is a fundamental, missing layer in the Internet architecture today and its existence, more than just enabling the obvious lower latency feed readers, would enable many cool applications, most of which we can't even imagine. But we're looking forward to decentralized social networking.



Table of Contents

1.  Notation and Conventions
2.  Definitions
3.  High-level protocol flow
4.  Discovery
5.  Subscribing and Unsubscribing
    5.1.  Subscriber Sends Subscription Request
    5.2.  Subscription Validation
    5.3.  Hub Verifies Intent of the Subscriber
6.  Publishing
7.  Content Distribution
8.  Authenticated Content Distribution
9.  References
Appendix A.  Specification Feedback
§  Authors' Addresses




 TOC 

1.  Notation and Conventions

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in [RFC2119] (Bradner, B., “Key words for use in RFCs to Indicate Requirement Levels,” .). Domain name examples use [RFC2606] (Eastlake, D. and A. Panitz, “Reserved Top Level DNS Names,” .).



 TOC 

2.  Definitions

Topic:
An HTTP (Fielding, R., Gettys, J., Mogul, J., Frystyk, H., Masinter, L., Leach, P., and T. Berners-Lee, “Hypertext Transfer Protocol -- HTTP/1.1,” .) [RFC2616] resource URL. The unit to which one can subscribe to changes.
Hub ("the hub"):
The server (URL (Berners-Lee, T., “Uniform Resource Identifiers (URI): Generic Syntax,” .) [RFC3986]) which implements both sides of this protocol. Any hub MAY implement its own policies on who can use it.
Publisher:
An owner of a topic. Notifies the hub when the topic feed has been updated. As in almost all pubsub systems, the publisher is unaware of the subscribers, if any. Other pubsub systems might call the publisher the "source".
Subscriber:
An entity (person or program) that wants to be notified of changes on a topic. The subscriber must be directly network-accessible and is identified by its Subscriber Callback URL.
Subscription:
A unique relation to a topic by a subscriber that indicates it should receive updates for that topic. A subscription's unique key is the tuple (Topic URL, Subscriber Callback URL). Subscriptions may (at the hub's decision) have expiration times akin to DHCP leases which must be periodically renewed.
Subscriber Callback URL:
The URL (Berners-Lee, T., “Uniform Resource Identifiers (URI): Generic Syntax,” .) [RFC3986] at which a subscriber wishes to receive notifications.
Event:
An event that causes updates to multiple topics. For each event that happens (e.g. "Brad posted to the Linux Community."), multiple topics could be affected (e.g. "Brad posted." and "Linux community has new post"). Publisher events cause topics to be updated and the hub looks up all subscriptions for affected topics, sending out notifications to subscribers.
Notification:
A payload describing how a topic's contents have changed, or the full updated content. Depending on the topic's content type, the difference (or "delta") may be computed by the hub and sent to all subscribers.


 TOC 

3.  High-level protocol flow

(This section is non-normative.)



 TOC 

4.  Discovery

A potential subscriber initiates discovery by retrieving (GET or HEAD request) the topic to which it wants to subscribe. The HTTP (Fielding, R., Gettys, J., Mogul, J., Frystyk, H., Masinter, L., Leach, P., and T. Berners-Lee, “Hypertext Transfer Protocol -- HTTP/1.1,” .) [RFC2616] response from the publisher MUST include at least one Link Header (Nottingham, M., “Web Linking,” October 2010.) [RFC5988] with rel=hub (a hub link header) as well as exactly one Link Header (Nottingham, M., “Web Linking,” October 2010.) [RFC5988] with rel=self (the self link header). The former MUST indicate the exact URL of a PubSubHubbub hub designated by the publisher. If more than one URL is specified, it is expected that the publisher pings each of these URLs, so the subscriber may subscribe to one or more of these. The latter will point to the permanent URL for the resource being polled.

In the absence of HTTP (Fielding, R., Gettys, J., Mogul, J., Frystyk, H., Masinter, L., Leach, P., and T. Berners-Lee, “Hypertext Transfer Protocol -- HTTP/1.1,” .) [RFC2616] Link headers, subscribers MAY fall back to other methods to discover the hub(s) and the canonical URI of the topic. If the topic is an XML based feed, it MAY use embedded link elements as described in Appendix B of Web Linking (Nottingham, M., “Web Linking,” October 2010.) [RFC5988]. Similarly, for HTML pages, it MAY use embedded link elements as described in Appendix A of Web Linking (Nottingham, M., “Web Linking,” October 2010.) [RFC5988]. Finally, publishers MAY also use the Well-Known Uniform Resource Identifiers (Nottingham, M. and E. Hammer-Lahav, “Defining Well-Known Uniform Resource Identifiers (URIs),” .) [RFC5785] .host-meta to include the <Link> element with rel="hub".



 TOC 

5.  Subscribing and Unsubscribing

Subscribing to a topic URL consists of four parts that may occur immediately in sequence or have a delay.

Unsubscribing works in the same way, except with a single parameter changed to indicate the desire to unsubscribe. Also, the Hub will not validate unsubscription requests with the publisher.



 TOC 

5.1.  Subscriber Sends Subscription Request

Subscription is initiated by the subscriber making an HTTPS (Fielding, R., Gettys, J., Mogul, J., Frystyk, H., Masinter, L., Leach, P., and T. Berners-Lee, “Hypertext Transfer Protocol -- HTTP/1.1,” .) [RFC2616] or HTTP (Fielding, R., Gettys, J., Mogul, J., Frystyk, H., Masinter, L., Leach, P., and T. Berners-Lee, “Hypertext Transfer Protocol -- HTTP/1.1,” .) [RFC2616] POST request to the hub URL. This request has a Content-Type of application/x-www-form-urlencoded (described in Section 17.13.4 of [W3C.REC‑html401‑19991224] (Raggett, D., Hors, A., and I. Jacobs, “HTML 4.01 Specification,” December 1999.)) and the following parameters in its body:

hub.callback
REQUIRED. The subscriber's callback URL where notifications should be delivered. It is considered good practice to use a unique callback URL for each subscription.
hub.mode
REQUIRED. The literal string "subscribe" or "unsubscribe", depending on the goal of the request.
hub.topic
REQUIRED. The topic URL that the subscriber wishes to subscribe to or unsubscribe from.
hub.lease_seconds
OPTIONAL. Number of seconds for which the subscriber would like to have the subscription active. Hubs MAY choose to respect this value or not, depending on their own policies. This parameter MAY be present for unsubscription requests and MUST be ignored by the hub in that case.
hub.secret
OPTIONAL. A subscriber-provided secret string that will be used to compute an HMAC digest for authorized content distribution (Authenticated Content Distribution). If not supplied, the HMAC digest will not be present for content distribution requests. This parameter SHOULD only be specified when the request was made over HTTPS (Rescorla, E., “HTTP Over TLS,” May 2000.) [RFC2818]. This parameter MUST be less than 200 bytes in length.

Subscribers MAY also include additional HTTP (Fielding, R., Gettys, J., Mogul, J., Frystyk, H., Masinter, L., Leach, P., and T. Berners-Lee, “Hypertext Transfer Protocol -- HTTP/1.1,” .) [RFC2616] request parameters, as well as HTTP (Fielding, R., Gettys, J., Mogul, J., Frystyk, H., Masinter, L., Leach, P., and T. Berners-Lee, “Hypertext Transfer Protocol -- HTTP/1.1,” .) [RFC2616] Headers if they are required by the hub. In the context of social web applications, it is considered good practice to include a From HTTP (Fielding, R., Gettys, J., Mogul, J., Frystyk, H., Masinter, L., Leach, P., and T. Berners-Lee, “Hypertext Transfer Protocol -- HTTP/1.1,” .) [RFC2616] header (as described in section 14.22 of Hypertext Transfer Protocol (Fielding, R., Gettys, J., Mogul, J., Frystyk, H., Masinter, L., Leach, P., and T. Berners-Lee, “Hypertext Transfer Protocol -- HTTP/1.1,” .) [RFC2616]) to indicate on behalf of which user the subscription is being performed.

Hubs MUST ignore additional request parameters they do not understand.

Hubs MUST allow subscribers to re-request subscriptions that are already activated. Each subsequent request to a hub to subscribe or unsubscribe MUST override the previous subscription state for a specific topic URL and callback URL combination once the action is verified. Any failures to confirm the subscription action MUST leave the subscription state unchanged. This is required so subscribers can renew their subscriptions before the lease seconds period is over without any interruption.



 TOC 

5.1.1.  Subscription Parameter Details

The topic and callback URLs MAY use HTTP (Fielding, R., Gettys, J., Mogul, J., Frystyk, H., Masinter, L., Leach, P., and T. Berners-Lee, “Hypertext Transfer Protocol -- HTTP/1.1,” .) [RFC2616] or HTTPS (Rescorla, E., “HTTP Over TLS,” May 2000.) [RFC2818] schemes. The topic URL MUST be the one advertised by the publisher in a Self Link Header during the discovery phase. (See Section 4 (Discovery)). Hubs MAY refuse subscriptions if the topic URL does not correspond to the one advertised by the publisher. The topic URL can otherwise be free-form following the URI spec (Berners-Lee, T., “Uniform Resource Identifiers (URI): Generic Syntax,” .) [RFC3986]. Hubs MUST always decode non-reserved characters for these URL parameters; see section 2.4 on "When to Encode or Decode" in the URI spec (Berners-Lee, T., “Uniform Resource Identifiers (URI): Generic Syntax,” .) [RFC3986].

The callback URL MAY contain arbitrary query string parameters (e.g., ?foo=bar&red=fish). Hubs MUST preserve the query string during subscription verification by appending new parameters to the end of the list using the & (ampersand) character to join. Existing parameters with names that overlap with those used by verification requests will not be overwritten. For event notification, the callback URL will be POSTed to including any query-string parameters in the URL portion of the request, not as POST body parameters.



 TOC 

5.1.2.  Subscription Response Details

The hub MUST respond to a subscription request with an HTTP (Fielding, R., Gettys, J., Mogul, J., Frystyk, H., Masinter, L., Leach, P., and T. Berners-Lee, “Hypertext Transfer Protocol -- HTTP/1.1,” .) [RFC2616] 202 "Accepted" response to indicate that the request was received and will now be verified (Section 5.3 (Hub Verifies Intent of the Subscriber)) and validated (Section 5.2 (Subscription Validation)) by the hub. The hub SHOULD perform the verification and validation of intent as soon as possible.

If a hub finds any errors in the subscription request, an appropriate HTTP (Fielding, R., Gettys, J., Mogul, J., Frystyk, H., Masinter, L., Leach, P., and T. Berners-Lee, “Hypertext Transfer Protocol -- HTTP/1.1,” .) [RFC2616] error response code (4xx or 5xx) MUST be returned. In the event of an error, hubs SHOULD return a description of the error in the response body as plain text. Hubs MAY decide to reject some callback URLs or topic URLs based on their own policies (e.g., domain authorization, topic URL port numbers).



 TOC 

5.2.  Subscription Validation

Subscriptions MAY be validated by the Hubs who may require more details to accept or refuse a subscription. The Hub MAY also check with the publisher whether the subscription should be accepted.

If (and when), the subscription is accepted, the hub MUST perform the verification of intent (Hub Verifies Intent of the Subscriber) of the subscriber.

If (and when), the subscription is denied, the hub MUST inform the subscriber by sending an HTTP (Fielding, R., Gettys, J., Mogul, J., Frystyk, H., Masinter, L., Leach, P., and T. Berners-Lee, “Hypertext Transfer Protocol -- HTTP/1.1,” .) [RFC2616] GET request to the subscriber's callback URL as given in the subscription request. This request has the following query string arguments appended (format described in Section 17.13.4 of [W3C.REC‑html401‑19991224] (Raggett, D., Hors, A., and I. Jacobs, “HTML 4.01 Specification,” December 1999.)):

hub.mode
REQUIRED. The literal string "denied".
hub.topic
REQUIRED. The topic URL given in the corresponding subscription request.
hub.reason
OPTIONAL. The hub may include a reason for which the subscription has been denied.

Hubs may provide an additional HTTP (Fielding, R., Gettys, J., Mogul, J., Frystyk, H., Masinter, L., Leach, P., and T. Berners-Lee, “Hypertext Transfer Protocol -- HTTP/1.1,” .) [RFC2616] Location header (as described in section 14.30 of Hypertext Transfer Protocol (Fielding, R., Gettys, J., Mogul, J., Frystyk, H., Masinter, L., Leach, P., and T. Berners-Lee, “Hypertext Transfer Protocol -- HTTP/1.1,” .) [RFC2616]) to indicate that the subscriber may retry subscribing to a different hub.topic. This allows for limited distribution to specific groups or users in the context of social web applications.

The subscription MAY be denied by the hub at any point (even if it was previously accepted). The Subscriber SHOULD then consider that the subscription is not possible anymore.



 TOC 

5.3.  Hub Verifies Intent of the Subscriber

In order to prevent an attacker from creating unwanted subscriptions on behalf of a subscriber (or unsubscribing desired ones), a hub must ensure that the subscriber did indeed send the subscription request.

The hub verifies a subscription request by sending an HTTP (Fielding, R., Gettys, J., Mogul, J., Frystyk, H., Masinter, L., Leach, P., and T. Berners-Lee, “Hypertext Transfer Protocol -- HTTP/1.1,” .) [RFC2616] GET request to the subscriber's callback URL as given in the subscription request. This request has the following query string arguments appended (format described in Section 17.13.4 of [W3C.REC‑html401‑19991224] (Raggett, D., Hors, A., and I. Jacobs, “HTML 4.01 Specification,” December 1999.)):

hub.mode
REQUIRED. The literal string "subscribe" or "unsubscribe", which matches the original request to the hub from the subscriber.
hub.topic
REQUIRED. The topic URL given in the corresponding subscription request.
hub.challenge
REQUIRED. A hub-generated, random string that MUST be echoed by the subscriber to verify the subscription.
hub.lease_seconds
REQUIRED/OPTIONAL. The hub-determined number of seconds that the subscription will stay active before expiring, measured from the time the verification request was made from the hub to the subscriber. Hubs MUST supply this parameter for subscription requests. This parameter MAY be present for unsubscribe requests and MUST be ignored by subscribers during unsubscription.


 TOC 

5.3.1.  Verification Details

The subscriber MUST confirm that the hub.topic corresponds to a pending subscription or unsubscription that it wishes to carry out. If so, the subscriber MUST respond with an HTTP success (2xx) code with a response body equal to the hub.challenge parameter. If the subscriber does not agree with the action, the subscriber MUST respond with a 404 "Not Found" response.

The hub MUST consider other server response codes (3xx, 4xx, 5xx) to mean that the verification request has failed. If the subscriber returns an HTTP (Fielding, R., Gettys, J., Mogul, J., Frystyk, H., Masinter, L., Leach, P., and T. Berners-Lee, “Hypertext Transfer Protocol -- HTTP/1.1,” .) [RFC2616] success (2xx) but the content body does not match the hub.challenge parameter, the hub MUST also consider verification to have failed.

Hubs MAY make the hub.lease_seconds equal to the value the subscriber passed in their subscription request but MAY change the value depending on the hub's policies. To sustain a subscription, the subscriber MUST re-request the subscription on the hub before hub.lease_seconds seconds has elapsed.



 TOC 

6.  Publishing

The publisher MUST inform the hubs it previously designated when a topic has been updated. The hub and the publisher can agree on any mechanism, as long as the hub is eventually able send the updated payload to the subscribers.



 TOC 

7.  Content Distribution

A content distribution request is an HTTP (Fielding, R., Gettys, J., Mogul, J., Frystyk, H., Masinter, L., Leach, P., and T. Berners-Lee, “Hypertext Transfer Protocol -- HTTP/1.1,” .) [RFC2616] POST request from hub to the subscriber's callback URL with the payload of the notification. This request MUST have a Content-Type corresponding to the type of the topic. The hub MAY reduce the payload to a diff between two consecutive versions if its format allows it.

The request MUST include a Link Header (Nottingham, M., “Web Linking,” October 2010.) [RFC5988] with rel=hub pointing to the Hub as well as a Link Header (Nottingham, M., “Web Linking,” October 2010.) [RFC5988] with rel=self set to the topic that's being updated. The Hub SHOULD combine both headers into a single Link Header (Nottingham, M., “Web Linking,” October 2010.) [RFC5988].

The successful response from the subscriber's callback URL MUST be an HTTP (Fielding, R., Gettys, J., Mogul, J., Frystyk, H., Masinter, L., Leach, P., and T. Berners-Lee, “Hypertext Transfer Protocol -- HTTP/1.1,” .) [RFC2616] success (2xx) code. The hub MUST consider all other subscriber response codes as failures; that means subscribers MUST NOT use HTTP redirects for moving subscriptions. The response body from the subscriber MUST be ignored by the hub. Hubs SHOULD retry notifications repeatedly until successful (up to some reasonable maximum over a reasonable time period). Subscribers SHOULD respond to notifications as quickly as possible; their success response code SHOULD only indicate receipt of the message, not acknowledgment that it was successfully processed by the subscriber.



 TOC 

8.  Authenticated Content Distribution

If the subscriber supplied a value for hub.secret in their subscription request, the hub MUST generate an HMAC signature of the payload and include that signature in the request headers of the content distribution request. The X-Hub-Signature header's value MUST be in the form sha1=signature where signature is a 40-byte, hexadecimal representation of a SHA1 signature (Eastlake, D. and P. Jones, “US Secure Hash Algorithm 1 (SHA1),” September 2001.) [RFC3174]. The signature MUST be computed using the HMAC algorithm (Krawczyk, H., Bellare, M., and R. Canetti, “HMAC: Keyed-Hashing for Message Authentication,” .) [RFC2104] with the request body as the data and the hub.secret as the key.

When subscribers receive a content distribution request with the X-Hub-Signature header specified, they SHOULD recompute the SHA1 signature with the shared secret using the same method as the hub. If the signature does not match, subscribers MUST still return a 2xx success response to acknowledge receipt, but locally ignore the message as invalid. Using this technique along with HTTPS (Rescorla, E., “HTTP Over TLS,” May 2000.) [RFC2818] for subscription requests enables simple subscribers to receive authenticated notifications from hubs without the need for subscribers to run an HTTPS (Rescorla, E., “HTTP Over TLS,” May 2000.) [RFC2818] server.

Please note however that this signature only ensures that the payload was not forged. Since the notification also includes headers, these should not be considered as safe by the subscriber, unless of course the subscriber uses HTTPS (Rescorla, E., “HTTP Over TLS,” May 2000.) [RFC2818] callbacks.



 TOC 

9. References

[RFC2104] Krawczyk, H., Bellare, M., and R. Canetti, “HMAC: Keyed-Hashing for Message Authentication,” RFC 2104.
[RFC2119] Bradner, B., “Key words for use in RFCs to Indicate Requirement Levels,” RFC 2119.
[RFC2606] Eastlake, D. and A. Panitz, “Reserved Top Level DNS Names,” RFC 2606.
[RFC2616] Fielding, R., Gettys, J., Mogul, J., Frystyk, H., Masinter, L., Leach, P., and T. Berners-Lee, “Hypertext Transfer Protocol -- HTTP/1.1,” RFC 2616.
[RFC2818] Rescorla, E., “HTTP Over TLS,” RFC 2818, May 2000 (TXT).
[RFC3174] Eastlake, D. and P. Jones, “US Secure Hash Algorithm 1 (SHA1),” RFC 3174, September 2001 (TXT).
[RFC3986] Berners-Lee, T., “Uniform Resource Identifiers (URI): Generic Syntax,” RFC 3986.
[RFC5785] Nottingham, M. and E. Hammer-Lahav, “Defining Well-Known Uniform Resource Identifiers (URIs),” RFC 5785.
[RFC5988] Nottingham, M., “Web Linking,” RFC 5988, October 2010 (TXT).
[W3C.REC-html401-19991224] Raggett, D., Hors, A., and I. Jacobs, “HTML 4.01 Specification,” World Wide Web Consortium Recommendation REC-html401-19991224, December 1999 (HTML).
[XEP-0060] Millard, P., Saint-Andre, P., and R. Meijer, “Publish-Subscribe,” XSF XEP 0060.


 TOC 

Appendix A.  Specification Feedback

Feedback on this specification is welcomed via the PubSubHubbub W3C Community Group. For more information, see the W3C PubSubHubbub Community Group.



 TOC 

Authors' Addresses

  Brad Fitzpatrick
  Google, Inc
Email:  brad@danga.com
  
  Brett Slatkin
  Google, Inc
Email:  bslatkin@gmail.com
  
  Martin Atkins
  Six Apart Ltd.
Email:  mart@degeneration.co.uk
  
  Julien Genestoux
  Notifixious Inc.
Email:  julien@superfeedr.com