http2: Introducing http/2 by jasnell · Pull Request #14239 · nodejs/node

jasnell · 2017-07-14T19:57:36Z

At long last: The initial experimental implementation of HTTP/2.

This is an accumulation of the work that has been done in the nodejs/http2
repository, squashed down to a couple of commits. The original commit
history has been preserved in the nodejs/http2 repository.

Contributors to this work include:

Anna Henningsen / @addaleax
Colin Ihrig / @cjihrig
Daniel Bevenius / @danbev
James M Snell / @jasnell
Jun Mukai / @jmuk
Kelvin Jin / @kjin
Matteo Collina / @mcollina
Robert Kowalski / @robertkowalski
Santiago Gimeno / @santigimeno
Sebastiaan Deckers / @sebdeckers
Yosuke Furukawa / @yosuke-furukawa

This PR introduces the nghttp2 C library as a new dependency. This library
provides the majority of the HTTP/2 protocol implementation, with the rest
of the code here providing the mapping of the library into a usable JS API.

Within src, a handful of new node_http2_.c and node_http2_.h files are
introduced. These provide the internal mechanisms that interface with nghttp
and define the process.binding('http2') interface.

The JS API is defined within internal/http2/*.js.

There are two APIs provided: Core and Compat.

The Core API is HTTP/2 specific and is designed to be as minimal and as
efficient as possible.

The Compat API is intended to be as close to the existing HTTP/1 API as
possible, with some exceptions.

Tests, documentation and initial benchmarks are included.

The http2 module is gated by a new --expose-http2 command line flag.
When used, require('http2') will be exposed to users. Note that there
is an existing http2 module on npm that would be impacted by the introduction
of this module, which is the main reason for gating this behind a flag.

When using require('http2') the first time, a process warning will be
emitted indicating that an experimental feature is being used.

To run the benchmarks, the h2load tool (part of the nghttp project) is
required: ./node benchmarks/http2/simple.js benchmarker=h2load. Only
two benchmarks are currently available.

Additional configuration options to enable verbose debugging are provided:

$ ./configure --debug-http2 --debug-nghttp2
$ NODE_DEBUG=http2 ./node

The --debug-http2 configuration option enables verbose debug statements
from the src/node_http2_* files. The --debug-nghttp2 enables the nghttp
library's own verbose debug output. The NODE_DEBUG=http2 enables JS-level
debug output.

The following illustrates as simple HTTP/2 server and client interaction:

(The HTTP/2 client and server support both plain text and TLS connections)

const req = client.request({ ':path': '/some/path' });
req.on('data', (chunk) => { /* do something with the data */ });
req.on('end', () => {
  client.destroy();
});

// Plain text (non-TLS server)
const server = http2.createServer();
server.on('stream', (stream, requestHeaders) => {
  stream.respond({ ':status': 200 });
  stream.write('hello ');
  stream.end('world');
});
server.listen(80);

const http2 = require('http2');
const client = http2.connect('http://localhost');

Checklist

make -j4 test (UNIX), or vcbuild test (Windows) passes
tests and/or benchmarks are included
documentation is changed or added
commit message follows commit guidelines

Affected core subsystem(s)

http2

[Edit:ofrobots: fixed github handle for Kelvin Jin]

Fishrock123 · 2017-07-14T20:04:18Z

Just a thought: separating the new dep checkin to a separate commit might make review easier?

jasnell · 2017-07-14T20:05:14Z

This thing is going to be massive and difficult to review no matter how it is split up, unfortunately.

lucaslago · 2017-07-14T20:15:52Z

deps/nghttp2/lib/nghttp2_pq.h

Nitpick: typo on this comment

That's in the upstream dependency so it would need to be fixed there :-) https://github.com/nghttp2/nghttp2

Fixed upstream nghttp2/nghttp2#958

vsemozhetbyt · 2017-07-14T20:36:36Z

test/parallel/test-tls-disable-renegotiation.js

A nit: redundant after #14021

thinking about it, I think I need to go add the crypto skip in a couple of the other tests.

@lucaslago

Came up in downstream code review by @lucaslago nodejs/node#14239 (comment)

addaleax · 2017-07-15T00:00:41Z

@jasnell We also split out the deps commits for node-inspect and for the inspector iirc; that worked out pretty well and did help with reviewing. :)

addaleax · 2017-07-15T00:04:09Z

Also, for N-API, async_hooks, and some other PRs, we used Author: name <email> metadata (example: 56e881d) for specifying multiple authors. Feel invited to adapt that format. :) (That has the nice side effect of not pinging everybody everytime you rebase/the commit is landed/etc.)

addaleax · 2017-07-15T01:46:39Z

doc/api/http2.md

This does the same thing as stream.priority(options), right? In that case I think I’d prefer not to expose it (same thing for rstStream)

Currently, yes, but eventually this will support the ability to pass in a numeric stream identifier rather than the Http2Stream object. The HTTP/2 spec allows priorities and rst_stream frames to be sent independently, even without a stream having been previously established.

yoshuawuyts · 2017-07-15T12:17:50Z

The colon prefix in :status looks a bit odd to me. I'm sure there have been careful considerations made as to why that makes sense. Would you mind sharing the reasoning behind using colon prefixes? Thanks!

mcollina · 2017-07-15T12:46:59Z

@yoshuawuyts it's part of the HTTP2 spec: https://http2.github.io/http2-spec/#HttpResponse

yoshuawuyts · 2017-07-15T13:04:30Z

@mcollina cool, thanks for answering!

…

On Sat, Jul 15, 2017, 14:47 Matteo Collina ***@***.***> wrote: @yoshuawuyts <https://github.com/yoshuawuyts> it's part of the HTTP2 spec: https://http2.github.io/http2-spec/#HttpResponse — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#14239 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ACWleuhN8XM3Dk7G-OD3ZsFjZciMOVUsks5sOLThgaJpZM4OYrK1> .

dougwilson · 2017-07-15T17:39:26Z

Awesome work! Above you said the following:

The Compat API is intended to be as close to the existing HTTP/1 API as possible, with some exceptions.

I tried looking in the documentation included in this PR, but couldn't figure out what these exceptions are. Can you ellaborate? Is the compatibility API just the same how the "spdy" module does the HTTP/2 -> 1 compatibility API or different somehow? I'm not really looking for a comparison to the "spdy" module, but thought if you knew that would help, otherwise would love to see the docs around what these differences are.

jasnell · 2017-07-15T17:42:49Z

@dougwilson .... I'm hoping to get that compat api documented this next week... Including those caveats.

vsemozhetbyt · 2017-07-15T17:59:18Z

http2 should also be added to all.md.

vsemozhetbyt · 2017-07-15T18:09:59Z

doc/api/errors.md

`RST_STREAM`?

vsemozhetbyt · 2017-07-15T18:13:15Z

doc/guides/writing-and-running-benchmarks.md

nghttp2.org

dougwilson · 2017-07-15T18:13:17Z

Thanks @jasnell ! Please feel free to ping me when you do, I would love to take a look.

vsemozhetbyt · 2017-07-16T00:44:28Z

doc/api/http2.md

missing ]

vsemozhetbyt · 2017-07-16T00:59:04Z

doc/api/http2.md

According to using in text,

[ServerHttp2Stream]: [ClientHttp2Stream]: [Http2Stream]:

should be:

[`ServerHttp2Stream`]: [`ClientHttp2Stream`]: [`Http2Stream`]:

vsemozhetbyt · 2017-07-16T01:00:22Z

doc/api/http2.md

Missing reference for [Buffer][]

vsemozhetbyt · 2017-07-16T01:03:12Z

doc/api/http2.md

Missing ]

vsemozhetbyt · 2017-07-16T14:02:02Z

Issue with the first examples from the main doc: #14304

sebdeckers · 2017-07-16T15:45:39Z

@dougwilson Some notable differences, off the top of my head:

Most private variables are not supported. Only the documented methods, properties, and events are supported. Code that relies on undocumented behaviour is not supported by the compat api. E.g. no more _header checking; use headersSent instead.
The request and response objects do not inherit from http.IncomingMessage and http.ServerResponse respectively. This poses a challenge for express@4, in my experience.
No status message in sendHead because HTTP/2 dropped the concept of text status messages. Status code is now the :status header. A process warning is emitted if the (optional) status message is passed. The message itself is ignored. Status code and rest of the headers are sent.
The socket property should not be written to. HTTP/2 has a framing layer and does not directly expose the socket. Only the low-level nghttp2 library deals with the socket. However meta properties on the socket can be useful, like detecting http vs https, address/port, TLS info, etc.
Only status codes 1xx-5xx are supported. This follows the HTTP spec. Previously in the HTTP/1 API status codes 1xx-9xx were allowed.
Extensions:
- The [req|res].stream property exposes the HTTP/2 stream, which in turn exposes the HTTP/2 session.
- res.createPushResponse(headers, cb) can use used to send a PUSH_PROMISE frame. The callback receives the corresponding Http2ServerResponse instance via which headers and a body can be sent.