The bare minimum a distributed system developer should know aboutHTTPS Negotiation
I mentioned in a previous post that an SSL connection will typically use a Server Name Indication in the initial (unencrypted) packet to let the server know which address it is interested in. This allow the server to do things such as select the appropriate certificate to answer this initial challenge.
A more interesting scenario is when you want to force your users to always use HTTPS. That is pretty trivial, you setup a website to listen on port 80 and port 443 and redirect all HTTP traffic from port 80 to port 443 as HTTPS. Pretty much any web server under the sun already have some sort of easy to use configuration for that that. Let us see how this will look like if we were writing this using bare bones Kestrel.
This is pretty easy, right? We setup a connection adapter on port 80, so we can detect that this is using the wrong port and then just redirect it. Notice that there is some magic that we need to apply here. At the connection adapter, we deal with raw TCP socket, but we don’t want to mess around with that, so we just pass the decision up the chain until we get to the part that deal with HTTP and let it send the redirect.
Pretty easy, right? But about about when a user does something like this?
http://my-awesome-service:443
Note that in this case, we are using the HTTP protocol and not the HTTPS protocol. At that point, things are a mess. A client will make a request and send a TCP packet containing HTTP request data, but the server is trying to parse that as an SSL client help message. What will usually happen is that the server will look at the incoming packet, decide that this is garbage and just close the connection. That lead to some really hard to figure out errors and much forehead slapping when you figure out what the issue is.
Now, I’m sure that you’ll agree that anyone seeing a URL as listed about will be a bit suspicious. But what about these ones?
- http://my-awesome-service:8080
- https://my-awesome-service:8080
Unlike before, where we would probably notice that :443 is the HTTPS port and we are using HTTP, here there is no additional indication about what the problem is. So we need to try both. And if a user is getting connection dropped error when trying the connection, there is very little chance that they’ll consider switching to HTTPS. It is far more likely that they will start looking at the firewall rules.
So now, we need to do protocol sniffing and figure out what to do from there. Let us see how this will look like in code:
We read the first few bytes of the request and see if this is the start of an SSL TCP connection. If it is, we forward the call to the usual Kestrel HTTPS behavior. If it isn’t, we mark the request as must redirect and pass it, as is, to the request parsed and ready for action and then send the redirect back.
In this way, any request on port 80 will be sent to port 443 and an HTTP request on a port that listens to HTTPS will be told that it needs to switch.
One note about the code in this post. This was written at 1:30 AM as a proof of concept only. I’m pretty sure that I’m heavily abusing the connection adapter system, especially with regards to the reflection bits there.
More posts in "The bare minimum a distributed system developer should know about" series:
- (20 Nov 2017) Binding to IP addresses
- (15 Nov 2017) HTTPS Negotiation
- (06 Nov 2017) DNS
- (03 Nov 2017) Certificates
- (01 Nov 2017) Transport level security
- (31 Oct 2017) networking
Comments
Oleg, Suggesting specific topics for discussion is the best way to ensure that there will be more content.
Technically speaking, you can detect if an incoming connection is HTTP or HTTPS. but you must reply in the same protocol. That isn't trivial, see this post:
https://ayende.com/blog/180513/the-bare-minimum-a-distributed-system-developer-should-know-about-https-negotiation?key=6329dd9044194fbcb7017d018491f05b
Oren thanks, that is one of the pieces I am most interested in around HTTPS adoption!!
Great to see a solid solution, making worlds better and simpler.
I wonder if this should be a standard (optional?) feature of Kestrel, and other implementations alike?
Oleg, I have in the blog queue something that is actually even cheaper than this and plays nicely with Kestrel. That can be made a part of this directly, yes.
Comment preview