Warm tip: This article is reproduced from stackoverflow.com, please click
docker ssl go proxy grpc

gPRC behind goproxy returns certificate error, works fine without proxy

发布于 2020-03-27 10:26:35

I have a gRPC client and server, both secured with an ssl certificate. Without a proxy inbetween these work great. As a test, when I purposely create faulty certificates it fails. Proving later in this post it's not a certificate issue.

gRPC server code:

// Creates a new gRPC server
// Create the TLS credentials
creds, err := credentials.NewServerTLSFromFile("configs/cert/servercert.pem", "configs/cert/serverkey.pem")
if err != nil {
    log.Fatalf("could not load TLS keys: %s", err)
}
// Create an array of gRPC options with the credentials
opts := []grpc.ServerOption{grpc.Creds(creds)}
// create a gRPC server object
s := grpc.NewServer(opts...)

gRPC client code:

// Create the client TLS credentials
creds, err := credentials.NewClientTLSFromFile("configs/cert/servercert.pem", "")
if err != nil {
    log.Fatalf("could not load tls cert: %s", err)
}

conn, err := grpc.Dial(grpcUri, grpc.WithTransportCredentials(creds))
if err != nil {
    log.Fatalf("Unable to connect: %v", err)
}

now I am trying to use a forward proxy (which ive tested and works fine on normal HTTP api requests). However it constantly fails on gRPC requests through the proxy.

I am using cuttle which internally uses goproxy with the following setup. Do note that the InsecureSkipVerify boolean has been tried both true and false. With my (limited) understanding of SSL that this needs to be false as it will check online for the certificate, and these are self-signed so naturally it would fail. However, again, i tried both true and false

// Config proxy.
proxy := goproxy.NewProxyHttpServer()
proxy.Tr = &http.Transport{
    // Config TLS cert verification.
    TLSClientConfig: &tls.Config{InsecureSkipVerify: !cfg.TLSVerify},
    Proxy:           http.ProxyFromEnvironment,
}

Running a proxy between the gRPC client and server results in the following error:

transport: authentication handshake failed: x509: certificate signed by unknown authority (possibly because of "x509: invalid signature: parent certificate cannot sign this kind of certificate" while trying to verify candidate authority certificate "test server"

Which would indicate it's a certificate issue, however, gRPC works flawless without the proxy, as stated and tested earlier.

also note: I dont want to run gRPC behind a proxy but am forced to due to development environment. The gRPC server and proxy run on the same docker machine. Having the same IP address would result in the following configuration which would just cancel each other out (trust me i tried anyway).

ENV http_proxy 192.168.99.100:3128
ENV https_proxy 192.168.99.100:3128
ENV no_proxy 192.168.99.100 # <- this would be the gRPC server IP, which is the same as the proxy. resulting in nothing being run through a proxy.

Splitting the ip addressed in docker would solve this issue, however, I would learn nothing and would like to solve this. I tried configs like answered here to set different docker internal IP's however, the ip would remain empty (only the network would be set) and accessing on the new IP would just timeout.

Questioner
Rien
Viewed
152
colminator 2019-08-07 04:30

Background:

Each end of a TLS connection needs a pre-arranged trust. Most clients use the system trust chain when connecting to a remote host (GeoTrust, DigiCert CA's trusted certs are all listed there and allow you to safely get to sites like https://facebook.com, https://google.com etc.)

go, when using TLS, will default to the system-trust-chain when contacting servers. When developing custom solutions, chances are your application server's public cert is not in this system-trust-chain. So you have two options:

  • Disable trust via InsecureSkipVerify: true (DON'T do this!)
  • add a custom trust to your client

Most likely you application server has a self-signed certificate, so it's easy to get the public cert portion of this. You can also see a server's public certs using tools like openssl - using the linked solution you can grab public certs for not only your own development servers but any other remote service - just provide the hostname and port.


So just to summarize your situation. You have:

Client <- TLS -> Server

But want:

Client <-TLS-> Proxy <-TLS-> Server

So your client now, instead of trusting the Server now needs to just trust the proxy instead - as it is only ever talking directly to the proxy. The proxy will most likely have a self-signed cert (see above on how to extract the trust cert). Once you have this, update your go code to use this custom trust-file like so:

// Get the SystemCertPool, continue with an empty pool on error
rootCAs, err := x509.SystemCertPool() // <- probably not needed, if we're only ever talking to this single proxy
if err != nil || rootCAs == nil {
    rootCAs = x509.NewCertPool()
}

// Read in the custom trust file
certs, err := ioutil.ReadFile(localTrustFile)
if err != nil {
    log.Fatalf("Failed to append %q to RootCAs: %v", localTrustFile, err)
}

// Append our cert to the system pool
if ok := rootCAs.AppendCertsFromPEM(certs); !ok {
    log.Fatalf("failed to append custom cert")
}

tlsConfig := &tls.Config{
    RootCAs: rootCAs,
}

The proxy also will need to trust the server - so if the server's cert is not in the system-trust-chain, then it will need a similar tls.Config setup like above.