#1753 https-inspection, Error: Issuer key doesn't match issuer certificate, follow-up to #1750

[Gergely]

This is a follow-up to https://sourceforge.net/p/ijbswa/support-requests/1750/, on redirects and https inspection.

As Fabian explained, "There is no way for Privoxy to ask the client to accept the certificate."
Now I try to use the Let's Encrypt service and certificate for a domain that I own, say, example.com.
The reported certificate name is example.com,
and the hosts are *.example.com,example.com

This example.com is used in the DNS in the/etc/hosts.
127.0.0.1 local.example.com

What I download the Let's Encrypt certificate from the example.com's admin interface, which reads

-----BEGIN EC PRIVATE KEY-----
...
-----END EC PRIVATE KEY-----
-----BEGIN CERTIFICATE-----
...
-----END CERTIFICATE-----

This is saved as my.crt

I copy the certificate to
ca-directory /usr/local/etc/privoxy/CA

Now following the #1750 setup,

curl output

curl -x local.example.com:8118 -v --head https://www.facebook.com/ --cacert my.crt

*   Trying 127.0.0.1:8118...
* Connected to local.route627.com (127.0.0.1) port 8118 (#0)
* allocate connect buffer!
* Establish HTTP proxy tunnel to www.facebook.com:443
> CONNECT www.facebook.com:443 HTTP/1.1
> Host: www.facebook.com:443
> User-Agent: curl/7.71.1
> Proxy-Connection: Keep-Alive
> 
< HTTP/1.1 200 Connection established
HTTP/1.1 200 Connection established
< 


* Proxy replied 200 to CONNECT request
* CONNECT phase completed!
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /.../my.crt
  CApath: none
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* CONNECT phase completed!
* CONNECT phase completed!
* OpenSSL SSL_connect: SSL_ERROR_SYSCALL in connection to www.facebook.com:443 
* Closing connection 0
curl: (35) OpenSSL SSL_connect: SSL_ERROR_SYSCALL in connection to www.facebook.com:443

log output

privoxy    | 2022-11-25 13:14:32.548 7f6f05025700 Connect: Accepted connection from 127.0.0.1 on socket 4
privoxy    | 2022-11-25 13:14:32.548 7f6f05025700 Header: scan: CONNECT www.facebook.com:443 HTTP/1.1
privoxy    | 2022-11-25 13:14:32.548 7f6f05025700 Header: scan: Host: www.facebook.com:443
privoxy    | 2022-11-25 13:14:32.548 7f6f05025700 Header: scan: User-Agent: curl/7.71.1
privoxy    | 2022-11-25 13:14:32.548 7f6f05025700 Header: scan: Proxy-Connection: Keep-Alive
privoxy    | 2022-11-25 13:14:32.549 7f6f05025700 Header: crumble crunched: Proxy-Connection: Keep-Alive!
privoxy    | 2022-11-25 13:14:32.549 7f6f05025700 Writing: to socket 4: HTTP/1.1 200 Connection established\x0d\x0a\x0d\x0a
privoxy    | 2022-11-25 13:14:32.549 7f6f05025700 Error: A website key already exists but there's no matching certificate. Removing /usr/local/var/privoxy/certs/660328a7f9004d462085aa67a82065db.pem before creating a new key and certificate.
privoxy    | 2022-11-25 13:14:32.644 7f6f05025700 Error: Issuer key doesn't match issuer certificate
privoxy    | 2022-11-25 13:14:32.644 7f6f05025700 Error: generate_host_certificate failed: -1
privoxy    | 2022-11-25 13:14:32.644 7f6f05025700 Error: Failed to open a secure connection with the client

notes

I observed that running the curl command also generates a
/usr/local/var/privoxy/certs/660328a7f9004d462085aa67a82065db.pem
This contains only a

-----BEGIN RSA PRIVATE KEY--
...
-----END RSA PRIVATE KEY--

question

How to get the certificate accepted instead of the above Error: Issuer key doesn't match issuer certificate?

[Fabian Keil]

I'm not sure I understand what you are trying to achieve. Are you trying to let Privoxy use the Let's Encrypt private key and certificate when generating a key and certificate for example.com?

This is not expected to work because Let's Encrypt doesn't distribute its issuer key.

Also Privoxy has a ca-cert-file and ca-key-file directive so the ca key and certificate should not be in the same file.

How did you configure Privoxy ca-related directives?

[Gergely]

The goal is the same as described in #1750. Getting https redirects work, without any manual install of certificates on the client side.

Instead of using a self-signed certificate generated for privoxy, and manually importing it into the browsers (as described in ticket #1750), I would like to use a trusted certificate for privoxy redirects in order to avoid manual imports.
I assumed that

1. using the Let's Encrypt generated certificate of a domain I own (let's say example.com),
2. configuring local DNS for example.com pointing to localhost and
3. using example.com:8118 as proxy

will make self-signed certificates and manually importing them redundant.
Regarding your question, I would like to use Let's Encrypt private key and certificate for privoxy listening on example.com:8118 and not generate new ones.
Privoxy config is the same as described in #1750 and I changed the ca-cert and ca-key files for those of "example.com".

[Fabian Keil]

To get a client to accept a redirect when requesting https://facebook.com/ you need certificates that are accepted for facebook.com and not the URL the redirect points to.

If you use Let's Encrypt for a domain you control the generated certificate and key will only be valid for that domain and not for facebook.com.

While you can trick Privoxy into using a certificate and key generated with Let's Encrypt (by putting them into the certificate-directory using the right name), the client will only accept them for the domain they have been generated for.

[Gergely]

This tricking is what I wanted to do. My config is

ca-cert-file cacert.crt
ca-key-file cakey.pem
# ca-password - we do not have a password
certificate-directory /usr/local/var/privoxy/certs
ca-directory /usr/local/etc/privoxy/CA

and I add the files,

cakey.pem
-----BEGIN EC PRIVATE KEY-----
...
-----END EC PRIVATE KEY-----
cacert.crt
-----BEGIN CERTIFICATE-----
...
-----END CERTIFICATE-----

to both /usr/local/var/privoxy/certs and /usr/local/etc/privoxy/CA. I expected this to work because following the local dns logic, privoxy runs on the domain that matches with the let's encrypt certificate (local.example.com). When we get the

Error: Issuer key doesn't match issuer certificate

then which key is compared with which issuer certificate?

I do see certificates being generated in /usr/local/var/privoxy/certs. Could you share info on these as well?

[Fabian Keil]

How did you create cacert.crt and cakey.pem?

The fact that you are not using a password seems to imply that you didn't use the command suggested in the documentation at:
https://config.privoxy.org/user-manual/config.html#CA-CERT-FILE

The "Error: Issuer key doesn't match issuer certificate" is emitted after checking the cacert.crt and the cakey.pem.

The certificates in the certificate-directory are generated by Privoxy and presented to the client when impersonating the website that is requested. If you want to trick Privoxy into using a key and certificate generated with Let's Encrypt you need to put the files there using the right names (which you can get from the logs).

[Gergely]

I am trying to do exactly what you have described in your last sentence: tricking Privoxy into using a key and certificate generated with Let's Encrypt, but I am not sure how to do that properly.
You wrote „you need to put the files there using the right names (which you can get from the logs)”. Would you please elaborate, which files I have to put where and where I get the the correct names from?

[Fabian Keil]

If you set "debug 2" in the config file Privoxy will log the hash of the host name for the requests:

17:12:14.838 013 Connect: Performing the TLS/SSL handshake with client. Hash of host: 6db5da8a16c246d1bd8c0fa7cd160a5b

The hash of the host is used to build the file name for certificates (file ending ".crt") and keys (file ending ".pem") for the host in the certificate-directory. If a valid key and certificate exists Privoxy will reuse them so you can replace the them with certificates and keys you generated with Let's Encrypt.

As you can only use Let's Encrypt for domains you control this is of limited use, though.