How to trust self-signed certificate in cURL command line?

Question

I've created a self-signed certificate for foo.localhost using a Let's Encrypt recommendation using this Makefile:

include ../.env

configuration = csr.cnf
certificate = self-signed.crt
key = self-signed.key

.PHONY: all
all: $(certificate)

$(certificate): $(configuration)
    openssl req -x509 -out $@ -keyout $(key) -newkey rsa:2048 -nodes -sha256 -subj '/CN=$(HOSTNAME)' -extensions EXT -config $(configuration)

$(configuration):
    printf "[dn]\nCN=$(HOSTNAME)\n[req]\ndistinguished_name = dn\n[EXT]\nsubjectAltName=DNS:$(HOSTNAME)\nkeyUsage=digitalSignature\nextendedKeyUsage=serverAuth" > $@

.PHONY: clean
clean:
    $(RM) $(configuration)

I've then assigned that to a web server. I've verified that the server returns the relevant certificate:

$ openssl s_client -showcerts -connect foo.localhost:8443 < /dev/null
CONNECTED(00000003)
depth=0 CN = foo.localhost
verify error:num=20:unable to get local issuer certificate
verify return:1
depth=0 CN = foo.localhost
verify error:num=21:unable to verify the first certificate
verify return:1
---
Certificate chain
 0 s:/CN=foo.localhost
   i:/CN=foo.localhost
-----BEGIN CERTIFICATE-----
[…]
-----END CERTIFICATE-----
---
Server certificate
subject=/CN=foo.localhost
issuer=/CN=foo.localhost
---
No client certificate CA names sent
Peer signing digest: SHA512
Server Temp Key: X25519, 253 bits
---
SSL handshake has read 1330 bytes and written 269 bytes
Verification error: unable to verify the first certificate
---
New, TLSv1.2, Cipher is ECDHE-RSA-AES128-GCM-SHA256
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : ECDHE-RSA-AES128-GCM-SHA256
    Session-ID: […]
    Session-ID-ctx: 
    Master-Key: […]
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    TLS session ticket:
    […]

    Start Time: 1529622990
    Timeout   : 7200 (sec)
    Verify return code: 21 (unable to verify the first certificate)
    Extended master secret: no
---
DONE

How do I make cURL trust it without modifying anything in /etc? --cacert does not work, presumably because there is no CA:

$ curl --cacert tls/foo.localhost.crt 'https://foo.localhost:8443/'
curl: (60) SSL certificate problem: unable to get local issuer certificate
More details here: https://curl.haxx.se/docs/sslcerts.html

curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above.

The goal is to enable HTTPS during development:

I can't have a completely production-like certificate without a lot of work to enable DNS verification in all development environments. Therefore I have to use a self-signed certificate.
I still obviously want to make my development environment as similar as possible to production, so I can't simply ignore any and all certificate issues. curl -k is like catch (Exception e) {} in this case - nothing at all like a browser talking to a web server.

In other words, when running curl [something] https://project.local/api/foo I want to be confident that

if TLS is configured properly except for having a self-signed certificate the command will succeed and
if I have any issues with my TLS configuration except for having a self-signed certificate the command will fail.

Using HTTP or --insecure fails the second criterion.

There seems to be a solution there: https://stackoverflow.com/a/21262787/6368697 — Patrick Mevzek, Jun 22 '18 at 02:20
@PatrickMevzek No, "without modifying anything in /etc" is not satisfied by that solution. — l0b0, Jun 22 '18 at 02:58
Although there's no real CA, a selfsigned cert is effectively treated as its own CA for validation purposes. Try `openssl x509 — dave_thompson_085, Jun 22 '18 at 08:44
What do you mean by "make cURL trust it"? In general there is no notion of "trust" for self-signed certificates since anyone can make them. What is that you want? Only to accept that one certificate's fingerprint? Only a certain certificate including the extensions? Something else? — V13, Oct 17 '18 at 22:49
I'm having a similar issue. I get the certificate chain of a self-signed CA of our corporate proxy using the `openssl s_client -showcerts` answer, but `curl -v --cacert cacert.pem URL` won't add the self-signed CA as an explicit whitelisting of trust with `CERT_TRUST_REVOCATION_STATUS_UNKNOWN`. — Josh Peak, Oct 24 '18 at 23:13
I know that's not exactly the answer to the question, but it's an alternative approach that can work for anyone looking for a solution to the problem (for development environment). You can use Let's Encrypt certificates for free, making it trusted in most browsers and when using curl. You only need to have a public Ip for your environment and a DNS. — danilo, Apr 25 '20 at 15:54

score 69 · Answer 1 · answered Oct 16 '18 at 23:55

69

Try -k:

curl -k https://yourhost/

It should "accept" self-signed certificates

answered Oct 16 '18 at 23:55

V13

4,551
1
15
20

76

The question is how to **trust** self-signed certificates, not how to bypass certificate validation. – l0b0 Oct 17 '18 at 00:08
2

@l0b0: To make **curl** trust self-signed certificates. And it also says: "The goal is to enable HTTPS during development". `curl -k` achieves both. There is no validation in self-signed certificates, unless you are implying that you want to accept only a certain self-signed certificate, but this is not what the question says. Can you explain what is your objection? – V13 Oct 17 '18 at 00:14
9

I wrote "How do I make cURL trust **it**". If I asked you how to open SSH to a specific IP, would you tell me to open it to every IP? – l0b0 Oct 17 '18 at 23:37
What does "trust it" means in your mind? Is that a CA certificate (i.e. has the CA extensions) that you want to trust as CA? or is it a plain certificate and you want to make sure that this is the one you're receiving? Do you just want to look at the fingerprint of the key or the whole set of extensions? Do you maybe care just about the CN? – V13 Oct 17 '18 at 23:48
You can see from the certificate that it does not have CA extensions. It's a self-signed certificate. I want cURL to do the same validations it does for any certificate. As `man curl` puts it "The server connection is verified by making sure the server's certificate contains the right name and verifies successfully using the cert store." Maybe that means I need a way to create a cert store for a self-signed certificate. – l0b0 Oct 18 '18 at 00:03
3

I cannot see that from your post. There isn't a dump of the certificate in it. Curl probably relies on openssl to do the validations. The validations (may) include the proper flags for use (e.g. ssl server), CN name, date, chain validation, revocation check via CRL, revocation check via OCSP and probably something else that I'm forgetting. Your post doesn't mention any of these, nor it shows the certificate, and you keep updating it. I'd suggest forming a good question from scratch and taking the answers a bit more seriously instead of being rude to everyone that tries to help you. – V13 Oct 18 '18 at 00:17
1

I see that you have listed the generation of the cert. Cert validation needs to start from a CA and you don't use a CA, so you cannot possibly validate the cert. If you are looking for something like "openssl verify"'s -trusted then I don't think that exists. – V13 Oct 18 '18 at 00:34
1

Fantastic, that actually gets me somewhere. I'll try creating a CA. Do you want to post that as an answer? I'm afraid I find TLS certificates and the surrounding tools frustratingly obtuse and weird, and I really thought my original question was easy to answer because it's one I've encountered in several jobs and every "solution" has been to just ignore the problem until the configuration is in production and everything falls apart. Basically I just want to get as close to production as possible given the limitation of having to generate self-signed certificates. – l0b0 Oct 18 '18 at 01:00
does not make since to answer oppositely to the question. The question is about trusting certificate and you are ignoring it !!! – Abdennour TOUMI Apr 06 '19 at 20:37
1

You cannot "trust" a self-signed certificate, unless you want to treat it as a trusted certificate itself for validation purposes, which would be the equivalent of a CA cert. In that case you'd use the same certificate as a CA cert, but that will also require the cert to be tagged as a CA. Please read the rest of the comments. Also, the question has been altered since I posted this question. My response was to an earlier version which did not have the clarifications. – V13 Apr 07 '19 at 13:45

score 20 · Answer 2 · answered Sep 11 '18 at 19:19

20

Following these steps should solve your issue:

Download and save the self-signed certificate: echo quit | openssl s_client -showcerts -servername "${API_HOST}" -connect "${API_HOST}":443 > cacert.pem
Tell the curl client about it: curl --cacert cacert.pem --location --silent https://${API_HOST}

Also one could use wget and ignore certificates with: wget --no-check-certificate https://${API_HOST}

answered Sep 11 '18 at 19:19

Robert Brisita

1,023
8
8

20

-1 I already tried `--cacert`. And I'm definitely not interested in *ignoring* the certificate. This is terrible advice. – l0b0 Sep 11 '18 at 19:35
I understand your frustration. Did you try the openssl command and save the cert from the server you are connecting to? The ignore is just another option for someone else to try if they don't really need to test with HTTPS. – Robert Brisita Sep 11 '18 at 23:55
I already have the certificate. Please read my question. – l0b0 Sep 12 '18 at 01:39
That is where the problem lies. I believe you are using the certificate from the `--out` argument rather than following the steps in my answer. – Robert Brisita Sep 12 '18 at 16:55
I *created* the certificate, so I don't need to retrieve it. That is not what the question is about, and rather than assuming that I don't have the certificate I would suggest using comments to ask such follow-up questions. – l0b0 Sep 12 '18 at 19:27
Is there some reason to believe that using `openssl` to download the certificate from the server will be different than just using the copy already on disk from when it was generated? – Michael Mrozek Sep 12 '18 at 20:20
Yes. It is different formats. I give you what worked for me. Do what you will. Have a good day. – Robert Brisita Sep 13 '18 at 12:21
OK, I did exactly what you wrote, and now it says "SSL certificate problem: unable to get local issuer certificate". – l0b0 Oct 17 '18 at 00:23
1

I have never encountered that error message but a different error message is good. Which web server are you using? It might be misconfigured to serve the certificate. For instance in a virtual host ssl Apache2 config file there would be: `SSLCertificateFile ${CRT_OUT}` `SSLCertificateKeyFile ${KEY_OUT}` The variables are file paths to where the generated keys live on the server. – Robert Brisita Oct 17 '18 at 21:32
I'm terminating TLS with Traefik, and that works fine with Let's Encrypt certificates. – l0b0 Oct 17 '18 at 23:32
1

I have zero experience with Traefik. I am assuming Traefik is trying to verify the self-signed certificate. My development environments are local and AWS EC2 instances with either Apache2 or Nginx. I would suggest reviewing how Let's Encrypt (LE) configures Traefik and where LE tells Traefik where it stores it's certificate and key files. That is where you want to point to the respective files. I would also suggest to update your question with the software that you are working with. I incorrectly assumed you were using a popular web-server to validate the certificates. – Robert Brisita Oct 19 '18 at 11:14

TrentP · Accepted Answer · 2020-11-23T02:23:41.980

I had this issue, exact same problem and error messages, but I used GNUTLS's certtool to generate my cert rather than openssl.

My problem was that I had not made my self signed cert a CA. It was only configured to act as a web server cert. Which is all I wanted to do with it and I wasn't going to use it as a CA to sign other certs.

But when you want to add a cert into the trust chain as the Issuer of other certs, that cert must be a CA, or it's rejected by openssl!

With certtool -i < mycert.crt, one needs to see this:

    Extensions:
            Basic Constraints (critical):
                    Certificate Authority (CA): TRUE

Try adding -addext basicConstraints=critical,CA:TRUE,pathlen:1 to your openssl command or modifying your cnf file to the same effect.

Or, use certtool, it's much easier for one-off cert generation:

certtool -p --outfile localhost.key
certtool -s --load-privkey localhost.key --outfile localhost.crt

And then answer the prompts to supply the cert's CN and so on. And say yes when asked if it's for a certificate authority!

So far this looks like the only workable solution, so I'll mark it as accepted. Unfortunately the relevant project is long done, so it's unlikely I'll have opportunity to test this in the foreseeable future. — l0b0, Nov 22 '20 at 18:39
Also you need to add -addext keyUsage=keyCertSign, from this [link](https://stackoverflow.com/a/59041553/2023035) but this worked for me, Thanks! — Ana Franco, Oct 07 '21 at 02:20

score 4 · Answer 4 · edited Feb 07 '22 at 16:15

The solution provided using trustme works perfectly, but if anyone is interested in creating the certificates for learning purposes, these are the steps that worked for me:

Create the root CA:

openssl req -x509 -sha256 -days 1825 -newkey rsa:2048 -keyout rootCA.key -out rootCA.crt

Create the server private key:
```
openssl genrsa -out localhost.key 2048
```

Create the certificate signing request:

openssl req -key localhost.key -new -out localhost.csr

Create the localhost.ext file with the following content:
```
authorityKeyIdentifier=keyid,issuer
basicConstraints=CA:FALSE
subjectAltName = @alt_names
[alt_names]
DNS.1 = localhost
```
This file has the DNS configuration for the certificate. This is important as the TLS check won't pass if the name in the URL doesn't match the DNS in the certificate.

Sign the server certificate with the root CA that was created before:

openssl x509 -req -CA rootCA.crt -CAkey rootCA.key -in localhost.csr -out localhost.crt -days 365 -CAcreateserial -extfile localhost.ext

Convert the crt files to the pem format:

openssl x509 -in localhost.crt -out localhost.pem
openssl x509 -in rootCA.crt -out rootCA.pem

Now you should be able to use the localhost.pem and the localhost.key in the server. The client should use the rootCA.pem to connect to the server.

In my case, I had to use a jks file. The pem file can be converted to jks as follows:

//first convert it to pkcs12
openssl pkcs12 -export -in localhost.pem -inkey localhost.key -out serverCertificate.p12 -name "serverCertificate"

//then convert it from pkcs12 to jks
keytool -importkeystore -srckeystore serverCertificate.p12 -srcstoretype pkcs12 -destkeystore localhost.jks

The entries in the keystore can be seen using this command:

keytool -keystore localhost.jks -list

The localhost.jks should be used in the server and the connection can be tested using:

curl --cacert rootCA.pem https://localhost:<port>

References:

score 3 · Answer 5 · answered Oct 19 '21 at 02:46

trustme, used by urllib3, looks like a good option. Per their readme:

$ # ----- Creating certs -----
$ python -m trustme
Generated a certificate for 'localhost', '127.0.0.1', '::1'
Configure your server to use the following files:
  cert=/tmp/server.pem
  key=/tmp/server.key
Configure your client to use the following files:
  cert=/tmp/client.pem
$ # ----- Using certs -----
$ gunicorn --keyfile server.key --certfile server.pem app:app
$ curl --cacert client.pem https://localhost:8000/
Hello, world!

I wasn't able to get `certtool` to make a certificate that curl actually accepts ("unsupported certificate purpose"), so this one is the first answer I'm seeing that actually works. Thanks! — Benno, Sep 12 '22 at 17:45

score 2 · Answer 6 · 2019-06-19T15:58:30.497

It is not valid to have a trust chain that include a self-signed cert. If that were the case anyone could provide a (made up) valid trust chain. If a self-signed cert appears in a trust chain it must be ignored. A self-signed cert could only be valid in a local directory (controlled by the computer owner). The cert given to any server must be chained to the self-signed cert.

A general guide without most of the little details.

Your output of the openssl s_client command is showing two errors:
```
verify error:num=20:unable to get local issuer certificate
verify error:num=21:unable to verify the first certificate
```
That means that the default cert store in your machine is missing a cert that validates the chain given from the web site you used. You need a directory with a self-signed cert and a cert chained to that for the web server.

Steps:

You could build a new directory (anywhere), process it with the c_rehash script and tell openssl to use it to verify the certs with the option -CApath Directory. Make changes until you get rid of both errors while using the -CApath option.
Generate a chained cert for the web server.
Then, tell curl about the certificate directory with:
```
curl --capath <dir>
```
and all the other options needed.

That will clear both errors.

When creating self-signed certificates there is no separate root certificate. I've added the certificate creation Makefile to my question to clarify this. — l0b0, Oct 17 '18 at 23:35
If I need to create a self-signed CA certificate to do this, that would be fine. I just thought (naively, it seems) that there would be a simple way to do this. — l0b0, Oct 18 '18 at 00:06

score 0 · Answer 7 · answered May 04 '20 at 20:58

0

If you save off the self-signed.crt from your server, you can pass it to curl via "--cacert self-signed.crt" and curl will validate the certificate of your server using the given CA Cert.

answered May 04 '20 at 20:58

user2679859

1

2

This looks the same as [this answer](https://unix.stackexchange.com/a/468360/3645). If it's not, can you please give some details why it's not? – l0b0 May 04 '20 at 20:59
I think it's similar. Your comment on that answer is that you created the cert and that answer wasnt working. I have a very similar system. I have a web server that, if not configured, auto-creates it's own ca cert, then server cert and signs the server cert with the ca cert. So in my system, I just directly point curl at my ca cert using the --cacert option and it all works well for me. – user2679859 May 06 '20 at 14:24

How to trust self-signed certificate in cURL command line?

7 Answers7

Linked