How to use "hysteria" without domain name: Self-Signed certificates and configurations

[LazarenkoA]

Can I use "hysteria" without specifying the "acme" parameter in the config? I simply don't have a domain name, just a public IP address. If the only solution is to generate my own certificates and use the 'tls' parameter, then another question: can I use self-signed certificates?

[tobyxdd]

Yes and yes

[haruue]

You can even self-sign certificates for an IP address.

For example, assuming your server's IP address is 192.0.2.123, the following script will generate a self-signed certificate for that IP address.

#!/bin/bash

ip="192.0.2.123"

openssl genrsa -out hysteria.ca.key 2048

openssl req -new -x509 -days 3650 -key hysteria.ca.key -subj "/CN=Hysteria Root CA" -out hysteria.ca.crt

openssl req -newkey rsa:2048 -nodes -keyout hysteria.server.key -subj "/CN=$ip" -out hysteria.server.csr

openssl x509 -req -extfile <(printf "subjectAltName=IP:$ip") -days 3650 -in hysteria.server.csr -CA hysteria.ca.crt -CAkey hysteria.ca.key -CAcreateserial -out hysteria.server.crt

[meower1]

Heres a script i've written that automatically does that for you
https://github.com/meower1/Supernova

but if you want to know the details, heres the explaination:
so you bascially get a self signed certificate using openssl by executing the following command

openssl req -x509 -newkey rsa:4096 -keyout private.key -out cert.crt -sha256 -days 3650 -nodes -subj "/C=XX/ST=StateName/L=CityName/O=CompanyName/OU=CompanySectionName/CN=CommonNameOrHostname"

and afterwards it generates a cert.crt and a private.key for you which you can use in your config.
heres an example setup using docker

config.yaml

listen: :443
tls:
  cert: /etc/hysteria/certs/cert.crt
  key: /etc/hysteria/certs/private.key
auth:
  type: password
  password: Oq1jG5B2Dn0sGscTlqgY0OufjFYW5T
quic:
  initStreamReceiveWindow: 8388608
  maxStreamReceiveWindow: 8388608
  initConnReceiveWindow: 20971520
  maxConnReceiveWindow: 20971520
  maxIdleTimeout: 60s
  maxIncomingStreams: 1024
  disablePathMTUDiscovery: false
bandwidth:
  up: 1 gbps
  down: 1 gbps
ignoreClientBandwidth: true
disableUDP: false
udpIdleTimeout: 60s
resolver:
  type: https
  https:
    addr: 1.1.1.1:443
    timeout: 10s
    sni: cloudflare-dns.com
    insecure: false
acl:
  inline: 
    - reject(*.ir)
    - reject(all, udp/443)
    - reject(geoip:ir)
masquerade:
  type: proxy
  proxy:
    url: https://vipofilm.com
    rewriteHost: true 
  listenHTTP: :80 
  listenHTTPS: :443 
  forceHTTPS: true

make sure to put the your cert and private key to the correct path specified in the config file

tls:
  cert: /etc/hysteria/certs/cert.crt
  key: /etc/hysteria/certs/private.key

docker-compose.yaml:

version: "3.9"
services:
  hysteria:
    image: tobyxdd/hysteria
    container_name: hysteria
    restart: always
    network_mode: "host"
    volumes:
      - ./config.yaml:/etc/hysteria/config.yaml:ro
      - ../certs/cert.crt:/etc/hysteria/certs/cert.crt:ro
      - ../certs/private.key:/etc/hysteria/certs/private.key:ro
    command: ["server", "-c", "/etc/hysteria/config.yaml"]

then run the following command to run your proxy server
docker compose up -d