So i have a domain that I have been using solely for homelab and VPS services (domain.example).
I have my A and AAAA record for my VPS proxying through cloudflare (proxy.domain.example) and a DNS A record pointing towards my homelab for my home Wireguard (wg.domain.example) with no other records pointing home or anywhere. I have a couple of services at home with certificates for example (proxmox.domain.example, nas.domain.example, router.domain.example) that are using cloudflares API token but they do not have records listed at cloudflare
Now my issue is I specifically setup a Cloudflare WAF to block every continent/country except my own and this is now showing in the events that a crawler is attempting to access router.domain.example, nas.domain.example, homeassistant.domain.example. Do I have any reason to be concerned and also how would this web crawler only be searching for my home lab domains. None of these services are public facing.
Probably not. It’s most likely automated scanning and the subdomains seem common enough to be included in wordlists. Another possibility is that the subdomains have leaked somehow, do you use LetsEncrypt? If so, the existence of your subdomains is public knowledge and can easily be picked up by bots.
If anyone is interested in mitigation, the only way around this AFAIK is to start with a brand new domain, only use wildcard certs (with DNS validation), and don’t bundle multiple renewals into a single cert.
Also, don’t enter your domain or related IP address into dns reverse engineering tools (like dnsdumpster), and check certificate transparency logs (https://crt.sh) to see what information related to your cert renewals has been published.
This won’t stop automated bots from scanning your ip for domains, but should significantly reduce the amount of bots that discover them