Discussion summary
A tool searches for exposed files on domains, with users discussing its effectiveness and scope. Some mention restrictions on .gov domains and the nature of exposed data.
What the discussion says
- The tool is fast and straightforward, but some users question its coverage.
- Restrictions on .gov domains may limit results.
- Users are curious about the types of businesses exposed.
“My guess is they run selective searches on registered domains.”
“I blacklisted .gov from crawling, it wasn't worth the trouble.”
Join the discussion
Write your take first — we'll ask for email only when you're ready to publish.
- Hacker News
- Awesome tool, fast and right to the point!by sub-e12
- searching for .gov reveals 0 matches... doubtby cvadict
- I manually blacklisted .gov from being crawled on my side. felt like it wasn't worth the potential troubleby PatchRequest
- My guess is that they ran selective search on the domains which get registered with any registrar, thats the trigger to start the search. .gov domains are not managed by your typical registrar which is selling the domain registration information to all these downstream partners/scavengers (for lack of better word)by sandeepkd
- Nice tool. I’d like to understand what kinds of businesses the customers using this website are in.by Avery29
- CT logs are funny, not enough people know about them. Whenever I speak to people they're shocked to know that "internal" subdomains get exposed through them.by marysol5
- Wildcard certs were somewhat useful for hiding this. Not sure that's still true.by seethishat
- Is this just re-skinned leakix.net? One of my honeypots for them is showing the same results.by Elliott-Diy
- Nope, i crawl everything myself.by PatchRequest
- Its interesting and not interesting at the same time based on some of the search results
Almost all of them seem like home projects being deployed with ease in mind than security. The common thread seems to be the fact that most of them are phishing website, not sure if thats a business model to target here?
by sandeepkd - In the early days of the web you could do a search on google like
Sometimes there were even shadow passwd files with the hashes exposed on the web. Crazy days.path:/etc/passwdby keepamovin - i thought it was still possible!
Luckily security has come a long way, but as shown by the project of OP, we are not quite there yet.
by AndReics - I remember seeing examples like this in security courses.
It was always surprising how many servers accidentally exposed sensitive files.
- There's an astounding amount of .DS_Store showing up - I hadn't realised how common it apparently is for people to accidentally upload this.by technion
- It’s a terrible design from Apple to expose this metadata like this, it’s one of my biggest pet peeves.
- So is this the crawler that has been constantly hammering all my applications searching for these files from the very second I first issue a TLS cert for them? Thanks to you I've had to put fail2ban on all my public-facing web servers...
How about you be a good netizen and make it so people can request to be scanned and don't proactively do it, let alone constantly keep hammering them with requests?
by phoronixrly - Lots of crawlers do this. I have never seen a webserver that does not get a variety of these from obviously different sources. Even just an ssh port will get a lot of malicious login attempts.by graemep
- I need to protect against the malicious good guys, the shodans of the world. Peeping inside your windows and trying your front door handles. Querying every string imaginable and port pinging all 65536 of them.
And I need to protect against the actual criminals already inside my house looking for something to steal. Scouting out the easy target to setup their ransomwares.
by bblb