Birbla

Join the discussion

Write your take first — we'll ask for email only when you're ready to publish.

Sign in
  • Hacker News
  • > The trick everyone misses:

    Please just write the first sentence of your README yourself

    by timvdalen
  • >"The trick everyone misses: you don't need to keep the blocklist in RAM.

    Store the domains as sorted 40-bit hashes in flash and binary-search them.

    140,000+ domains fit in ~0.7 MB of flash..."

    Brilliant! (Although, what about potential hash collisions with domain names that shouldn't be blocked? Those are probably few and far between, so the utility of the block overall far outweighs any few false positives that might arise -- so I reiterate my claim of "brilliant"!)

    Also (if you're a compression nerd like I am), it may be possible to analyze this table of 40-bit hashes further to find the longest binary substring that repeats the most across all of them (or binary substrings that just repeat a whole lot), and design a Huffman-like (or other tree or tree-like) data structure around those, and possibly compress the data further...

    Of course, then the simplicity and elegance of the Binary Search might be lost, but it might be an interesting optimization exercise to know just how far that data could be compressed... "Hey Claude, try to optimize the compression of this data using alternative compression data structures, and report back to me!" (Or something like that! You know, get the AI coding agents to try different things -- or attempt to code it on your own (always better for understanding!), etc., etc.!)

    Anyway, brilliant!

    by peter_d_sherman
  • Cool idea, latency might be too high, wish the docs weren't all AI vomit.
    by fwip
  • I think it could fit even more domains using a Bloom Filter or similar probabilistic data structure. With a chance of false-positives of course. But that's a trade-off the project already makes.
    by waysa
  • Whitelist/allowlist is easier, e.g., it's smaller

    Depends on the user but not everyone is visiting new websites everyday

    Even for those that are, the number of domain-IP mappings needed will be relatively small

    Definitely under 140,000

    Most DNS data I use is "static", it rarely changes. As such most times I don't have to make DNS queries. I store the domain-IP mappings in proxy memory; this is faster than DNS

    No "blocklist" needed

    by 1vuio0pswjnm7
  • I'm a bit of a paranoid freak that likes lists, so I've got a PiHole that has a total list of 14-16M blocked domains.

    Great idea, and good for casual blocking, but I'm almost moving to an "allow list" mindset. This solution would probably work better for that, I wonder if the good parts of the internet would fit into an 140k list.

    by BLKNSLVR
  • If you're ever thinking about getting an ESP32-C3, do yourself a favor and get the variant that you can connect an external antenna to. The normal C3 has a built-in antenna that is extremely weak, making it useless for most things I was planning using it for.
    by yoavm
  • Weird how the readme talks about hash collisions, but not in the way I would expect. Two blocked domains with the same hash isn't a problem, they both need to be blocked.

    Where hash collisions matter is false positives, e.g. if hash(google.com) = hash(adserver.com). There does appear to be a web dashboard and /unblock api so should be straightforward to resolve.

    by muti

Explore Birbla archives