Birbla

Join the discussion

Write your take first — we'll ask for email only when you're ready to publish.

Sign in
  • Hacker News
  • I like the entire premise, the one thing stopping me from trying this is not knowing the time scales that I will need to set my computer aside for the processing of large folders of video frames, or my photos library's videos, some 12,000 videos
    by stephenitis
  • There are some specific point to watch out for when implementing an on-device photo processing pipeline—points I encountered firsthand and that you might want to check.

    EXIF Orientation: If you decode an image without applying the orientation information, the resulting embeddings will be calculated based on the rotated image. CLIP is surprisingly sensitive to this, and OCR generally fails to work on text rotated by 90 degrees.

    by tomsonoda
  • How well do you think this would work on stock photography on m1 mac with 32GB ram? For example I'd like to be able to search a folder of ~2k photos for houses with palm trees. Or find photos of kitchens, or find photos of desert southwest landscapes.
    by measure2xcut1x
  • Why chose CLIP to do this. Have you tried small VLMs like Qwen-VL? I believe those models have video encoders can better perform at this scenario.
    by yt1998
  • possibly off-topic, but for anyone interested in this on a more cross-platform / holistic basis, Immich does this

    (& by "this" I mean an approximate AI search for photos & videos - I can't account for the "every frame", nor for the comparative search quality)

    by lucideer
  • Having built something similar with CLIP on an M1, frame sampling rate is the whole ballgame. One frame a second on 12k videos is days, keyframes only got me to an overnight run.
    by hn3ufz62f7
  • Slightly offtopic, but made me wonder.

    Can you copyright things like this now that LLMs exist? I mean, up until now if a small startup has a great idea they will get bought out by big tech which will integrate (or kill) their tech. But now with LLMs can the likes of OpenAI just tell their model to make something that works similar to X (such as this project) and then get round copying laws and negate being behind the curve?

    EDIT: switched to the correct spelling of copyright.

    by alt227
  • Since this is for the mac you really should be using apple's vision framework for OCR. It smokes tesseract in both speed and accuracy.

    Edit: I'm curious which LLM was used to generate the code. I fed the title of your post to claude/deepseek/qwen/codex asking to recommend a stack for this project, expecting to frown thinking that they still recommend tesseract. However, I found that they all recommend apple's vision framework. In fact the latest model to recommend Tesseract is gpt-4.1.

    by postalcoder

Explore Birbla archives