Join the discussion
Write your take first — we'll ask for email only when you're ready to publish.
- Hacker News
- How does it run the LLMs? Or does it call a API to llama.cpp/ollama/etc. ?
During normal usage, how often does it try to parse info from the screen capture? Once a minute?
by jboss10 - You can decide which AI provider you use, either local through Ollama for example, using your own cloud API key, or using Screenpipe cloud. We also support confidential inference through tinfoil.sh
Screenpipe captures accessibility tree and screenshot when you perform a meaningful action. It doesn't use AI at recording time (except PII removal and OCR infrequently). So it can be every few hundred milliseconds to few minutes or more (if screen idle or sleeping)
We usually benchmark CPU usage on $200 Windows/MacOS laptop
by louis030195 - How do you distinguish durable user preferences from transient screen context before an agent turns recorded activity into an automation? I'm especially curious whether each inferred memory keeps provenance and an expiry or confidence signal so stale behavior does not become a permanent rule.by xms17189
- yes we keep track of memories confidence level and timestamp,
we found that in order to find automation opportunities it's better to have agents segment, classify (for example personal vs pro), enrich (which customers/people are related to this workflow) and organize the data first, then it's much easier to find repetitive tasks
by louis030195 - I tried this ~ a week ago, tried one of the suggested automations and it pretty immediately started sending my local api keys to an endpoint… afk right now but can share more when I’m backby basketbla
- Would love to get in a call and help you setup the privacy settings properly, although default are filtering API keys, my cal link if you want:
https://cal.com/team/screenpipe/chat
Feel free to configure local LLM like Ollama, btw
by louis030195 - Implementation specifics aside, I increasingly think life and death cycles are preferable. The opposite of omnipotent context, I want cycles of clean slates so that baggage - of all kinds - does not weigh down the new.by apsurd
- I think this is true for some tasks where you want a clean slate, but increasingly I find myself constantly referencing specific skills, or having files with prompts I use all the time. I'd much rather have an agent that remembers things about me and specifics about my workflows and "taste" - of course having the ability to reset context sometimes or prune memory is valuable but I think the future of agentic AI is blocked on "omnipotent context"by 3s
- Interesting, I partly agree. It sounds like how the human brain works, but what nature learned is not necessarily what we should keep
We capture everything and gives you and option for a flexible data retention policy, so that you can prune mp4 files for example or verbose accessibility trees, while our agents generate high level memories referencing low level data
It still not perfect, but ideally Screenpipe would record forever so that you have infinite memory of every tiny details, while having high level structured memories for humans and agents, individually or as a company
by louis030195 - I tried this a couple months back. While I found it useful, I also found on my MacBook M1, the fact it was recording all the time, made my CPU melt at high temps all the time. I eventually had to shut it off and stop using because of that.
Curious how it runs on more modern hardware? Is it pretty lightweight these days?
by kerv - Hey sorry about that, we made a lot of work on improving CPU usage, we benchmark it on $200 laptop now and it runs smoothly
You can also configure settings like changing transcription engine or only record meetings or turning off some features to reduce resource usage
by louis030195 - The 'record everything on your screen' pitch keeps hitting a privacy wall for teams — even local-only storage triggers a brutal compliance conversation. What's the story for an engineer whose company blocks unaudited desktop recorders by policy? Curious if there's a scoped mode that only captures within a designated app boundary."by russelmelroy
- You can turn off screenshot recording and only capture accessibility data, it still captures most of the meaningful data
Otherwise we usually collaborate with the company to allow screenpipe
by louis030195 - Congrats on the launch. The arc you describe — fine-tuning, then tool-calling, then MCP, each one still "too static" — tracks with what's happened across the whole PKM space, not just agent tooling. But I'd push on one thing: MCP didn't fail because the protocol is weak, it's that access to context and surfacing of context are two different problems. An agent with MCP still has to decide to ask, at the right moment, in the right words. That's the same gap that's plagued Obsidian/Notion-style PKM forever — the archive is complete, but retrieval is still pull-based, so it only helps you when you remember to go looking.
The systems that actually feel like a second brain flip that: they push relevant history, decisions, and prior context at you before you ask, and — this part matters more than people give it credit for — they show their work. Cite where a given answer came from (which note, which meeting, which thread), not just serve up a synthesized blob. Looking at the pushback in this thread (the harvesting claim, the "zero chance I trust cloud SaaS" comments), I don't think that's really an objection to always-on capture — it's an objection to opacity. People will tolerate a lot of passive collection if they can see the receipts on what it's doing with it and where an answer traces back to. Local-only storage helps with the trust problem, but provenance is what actually earns it over time.
by jpardilla - Been a user of Screenpipe to build some "Ai-buddies" for me since last year. Core of that is giving AI a look at what is happening in time space. Without Screenpipe this was rather hard to do at the performance screenpipe gives.by fillskills
- That's interesting, thanks! Indeed we did a lot of work to optimize performance and reliability.
I'm curious what does your AI buddies do more specifically and how do you use them?
by louis030195 - I would keep the capture log and the memory layer as two different things. The capture log is chronological evidence. Memory is a set of derived, revisable claims about projects, people, decisions, and habits. Treating every captured event as memory will make retrieval noisy, while replacing the events with summaries will make the result hard to audit.
Each derived claim should point back to the exact screen or audio spans that support it, record when it was inferred, and say whether it is current, disputed, or superseded. If a meeting moves a launch from Friday to Monday, the Friday record should remain in the history but stop being returned as the current plan. A later answer can then explain both what changed and where the change came from.
Deletion also has to follow that lineage. Removing a sensitive interval should invalidate embeddings, entity records, summaries, and cached agent context derived from it, not just hide the original frames. Otherwise the visible timeline says the data is gone while the useful representation of it remains searchable.
A good memory test would be a correction followed by a deletion. Ask for the current fact, ask why the earlier answer changed, delete the supporting interval, and ask again. The system should answer the first two from traceable evidence and stop claiming the fact after its remaining support disappears.
- Once Screenpipe has months of screen and audio indexed locally, what stops a compromised or over-permissioned agent from querying the whole history through the port 3030 API instead of just the current task's context? Continuous recording solves the memory problem, but it turns "give the agent access to my second brain" into a much bigger blast radius than giving it access to one document. Is there per-query scoping or a time-window restriction on what an agent can pull back, or is authentication the only gate?by rahulladumor
- We have a permission system that allows you to define rules in the markdown header, such as only allowing a time range to be queried, or modalityby louis030195
- How are you planing to segregate between professional use and personal use, I dont want any agent or any llm to know all the time what i have been doing on my system, it would be privacy nightmare and most of the time the screencapture is not meaningful. People may use their work laptop or devices to checkout reddit or hackernews occasionally.
- You can define websites or apps to filter out. You can also define a recording schedule, for example stop recording on weekendby louis030195
- Funny timing. I've been building something similar in my spare time called Daydream. There’s a lot of overlap: local screen/audio capture, OCR and transcription, window and activity context, SQLite, and a searchable memory of what happened.
The main difference is the product direction. Screenpipe seems focused on continuously giving agents context through APIs, MCP, and skills. Daydream is more narrowly built around answering "what did I do today?" through a timeline you can inspect, replay, search, and turn into a daily digest.
I'm also treating deletion as part of the data model. If you cut a sensitive span, its frames, audio, OCR, transcripts, embeddings, and summaries should be deleted or invalidated too.
Mine is still early and Linux-first. I'm open-sourcing it in case anyone wants to contribute, poke around, or use it as a starting point. It’s built with Tauri, a Rust backend, React/TypeScript, SQLite, GStreamer, Whisper, OCR, and VLM processing.
I genuinely didn’t know you were building this when I started. Apparently personal memory capture is becoming a SaaS category too lol.
Code is here: https://github.com/snackbit/daydream
- Awesome thanks for sharing, will follow closely :)by louis030195