Birbla

Join the discussion

Write your take first — we'll ask for email only when you're ready to publish.

Sign in
  • Hacker News
  • The idea is that AI slop is non-creative boring crap, to really determine if you are able to identify AI slop then it should be determined if you misidentify human slop as AI slop.

    Idea 1: Identify the worst most boring human marketing, organizational, bureaucratic texts from the a time before AI was writing it, anonymize this text to make sure there is no reference to current events that can be used to determine that it is not AI. And then see if the AI will say hey, that is not AI slop.

    Idea 2: Have people parody AI slop. Can it determine the parody is still not AI slop?

    by bryanrasmussen
  • Structure alone can differentiate AI content, interesting approach.
    by DylanMerigaud
  • Immediately saw false positives on content written before ChatGPT. Not difficult to see how the methodology is wrong when it considers restating the thesis in the conclusion to be signal.
    by evantbyrne
  • Some people are just very robotic.

    Today I spent 20 minutes in a call to customer support. The representative spoke so robotic-ally (both cadence and word choice), I was extremely confident they were an AI for the first 15 minutes. Was quite certain the automated AI had transferred me to a more advanced one…

    Turns out they were just likely in a low cost Latin American country and trying really hard. I felt bad one could be reduced to such…

    by BobbyTables2
  • The idea is interesting but in looking at methods and github I feel the tooling leaves me wanting. I mean you are trusting LLMs here to establish, vet, and detect your various thresholds that were then used to train the classifier. I'd rather see this sort of thing done deterministically with actual code vs lossy human english prompts and a dependency on token spend to a single third party (who will probably pull the underlying model used in what a few short years probably) to replicate the results or try and use different training data.
    by asdff
  • The “tells you the same thing three times” finding really rings true. That’s the structure every content-marketing course taught long before LLMs, and the models seem to have learned the most average version of it.

    What I like about structural features is that they make for a good eval, not just a detector. If you fine-tune a model to write for your brand, “did the new version get less sloppy than the one we’re running?” becomes a measurable question.

    Disclosure: I work on Ookami (github.com/KatyarAILabs/Ookami), an open-source layer for serving and fine-tuning models. It only puts a fine-tuned model live if it beats the current one on evaluators you plug in.

    Since Slopshape is open source, it could be one of those evaluators: fine-tune on your best human-written posts, and the gate rejects any version whose outputs score as more AI-shaped than the live model.

    I haven’t tried it yet, but that’s the kind of use case I find interesting.

    One question: were the 19 misclassifications writing to SEO templates (listicles, “ultimate guides,” etc.)? If so, the classifier might be detecting “written to a template” more than “written by a model,” which is arguably the more useful signal.

    by katyarailabs

Explore Birbla archives