Birbla

Join the discussion

Write your take first — we'll ask for email only when you're ready to publish.

Sign in
  • Hacker News
  • I mean yah... host glm and kimi and I am game.
    by _pdp_
  • Interesting. I could see them perhaps coming in competitive for models that fit into single cards? Less so playing in the big model serving league...climbing into that esp right now would be madness
    by Havoc
  • Why do you think this would be madness? It seems like, at least in the EU, there is barely competition for the big open-weight models?
    by jonas_scholz
  • This is interesting because I thought Hetzner was anti-crypto? LLMs aren't the same but they're often lumped in with crypto as "things no one wants."
    by NetOpWibby
  • Crypto DAU: 10’s of k LLMaaS DAU: 100’s of k
    by cousinbryce
  • Who lumps LLMs with crypto? The former has actual usage and purpose, the latter doesn't. Anyone who does must think any new technology is all the same.
    by satvikpendem
  • Excited to see the price of every other product they offer triple for no reason.
    by danlitt
  • gestures broadly at ~$1T+ in AI capex spend
    by toomuchtodo
  • "no reason" like hardware prices going through the roof?
    by jonas_scholz
  • Whats definitely missing: a solid (non Mistral) GDPR compliant coding plan / subscription. All offerings are either US or China based. With the newest open weights models this became really interesting imo.
    by perelin
  • > The enable_thinking option is worth mentioning. Without it, the model can spend a surprising amount of the completion budget reasoning before it returns a visible answer.

    Straight up the opposite, which the name makes abundantly clear, with the option it does reasoning, without it it doesn't...

    by embedding-shape
  • My writing wasnt clear here I think, without the option it defaults to reasoning enabled. With "without it" I meant without enable_thinking=False!
    by jonas_scholz
  • Very soon we will be having reseller programs for inference, this will be just like web hosting reseller. After big players, small players will also start entering in this field.

    I'm waiting for that day so that inference will be affordable just like web hosting. 200$ per month is in no way affordable by everyone.

    by pmg1991
  • This seems like a smart move, given their ability to host efficiently. I approve of efforts to make the cost of inference for smaller useful models slowly approach 'close to zero' and there are many good paths for getting there. It is useful for companies to get fast hosting for the class of smaller models they may end up hosting in house.
    by mark_l_watson
  • Problem is right now the biggest GPU boxes they have is single rtx pro 6000s.
    by mips_avatar
  • I really hope they dont stop at the small models though! The bigger ones that dont fit on a single GPU are more interesting I think
    by jonas_scholz
  • Good to see more developments in this space. I quite like this service, which is a little further than Hetzner and has several models to choose from: https://www.infomaniak.com/en/hosting/ai-services
    by ano-ther
  • lurus.ai and cortecs.ai are worth a try too!
    by Aldipower
  • Infomaniak is such a shitty company, I had to use them on a previous job I worked at and dealing with them was awful.
    by archerx
  • Interesting, didn’t know about them! Weird model selection though, no glm or deepseek?
    by jonas_scholz
  • Important note on Infomaniak's offering - when they first launched it, I gave it a try but despite claims that it's OpenAI API compatible, even basic things like sending a base64 encoded image were broken. I reached out to their support, who replied with (verbatim): "Unfortunately, we do not provide support for our AI Service, as the solution is highly unmanaged and uses our API."

    "highly unmanaged" didn't fill me with confidence, and "uses our API" is a very weird reason to give for not offering support. I wrote off ever using them for anything beyond email after that reply.

    by drcongo
  • It would certainly be interesting to have a highly respected EU-native inference provider, if only to make the regulatory gods happy
    by swiftcoder
  • To all mentions of IONOS / StackIT / OVH:

    All of these are lightyears behind US/China LLM offerings. None of them offer any model close to open-source SOTA. Runtime and reliability is a disaster and sure enough you have to pay a "sovereignty" mark up.

    by beernet
  • agree. I have a usecase for a bigger open-weight model hosted by an EU company and the selection isnt really great, hard to make the regulatory gods (and the devs) happy at the same time right now
    by jonas_scholz
  • IONOS (1&1) also offers inference hosted in europe/germany: https://cloud.ionos.com/managed/ai-model-hub
    by someone4958923
  • Scaleway does that already but competition is always good.
    by hoppp
  • Scaleway have two separate (one fully managed one a bit less) services for that:

    https://www.scaleway.com/en/generative-apis/

    https://www.scaleway.com/en/inference/

    by sofixa
  • OVH already provides this → https://www.ovhcloud.com/en/public-cloud/ai-endpoints/catalo...

    It works great.

    by eliaskg