Join the discussion
Write your take first — we'll ask for email only when you're ready to publish.
- Hacker News
- I will try this thing. if give free month
- Hi danger dingo,
Sure, sign up and ping me in discord
by yolo-auto - What type of hardware, GPU are you utilizing? I have the same model self hosted at homeby lobster45
- 1x MI300x , rocm/sglang , highly optimized image, FP8, FP16 Cache (this fixes looping issues with qwen3.6-35b
~100tok/s average
by yolo-auto - I wanted to start using this but I can't since I can't use any email/password combination I want. By requiring an existing account from Google/GitHub/Discord you are just contributing to corporate-owned dystopia.by atmanactive
- i know, but im scared to own our own auth, when there's money involved... too cheap to pay for a service. So we lean on the back of the giants for now.
If it helps, there's really no reason to ever log into our site after you get an API Key. I guess until you want to make another api key
by yolo-auto - Got me down a rabbit hole of trying to find the ROI of doing something like this. This is what every GM at AWS thinks about. So, a back of the napkin calculation of running at FP8 and approx 35GB. You can fit that on a g6e.xlarge @ 48GB for about $900/year at reserved capacity.
Assuming you are netting at $5.5 (post stripe fees), and maybe $100/month for API/auth box for egress, you probably need about 170 paying customers to breakeven. This is assuming light/bursty usage.
Kudos! This is a a win win for both you and customer using it as it removes the undifferentiated heavy lifting.
by anoop_kumar - actually the math is more like $1500 a month on a MI300x on a highly optimized rocm/sglang image and everyone's getting 100+ tok/sec and we have 100 active people and plenty of room for others.
but were losing money right now obviously.
we did come from 4x 3090s could actually make money, but we thought a couple hundred bucks a month isnt worth a 100 degree office.
People are dumping 128k cold documents in like its their job. Some dudes hit 2 billion + tokens a week.
its been fun though. you should check it out
by yolo-auto