Join the discussion
Write your take first — we'll ask for email only when you're ready to publish.
- Hacker News
- I’ve tried so hard to figure out what the hell oxide even is, but the website is utterly impenetrable. I guess it’s an “integrated platform” and does “AI” and it’s “purpose built for frontier workloads”, but… I have absolutely no idea what it is, still.
I guess it’s a bunch of computers. It says “AMD” in a bunch of places so, I guess they’re x86. Where are the GPU’s? If it’s for “frontier workloads”, wouldn’t those be sorta important? Is it its own OS? No idea. I see people adjacent to Oxide mention IllumOS sometimes[0], so does that mean it’s a solaris-like OS? Why would I want to run k8s (presumably with linux containers) on a not-linux OS? Or does it virtualize linux instances?
Do actual CTO’s go for this type of marketing, with zero details and nothing but pure fluff about “solutions”?
[0] This of course is from HN comments, I see no mention of any operating system whatsoever on the website, so if it is IllumOS, it’s not like you can easily find that info anywhere.
by ninkendo - It's a private cloud. I guess you might not be a target customer.by zdkaster
- I wrote this back in 2022, but it's still relevant https://news.ycombinator.com/item?id=30678324by steveklabnik
- Here's the docs:by esseph
- It's a rack-sale computer that uses hardware and software co-design to achieve things a stack of disparate 1U servers and network switches can't. You plug in power. You connect it to your network. You get a cloud provider like control plane API and web console to provision VMs, disks, and VPCs. That's it papa.
The host OS is an implementation detail since you, the customer, aren't running workloads directly on the host OS. The VMs running on Oxide run on our host OS, much like other cloud providers.
That rack-scale hardware and software co-design allows Oxide to provide higher CPU and memory density in the same footprint at lower power than competitors. That's super helpful for companies operating at scale where data center power matters. The co-design allows us tackle security problems by eliminating the BIOS, providing attestation from the firmware up to the guest VM, and actually updating the firmware that's on Oxide rather than letting it rot like most customers do today.
This is why when people ask which verticals Oxide targets we kinda say "all of them" because different customers benefit from different Oxide features, but all customers need the core VM, disk, VPC abstraction. Some customers come to us because they don't have an API to manage their on-premises compute and Oxide solves that. Others come to us because they need absolute confidence that there's no malicious firmware running in their compute stack. Others come to us because they are tired of paying exorbitant amounts of recurring money just to run on-premises compute.
Our job is to make Oxide an appealing on-premises computing platform for customers to run their public cloud provider workloads and on-premises workloads without it feeling like it's an entirely different platform than you're used to.
by sudomateo - How are folks managing stateful workloads like databases on Oxide?by redwood
- I cover this a bit in the post. Today they are running something like Longhorn backed by Oxide storage. When we release our CSI plugin they can use that instead.by sudomateo
- I am wondering when would you use kubernetes on oxide vs running kubernetes with kubevirt on baremetal?
At first look, it feels like oxide is equivalent to proxmox or some virtualization tool, may be it uses qemu stuff underneath.
Just curious.
The reason I am asking this is that, we have lot onprem scenarios in our business. We are tightly coupled with k8s, to solve this we started building an internal project that is kubernetes API compatible [1] but runs containerd or WASM or our platform natively.
Just curious how oxides work in this scenario
by bitlad - When oxide provide your metal?by e12e
- Yeah me too. Like I kinda compare oxide to coreweave and coreweave is built on top of k8s afaik. I'm wondering what the underlying tech oxide leans on. If I was doing this I would be using talos and k8s on baremetal and building on top of that. Container first rather than VM first seems like a big advantage imo. Most workloads will be containers.by bakies
- Oxide is shipping the baremetal server as well, with integrated hypervisor - so running kubernetes with kubevirt on it makes less sense than integrating with the host hypervisorby p_l
- You would use Oxide if you hate Dell/HPE/Lenovo/Supermicro. Also most people won't run k8s on bare metal because they want to dynamically provision a bunch of clusters.by wmf
- Good question. We use our own hypervisor[0] that's not KVM/QEMU. We don't have nested virtualization today so we don't follow the KubeVirt model, though we are discussing what CRDs like OxideInstance would look like for those that want to operate solely in Kubernetes manifests.
The core primitive on Oxide is the instance (virtual machine). We could support some container primitive, but that's a larger product direction discussion. Our host OS is Helios (Illumos) so there are details to iron out there regarding what abstractions we would build and expose to the users. Not impossible but not something we're currently pursuing either given that we have other immediate product asks.
If you're at a scale where compute density, power efficiency, security, and rack-level API management matters then that's where Oxide makes sense for you.
by sudomateo - So you’d be attaching a new volume to the running worker VM for each PVC? This seems a little odd to me. Could you attach a single large volume, and do path-based provisioning on that?by wolttam
- Does seem like that could be an issue for RWX PVCs, unless those volumes can be mounted to multiple nodes. Depending on the workload, you might be better off using NFS.by itintheory
- > So you’d be attaching a new volume to the running worker VM for each PVC?
That's what we prototyped before local disk was released and before we started disk hot-plug work.
> Could you attach a single large volume, and do path-based provisioning on that?
Possibly. We'd still want disk hot-plug first. Otherwise, customers would have to create their cluster in a certain shape before using PVCs.
by sudomateo - Bug report I guess, but the pages top navigation seems completely inoperable to me on my iPad. Clicking with my finger does not work. Clicking with my Apple Pencil does not work. Any sort of hover state that might be there does not work (with the pencil)by donatj
- Reported internally. Thank you!by sudomateo
- Kubernetes is where Oxide, from my armchair, doesn't yet quite match the public cloud.
On AWS EKS Fargate, each pod runs in its own dedicated VM. With Oxide each k8s node is a VM, so you still need something like Talos.
On networking it looks like it is getting closer. Where you can have external subnet give pod routed IPs without overlay. But the gap, as marked by the article, is also the load balancing.
It would be nice if Kubernetes were a native feature out-of-the-box. Also integrated within the existing user/access control.
by whazor - We're discussing what "native Kubernetes" looks like on Oxide in the limit. There's a bunch to build, some of which is blocked on product gaps. We'll get there though!
I don't necessarily want to match the public cloud experience if there's an opportunity for Oxide to exceed the public cloud experience. Eliminating the overlay is a good example of this. We have customers using external subnets to eliminate the overlay but we haven't integrated that into our controllers yet.
by sudomateo - Love to see the CAPOx provider and buy in to Cluster APIby moondev
- Yeah me too, unfortunately I cant justify huge 100K USD servers so ill continue with cluster api provider hetzner and metal3. The whole CAPI ecosystem is quite cool.by preisschild
- Disclaimer: I'm totally biased here.
I talked with a colleague from Oxide in 2024 about your Kubernetes story and he said back then "not yet but soon-ish". Seems like soon-ish is now :)
We said we'd talk again when that happens but he's since left Oxide. If you (or well...your customers) are interested in a Kubernetes native data platform 100% open source we'd be very happy to talk about how that could work easily. As it's "just" Kubernetes it should be trivial but we'd be happy to test and add you to our list: https://docs.stackable.tech/home/stable/kubernetes/#supporte...
The offer stands. If you're interested, my mail is in my HN profile. https://stackable.tech
by lars_francke - Great post.
ClusterAPI never got the love that it should. I spent a good amount of time on it at VMware, as Tanzu heavily leverages it for cluster deployment (mostly CAP-A and CAP-V). It's basically kubeadm + the spirit of Terraform, Kubernetes controller edition. There's lots of great enterprise-ready options for centralized k8s cluster fleet management these days, but it works really well for folks that are all-in on GitOps and such.
by nunez - I would absolutely kill for them to open source their documentation system.by overflowy
- Are you referring to our RFD system?[0][1] Because if so, good news![2]
[0] https://rfd.shared.oxide.computer/rfd/0001
[1] https://oxide-and-friends.transistor.fm/episodes/rfds-the-ba...
by bcantrill - I have seldom wanted anything as much as I want an Oxide rack at home. Maybe in 40 years we'll start to see the first ones show up in surplus auctions...by pianoben
- fwiw, I am very satisfied with talos and k8s at home.by bakies
- Every single Oxide article has comments about people wanting one at home.
A great example of completely misdirected marketing and/or engineering. Oxide should have developed home microcomputers, and they would have sold like cupcakes, with their ASCII art marketing. Instead they do million dollar "mainframes" that no one (except VCs) wants to buy.
by lkasjdas - You can run the control plane at home. I have a video on how to do it. Until we make a smaller footprint that's the only way you'd get Oxide at home.by sudomateo