Join the discussion
Write your take first — we'll ask for email only when you're ready to publish.
- Hacker News
- Very interesting article. I'm looking to get my first robot arm soon-ish, probably something in the SO-101 category. Can someone get reasonably far using recorded sessions compared to training in a simulated environment¹? Do you have any experience with third-party or DIY attachments for robot arms? I assume it's going to be more difficult for something like the Ufactory arm vs. the open models.by dracotomes
- This is so cool and looks so fun. I want to play with robots. I just play with software :(
I wanted to build something making bows automatically and damn, pretty complex and expensive to buy the parts.
I was like, I can probably just buy a 3D printer, print all the parts and buy some motors, but it seems it's way more complicated than that.
Like to play with a single hand robot. It looks like you need 10k+$. I wanted to spend max 1k, 3D print parts, buy motorized parts on Alibaba, and code on my Mac + spare GPUs I have access to. I'll have to save a little :(
by Frannky - Ah nice I've tinkered a bit with robotics few years ago, e.g. https://twitter-archive.benetou.fr/utopiah/status/1760260544... and it's pretty empowering... but also showcases how amazing the human body is. Our dexterity, ability to sense, etc.
Reminds me of https://rodneybrooks.com/why-todays-humanoids-wont-learn-dex... which is basically a stark warning against the hype.
by utopiah - Really cool. Good move on writing your own stack. I was weighing using ros2 or building our own, but ended up going the ros2 route to “save time”. We are working with autonomous mobility robots in nature, so we figured the ecosystem around ros2 would be worth the compromise. It was not. All the time we saved setting things up initially, we are paying for now.
There should exist a minimal, clear, robotics library like what you’ve built. The Flask of the robotics world.
by delbronski - Hey this is cool! I am doing something similar myself with the SO101 arm robot from Robot Studio using a patchwork of my own code and LeRobot. Would love to collaborate with you if you are open to it. You can find me on Discord as `.avilay`. https://www.linkedin.com/posts/avilay_lerobot-huggingface-ro...
Would like to know your reasoning on not going with LeRobot.
by avilay - - A single arm is sufficient for validating basic Pick/Place tasks, but more complex scenarios require Bi-arm
- Calibration is not required for VLA models.
- RGB or Stereo RGB inputs are sufficient for ACT, DP, and PI0/PI05.
- ROS2 is not strictly required, but it can be useful for sharing/co-developing codes. For instance, the Stanford team built a custom framework for diffusion policy instead. I also developed similar framework because ROS2 is not optimized for bi-manual manipulation or VLA workloads.
by b89kim - Good move getting a nice robot! I'm doing something similar to you, but I went with a cheap robot, the "HIWONDER 6DOF Robotic Arm Kit". It was only $600, but wow is it bad. The precision and repeatability are both "are you drunk?" level. I can hear the gears grind when it moves. I suppose I should upgrade. But if my system can work with a terrible robot, I assume it would work even better with a nice one!
My project is https://github.com/colinator/Ariel - basically, no VLAs - instead, "just write code". Or have the agents do it.
I don't have a writeup yet about applying Ariel to _this_ robot, but this is for a previous one: https://colinator.github.io/Ariel/post1.html.
Excited to follow your progress!
by colinator - Cool stuff. At my previous (startup / research) job I had to set up similar system (but with franka arm and multi view camera) alone because I was the only one with robotics background.
>I do not intend to calibrate the camera’s extrinsics or intrinsics for now.
Sensible choice, although I suggest it's good in the long run to do at early stage in your setup, especially if you intend to collect data for policy learning.
Debugging trained policies for visual manipulation task can be a headache and having as much context on variables that can change is a good practice.
My previous setup was in Japan, a earthquake prone place and I wasted some time after realizing the camera got misaligned due to earthquake. A simple solution is just to place an Aruco marker on the table that tracks the relative extrinsic position of camera, and add it as metadata to collected teleoperation dataset.
by NalNezumi