Join the discussion
Write your take first — we'll ask for email only when you're ready to publish.
- Hacker News
- "we almost quit multiple times"
It was a three day small task?
by euph0ria - Probably because they have to deal with customer feedbacks and bugs on existing system. The stress would be huge, making a day like a week long, I guaranteeby novoreorx
- This didn't make sense to me either? If it only took three days for a complete rewrite to another language, what's the problem? Did I read they were getting interrupted for user requests? felt weird.by ehutch79
- I do a lot of glueware and semi-embedded stuff with Python... but my goto these days for anything networky is Elixir (LiveView if ux). If I need an event loop, async that is more than a patched on keyword, it just rocks. It is amazing to me how much Elixir does not have, and yet how capably it solves so many problems that other languages have had to add support for to solve.by travisgriggs
- > At this point, some people are probably screaming at their screens going: "just use FastAPI!" -- and we did indeed consider it.
Working with both sync Django and async FastAPI daily, it’s so easy to screw up async FastAPI and bring things to a halt. If async is such the huge key feature they seem to think it is for their product, then I would agree moving away from Python early while it’s still relatively easy is the right call.
> and we had actually already written our background worker service in Node,
Ok well that’s a little bizarre… why use Django to begin with if you are not going to use the huge ecosystem that comes with it. New Django has first-class support for background workers, not that Celery is difficult to get setup. It’s sounds like the engineering team just started building things in what they knew without any real technical planning and the async hiccup is more or less an excuse to get things in order after the fact.
by move-on-by - > it’s so easy to screw up async FastAPI and bring things to a halt
For example?
by kurtis_reed - I probably would have pushed for Hono as the underlying framework... That said, I've been a fan of Koa/Oak over Express for a very long time. For API usage, the swagger+zod integration is pretty decent, though it changes the typical patterns a bit.
All-in, there's no single silver bullet to solving a given issue. Python has a lot of ecosystem around it in terms of integrations that you may or may not need that might be harder with JS. It really just depends.
Glad your migration/switch went relatively smoothly all the same.
by tracker1 - It depends on your use case. Exactly. If you’re building big data intensive pipelines with lots of array manipulation or matrix multiplications you know what will shine. Building user facing APIs, use something with types and solid async.by davedx
- Django is massive and ton of baggage to be carrying if you are only doing REST APIs.
This sounds like standard case going with what developers know instead of evaluating tool for job.
by stackskipton - I worked at a mid-size startup that was still running on Python 2.7 and Django for their REST APIs, as late as 2022. It was pretty meh and felt like traveling back in time 10 years.by icedchai
- How would you compare this with spring?by e145bc455f1
- you can just run part of django. So the negatives of it being “massive” is really just the size of the library that will just be sitting there on disk. which is really not a big deal in most situations.
As far as going with what you know vs choosing the best tool for the job, that can be a bit of a balancing act. I generally believe that you should go with what the team knows if it is good enough, but you need to be willing to change your mind when it is no longer good enough.
by dec0dedab0de - I think the baggage goes both ways - Django has the advantage of being a "complete & proven recipe" vs. Node where you try to lego-together an app out of dependencies that have deprecation warnings even in their latest versions.by isodev
- "Python async sucks", then rants about django
"Python doesn't have native async file I/O." - like almost everybody, as "sane" file async IO on Linux is somehow new (io_uring)
Anyway ..
by JackSlateur - Sane async does not need io_uring. All the non-linear-time select(2) alternatives are good enough.by antirez
- Python async does suck thoughby morshu9001
- libuv has provided an async interface for io using a worker thread pool for a decade, no dependency on io_uring required. I guess the threadpool they mention that aiofiles uses is written in python, so it gives concurrency but retains the GIL, so no parallelism. Node's libuv async stuff moves all the work off the main thread into c/c++ land until results are ready, only when dealing with the completed data read event does it re-enter the NodeJS "GIL" JavaScript thread.by jitl
- I don't see it mentioned enough in the comments here, but not considering Celery as an alternative to Django + async really is the missing puzzle piece here. Aside from application-level options that weren't explored, I'm wondering whether handling some of the file IO stuff with, for instance, nginx, might be a better fit for their use case.
Once you're in the situation of supporting a production system with some of the limitations mentioned, you also owe it to yourself to truly evaluate all available options. A rewrite is rarely the right solution. From an engineering standpoint, assuming you knew the requirements pretty early on, painting yourself into a bad enough corner to scrap the whole thing and pick a new language gives me significant pause for thought.
In all honesty I consider a lot of this blog post to be a real cause for concern -- the tone, the conflating arguments (if your tests were bad before, just revisit them), the premature concern around scaling. It really feels like they may have jumped to an expensive conclusion without adequate research.
In an interview, I would not advance a candidate like this. If I had a report who exhibited this kind of reasoning, I'd be drilling them on fundamentals and double-checking their work through the entire engineering process.
- Hi! Could you please tell me what use cases would nginx be better for, outside of serving static files?
Moreover, having worked with Django a bit (I certainly don't have as much experience as you do), it seems to me that anything that benefits from asynchrony and is trivial in Node is indeed a pain in Django. Good observability is much harder to achieve (tools generally support Node and its asynchrony out of the box, async python not so much), Celery is decent for long running, background, or fire and forget tasks, but e.g. using it to do some quick parallel work, that'd be a simple Promise.all() is much less performant (serialize your args, put it in redis, wait for a worker to pick it up, etc), doing anything that blocks a thread for a little bit, whether in Django or Celery,is a problem, because you've got a very finite amount of threads (unless you use gevent, which patches stdlib, which is a huge smell in itself), and it's easy to run out of them... Sure, you can work around anything, but with Node you don't have to think about any of this, it just works.
When you're still small, isn't taking a week to move to Node a better choice than first evaluating a solution to each problem, implementing solutions, each of which can be more or less smelly (which is something each of your engs will have to learn and maintain... We use celery for this, nginx for that, also gevent here because yada yada, etc etc), which in total might take more days and put a much bigger strain on you in the long term? Whereas with Node, you spend a week, and it all just works in a standard way that everyone understands. It seems to me that exploring other options first would indeed be a better choice, but for a bigger project, not when the rewrite is that small.
Thank you for your answers!
by ksclk - I tried to use Celery for something extremely trivial (granted, 5+ years ago). It was so badly documented and failed to do basic things I would expect from a task queue (like progress reporting) I have no idea why it was and still is popular.by Kwpolska
- As a long-time Django user, I would not use Django for this. Django async is probably never the right choice for a green-field project. I would still pick FastAPI/SQLAlchemy over Express and PostHog. There is no way 15 different Node ORMs are going to survive in the long run, plus Drizzle and Prisma seem to be the leaders for now.by megaman821
- I'd skip the ORM, Postgres is already designed for direct use on backends.by morshu9001
- Agree
Django is great but sometimes it seems it just tries to overdo things and make them harder
Trying to async Django is like trying to do skateboard tricks with a shopping cart. Just don't
by raverbashing - FastAPI/SQLAlchemy won’t be more scalable than a typical Django setup. The real bottleneck is the threading model, not the few microseconds the framework spends before handing off to user code. Django running under uWSGI with green threads can outperform Go-based services in some scenarios, largely thanks to how efficient Python’s C ABI is compared to Go.by ccanassa
- Same experience working on FastAPI projects. I don’t know how big production apps are maintained (and supported operationally) with the mess that is python+async+types.
Conversely all the node+typescript projects, big and small, have been pretty great the last 10+ years or so. (And the C# .NET ones).
I use python for real data projects, for APIs there are about half a dozen other tech stacks I’d reach for first. I’ll die on this hill these days.
by davedx - > Same experience working on FastAPI projects. I don’t know how big production apps are maintained (and supported operationally) with the mess that is python+async+types.
Very painfully.
I avoid the async libs where possible. I'm not interested in coloring my entire code-base just for convenience.
by all2 - Hell I write ETL pipelines in Typescript since it's just... way easier to deal with. Not doing any crazy ML processing. But the Node ecosystem is giant and I'm a huge fan of the ergonomics of Typescript. And since Typescript is so popular it's very easy for other developers to make changes in my code.by 65
- What's messy about python+async+types?by kurtis_reed
- Yeah I did my first project with FastAPI earlier this year with experience in other languages and I couldn't believe how bad it was.by simlevesque
- >Same experience working on FastAPI projects. I don’t know how big production apps are maintained (and supported operationally) with the mess that is python+async+types.
In my experience async is something that node.js engineers try to develop/use when they come from node.js, and it's not something that python developers use at all. (with the exception of python engineers that add ASGI support to make the language enticing to node developers.)
by TZubiri