Join the discussion
Write your take first — we'll ask for email only when you're ready to publish.
- Hacker News
- Overall this looks very interesting - I always thought more could have been done with RPython, but there was never any documentation for it. I do have some nits though:
> The following silly example happily passes mypy
In fairness that's because Mypy is shit. Pyright catches this mistake.
> As such, we need to treat Python type checkers more like linters than actual theorem provers
Again I think this is true of Mypy but Pyright is much closer to a sound type checker.
> redshifting
This is just constant propagation isn't it? C compilers have been doing this for decades. I don't think we need a silly new term for it. Let's not cause another "tree shaking" situation.
> So far, this is not different than usual constant folding, with the difference that it's guaranteed to happen. What makes it more powerful is the ability to mark some functions as @blue.
That's not different. It's just constant folding with C++'s `consteval`. And @blue is an absolutely abysmal name.
It would be much clearer if @blue were changed to @consteval or @comptime (either's good I think), and you just call it "compile time evaluation" or "constant propagation" instead of "redshifting".
by IshKebab - Reminds me of Shed Skin, but no mention of it. I wonder how they compare.by kgeist
- It is has been a hot minute and no one has mentioned ChocoPy, which is basically this project. https://chocopy.org/
The author invents so many terms, I don't want to uncharitably ascribe this to anything, but they should use existing terminology.
by seg_lol - The problem with this is that the main value of Python is its ecosystem. SPy aims to be able to import Python libraries, but also not implement all Python features. If you are not 100% compatible how can you reliably import libraries?
SPy seems most likely to be more likely to be appealing as a more Pythonic alternative to Cython rather than a Python replacement.
by graemep - I did a similar project, a typed perl. cperl. I could import most the modules, and did add types to some of the important modules. Eg testing was 2x faster. I needed typing patches for about 10% for most CPAN packages.
A type is a contract, not a hint!
by rurban - hello, author of the blog post and author of SPy here.
> how can you reliably import libraries?
the blog post specifies it but probably not in great level of detail. Calling python libs from spy will go through libpython.so (so essentially we will embed CPython). So CPython will import the library, and there will be a SPy<=>CPython interop layer to convert/proxy objects on the two worlds.
by antocuni - As a prospective user the first thing I look for is lists, list comprehension, list slices, string methods, zip, zip*, tuple destructuring, generators, dicts and sets. These data structures, their syntax, and associated methods are what make Python a joy to use. They are what make it possible to solve a puzzle in a one-liner.
The examples in the playground are neat but I don’t see any of the Python bread and butter usage. I see C with Python syntax.
I’d like to see Python syntax solving a problem in a pythonic way, albeit with type annotations. And then I’d like to see the compiled version run 30x-100x faster. Is that ultimately the promise of this project?
by intalentive - yes (work in progress :))by antocuni
- If in the end we can just have .spy on some files that have performance critical functions in them and the rest is just normal python, this could be down right amazing.
We recently swapped out mypyc optimised module for a rust implementation to get a 2-6x speed up, and not having to do that would be great.
by ic_fly2 - See my other comment in the thread. I would argue that anything that uses arcane dynamic stuff in python should be renamed to .dpy and the vast majority of the commonly used constructs retain .py
The issue in HN threads like this is that everyone is out to promote their own favorite language or their favorite python framework that uses dynamic stuff. The majoritarian and hacker-ethos of python don't always line up.
Like Chris Lattner was saying on a recent podcast, he wrote much of Swift at home on nights/weekends over 18 months. We need someone like that do this for spy.
by adsharma - +1 this.
Also, when I read about the language features which make Python intrinsically slow, generally I think "I never use that." e.g. operator overloading meaning you need to do all these pointer dereferences just to add two numbers together. Yes, I get that pytorch and numpy rely critically on these. But it makes me wonder...
Could you disable these language features on a module-by-module basis? That is, declare "in this sub-tree of the source code, there shalt be no monkey-patching, and no operator overloading" and therefore the compiler can do a better job. If anybody tries to do the disallowed stuff, then it's a RuntimeError. Would that work?
by oofbey - This seems to be going for a somewhat similar goal to Mojo [0] - anyone here who used both and is willing to offer a comparison?by falcor84
- Mojo is such an interesting project but it's disappointing they pivoted so hard to focusing on GPU kernel programming and seemingly abandoned the objective of a general purpose python alternativeby nylonstrung
- There are even older languages with similar goals. At least three are called viper and are older than 6 years. Two linked below and much older one with ocaml-ish features that I could not find a link for.by emmelaich
- Based on my understanding, Mojo aims to make number crunch computation faster (GPU), while as SPy aims to make generic Python application logic faster. Very similar, but different sweet spots and use cases.by miohtama
- Time for me to remind everyone of the Shedskin Python compiler.
- Common Lisp also allows you to redefine everything at runtime but doesn't suffer from the same performance issues that Python has, does it?
Doe anyone have insight into this?
by cardanome - Just like Smalltalk and SELF, also Lisp Machines and Interlisp-D.
Usually comes down from a urban myth that Python is special and there was no other dynamic language before it came to be.
The JIT research on those platforms is what gave us leading JIT capabilities on modern runtimes, OpenJDK HotSpot traces back to Smalltalk and StrongTalk, while V8 traces back to SELF.
Especially in Smalltalk and SELF, you can change anything at any time across the whole image, and have the JIT pick up on that and re-optimize.
Granted what messes up Python, or better said CPython implemenation, is that C extensions are allowed to mess up with its internals thus making void many possible optimizations that would be otherwise available.
A reason why JVM, CLR, V8, ART make use of handles and have marshaling layers not allowing such kind of liberties with native extensions.
by pjmlp - Common Lisp is not a runtime, it’s a specification. Implementations are free to compile everything to fast native code, or to interpret everything. Various available implementations do that and everything in between. That said , SBCL and the commercial implementations can be extremely fast, especially if you specify types on tight loops. SBCL comes with a disassembler that shows you right in the REPL the Assembly a function compiles to so you can even get close to C performance.by brabel
- Common Lisp doesn't use (expensive) CLOS dispatch in the core language, e.g. to add two numbers or find the right equality operator. That's one known pain point due to CLOS having been "bolted-on" rather than part of the language which makes the divide between internal (using typecase and similar) and external (generic functions) dispatch pretty ugly; and gave use the eql/equal/equalp/etc... hell.
Thing is that you need a complex JIT like Julia's or stuff like https://github.com/marcoheisig/fast-generic-functions to offset the cost of constant dynamic dispatch.
I actually had such a conversation on that comparison earlier this year: https://lwn.net/Articles/1032617/
- Neat idea! Author’s ideas about different subsets of Python are worth the price of admission. What you can express in the type system, what performs well under JIT, what’s basically same and reasonable, may not be precisely specified, but are still useful and distinct ideas.by sevensor
- Antonio Cuni gave a great talk about SPy at EuroPython 2024: https://ep2024.europython.eu/session/spy-static-python-lang-...by codethief
- Looks very interesting!
I remember chatting with one of the creators of PyPy (not the author of TFA) a number of years ago at HPI. He had just given a talk about how RPython was used in PyPy development, and I was fascinated.
To me, it seemed completely obvious that RPython itself seemed like a really interesting standalone language, but he would have none of it.
Whenever I suggested that RPython might have advantages over PyPy he insisted that PyPy was better and, more strangely, just as fast. Which was sort of puzzling, because the reason given for RPython was speed. When I then suggested that they could (after bootstrap) just use PyPy without the need for RPython, he insisted that PyPy was too slow for that to be feasible.
The fact that both of these statements could not really be true at the same time did not register.
by mpweiher - I had understood that the only reason for RPython's existence was that bootstrapping was (or at least seemed) impossible without it... ? Although I didn't dig into that claim, either.by zahlman
- I'm not quite seeing the contradiction either? I sort of get that you're pointing out some kind of tension, but it's not obvious that there's a contradiction. The statements involved don't seem to be interpretable in a self-contained way.by getnormality
- I have asked about using RPython as a generic standalone language before. I think the official statement is that is was never intended to become one, and it's really a very minimal subset of Python (so basically no existing Python code will run, it would require heavy refactoring or complete rewrite), and it's only specifically those features that they currently need, and it might also be a moving target, and they don't want to give certain guarantees on stability of the language etc.
Once you consider that you anyway need to write very different kind of code for RPython, then maybe just using Nim or some other language is a better idea?
by albertzeyer - It's weird, but the pain of moving to a whole new language seems less than the pain of using one of these language subsets. I guess it might be because of "loss aversion": we dislike losing something we have more the value we put on gaining it. In a new language each newly added feature is a gain, but in a language subset you're always encountering things you would have in the full language. So no matter how good the rational case is, there is a bias against them. I can't think of a successful one actually - anyone?by ajb
- I would say that Rpython is successful. However, it's primary goal is to be whatever the PyPy developers need to write PyPy.
I totally agree about subsets though, I would much rather have a superset like Cython, but that has it's own challenges for compatibility if you wanted to have a "pure python" version of the same library. Which is why I really like that Cython is using typehints now.
by dec0dedab0de - Throwing my hands up and moving to Nim was downright easy next to the excessive effort I put into trying out Nuitka, Numba, and PyInstaller for my use case. If you want static compilation, use a language and libraries built with that assumption as a ground rule. The herculean effort of building a half-compatible compiler for a dynamic language seems like a fool's errand, and would be a fun curiosity if so many people hadn't already tried it, especially with Python.by netbioserror
- One of the standard architectures for complex applications is to put together a scripting language and an systems programming language. You can either look at the scripting language as primary with the systems language used for performance or hardware interfacing or you can look at the systems language being primary and the scripting being used around the edges.
The model of having a closely related scripting and systems language would be an optimization of this and SPy seems like an answer.
[1] AAA games that have a scripting engine on top of C/C++, a finite element solver that lets you set up a problem with Lua and solve it with FORTRAN, etc.
by PaulHoule