Seems like a good time to throw out a reminder regarding "Roads and Bridges: The Unseen Labor Behind Our Digital Infrastructure" by Nadia Asparouhova. While she may have published it in 2016, it's still relevant today and speaks to the need for the private sector generally (looking at you VC firms) to support and understand the open source work, hours of unfunded labor, powering our societies.
Really simple fix: social pressure and expectations should be that every company that uses open source pays a fixed amount of their revenue (is 0.1% low enough to be negligible for the companies). Companies that don't should shunned.
To a large extent they do and always have. It's not as broad or fair as it should be[1], but for almost any economically important project all the major contributors and maintainers are on the payroll of one of the big tech interests or a foundation funded by them.
The hippies writing that software may not be compensated at the level you'd expect given the value they provide, but they'll never go hungry.
[1] LLVM and Linux get more cash than they can spend. GNU stuff is comparatively impoverished because everyone assumes they'd do it for free anyway. Stuff that ships on a Canonical desktop or RHEL default install gets lots of cash but community favorites like KDE need to make their own way, etc... Also just to be clear: node is filled with povertyware and you should be extremely careful what you grab from npm.
> but for almost any economically important project all the major contributors and maintainers are on the payroll of one of the big tech interests or a foundation funded by them.
"almost" is the load bearing word here, and/or a weasel word. Define what an "economically important project" is.
> Also just to be clear: node is filled with povertyware and you should be extremely careful what you grab from npm.
Is "povertyware" what we call software written by people and released for free now?
> "almost" is the load bearing word here, and/or a weasel word. Define what an "economically important project" is.
Linux, clang, python, react, blink, v8, openssl... You know what I mean. I stand by what I said. Do you have a counterexample you think is clearly unfunded? They exist[1], but they're rare.
> Is "povertyware" what we call software written by people and released for free now?
It's software subject to economic coercion owing to the lack of means of its maintainership. It's 100% fine for you to write and release software for free, but if a third party bets their own product on it they're subject to an attack where I hand you $7M to look the other way while I borrow your shell.
[1] The xz-utils attack is the flag bearer for this kind of messup, obviously.
This makes sense given how much of the current AI ecosystem is built on top of Python. I hope this helps the foundation improve security for everyone who relies on these libraries.
For anyone who isn’t aware/remembering, this is certainly made with the security of PyPi in mind, python’s main package repository.
NPM is the other major source of issues (congrats for now, `cargo`!), and TIL that NPM is A) a for-profit startup (??) and B) acquired by Microsoft (????). In that light, this gift seems even more important, as it may help ensure that relative funding differences going forward don’t make PyPi an outsized target!
(Also makes me wonder if they still have a Microsoft employee running the PSF… always thought that was odd.)
AFAIU the actual PSF development team is pretty small and focused on CPython (aka language internals), so I’m curious how $750,000/year changes that in the short term…
EDIT: there’s a link below with a ton more info. This gift augments existing gifts from Amazon, Google, Microsoft, and Citi, and they soft-commit to a cause:
Planned projects include creating new tools for automated proactive review of all packages uploaded to PyPI, improving on the current process of reactive-only review. We intend to create a new dataset of known malware that will allow us to design these novel tools, relying on capability analysis.
Poor management has played a role. They refused to invest in packaging to the extent that a separate company (astral) had to do it for them. Bugs closed for years with the excuse “we’re only volunteers.” Meanwhile, “outreach” was funded for several million a year. Not confidence inspiring. Maybe would have improved if the funds had been spent more appropriately.
Where are you getting these numbers? Looking at the PSFs Report for 2024 [0], 50% of their expenses went to pycon. Would you consider that outreach? I believe conferences are very important as part of the health of a language, and reading the definition of outreach[1], I would not classify the conference as that. The second highest amount of expenses (27.1%) went to (surprise!) "Packaging Work Group/Infrastructure/Other", i.e. pypi, pip etc... "Outreach & Education" was only 2.8% of 12.9% of expenses, i.e. 0.3612%, which is $17846 (actual dollars, not thousands like in the report.)
The assertions above are my memory from pre-covid, I’d look at 2019 perhaps. Many things changed after that and council but it takes a while to change perception.
I don't know much about the Linux Foundation if I'm being honest, even though I've been a 24/7 Linux user for decades, but they seemingly don't have the same image in the ecosystem, at least not close to how people see Mozilla today.
Why is that? Is there lessons to be learned from the Linux Foundation how to actually effectively and responsibly manage that sort of money, in those types of projects?
Kinda crazy that the top level "Visionary Sponsor" is a donation level of $160k. There's also 0 sponsors at the $100k level. I was also surprised to see Netflix at $5k and Jane Street at $17k. Maybe they should give more but there's a lot of names absent and that says more
Is it so hard to imagine that they do it because the PSF's work is important and they want to support them? All the AI labs depend hugely on the Python ecosystem and infrastructure. Startups burning cash spend on many things that are important to them.
My wife's previous job was as an accountant with the endowment foundation at a mid-sized public university (San Jose State University). A lot of her time was spent making sure that the spending from the endowments many different funds corresponded to the rules that the donors had given when donating that money. Much of that was working with groups to shift spending around between accounts when they invariably made "mistakes".
One of her biggest projects was shepherding a large group of very old donations through a legal process to remove provisions in the donation agreements that were now illegal. In these cases the donors were long deceased, and the most common rule that needed to be changed was targeting race or ethnicity (e.g.: funds setup to help black people, or Irish, etc...).
The sheer number of different variations on "donor intent", or even just the wording on that legal document was astounding. There was always a tension between my wife's group and the group that was bringing in the money ("stewardship"), her group wanted things to be simpler and the "stewarding" group wanted nothing to get in the way of donations. It was remarkably similar to the tensions between sales and engineering in many software firms.
Of course you can. The vast majority of donations of this magnitude come with strings attached, be it how the money is spent, access to leadership/events, etc
It's super common with non-profits. Obviously they would prefer no strings attached but some light strings are usually not a problem for most non-profits.
And they come in a variety of bindingness. I didn’t notice any details in this link which makes me think this is mostly a handshake deal, but it wouldn’t be at all unusual for there to be some auditing mechanisms on a quarterly/yearly cycle.
For example, Wikimedia just recently claimed that they can’t chase some political project that critics wanted them to because most of their funds are earmarked-for/invested-in specific projects. So it does happen with US-based tech non-profits to at least some extent.
The vast majority of donations to, say, universities are made with a specific purpose, and that happens with a lot of non-profits too. The recipient doesn't have to accept the donation, of course, but if they do they track exactly how it was spent.
Which seems intellectually frustrating. The python foundation was only short money because they refused to accept a 1.5 million dollar federal grant from the Trump admin for political reasons (I believe a condition of the money was it couldn't be used for DEI). They have now received 1.5 million from Anthropic, which is VC funded and burning cash.
I find these matters are often more complex than I can understand from a headline but this feels like Anthropic bailed out the PSF because PSF is making bad management decisions, and bailing them out might be a bad long-term play.
Just to clarify: the NSF grant was refused because it required the PSF to abandon all DEI efforts, not just that the grant itself couldn't be used for DEI. Accepting the NSF grant would have required the PSF to forgo one of its core principles. It was the right decision, not bad management.
I don't agree that it was a "bad management decision". The Trump administration has demonstrated that it will play dirty with grants if they perceive that the receiving organization is not towing their political line as closely as they want.
Not only will they not grant future funds, but they have shown that they will not pay out previously agreed monies, and will even try (with government layers) to pull back funds from groups they have decided "do not align with the governments interests", for however they define that at that moment. There are a long list of court findings that these have been arbitrary and capricious, but every one of those findings (wins) cost the grant receivers a lot of money in court and later fees.
So any money taken from them is incurring a risk. You can disagree with the Python Foundation's calculus on this (saying it was not that large a risk), but please don't pretend that it was not an actual risk.
Just recently I heard that they can donate to “typed languages” too, a donation to one language does’t preclude other donations, and given their cash injections they have a few $1.5m’s to spare.
For any programming really, but I think Python got big due to
a) the huge influx of beginners into IT,
b) lots of intro material available in Python and
c) having a simple way to run your script and get feedback (same as PHP)
I say that as someone urging people to look beyond Python when they master the basics of programming.
Python has a terseness that is hard to rival. I think that was a major selling point: its constructs and use of whitespace mean that a valid Python program looks pretty close to the pseudo-code one might write to reason out the problem before writing it in another language.
Python type hints are static - at the moment, they are advisory only, but there is an obvious route forward to making Python an (optionally) fully statically typed language by using static type checking on programs before execution.
Yes then you might as well use some other language that uses types but also gets you performance. I agree the ecosystem is missing but hey we have LLMs now
Performance isn’t the only important metric. There are other pros to weigh. For many apps a language might be performant enough, and bring other pros that make it more appealing than more performant alternatives.
> Python type hints allow checking correctness statically
Not really. You can do some basic checking, like ensuring you don't pass a string into where an integer is expected, but your tests required to make sure that you're properly dealing with those integers — of which Python's type hints are not nearly capable enough to forgo such tests — would catch that anyway. The LLM doesn't care if the error comes from a type checker or test suite.
When you get into real statically typed languages there isn't much consideration for Python. Perhaps you can prompt an LLM to build you an extractor, but otherwise, based on what already exists, your best bet is likely Lean extracted to C, imported as a Python module.
If you are satisfied with the SMT middle-ground, Dafny does support Python as a target. But as the earlier commenter said: Types are best.
For a lot of the business world, code flexibility is much more important than speed because speed is bottlenecked not on the architecture but on the humans in the process; your database queries going from two seconds to one second matters little if the human with their squishy eyeballs takes eight seconds to digest and understand the output anyway. But when the business's needs change, you want to change the code supporting them now, and types make it much easier to do that with confidence you aren't breaking some other piece of the problem domain's current solution you weren't thinking about right now (especially if your business is supported by a team of dozens to hundreds of engineers and they each have their own mental model of how it all works).
Besides... Regarding performance, there is a tiny hit to performance in Python for including the types (not very much at all, having more to do with space efficiency than runtime). Not only do most typed languages not suffer performance hindrance from typing, the typing actually enables their compilation-time performance optimizations. A language that knows "this variable is an int and only and int and always an int" doesn't need any runtime checks to confirm that nobody's trying to squash a string in there because the compiler already did that work by verifying every read and write of the variable to ensure the rules are followed. All that type data is tossed out when the final binary gets built.
If your code is talking to an LLM, the performance difference between rust and python represents < 0.1% of the time you spend waiting for computers to do stuff. It's just not an important difference.
It's true; mypy won't make your Python faster. To get something like that, you'd want to use Common LISP and SBCL; the SBCL compiler can use type assertions to actually throw away code-paths that would verify type expectations at runtime (introducing undefined behavior if you violate the type assertions).
It's pretty great, because you can run it in debug mode where it will assert-fail if your static type assertions are violated, or in optimized mode where those checks (and the code to support multiple types in a variable) go away and instead the program just blows up like a C program with a bad cast does.
Why is this getting downvoted... it is true. Also it is true that dynamic languages (like Ruby ;) and Python) are more efficient with tokens, like significantly then types like C, C++ or such. But Javascript and Typescript are using twice the tokens of Ruby for example and Clojure is even more efficient, obviosly I would add.
AFAICT Python basically is a [statically-]typed language nowadays. Most people are using MyPy or an alternative typechecker, and the community frowns on those who aren’t.
> Most people are using MyPy or an alternative typechecker, and the community frowns on those who aren’t.
That's not like a widespread/by-default/de-facto standard across the ecosystem, by a wide margin. Browse popular/trending Python repositories and GitHub sometime and I guess you can see.
Most of the AI stuff released is still basically using conda or pip for dependencies, more times than not, they don't even share/say what Python version they used. It's basically still the wild west out there.
Never had anyone "frown" towards me for not using MyPy or any typechecker either, although I get plenty of that from TS fans when I refuse to adopt TS.
> Never had anyone "frown" towards me for not using MyPy or any typechecker either
I’ve seen it many times. Here’s one of the more extreme examples, a highly-upvoted comment that describes not using type hints as “catastrophically unprofessional”:
But yeah, that's reddit, people/bots rejoice over anything being cargoculted there, and you really can't take any upvote/downvote numbers on reddit seriously, it's all manipulated today.
Don't read stuff on reddit and use whatever you've "learned" there elsewhere, because it's basically run by moderators who try to profit of their communities these days, hardly any humans left on the subreddits.
Edit: I really can't stress this enough, don't use upvotes/likes/stars/whatever as an indicator that a person on the internet is right and has a good point, especially not on reddit but I would advice people to not do so on HN either, or any other place. But again, especially on reddit, the upvotes literally count for nothing. Don't pick up advice based on upvoted comments on reddit!
Generally you only get frowned at if you're not using type hints while contributing to a project whose coding standards say "we use type hints here."
If you're working on a project that doesn't use type hints, there's also plenty of frowning, but that's just because coding without a type checker is kind of painful.
> Generally you only get frowned at if you're not using type hints while contributing to a project whose coding standards say "we use type hints here."
Yeah, that obviously makes sense, not following the code guidelines of a project should be frowned upon.
I think in the case of TS, it's more that JavaScript itself is notoriously trash (I'm not being subjective; see https://www.destroyallsoftware.com/talks/wat), and TypeScript helps paper over like 90% of the holes in JavaScript.
Python typed or untyped feels like a taste / flexibility / prototyping tradeoff; TypeScript vs. JavaScript feels like "Do you want to get work done or do you want to wrap barbed wire around your ankle and pull?" And I say this as someone who will happily grab JS sometimes (for <1,000 LOC projects that I don't plan to maintain indefinitely or share with other people).
Plus, TypeScript isn't a strict superset of JavaScript, so choice at the beginning matters; if you start in JS and decide to use TS later, you're going to have to port your code.
Typed Python vs untyped Python is literally the same as TS vs JS, don't let others fool you into thinking somehow it's different.
> TypeScript helps paper over like 90% of the holes in JavaScript
Always kind of baffles me when people say this, how are you actually programming where 90% of the errors/bugs you have are related to types and other things TS addresses? I must be doing something very different when writing JS because while those things happen sometime (once or twice a year maybe?), 90% of the issues I have while programming are domain/logic bugs, and wouldn't be solved by TS in any way.
It's a pretty nice best-of-both-worlds arrangement. The type information is there, but the program still runs without it (unless one is doing something really fancy, since it does actually make a runtime construct that can be introspected; some ORMs use the static type data to figure out database-to-object bindings). So you can go without types for prototyping, and then when you're happy with your prototype you can let mypy beat you up until the types are sound. There is a small nonzero cost to using the types at runtime (since they do create metadata that doesn't get dropped like in most languages with a static compilation step, like C++ or TypeScript).
I can name an absolute handful of languages I've used that have that flexibility. Common LISP comes to mind. But in general you get one or the other option.
> It's a pretty nice best-of-both-worlds arrangement
It’s also a worst-of-both-worlds arrangement, in that you have to do the extra work to satisfy the type checker but don’t get the benefits of a compiled language in terms of performance and ease-of-deployment, and only partial benefits in terms of correctness (because the type system is unsound).
AFAIK the Dart team felt this way about optional typing in Dart 1.x, which is why they changed to sound static typing for Dart 2.
I must be the only one in here who thinks $1.5M is a small sum compared to Anthropic's size and the amount of value they have gotten out of Python. Good press is cheaper than I thought.
Every single financial institution on Wall Street, the City of London, Amsterdam, Tokyo, Dubai and so on, uses Python. Very few contribute.
I've worked at a few that use the 'mold' linker to dramatically reduce their build times. Again, very few contribute. In this particular case, I managed to get one former employer to make a donation.
that was my first thought too, $1.5M is peanuts for Anthropic, however $1.5M is better than nothing, so it worth some PR too. Good they do, I think we have to encourage companies to do it, shaming will not help.
https://www.fordfoundation.org/learning/library/research-rep...
The hippies writing that software may not be compensated at the level you'd expect given the value they provide, but they'll never go hungry.
[1] LLVM and Linux get more cash than they can spend. GNU stuff is comparatively impoverished because everyone assumes they'd do it for free anyway. Stuff that ships on a Canonical desktop or RHEL default install gets lots of cash but community favorites like KDE need to make their own way, etc... Also just to be clear: node is filled with povertyware and you should be extremely careful what you grab from npm.
"almost" is the load bearing word here, and/or a weasel word. Define what an "economically important project" is.
> Also just to be clear: node is filled with povertyware and you should be extremely careful what you grab from npm.
Is "povertyware" what we call software written by people and released for free now?
Linux, clang, python, react, blink, v8, openssl... You know what I mean. I stand by what I said. Do you have a counterexample you think is clearly unfunded? They exist[1], but they're rare.
> Is "povertyware" what we call software written by people and released for free now?
It's software subject to economic coercion owing to the lack of means of its maintainership. It's 100% fine for you to write and release software for free, but if a third party bets their own product on it they're subject to an attack where I hand you $7M to look the other way while I borrow your shell.
[1] The xz-utils attack is the flag bearer for this kind of messup, obviously.
EDIT: or are you rather thinking about the book Working in Public: The Making and Maintenance of Open Source Software?
From a 2022 email:
> (P.S. I have a new last name! Still transitioning everything over, but I’m now Nadia Asparouhova.)
Here the website of the author: https://nadia.xyz/
NPM is the other major source of issues (congrats for now, `cargo`!), and TIL that NPM is A) a for-profit startup (??) and B) acquired by Microsoft (????). In that light, this gift seems even more important, as it may help ensure that relative funding differences going forward don’t make PyPi an outsized target!
(Also makes me wonder if they still have a Microsoft employee running the PSF… always thought that was odd.)
AFAIU the actual PSF development team is pretty small and focused on CPython (aka language internals), so I’m curious how $750,000/year changes that in the short term…
EDIT: there’s a link below with a ton more info. This gift augments existing gifts from Amazon, Google, Microsoft, and Citi, and they soft-commit to a cause:
Similar story with Mozilla.
[0] https://www.python.org/psf/annual-report/2024/ [1] https://en.wikipedia.org/wiki/Outreach
Why is that? Is there lessons to be learned from the Linux Foundation how to actually effectively and responsibly manage that sort of money, in those types of projects?
But also they rely heavily on Python and want to support the ecosystem.
One of her biggest projects was shepherding a large group of very old donations through a legal process to remove provisions in the donation agreements that were now illegal. In these cases the donors were long deceased, and the most common rule that needed to be changed was targeting race or ethnicity (e.g.: funds setup to help black people, or Irish, etc...).
The sheer number of different variations on "donor intent", or even just the wording on that legal document was astounding. There was always a tension between my wife's group and the group that was bringing in the money ("stewardship"), her group wanted things to be simpler and the "stewarding" group wanted nothing to get in the way of donations. It was remarkably similar to the tensions between sales and engineering in many software firms.
For example, Wikimedia just recently claimed that they can’t chase some political project that critics wanted them to because most of their funds are earmarked-for/invested-in specific projects. So it does happen with US-based tech non-profits to at least some extent.
> $1.5 million over two years would have been quite a lot of money for us, and easily the largest grant we’d ever received.
If you missed it, they bought Bun a while back, which is what Claude Code is built in: https://bun.sh/blog/bun-joins-anthropic
I find these matters are often more complex than I can understand from a headline but this feels like Anthropic bailed out the PSF because PSF is making bad management decisions, and bailing them out might be a bad long-term play.
Not only will they not grant future funds, but they have shown that they will not pay out previously agreed monies, and will even try (with government layers) to pull back funds from groups they have decided "do not align with the governments interests", for however they define that at that moment. There are a long list of court findings that these have been arbitrary and capricious, but every one of those findings (wins) cost the grant receivers a lot of money in court and later fees.
So any money taken from them is incurring a risk. You can disagree with the Python Foundation's calculus on this (saying it was not that large a risk), but please don't pretend that it was not an actual risk.
Drive-by insinuation rather than argumentation.
This is a morally depraved condition, kudos on them for turning it down
Not really. You can do some basic checking, like ensuring you don't pass a string into where an integer is expected, but your tests required to make sure that you're properly dealing with those integers — of which Python's type hints are not nearly capable enough to forgo such tests — would catch that anyway. The LLM doesn't care if the error comes from a type checker or test suite.
When you get into real statically typed languages there isn't much consideration for Python. Perhaps you can prompt an LLM to build you an extractor, but otherwise, based on what already exists, your best bet is likely Lean extracted to C, imported as a Python module.
If you are satisfied with the SMT middle-ground, Dafny does support Python as a target. But as the earlier commenter said: Types are best.
For a lot of the business world, code flexibility is much more important than speed because speed is bottlenecked not on the architecture but on the humans in the process; your database queries going from two seconds to one second matters little if the human with their squishy eyeballs takes eight seconds to digest and understand the output anyway. But when the business's needs change, you want to change the code supporting them now, and types make it much easier to do that with confidence you aren't breaking some other piece of the problem domain's current solution you weren't thinking about right now (especially if your business is supported by a team of dozens to hundreds of engineers and they each have their own mental model of how it all works).
Besides... Regarding performance, there is a tiny hit to performance in Python for including the types (not very much at all, having more to do with space efficiency than runtime). Not only do most typed languages not suffer performance hindrance from typing, the typing actually enables their compilation-time performance optimizations. A language that knows "this variable is an int and only and int and always an int" doesn't need any runtime checks to confirm that nobody's trying to squash a string in there because the compiler already did that work by verifying every read and write of the variable to ensure the rules are followed. All that type data is tossed out when the final binary gets built.
It's pretty great, because you can run it in debug mode where it will assert-fail if your static type assertions are violated, or in optimized mode where those checks (and the code to support multiple types in a variable) go away and instead the program just blows up like a C program with a bad cast does.
That's not like a widespread/by-default/de-facto standard across the ecosystem, by a wide margin. Browse popular/trending Python repositories and GitHub sometime and I guess you can see.
Most of the AI stuff released is still basically using conda or pip for dependencies, more times than not, they don't even share/say what Python version they used. It's basically still the wild west out there.
Never had anyone "frown" towards me for not using MyPy or any typechecker either, although I get plenty of that from TS fans when I refuse to adopt TS.
I’ve seen it many times. Here’s one of the more extreme examples, a highly-upvoted comment that describes not using type hints as “catastrophically unprofessional”:
https://www.reddit.com/r/Python/comments/1iqytkf/python_type...
Don't read stuff on reddit and use whatever you've "learned" there elsewhere, because it's basically run by moderators who try to profit of their communities these days, hardly any humans left on the subreddits.
Edit: I really can't stress this enough, don't use upvotes/likes/stars/whatever as an indicator that a person on the internet is right and has a good point, especially not on reddit but I would advice people to not do so on HN either, or any other place. But again, especially on reddit, the upvotes literally count for nothing. Don't pick up advice based on upvoted comments on reddit!
If you're working on a project that doesn't use type hints, there's also plenty of frowning, but that's just because coding without a type checker is kind of painful.
Yeah, that obviously makes sense, not following the code guidelines of a project should be frowned upon.
Python typed or untyped feels like a taste / flexibility / prototyping tradeoff; TypeScript vs. JavaScript feels like "Do you want to get work done or do you want to wrap barbed wire around your ankle and pull?" And I say this as someone who will happily grab JS sometimes (for <1,000 LOC projects that I don't plan to maintain indefinitely or share with other people).
Plus, TypeScript isn't a strict superset of JavaScript, so choice at the beginning matters; if you start in JS and decide to use TS later, you're going to have to port your code.
> TypeScript helps paper over like 90% of the holes in JavaScript
Always kind of baffles me when people say this, how are you actually programming where 90% of the errors/bugs you have are related to types and other things TS addresses? I must be doing something very different when writing JS because while those things happen sometime (once or twice a year maybe?), 90% of the issues I have while programming are domain/logic bugs, and wouldn't be solved by TS in any way.
I can name an absolute handful of languages I've used that have that flexibility. Common LISP comes to mind. But in general you get one or the other option.
It’s also a worst-of-both-worlds arrangement, in that you have to do the extra work to satisfy the type checker but don’t get the benefits of a compiled language in terms of performance and ease-of-deployment, and only partial benefits in terms of correctness (because the type system is unsound).
AFAIK the Dart team felt this way about optional typing in Dart 1.x, which is why they changed to sound static typing for Dart 2.
We should applaud their donation today, and at another time assess the meager contributions of many companies that should be shamed.
I've worked at a few that use the 'mold' linker to dramatically reduce their build times. Again, very few contribute. In this particular case, I managed to get one former employer to make a donation.
But the list goes on.
Short arms, deep pockets, as the saying goes.
If python wants to require money for updates or for customers over $X in revenue, they can!
If companies don’t want to donate, they don’t have to just as python contributors don’t have to if they’re annoyed at how it’s used.