Please do the opposite. Let all deprecation warnings last at least a decade, just include in the warning that it is not maintained.
But more to the point, go out of your way to avoid breaking backwards compatibility. If it's possible to achieve the same functionality a different way, just modify the deprecated function to use the new function in the background.
My biggest problem with the whole static typing trend is that it makes developers feel empowered to break backwards compatibility when it would be trivial to keep things working.
edit:
Not that it is always trivial to avoid breaking backwards compatibility, but there are so many times that it would be.
> just include in the warning that it is not maintained.
I'm convinced this isn't possible in practice. It doesn't matter how often you declare that something isn't maintained, the second it causes an issue with a [bigger|more important|business critical] team it suddenly needs become maintained again.
And here's where your business can contact me to talk about a support contract.
If it's important, they'll pay. Often you find out it wasn't that important, and they're happy to figure it out.
It sounds like you're imagining open source whereas the comment you're replying to is imagining more intra-company dependencies.
I think deprecation in intra-company code is a completely different beast. You either have a business case for the code or not. And if something is deprecated and a downstream project needs it, it should probably have the budget to support it (or code around the deprecation).
In many ways, the decision is easier because it should be based on a business use case or budget reason.
The business case is the easy part, the quagmire is in getting the different teams to agree who should support the business case, why it's more important than the business cases they wanted to spend cycles on instead, and how much of the pie supporting it takes on the budget side. Less so when the place is small enough everyone knows everyone's name, more so when it's large enough they really don't care what your business case is much even though it'd be 10x easier to support from their side instead of another.
Oh. But that is a solved problem. The users of the library just copy the code from before the deprecation and then stick it in their codebase not to be maintained anymore. Problem solved. /s
> I'm convinced this isn't possible in practice.
I don't agree. Some programming languages started supporting a deprecated/obsolete tagging mechanism that is designed to trigger warnings in downstream dependencies featuring a custom message. These are one-liners that change nothing in the code. Anyone who cares about deprecating something has the low-level mechanisms to do so.
For sure, but for this to work you need someone downstream to notice those messages and prioritize the work to migrate off the deprecated code paths. Some teams will respond, but many won't. No matter how loudly you declare that the code is deprecated, you'll still have people using it up to the point it stops working.
It's far better to plan the removal of the code (and the inevitable breaking of downstream users systems) on your own schedule than to let entropy surprise you at some random point in the future.
I don't know that I see why/how this is a problem? You would do the same with any other thing in your life?
More, in many things, we have actively decided not to do something anymore, and also highly suggest people not mess with older things that did use it. See asbestos. Removing it from a building is not cheap and can be very dangerous.
It also keeps slowing down development as getting a green global compile will make you still update "deprecated" functions that face breaking API changes.
> My biggest problem with the whole static typing trend is that it makes developers feel empowered to break backwards compatibility when it would be trivial to keep things working.
I don't see the connection you're drawing here.
Disclaimer: I’m a strong advocate for static typing.
I absolutely see the connection. One of the advantages of static typing is that it makes a lot of refactoring trivial (or much more than it would be otherwise). One of the side effects of making anything more trivial is that people will be more inclined to do it, without thinking as much about the consequences. It shouldn’t be a surprise that, absent other safeguards to discourage it, people will translate trivial refactoring into unexpected breaking changes.
Moreover, they may do this consciously, on the basis that “it was trivial for me to refactor, it should be trivial to adapt downstream.” I’ll even admit to making exactly that judgment call, in exactly those terms. Granted I’m much less cavalier about it when the breaking changes affect people I don’t interface with on a regular basis. But I’m much less cavalier about that sort of impact across the board than I’ve observed in many of my peers.
Because it lays out the contract you have to meet on the interface. No contract? No enforced compatibility.
It is a way to signal to other developers that your code changed, which is better than not saying anything.
But it seems to make library developers more comfortable with making breaking changes. It's like they're thinking 'well it's documented, they can just change their code when they update and get errors in their type checker/linter.' When I think they should be thinking, 'I wonder what I could do to make this update as silent and easy as possible.'
Of course, we all have different goals, and I'm grateful to have access to so many quality libraries for free. It's just annoying to have to spend time making changes to accommodate the aesthetic value of someone else's code.
But it also makes it easier for library developers to identify that a change is breaking. Without static types you can easily make a change you think is fine, but it ends up breaking user code because they depended on something you didn't expect. Granted, static typing doesn't completely solve that problem, there can still be changes in behavior that break things, but it is a useful tool in identifying certain classes of breaking changes.
yes, that is my point.
I think the only time I’ve ever seen this is in a JS library, where each major version bump is practically a different library altogether because it’s a complete rewrite, and these days is a wrapper around some Rust thing.
Not even JS alone. I blame the enforcement of semantic versioning, as if a version of code simply had to be a sequence of meaningful numbers.
If you are using a language with a complete type system, sure. But who uses those?
When using the languages people actually use in the real world, not really. Consider a simple example where the contract is that you return an integer value from 1 to 10. In most languages people actually use, you're going to be limited to using an integer type that is only constrained by how many bits it is defined to hold, which can be exploited later to return 11, unbeknownst to the caller's expectations. There are a small number of actually-used languages that do support constraining numeric types to a limited set of values, but even they fall apart as soon as you need something slightly more complex.
This is what tests are for. They lay out the contract with validation of it being met, while also handily providing examples for the user of your API to best understand how it is intended to be used.
More than anything, this reveals your lack of understanding of modern languages and type systems.
TypeScript, for example, is one of the most widely used languages in the world. It has an incredibly powerful type system which you can use to model a lot of your invariants. By leaning on patterns such as correct-by-construction and branding, you can carry around type-level evidence that e.g. a number is within a certain range, or that the string you are carrying around is in fact a `UserId` and not just any other random string.
Can you intentionally break these guarantees if you go out of your way? Of course. But that's irrelevant, in the same way it is irrelevant that `any` can be used to break the type system guarantees. In practice, types are validated at the boundaries and everything inside can lean on those guarantees. The fact that someone can reach in and destroy those guarantees intentionally doesn't matter in practice.
Typescript is one of the actually-used languages that supports constraining the integer, albeit in a pretty awkward way.
But now try defining a type that enforces a RFC-compliant email address...
There are languages with proper type systems that can define full contracts, but Typescript is not among them. Without that, you haven't really defined a usable contract as it pertains to the discussion here. You have to rely on testing to define the contract (e.g. assert the result of the 'generate email' function is RFC-complaint).
And Typescript most definitely does. Testing is central to a Typescript application. It may have one of the most advanced type systems found in languages actually used, but that type system is still much too incomplete to serve as the contract. Hence why the ecosystem is full of testing frameworks to help with defining the contract in tests.
You clearly don't understand the patterns I described. Look up branding and try reading e.g. "parse, don't validate".
"Parse, don't validate", while familiar to all HN users (it feels like it gets posted here every week), is orthogonal and does not address what we are talking about. To extrapolate, let's stay with the email address type example.
In Typescript, you can define an EmailAddress type as:
type EmailAddress = string & { __brand: "EmailAddress" }
If you are feeling saucy, you can even define it as:
type EmailAddress = `${string}@${string}` & { __brand: "EmailAddress" }
But nether of these prove to me, the user of your API, that an EmailAddress value is actually a RFC-compliant email address. Your "parse" function, if you want to think in those terms, is quite free to slip in noncompliant characters (even if only by accident) that I, the consumer, am not expecting. The only way for me to have confidence in your promise that EmailAddress is RFC-compliant is to lean on your tests written around EmailAddress production.
That isn't true for languages with better type systems. In those you can define EmailAddress such that it is impossible for you to produce anything that isn't RFC-compliant. But Typescript does not fit into the category of those languages. It has to rely on testing to define the contract.
Let's imagine we are working with some hypothetical language in which what you describe is possible.
At some point, you will have to write a function where you validate/parse some arbitrary string, and it then returns some sort of `Email` type as a result. That function will probably return something like `Option<Email>` because you could feed it an invalid email.
The implementation for that function can also be wrong, in exactly the same way the implementation for the typescript equivalent could be wrong. You would have to test it just the same. The guarantees provided by the typescript function are exactly equivalent, except for the fact that you do technically have an escape hatch where you can "force" the creation of a branded `Email` without using the provided safe constructor, where the other language might completely prevent this - but I've already addressed this. In practice, it doesn't matter. You only make the safe constructor available to the user, so they would have to explicitly go out of their way to construct an invalid branded `Email`, and if they do, well, that's not really your problem.
I could see some maintainers being encouraged by static typing to introduce breaking changes, as long as the changes ensure compile-time breakage. However, that’s only an option in languages without dynamic linking. So it’s the combination of static typing and static linking that would encourage that.
Indeed. C# is statically typed. Python is not. Which is the one with a notorious backwards-incompatibility story?
We have already ported many projects from .NET8 and .NET9 to .NET10 with close to zero issues.
Guessing you never used OpenAPI or Razor then. Which were core features in 9 and no longer existed in 10.
What are you talking about?? OpenAPI and Razor are both still present in .NET10.
Yeah, I'm confused at these examples, along with the implicit idea that the existence of breaking changes means C# had equivalent compatibility challenges to, say, python3.
.NET Framework -> Core was more persuasive, but I stand by the overall point that compatibility is more about project philosophy than "static vs dynamic typing" and indeed I think Framework/Core illustrates just that: Framework more heavily favored preserving compatibility compared to Core
edit:
I assume this was a superficial grasp onto "Deprecation of WithOpenApi extension method" ("WithOpenApi duplicated functionality now provided by the built-in OpenAPI document generation pipeline") and "Razor runtime compilation is obsolete" ("Razor runtime compilation has been replaced by Hot Reload, which has been the recommended approach for a few years now.")
Of course, its an entirely trivial process to completely replace serialisation everywhere in your product and swap out your own custom hot reloading compilation pipeline you created before the language created its own one incompatible with your API...
The .NET framework to .Net Core migration broke a bunch. Similarly java 6 to 8, and when they introduced the new module system in Java 11 broke a bunch in another case. You’d still need to fixed that they swapped the string types, renamed a bunch of core APIs etc. in a statically typed python
Those are good examples. But python is dynamically typed -- indeed, python 3 is what I was alluding to
I was hoping to spin it as an environmental concern. You change something "for reasons". If you're doing this in a popular tool/library you probably just burnt down a small forest of trees worth of energy from all of the work people will have to put in deal with it. 100s of thousands of users missed dinner with a kid or loved one because of the extra work you created. A product that would of shipped on time didn't. Things you wanted to exist didn't because 100k people were distracted dealing with your breakage.
> Not that it is always trivial to avoid breaking backwards compatibility, but there are so many times that it would be.
Wow. Why even remove it? It's just the thinnest wrapper around the dict and since the dict is now part of the public API these methods will work forever unmodified.
While I am happy to see types in Python and Javascript (as in Typescript) I see far more issues how people use these.
In 99% of the time, people just define things as "string" or if doesn't cover, "any". (or Map/Object etc)
Meanwhile most of these are Enum keys/values, or constants from dependencies. Whenever I see a `region: string` or a `stage: string`; a part of me dies. Because these need to be declared as `region: Region` or `stage: Stage`. Where the "Region" and "Stage" are proper enums or interfaces with clear values/variables/options. This helps with compile (or build) time validation and checking, preventing issues from propagating to the production (or to the runtime at all)...
Stripe do this in a cool way. Their REST API is version based on date, and each time they change it they add a stackable compatibility layer. So your decade old code will still work.
As with all things, this can be pushed too far. Microsoft was suffering a maintainability crisis before the transition to Windows XP; their years of bending the API to support customers (which, in the short run, did keep customers to their benefit) was making for a fairly unmaintainable mess of an API surface that then screamed under the strain when the Internet era hit and all those open, under-observed APIs became potential worm attack vectors.
Actually, I think backwards compatibility is the reason Windows still dominates. It isn't the best OS at anything and it's actively user hostile, but if you want to run Windows apps... there are decades of Windows apps and they basically all work.
this is the answer.
No matter what others say, the pipelines are long.
The delays for release, getting into a distribution, then living out its lifetime... they are significant.
One thing I've always hated was this idea of "bitrot." That software just spontaneously stops working after time and loses backwards compatibility. Like it's some force of nature.
It's not a force of nature. Bitrot is: many software developers deliberately choosing to break backward compatibility in very small ways over and over. Software written in 1995 should still work today. It's digital. It doesn't rot or fall apart. It doesn't degrade. The reason it doesn't work today is decisions that platforms and library maintainers deliberately made. Like OP. Deprecate like you mean it. That's a choice!
If we want to solve bitrot, we need to stop making that choice.
Progress is often only possible by breaking things. It's not a choice, it's the only way forward. We have to optimize for the future being better, even if it makes the present a little worse occasionally.
This is a huge reason why open source projects are often so much more successful than corporate clones: they actually iterate and innovate, something corporate america has forgotten how to do.
Some deprecations are required to make progress. Many others aren't, and those are the most frustrating.
For example, ruby deprecated the `File.exists?`, and changed it to `File.exist?`, because enough people felt that the version with the `s` didn't make sense grammatically (which I disagree with, but that is not germane to my point).
For a long time, you would get warning that `exists?` was deprecated and should be replaced by `exist?`.... but why? Why couldn't they just leave `exists?` as an alias to `exist?`? There was not cost, the functions are literally identical except for one letter. While the change was trivial to fix, it added annoyance for no reason.
Although, luckily for me, with Ruby I can just make exists? an alias myself, but why make me do that?!? What is the point of removing the method that has been there forever just because you think the s is wrong?
Imagine I am new to Ruby. I go looking to figure out if a file exists. Or I get a list of available methods from the file object. Now I have to research both `exist?` and `exists?` to figure out if I should be choosing one or the other.
Now multiply that by.... every past history of every API and it makes adopting something really difficult as a newcomer.
aka Common Lisp.
It doesn’t matter which one you do, the methods are literally identical other than the extra s. You can’t pick the wrong one.
This sort of choice is very common in Ruby, you can have different style choices for the same functionality. You can have blocks made with {} or do and end. You can do conditionals in the form of “a if b” or “if b then a”. You can call methods with or without parentheses.
These are all Ruby style choices, and we already have a way to enforce a particular style for a particular project with formatters like rubocop.
I'm not defending the Ruby decision, I literally said I wouldn't have done it. It reminds me of the disttools removal from python, which was also not something I'd have done nearly so aggressively if it were up to me. Multiple distros have patched it back in.
But it wasn't up to me. It's not my project. I'm using somebody else's project, and at the end of the day it's their decision, because they own it. Unless it's impossible to work around, I feel like I have to either respect that, or switch to an alternative.
You're free to maintain a patch on top of Ruby to add the alias and run that on your machines, btw. It would probably be very simple, although certainly not as simple as aforementioned sed command...
> You're free to maintain a patch on top of Ruby to add the alias and run that on your machines, btw.
Comments like this are honestly just asshole-ish.
Why? It's true. It's not even really that hard. My point is nobody is forcing them to accept it.
Because Ruby is nothing without the community around it. If they wanted to make Ruby and don’t care about the people using it, they should go private, shut down the project, or have someone else take over.
It’s wrong to shut down discussion like this with comments like “it’s their code”, “make your own fork”, etc. because Ruby is supposed to be part of the open source community, which implies collaboration, give and take, discussion of pros and cons, etc.
What you are doing is ignoring this major aspect of a programming language and taking this weird anti social stance on it.
I'm not shutting down any discussion, I'm simply stating my opinion. You're free to disagree.
I didn't say to fork it. Do you really not appreciate the difference between rebasing a trivial patch forever, and maintaining a wholesale fork forever?
Crowing on saying “it’s not your code, they can do what they want” is an attempt to do that.
Fork, patch, maintaining your private whatever, that’s not the point, and is a digression.
It's a perfectly valid point, you just disagree. I'm not the one shutting down discussion here.
Of course it is their decision, and they don’t OWE me an explanation.
But this is a discussion forum, and I am asking for people who agree with the decision to explain why they agree. Again, they don’t have to answer me if they don’t want to. I am just saying, “if anyone knows an argument for this type of change, I would love to hear it”
Saying they don’t have to explain their reasoning is true but not really relevant to our conversation. I am not asking THEM, I am asking HN readers.
Your example could be fixed in your entire codebase with a single sed invocation. While yes, if it were up to me I'd probably just have left the alias, the annoyance for the end user does seem quite minimal...
You get to use open source projects for free, and a lot of people do ongoing maintenance on them which you benefit from for free. In return, sometimes you are expected to modify your code which depends on those projects because it makes their maintainer's life easier.
Personally, I see that as a very reasonable trade-off.
The third option is to use things that don't break themselves. One of the many joys of Go is that I can leave personal projects untouched for years and then go back and everything is still working
> Your example could be fixed in your entire codebase with a single sed invocation
I also have to hope all the dependencies I use did that, too.
But my real question is why? Why make me do it at all?
Because they wanted to, and it's their project. Part of being an adult is accepting you won't agree with every decision that affects you.
You were free to show up and argue against it, as was anyone else. Did you?
Ok, I think you are arguing against something I am not saying.
I am not arguing that they don’t have the RIGHT to make the change, or that they owe me personally anything. I am not even THAT mad. I still love Ruby the most of any language, and generally think they make the right decisions.
I am simply annoyed by this decision.
And yes, I argued against this change when it was first proposed (as did many others). They obviously were not convinced.
Again, I am not arguing that they should be FORCED to do what I want, or that they did something shady or nefarious by making this change. I am not asking for any remedy.
I am simply saying I disagree with this type of change (changing a method name because some people feel the new name makes more grammatical sense, but not changing the method itself at all). The reason I commented was because this is not a “we have to deprecate things for progress” situation. They aren’t limited in any way by the current method, the syntax isn’t cumbersome (and isn’t changing), there is no other capability that is held back by having to maintain this method. It is literally just “I read it as asking ‘Does this file exist?’ rather than asking ‘This file exists?’”
Again, they are obviously free to disagree with me, which they do. I am simply arguing that we shouldn’t break syntax just because you like the way it reads better without an s. And I am asking for someone who disagrees with me (you) to explain why it is worth making this type of change.
It's important for code to be clean and free of old cruft, even if that means old code has to change. Readibility is one of the most important aspects of maintainable code. Arguably more important than testing, IMHO. This is another reason open source solutions so consistently win over corporate ones, the open source world understands this.
Are there changes I disagree with? Of course. But I'd rather live in a world that moves forward and occasionally breaks me, than one where I have perfect compatibility but am stuck on code lacking the new innovations my competitors benefit from.
The whole idea behind deprecating things is to give people time to make the changes before they become breaking.
I went and looked: exists? was marked as deprecated in 2013 and removed in 2022. That's enormously generous, my previous comparison with the disttools debacle in python was inaccurate. You had a decade!
You've moved the goal posts quite a bit from your previous post to this one. Originally, you said that "progress is often only possible by breaking things... it's the only way forward". But the Ruby example that cortesoft provided isn't necessary to make progress. The language would get by just fine without that deprecation (or even without making the change at all). Saying "but it's a trivial change" in response is irrelevant, because you weren't originally trying to justify changes on the basis that they were easy, but that they were necessary.
> Saying "but it's a trivial change" in response is irrelevant
Of course it's relevant. It's a laughably trivial example compared to the other one in this thread.
It's absolutely a choice. All software can progress while preserving backward compatibility for existing users. It's not always easy, but it's never impossible.
> All software can progress while preserving backward compatibility for existing users
That's an incredibly ignorant claim. Just run "git log" in glibc, it won't take you very long to prove yourself wrong.
glibc, which has had ABI compatibility for decades?
There have been plenty of build breaking changes over the past couple decades, generally they happen for very good reasons and only affect niche usecases.
So existing binaries will keep working?
Not necessarily if you go back far enough to have nss problems :)
Not always without a performance cost which is a no go in projects with very expensive hardware (ahem Cisco).
Actually this is the reason why Win32 is the stable ABI for Linux.
God, would I hate to be programming now the way I had to program in 1995.
Granted, modern coroutines do bring up some nostalgic feel for the days I had to support cooperative multitasking...
Isn't that arguing for keeping the maintenance cost high and also stagnating the design?
> That would probably not trigger anyone’s midnight pager, but it would make it clear that relying on the deprecated functionality is a bug lurking in the code.
How do you know? This is a wild assertion. This idea is terrible. I thought it was common knowledge that difficult to reproduce, seemingly random bugs are much more difficult to find and fix than compiler errors.
If you're ready to break your api, break your api. Don't play games with me. If more people actually removed deprecated APIs in a timely manner, then people will start taking it more seriously.
Last paragraph of the article:
> In case the sarcasm isn’t clear, it’s better to leave the warts. But it is also worthwhile to recognise that in terms of effectiveness for driving system change, signage and warnings are on the bottom of the tier list. We should not be surprised when they don’t work.
Yeah, totally a woosh moment for me. Read all the way up to the `* * *`. That's on me :)
That last bit was added to the article after your comment, as the author realised the sarcasm had been too subtle for most people to catch.
Most HN visitors won’t read to the last paragraph, so it’s a good thing to emphasize.
I thought I had read it. :) I thought the three `* * *` at the bottom was indicating I was about to start reading suggestions for the next article. So definitely a "Woosh" moment for me :D
Yeah, I agree - this sort of intermittent failure could be incredibly hard to track down, and will absolutely fuck with people's faith in their CI systems as well - a flappy test is the absolute worst kind of test.
I agree, maintainers should just break the API if they're going to do it.
At the same time, it's crazy that urllib (the library mentioned in the article), broke their API on a minor version. Python packaging documentation[1] provides the sensible guideline that API breaks should be on major versions.
This isn't a good idea regardless of why it's being deprecated.
If it's no longer being maintained then put a depreciation warning and let it break on its own. Changing a deprecated feature just means you could maintain it but don't want to.
Alternatively if you want to aggressively push people to migrate to the new version, have a clear development roadmap and force a hard error at the end of the depreciation window so you know in advance how long you can expect it to work and can document your code accordingly.
This wishy-washy half-broken behaviour doesn't help anyone
The post is a sarcasm, see the below.
Having a deprecated API just randomly return failures is an awful idea!
Better to give an actual timeline (future version & date) for when deprecated functionality / functions will be removed, and in the meantime, if the language supports it, mark those functions as deprecated (e.g. C++ [[deprecated]] attribute) so that developers see compilation warnings if they failed to read the release notes.
Yep. I'll admit I've acted faster to hard set dates than some "in the future" message. I've also seen some tools become really noisy about deprecation spanning many lines AND repeating. Please don't log the same message over and over for each instance. Color or add emoji if you must to grab attention, but once is enough. It's annoying when you can't do anything about it at that time and have to sift through this extra noise when hunting down another issue in the CI log. Add a link that goes over it in more detail and how to migrate for that specific deprecation.
I don't think that was a serious suggestion, the article says it is sarcasm near the end.
But yes, that would be the worst idea ever.
Random is bad, but scheduled windows of breakage is good.
How are you supposed to deal with recalcitrant users? I work for an organization that is ending support for several long running APIs. And by support I mean turn off the servers, you must move to an entirely new platform.
We’ve sent out industry alerts, updated documentation and emailed all user. The problem is the contact information goes stale. The developer who initially registered and set up the keys, has moved on. The service has been running in production for years without problems and we’ve maintained backwards compatibility.
So do we just turn it off? We’ve put messages in the responses. But if it’s got 200ok we know no one is looking at those. We’ve discussed doing brownouts where we fail everything for an hour with clear error messages as to what is happening.
Is there a better approach? I can’t imagine returning wrong data on purpose randomly. That seems insane.
> How are you supposed to deal with recalcitrant users?
Keep the servers running, but make the recalcitrant users pay for the costs and a then some more. It is actually a common strategy. Big, slow companies often have trouble with deprecation, but they also have deep pockets, and they will gladly pay a premium so that they can keep the API stable at least for some time.
If you ask for money, you will probably get more reactions too.
Step 1: Stop thinking of them as "recalcitrant". They're not recalcitrant. They bought (presumably for money) a product, and expect that product to keep working as long as they need it to! They don't expect the vendor to pull the rug out from under them and break it just because the API is old and icky and their software engineers are sad to keep it around.
Instead of "deprecate like you mean it" the article should be: "Release software like you mean it" and by that, I mean: Be serious. Be really, really sure that you are good with your API because users are going to want to use it for a lot longer than you might think.
> They bought (presumably for money) a product, and expect that product to keep working as long as they need it to!
This depents on the terms of the contract. Typically, termination of service is covered in a license. If the license terms are okay in the respective jurisdiction, there is no fundamental ethical obligation to run a server beyond that. There might exist specific cases where it would be inappropriate to follow the terms by the letter, but that also has its limits.
Contract terms usually define legal obligations, not ethical obligations. They create duties parties must perform or face legal consequences--they don't speak to what those parties should do ethically.
Following legal obligations is an important part of ethics. The law has also the purpose to relieving the individual of the burden of complex ethical considerations. This is the general situation, especially in a democracy under the rule of law.
There are, of course, exceptions and disagreement about specific regulations. But as long as you have the law on your side is a very strong indicator that what you are doing is also ethical more or less okay. It is very hard to say that one person is far off ethically, if two people agreed on something and the terms of their agreement are without doubt legally correct.
Most systemic evil in the modern world is done legally, IMO. There is everything legal but nothing ethical about the way John Deere screws farmers over, how big tech sells your data, how Amazon creates a consumerist race to the bottom, how United Healthcare denies you coverage capriciously, etc.
I appreciate your viewpoint, even though mine is the exact opposite!
Nothing lasts forever. The second you decide to use a new 3P API you have to understand it might disappear one hour after your production launch, and that's okay.
You also have to understand that there will be APIs that eagerly rugpull and APIs that don't, and if you're offering the former then your users will end up moving to your competitors all else being equal.
Software evolves over time, along with business needs. What seemed like (or even was!) a good idea at some point will almost surely cease to be a good idea at some point in the future. Breaking the API is totally fine if there's a good reason and it's carefully managed.
I agree, get the API right as much as possible so it doesn't need to be deprecated.
But, perfection isn't realistic. If you don't have a plan for when you get things wrong, you're failing to plan for the inevitable.
Sleep()s that increase exponentially every month seem like a good solution. When the API has a 10 second latency hopefully someone starts asking questions. If not I think brownouts are a decent idea.
> We’ve discussed doing brownouts where we fail everything for an hour with clear error messages as to what is happening.
That sounds like the best option. People are used to the idea that a service might be down, so if that happens, they’ll look at what the error is.
A technique I used on a project was to change the URL, and have the old URL return a 426 with an explanation, a new link, and a clear date when the moved API. This reliably breaks the API for clients so that they can't ignore it, while giving them an easy temporary fix.
Clients weren't happy, but ultimately they did all upgrade. Our last-to-upgrade client even paid us to keep the API open for them past the date we set--they upgraded 9 months behind schedule, but paid us $270k, so not much to complain about there.
So did you return the correct data with the 426 Code?
No--the goal was to break the API so users noticed, with an easy fix. A lot of users weren't even checking the HTTP status codes, so it was necessary to not return the data to make sure the API calls broke.
We did roll this out in our test environment a month in advance, so that users using our test environment saw the break before it went to prod, but predictably, none of the users who were ignoring the warnings for the year before were using our test environment (or if they were, they didn't email us about it until our breaking change went to prod).
That must have been one heck of a rug pull if $270k was considered more cost-effective than getting someone to work on it sooner.
The solution I've found is to make using the API a hard error with an explicitly temporary and obnoxiously-named workaround variable.
It's loud, there's an out if you need your code working right now, and when you finally act on the deprecation, if anyone complains, they don't really have legs to stand on.
Of course you can layer it with warnings as a first stage, but ultimately it's either this or remove the code outright (or never remove it and put up with whatever burden that imposes).
I heard a second-hand story about some team at Google who did this, and named the variable something like:
a year before the deadline. I would be mildly amused by adding
_AND_MY_USER_ID_IS="<user_id>"
Love this idea, as it is already implemented in a $FAANG company tool. (used company wide by 80k+ SDEs). I got used to seeing these in the logs and terminal. So much that my brain now automatically ignores it from my view/seeing like it does for my nose.
Don't make it return a different result.
Instead, if you must, add a sleep within the function for 1 ms in the first release, 2 ms in the second release, and so on. But maybe just fix the tooling instead to make deprecations actually visible.
I get the intention but it's a bad idea, same with the article
if people are meant to depend on your endpoints, they need to be able to depend on all of them
you will always have ppl who don't respond to deprecation notices, the best you can do is give them reliable information on what to expect -- if they hide the warnings and forget, that's their business
but intentionally making problems without indication that its intentional results in everyone (including your own team) being frustrated and doing more work
you cannot force ppl to update their code and trying to agitate them into doing it only serves to erode confidence in the product, it doesn't make the point ppl think it makes, even if the court of public opinion sides with you
cover your bases and make a good faith effort to notify and then deal with the inevitable commentary, there will always be some who miss the call to update
this actually seems like an reasonable technical solution to the non-technical problem that causes deprecations to be ignored in the first place
Degrading performance exponentially (1ms, 2ms, 4ms, 8ms...) WILL create a 'business need', without directly breaking critical functions. Without this degradation, there is no reason to remove the deprecated code, from a business perspective.
It should slow down your compile instead of the runtime. That way the pain is felt by the developer who can fix it.
It is sometimes a tooling issue, but far more often it's a "few teams have a discipline of squashing all toolchain warnings" issue.
I get that it's nice to be able to deprecate/remove things.
But intentionally breaking my users runtime in a way that's really hard and annoying to find? Is the author OK? This reads like a madman to me.
It would help a lot if core Python libraries like urllib, NumPy, and others used SemVer. Removing a function is a breaking change. The root of this post is from urllib breaking something in a minor release. Get rid of the pseudo SemVer where you can deprecate functions and then break in minor releases. Maybe after that the rest of the community could hope their code will work long term if they don’t increment major versions.
That just encourages people to keep using old, unmaintained, insecure versions of libraries. Then, when they're still on version 2.1.1, and your maintained version is 5.7.3, and somebody finds a major security bug in 2.1, they will come whining at you to release a 2.1.2.
Code that is not being maintained is not usually suitable for use, period.
Library maintainers have no right to police how people use their open source code, period. Maintainers are also not obligated to backport security fixes. Anything else is effectively against the concept of open source.
Notably, even this policing doesn’t fix the whining. The whining will just be about what TFA is whining about. You’re just moving the whining around.
It also does nothing to actually force people to upgrade. Instead, people can just cap against the minor version you broke your package on. Instead of being user hostile, why not make the user’s job easier?
Correctly following SemVer disincentivizes unnecessary breaking changes. That’s a very good thing for users and ultimately the health of the package. If you don’t want to backport security fixes, users are free to pay, do it themselves, or stop using the library.
And then you can offer them a support contract to produce an update for an out of support version
Python itself is making breaking changes between minor versions, so I think its natural that the ecosystem will follow.
I have not had any real problems yet myself, but its worrying.
Python has never claimed to use semver, despite everyone associating the 3 in front with the 2->3 migration.
That just means its always been a problem, instead of being a new problem.
It does use major.minor.bugfix versioning, but without clarity about when to expect breaking changes.
With the pace of 3.x releases it has become more of a problem.
> It is important to know that NumPy, like Python itself and most other well known scientific Python projects, does not use semantic versioning. Instead, backwards incompatible API changes require deprecation warnings for at least two releases.
Have they made breaking changes to the ABI in other versions?
I’m not sure how that is relevant. Are you trying to suggest that NumPy in fact does follow SemVer despite their own documentation saying they don’t?
> Get rid of the pseudo SemVer where you can deprecate functions and then break in minor releases.
I agree, but I think there's a bigger, cultural root cause here. This is the result of toxicity in the community.
The Python 2 to 3 transition was done properly, with real SemVer, and real tools to aid the transition. For a few years about 25% of my work as a Python dev was transitioning projects from 2 to 3. No project took more than 2 weeks (less than 40 hours of actual work), and most took a day.
And unfortunately, the Python team received a ton of hate (including threats) for it. As a natural reaction, it seems that they have a bit of PTSD, and since 3.0 they've been trying to trickle in the breaking changes instead of holding them for a 4.0 release.
I don't blame them--it's definitely a worse experience for Python users, but it's probably a better experience for the people working on Python to have the hate and threats trickle in at a manageable rate. I think the solution is for people like us who understand that breaking changes are necessary to pile love on doing it with real SemVer, and try to balance out the hate with support and
I had a client who in 2023 still was on 2.7.x, and when I found a few huge security holes in their code and told them I couldn't ethically continue to work on their product if they wouldn't upgrade Python, Django, and a few other packages, and they declined to renew my contract. As far as I know, they're still on 2.7.x. :shrug:
That’s probably true. I do think part of the anger is that a lot of the changes didn’t clearly improve the code around it. The obvious example is the change to print from a statement to a function. It makes the language a little cleaner, but it also breaks existing code for little practical benefit. More insidious was the breaks with strings and byte strings. It was a good and necessary change that also could lead to weird quiet breakages.
At least for me, the real blocker was broad package support.
Maintainers should think carefully about whether their change induces lots of downstream work for users. Users will be mad if they perceive that maintainers didn’t take that into account.
No. Breaking other people's code is why we have this broken software world where new bugs (and security holes) are added everywhere all the time since no software is allowed to just exist without monthly rewrites. We need a culture where breaking backwards compatibility in published APIs is almost unthinkable, not something anyone would want to be caught doing.
Deprecation warnings are precisely that --- warnings --- so that people have the option not to act on them. So, it is of little surprise that users exercise one of the two available use case scenarios, which is not acting.
There are sophisticated users who care about the quality of their code and care about it breaking as infrequently as possible. Those users follow warnings. Not only warnings that happen by default; they use additional tooling to get extra warnings. The follow up on warnings.
Deprecation warnings serve those people.
As for the others, who cares. "We generously told you this would be removed, for years".
People who ignore warnings related to compatibility and have a workflow whereby your dependencies are not pinned down to specific versions, in some project configuration file, so that they are always getting the latest dependencies, are choosing to inflict breakages on themselves.
> What if we intentionally made deprecated functions return the wrong result … sometimes? Every time it intentionally returns the wrong result, it logs the deprecation warning.1
What if we found that a highway overpass construction material was suboptimal, and we want people to use superior materials, so, every now and then, we send a chunk of concrete plummeting down to the ground, to kill a motorist?
Thanks to deprecating like we mean it, they're going to replace that overpass sooner than they would otherwise. You'll thank me later.
> This API was emitting warnings for over 3 years in a top-3 Python package by downloads urging libraries and users to stop using the API and that was not enough. We still received feedback from users that this removal was unexpected and was breaking dependent libraries.
Entirely predictable.
Even many of those who saw the deprecation logging, and bothered to make a conscious decision, didn't think you'd actually break the API.
> We ended up adding the APIs back and creating a hurried release to fix the issue.
Entirely predictable.
Save yourself some anguish, and don't break API unnecessarily. Treat it like a guarantee, as much as possible.
If it's a real problem for ongoing development, consider using SemVer and multiple versions, like the linked article suggests. (With the deprecated branch getting minimal maintenance: maybe only select bug fixes, or only critical security fixes, maybe with a sunset on even those, and a deprecation warning for the entire library when it's no longer supported.)
The various projects that say something is deprecated but then don't give a removal timeline or keep delaying the removal (or even explicitly say it won't be removed, just remain deprecated) are the cause of this problem.
IMO, any deprecation should go in the following steps:
1. Decide that you want to deprecate the thing. This also includes steps on how to migrate away from the thing, what to use instead, and how to keep the existing behaviour if needed. This step would also decide on the overall timeline, starting with the decision and ending with the removal.
2. Make the code give out big warnings for the deprecation. If there's a standard build system, it should have support for deprecation warnings.
3. Break the build in an easy to fix way. If there is too much red tape to take one of the recommended steps, the old API is still there, just under a `deprecated` flag or path. Importantly, this means that at this step, 'fixing' the build doesn't require any change in dependencies or (big) change in code. This should be a one line change to make it work.
4. Remove the deprecated thing. This step is NOT optional! Actually remove it. Keep it part of your compiler / library / etc in a way to give an error but still delete it. Fixing the build now requires some custom code or extra dependency. It is no longer a trivial fix (as trivial as the previous step at least).
Honestly, the build system should provide the tools for this. Being able to say that some item is deprecated and should warn or it is deprecated and should only be accessible if a flag is set or it is removed and the error message should say "function foo() was removed in v1.4.5. Refer to the following link:..." instead of just "function foo() not found"
If the build system has the option to treat warnings as errors, it should also have the option to ignore specific warnings from being treated as such (so that package updates can still happen while CI keeps getting the warning). The warning itself shouldn't be ignored.
> In case the sarcasm isn’t clear, it’s better to leave the warts. But it is also worthwhile to recognise that in terms of effectiveness for driving system change, signage and warnings are on the bottom of the tier list. We should not be surprised when they don’t work.
Looking at comments I guess everyone whooshed.
The sentiment is good but the solution is wrong.
The ideal flow, IMHO:
1: Emit warning, and include if possible a deadline ("will be removed in the next major version")
2: After X time, BEFORE the deadline, turn it into a compiler OR linter error, but keep the code
3: Do the appropriated notification in the channels about it of impending doom
4: Doom
If this is done correctly, we change the expectations from "warnings LOL!" to "warnings are serious business" that is missing in a lot of contexts
I thought the industry solved this by bundling breaking changes in major version updates.
V 1.0 - foo introduced
V 1.1 - foo deprecated, recommend bar
V 2.0 - foo removed, only bar
Users can stay on 1.x indefinitely, even if it never receives updates. Development continues on 2.x, eventually 3.x and so on. Users only experience breaking changes when they manually do a major version upgrade.
I had the same clever idea once. Deprecation warning, and it would (by the power of a C-Macro) auto-turn off when the relase x was reached, with louder and louder warnings before.
One day I came back from holidays. I had just broken a big go-live where the release number passed x. Date missed, next possibility in a few weeks. The team was pissed.
Yes they COULD have fixed the warnings. But breaking the go live was quite of of proportion for not doing so.
Could the clients have reasonably foreseen when the release x would occur?
Could they not have rolled back?
I have 2 better ideas;
1. Start charging money for/of deprecated APIs. Just like Oracle did it with Java 8.
- OR -
2. Add a timestamp-based RuntimeException to the very same code-path. If the date is greater than (>) the deprecation date, simply throw RuntimeException. for 100% of the requests, all the time.
Yes, it will page someone, and yes, it will get fixed immediately!
Please don’t do this. If you want to remove the function, remove it and insist on its removal when people complain. Don’t gradually break people.
For one customer I do maintenance on a piece of software.
Building it produces about two-three dozen deprecation warnings.
The whole software stack relies on a cluster of packages that stopped receiving updates 5 years ago.
The software is not facing end-users. But it does build using NPM and not via vendored packages.
To avoid those warnings, large parts of the code need to be rewritten using a different set of packages.
That doesn't get prioritised because it works.
The software sucks in many ways, but only from the perspective of an artisan.
The customer is happy to ignore warnings as long as the software does its job.
There isn't money in fixing things that work because it got old.
The incentives for the people putting the deprecation warnings in those packages don't align with the users of those packages. Their timelines and motives are different.
The article has a new note at the bottom:
> In case the sarcasm isn’t clear, it’s better to leave the warts.
Wow, that wasn't there from the start? That explains a lot of the comments here.
I don't think deprecation should come with hostile signalling like this, but if it did, it should be consistent, and escalating with subsequent releases, performance regressions on the deprecated path, starting at least one release after the deprecation warning, not wrong results.
And it should be explicitly mentioned in the deprecation warnings.
(You don't want to break systems, but you want something people who care about the system will investigate, and will quickly find and understand the source of and understand what to do.)
(Notwithstanding that this is a joke) Maybe it's just me, but I read this as a solution that would be implemented internally at a large company to distribute pain/accountability/tech-debt across time to a team which might have high turnover. i.e., a way to align incentives by punishing teams with bombs (via their metrics) in their code, before the bomb actually detonates.
Never do this on the job. Its called corporate sabotage and will result in a lot of trouble for you.
Its also a generally bad idea.
The whole point of deprecation is to let others choose a moment to switch over on their own moment of choosing. If you don't want to maintain it just remove it. It is far nicer behavior.
Don't force others in a debugging hell. It makes you a bad person.
This is a bad idea. But in the same spirit, a bad but less bad idea: require the user to actively enable the deprecated feature via config or something, but have a known schedule to eventually expire it (with a hard error, not randomly incorrect data) and force the user to re-enable it. Causes some pain but not random, hard to track down bug. More like dealing with expiring certs - eventually, the user will want a permanent fix.
In a bit roundabout way Java 11 added `forRemoval` attribute to `@Deprecated` annotation so developers could indicate they really mean it.
The underlying complaint seems to be about libraries, rather than user behavior.
Occasionally returning wrong results is irresponsible, of course, but how about making deprecated APIs grow gradually slower?
> The response.getheader method in urllib has been deprecated since 2023 because the response.headers dictionary is what should be used instead.
Then response.getheader should just do that. There is no reason to expose the implementation to the user unless performance is critical.
> The response.getheader method in urllib has been deprecated since 2023 … When the method was eventually removed, lots of code broke.
Two years doesn't seem long to me for a widely used project, unless you have an LTS version that people needing more stability can use, or you are upfront that your API support is two years or less. Of course API support of less than two years is fine, especially for a project that people aren't paying for, but personally I would be quite explicit from the outset (in fact I am with some bits I have out there: “this is a personal project and may change or vanish on a whim, use it in any workflow you depend on being stable at your own risk”). Or am I expecting a bit much there?
If using semver or similar you are fine to break the API at a major release point, that is what a major release means, though it would be preferable for you to not immediately stop all support for the previous major version.
> What if we intentionally made deprecated functions return the wrong result … sometimes?
Hell no. A complete break is far preferable. Making your entire API essentially a collection of undefined (or vaguely undefined) behaviours is basically evil. You effectively render all other projects that have yours as a dependency, also just collections of vaguely defined behaviours. If your API isn't set in stone, say so then people have nothing to complain about unless they specifically ask you to keep something stable (and by “ask you to keep something stable” I mean “offer to pay for your support in that matter”).
> Users that are very sensitive to the correctness of the results…
That is: any user with any sense.
> might want to swap the wrong result for an artificial delay instead.
That is definitely more palatable. A very short delay to start with, getting longer until final deprecation. How short/long is going to be very dependent on use case and might be very difficult to judge: the shortest needs to be long enough to be noticeable to someone paying attention and testing between updating dependencies and releasing, but short enough that it doesn't completely break anything if someone updates and releases quickly, perhaps to bring in a bugfix that has begun to affect their project.
This is still evil, IMO, but a much lesser evil. I'd still prefer the complete break, but then again I'm the sort of person who would pay attention to deprecation notices, so unless you are a hidden nested dependency I'd not be affected.
This is silly and ignores how things are "deprecated" in every other practice on earth. Do we still install knob and tube electricity? Absolutely not. Is there still some out there that somebody is supporting some? Of course.
Does this mean that people and places shouldn't migrate out of older practices? No. But people have different priorities. And sure, we may treat "squeaky wheel policies" as a bad idea, but quite frankly that is far and away the most common policy out there.
To that end, please don't go out of your way to insist that your priority is everyone else's priority.
[deleted]
Make the deprecated method slower with a function from the deprecation release so the longer it goes on until it gets to around a day.
then remove the function.
« When the method was eventually removed, lots of code broke. »
Well, removing it earlier would just mean that lots of code would break earlier...
> The response.getheader method in urllib has been deprecated since 2023 because the response.headers dictionary is what should be used instead.
Who cares? This is such a trivial example, where there is no maintenance cost associated with leaving a dummy method in that does a dictionary lookup. I understand there are times when maintenance costs are substantial, but it's hard to take someone seriously when this is their example.
The obsession with newness, and stigmatizing something that doesn't conform to your sense of chasing newness, is really silly.. can you believe it's been deprecated since 2023 and someone still uses it???? 2023 was not some kind of dark ages of distant history.
Chaotic Evil code.
[deleted]
Ugh, I am finding it hard to express how much I hate this.
Software developers have enough treadmills they need to stay on. Deliberately breaking backward compatibility in the name of "deprecating something old" doesn't have to be one of them. Please don't be that platform or library that deprecates and removes things and makes me have to dust off that old software I wrote in 2005 to move over to a different set of APIs just to keep it working.
Instead randomly trigger kill self process. That's a safer way than random wrong returns.
Wow, tell me you've never shipped anything important / heavily used without telling me that.
I expected this to suggest a tick-tock deprecation cycle, where one version deprecated and the next removed, but this is definitely an idea that belongs on the domain "entropicthoughts.com"
How about never deprecate
sure - because i rewrite all my code every year!
Well written, but author is at the risk of tripping over Poe's Law, especially if readers don't read to the bottom.
Author's point is that their modest proposal (and its sibling, introducing intentional delays in resolving the deprecated API pieces) is a bad idea. Instead, author suggests that making the API change at all is begging the question "Who does this API serve?" It is, perhaps, okay actually if the old system never gets deprecated.
> especially if readers don't read to the bottom.
That note at the bottom was only added after the article was posted.
What about a sleep? Each release that the feature gets more deprecated, the program adds a couple milliseconds of just blocking. Less destructive, and gives a nice slow ramp up with real but non-destructive consequences.
Profoundly awful idea.
Filed under: trying to solve another domain's problem with only the tools in your domain (this is almost always a bad idea)
Thats madness. Just remove deprecated functions after a reasonable time period.
Another idea is to make the deprecation warning increasingly scary sounding.
"DEPRECATED: i will literally kill you"
"Continuing to use this feature is a moral failure on your part and you should be ashamed, your parents should be ashamed, and dishonor will follow you wherever you go."
"If you don't migrate I will tell your mum"
sorry, i'm not afraid of this. i'll keep using the feature.
Code churn is an evil, negative value activity. Design sanely and carefully before settling upon a published, standard contract. Churn of external-facing APIs should be extremely rare to solve essential critical problems, not for pseudo-maintenance bikeshedding. Also, the "it's a hobby" excuse doesn't fly.
tl;dr: Stop changing and breaking shit unnecessarily.
Jesus, what a terrible idea. This is such a terrible idea that I would not hire this guy based on this post alone.
What I want from code is for it to a) work, and b) if that's not possible, to fail predictably and loudly.
Returning the wrong result is neither of the above. It doesn't draw attention to the deprecation warnings as OP intended--instead, it causes a mysterious and non-deterministic error, literally the worst kind of thing to debug. The idea that this is going to work out in any way calls into question the writer's judgment in general. Why on earth would you intentionally introduce the hardest kind of bug to debug into your codebase?
Please do the opposite. Let all deprecation warnings last at least a decade, just include in the warning that it is not maintained.
But more to the point, go out of your way to avoid breaking backwards compatibility. If it's possible to achieve the same functionality a different way, just modify the deprecated function to use the new function in the background.
My biggest problem with the whole static typing trend is that it makes developers feel empowered to break backwards compatibility when it would be trivial to keep things working.
edit: Not that it is always trivial to avoid breaking backwards compatibility, but there are so many times that it would be.
> just include in the warning that it is not maintained.
I'm convinced this isn't possible in practice. It doesn't matter how often you declare that something isn't maintained, the second it causes an issue with a [bigger|more important|business critical] team it suddenly needs become maintained again.
And here's where your business can contact me to talk about a support contract.
If it's important, they'll pay. Often you find out it wasn't that important, and they're happy to figure it out.
It sounds like you're imagining open source whereas the comment you're replying to is imagining more intra-company dependencies.
I think deprecation in intra-company code is a completely different beast. You either have a business case for the code or not. And if something is deprecated and a downstream project needs it, it should probably have the budget to support it (or code around the deprecation).
In many ways, the decision is easier because it should be based on a business use case or budget reason.
The business case is the easy part, the quagmire is in getting the different teams to agree who should support the business case, why it's more important than the business cases they wanted to spend cycles on instead, and how much of the pie supporting it takes on the budget side. Less so when the place is small enough everyone knows everyone's name, more so when it's large enough they really don't care what your business case is much even though it'd be 10x easier to support from their side instead of another.
Oh. But that is a solved problem. The users of the library just copy the code from before the deprecation and then stick it in their codebase not to be maintained anymore. Problem solved. /s
> I'm convinced this isn't possible in practice.
I don't agree. Some programming languages started supporting a deprecated/obsolete tagging mechanism that is designed to trigger warnings in downstream dependencies featuring a custom message. These are one-liners that change nothing in the code. Anyone who cares about deprecating something has the low-level mechanisms to do so.
For sure, but for this to work you need someone downstream to notice those messages and prioritize the work to migrate off the deprecated code paths. Some teams will respond, but many won't. No matter how loudly you declare that the code is deprecated, you'll still have people using it up to the point it stops working.
It's far better to plan the removal of the code (and the inevitable breaking of downstream users systems) on your own schedule than to let entropy surprise you at some random point in the future.
I don't know that I see why/how this is a problem? You would do the same with any other thing in your life?
More, in many things, we have actively decided not to do something anymore, and also highly suggest people not mess with older things that did use it. See asbestos. Removing it from a building is not cheap and can be very dangerous.
It also keeps slowing down development as getting a green global compile will make you still update "deprecated" functions that face breaking API changes.
> My biggest problem with the whole static typing trend is that it makes developers feel empowered to break backwards compatibility when it would be trivial to keep things working.
I don't see the connection you're drawing here.
Disclaimer: I’m a strong advocate for static typing.
I absolutely see the connection. One of the advantages of static typing is that it makes a lot of refactoring trivial (or much more than it would be otherwise). One of the side effects of making anything more trivial is that people will be more inclined to do it, without thinking as much about the consequences. It shouldn’t be a surprise that, absent other safeguards to discourage it, people will translate trivial refactoring into unexpected breaking changes.
Moreover, they may do this consciously, on the basis that “it was trivial for me to refactor, it should be trivial to adapt downstream.” I’ll even admit to making exactly that judgment call, in exactly those terms. Granted I’m much less cavalier about it when the breaking changes affect people I don’t interface with on a regular basis. But I’m much less cavalier about that sort of impact across the board than I’ve observed in many of my peers.
Rather, static typing empowers backwards compatibility, right?
Because it lays out the contract you have to meet on the interface. No contract? No enforced compatibility.
It is a way to signal to other developers that your code changed, which is better than not saying anything.
But it seems to make library developers more comfortable with making breaking changes. It's like they're thinking 'well it's documented, they can just change their code when they update and get errors in their type checker/linter.' When I think they should be thinking, 'I wonder what I could do to make this update as silent and easy as possible.'
Of course, we all have different goals, and I'm grateful to have access to so many quality libraries for free. It's just annoying to have to spend time making changes to accommodate the aesthetic value of someone else's code.
But it also makes it easier for library developers to identify that a change is breaking. Without static types you can easily make a change you think is fine, but it ends up breaking user code because they depended on something you didn't expect. Granted, static typing doesn't completely solve that problem, there can still be changes in behavior that break things, but it is a useful tool in identifying certain classes of breaking changes.
yes, that is my point.
I think the only time I’ve ever seen this is in a JS library, where each major version bump is practically a different library altogether because it’s a complete rewrite, and these days is a wrapper around some Rust thing.
Not even JS alone. I blame the enforcement of semantic versioning, as if a version of code simply had to be a sequence of meaningful numbers.
If you are using a language with a complete type system, sure. But who uses those?
When using the languages people actually use in the real world, not really. Consider a simple example where the contract is that you return an integer value from 1 to 10. In most languages people actually use, you're going to be limited to using an integer type that is only constrained by how many bits it is defined to hold, which can be exploited later to return 11, unbeknownst to the caller's expectations. There are a small number of actually-used languages that do support constraining numeric types to a limited set of values, but even they fall apart as soon as you need something slightly more complex.
This is what tests are for. They lay out the contract with validation of it being met, while also handily providing examples for the user of your API to best understand how it is intended to be used.
More than anything, this reveals your lack of understanding of modern languages and type systems.
TypeScript, for example, is one of the most widely used languages in the world. It has an incredibly powerful type system which you can use to model a lot of your invariants. By leaning on patterns such as correct-by-construction and branding, you can carry around type-level evidence that e.g. a number is within a certain range, or that the string you are carrying around is in fact a `UserId` and not just any other random string.
Can you intentionally break these guarantees if you go out of your way? Of course. But that's irrelevant, in the same way it is irrelevant that `any` can be used to break the type system guarantees. In practice, types are validated at the boundaries and everything inside can lean on those guarantees. The fact that someone can reach in and destroy those guarantees intentionally doesn't matter in practice.
Typescript is one of the actually-used languages that supports constraining the integer, albeit in a pretty awkward way.
But now try defining a type that enforces a RFC-compliant email address...There are languages with proper type systems that can define full contracts, but Typescript is not among them. Without that, you haven't really defined a usable contract as it pertains to the discussion here. You have to rely on testing to define the contract (e.g. assert the result of the 'generate email' function is RFC-complaint).
And Typescript most definitely does. Testing is central to a Typescript application. It may have one of the most advanced type systems found in languages actually used, but that type system is still much too incomplete to serve as the contract. Hence why the ecosystem is full of testing frameworks to help with defining the contract in tests.
You clearly don't understand the patterns I described. Look up branding and try reading e.g. "parse, don't validate".
"Parse, don't validate", while familiar to all HN users (it feels like it gets posted here every week), is orthogonal and does not address what we are talking about. To extrapolate, let's stay with the email address type example.
In Typescript, you can define an EmailAddress type as:
If you are feeling saucy, you can even define it as: But nether of these prove to me, the user of your API, that an EmailAddress value is actually a RFC-compliant email address. Your "parse" function, if you want to think in those terms, is quite free to slip in noncompliant characters (even if only by accident) that I, the consumer, am not expecting. The only way for me to have confidence in your promise that EmailAddress is RFC-compliant is to lean on your tests written around EmailAddress production.That isn't true for languages with better type systems. In those you can define EmailAddress such that it is impossible for you to produce anything that isn't RFC-compliant. But Typescript does not fit into the category of those languages. It has to rely on testing to define the contract.
Let's imagine we are working with some hypothetical language in which what you describe is possible.
At some point, you will have to write a function where you validate/parse some arbitrary string, and it then returns some sort of `Email` type as a result. That function will probably return something like `Option<Email>` because you could feed it an invalid email.
The implementation for that function can also be wrong, in exactly the same way the implementation for the typescript equivalent could be wrong. You would have to test it just the same. The guarantees provided by the typescript function are exactly equivalent, except for the fact that you do technically have an escape hatch where you can "force" the creation of a branded `Email` without using the provided safe constructor, where the other language might completely prevent this - but I've already addressed this. In practice, it doesn't matter. You only make the safe constructor available to the user, so they would have to explicitly go out of their way to construct an invalid branded `Email`, and if they do, well, that's not really your problem.
I could see some maintainers being encouraged by static typing to introduce breaking changes, as long as the changes ensure compile-time breakage. However, that’s only an option in languages without dynamic linking. So it’s the combination of static typing and static linking that would encourage that.
Indeed. C# is statically typed. Python is not. Which is the one with a notorious backwards-incompatibility story?
Both.
https://learn.microsoft.com/en-us/dotnet/csharp/whats-new/br...
Come on, those are very rare and minor things that are very easy to fix.
Port me something from DotNet 9.0 and say that again.
https://learn.microsoft.com/en-us/dotnet/core/compatibility/...
We have already ported many projects from .NET8 and .NET9 to .NET10 with close to zero issues.
Guessing you never used OpenAPI or Razor then. Which were core features in 9 and no longer existed in 10.
What are you talking about?? OpenAPI and Razor are both still present in .NET10.
Yeah, I'm confused at these examples, along with the implicit idea that the existence of breaking changes means C# had equivalent compatibility challenges to, say, python3.
.NET Framework -> Core was more persuasive, but I stand by the overall point that compatibility is more about project philosophy than "static vs dynamic typing" and indeed I think Framework/Core illustrates just that: Framework more heavily favored preserving compatibility compared to Core
edit: I assume this was a superficial grasp onto "Deprecation of WithOpenApi extension method" ("WithOpenApi duplicated functionality now provided by the built-in OpenAPI document generation pipeline") and "Razor runtime compilation is obsolete" ("Razor runtime compilation has been replaced by Hot Reload, which has been the recommended approach for a few years now.")
Of course, its an entirely trivial process to completely replace serialisation everywhere in your product and swap out your own custom hot reloading compilation pipeline you created before the language created its own one incompatible with your API...
The .NET framework to .Net Core migration broke a bunch. Similarly java 6 to 8, and when they introduced the new module system in Java 11 broke a bunch in another case. You’d still need to fixed that they swapped the string types, renamed a bunch of core APIs etc. in a statically typed python
Those are good examples. But python is dynamically typed -- indeed, python 3 is what I was alluding to
That being said, I think backwards compatibility comes down to project philosophy and foresight (e.g. https://learn.microsoft.com/en-us/dotnet/standard/design-gui...), not static versus dynamic typing
I was hoping to spin it as an environmental concern. You change something "for reasons". If you're doing this in a popular tool/library you probably just burnt down a small forest of trees worth of energy from all of the work people will have to put in deal with it. 100s of thousands of users missed dinner with a kid or loved one because of the extra work you created. A product that would of shipped on time didn't. Things you wanted to exist didn't because 100k people were distracted dealing with your breakage.
> Not that it is always trivial to avoid breaking backwards compatibility, but there are so many times that it would be.
In this case it was 2 functions with 1 line of code each. https://github.com/urllib3/urllib3/pull/3732/files
Wow. Why even remove it? It's just the thinnest wrapper around the dict and since the dict is now part of the public API these methods will work forever unmodified.
Static Typing != Strongly Maintained Relationships.
While I am happy to see types in Python and Javascript (as in Typescript) I see far more issues how people use these.
In 99% of the time, people just define things as "string" or if doesn't cover, "any". (or Map/Object etc)
Meanwhile most of these are Enum keys/values, or constants from dependencies. Whenever I see a `region: string` or a `stage: string`; a part of me dies. Because these need to be declared as `region: Region` or `stage: Stage`. Where the "Region" and "Stage" are proper enums or interfaces with clear values/variables/options. This helps with compile (or build) time validation and checking, preventing issues from propagating to the production (or to the runtime at all)...
Stripe do this in a cool way. Their REST API is version based on date, and each time they change it they add a stackable compatibility layer. So your decade old code will still work.
https://stripe.com/blog/api-versioning
As with all things, this can be pushed too far. Microsoft was suffering a maintainability crisis before the transition to Windows XP; their years of bending the API to support customers (which, in the short run, did keep customers to their benefit) was making for a fairly unmaintainable mess of an API surface that then screamed under the strain when the Internet era hit and all those open, under-observed APIs became potential worm attack vectors.
Actually, I think backwards compatibility is the reason Windows still dominates. It isn't the best OS at anything and it's actively user hostile, but if you want to run Windows apps... there are decades of Windows apps and they basically all work.
this is the answer.
No matter what others say, the pipelines are long.
The delays for release, getting into a distribution, then living out its lifetime... they are significant.
One thing I've always hated was this idea of "bitrot." That software just spontaneously stops working after time and loses backwards compatibility. Like it's some force of nature.
It's not a force of nature. Bitrot is: many software developers deliberately choosing to break backward compatibility in very small ways over and over. Software written in 1995 should still work today. It's digital. It doesn't rot or fall apart. It doesn't degrade. The reason it doesn't work today is decisions that platforms and library maintainers deliberately made. Like OP. Deprecate like you mean it. That's a choice!
If we want to solve bitrot, we need to stop making that choice.
Progress is often only possible by breaking things. It's not a choice, it's the only way forward. We have to optimize for the future being better, even if it makes the present a little worse occasionally.
This is a huge reason why open source projects are often so much more successful than corporate clones: they actually iterate and innovate, something corporate america has forgotten how to do.
Some deprecations are required to make progress. Many others aren't, and those are the most frustrating.
For example, ruby deprecated the `File.exists?`, and changed it to `File.exist?`, because enough people felt that the version with the `s` didn't make sense grammatically (which I disagree with, but that is not germane to my point).
For a long time, you would get warning that `exists?` was deprecated and should be replaced by `exist?`.... but why? Why couldn't they just leave `exists?` as an alias to `exist?`? There was not cost, the functions are literally identical except for one letter. While the change was trivial to fix, it added annoyance for no reason.
Although, luckily for me, with Ruby I can just make exists? an alias myself, but why make me do that?!? What is the point of removing the method that has been there forever just because you think the s is wrong?
Imagine I am new to Ruby. I go looking to figure out if a file exists. Or I get a list of available methods from the file object. Now I have to research both `exist?` and `exists?` to figure out if I should be choosing one or the other.
Now multiply that by.... every past history of every API and it makes adopting something really difficult as a newcomer.
aka Common Lisp.
It doesn’t matter which one you do, the methods are literally identical other than the extra s. You can’t pick the wrong one.
This sort of choice is very common in Ruby, you can have different style choices for the same functionality. You can have blocks made with {} or do and end. You can do conditionals in the form of “a if b” or “if b then a”. You can call methods with or without parentheses.
These are all Ruby style choices, and we already have a way to enforce a particular style for a particular project with formatters like rubocop.
I'm not defending the Ruby decision, I literally said I wouldn't have done it. It reminds me of the disttools removal from python, which was also not something I'd have done nearly so aggressively if it were up to me. Multiple distros have patched it back in.
But it wasn't up to me. It's not my project. I'm using somebody else's project, and at the end of the day it's their decision, because they own it. Unless it's impossible to work around, I feel like I have to either respect that, or switch to an alternative.
You're free to maintain a patch on top of Ruby to add the alias and run that on your machines, btw. It would probably be very simple, although certainly not as simple as aforementioned sed command...
> You're free to maintain a patch on top of Ruby to add the alias and run that on your machines, btw.
Comments like this are honestly just asshole-ish.
Why? It's true. It's not even really that hard. My point is nobody is forcing them to accept it.
Because Ruby is nothing without the community around it. If they wanted to make Ruby and don’t care about the people using it, they should go private, shut down the project, or have someone else take over.
It’s wrong to shut down discussion like this with comments like “it’s their code”, “make your own fork”, etc. because Ruby is supposed to be part of the open source community, which implies collaboration, give and take, discussion of pros and cons, etc.
What you are doing is ignoring this major aspect of a programming language and taking this weird anti social stance on it.
I'm not shutting down any discussion, I'm simply stating my opinion. You're free to disagree.
I didn't say to fork it. Do you really not appreciate the difference between rebasing a trivial patch forever, and maintaining a wholesale fork forever?
Crowing on saying “it’s not your code, they can do what they want” is an attempt to do that.
Fork, patch, maintaining your private whatever, that’s not the point, and is a digression.
It's a perfectly valid point, you just disagree. I'm not the one shutting down discussion here.
Of course it is their decision, and they don’t OWE me an explanation.
But this is a discussion forum, and I am asking for people who agree with the decision to explain why they agree. Again, they don’t have to answer me if they don’t want to. I am just saying, “if anyone knows an argument for this type of change, I would love to hear it”
Saying they don’t have to explain their reasoning is true but not really relevant to our conversation. I am not asking THEM, I am asking HN readers.
Your example could be fixed in your entire codebase with a single sed invocation. While yes, if it were up to me I'd probably just have left the alias, the annoyance for the end user does seem quite minimal...
You get to use open source projects for free, and a lot of people do ongoing maintenance on them which you benefit from for free. In return, sometimes you are expected to modify your code which depends on those projects because it makes their maintainer's life easier.
Personally, I see that as a very reasonable trade-off.
The third option is to use things that don't break themselves. One of the many joys of Go is that I can leave personal projects untouched for years and then go back and everything is still working
> Your example could be fixed in your entire codebase with a single sed invocation
I also have to hope all the dependencies I use did that, too.
But my real question is why? Why make me do it at all?
Because they wanted to, and it's their project. Part of being an adult is accepting you won't agree with every decision that affects you.
You were free to show up and argue against it, as was anyone else. Did you?
Ok, I think you are arguing against something I am not saying.
I am not arguing that they don’t have the RIGHT to make the change, or that they owe me personally anything. I am not even THAT mad. I still love Ruby the most of any language, and generally think they make the right decisions.
I am simply annoyed by this decision.
And yes, I argued against this change when it was first proposed (as did many others). They obviously were not convinced.
Again, I am not arguing that they should be FORCED to do what I want, or that they did something shady or nefarious by making this change. I am not asking for any remedy.
I am simply saying I disagree with this type of change (changing a method name because some people feel the new name makes more grammatical sense, but not changing the method itself at all). The reason I commented was because this is not a “we have to deprecate things for progress” situation. They aren’t limited in any way by the current method, the syntax isn’t cumbersome (and isn’t changing), there is no other capability that is held back by having to maintain this method. It is literally just “I read it as asking ‘Does this file exist?’ rather than asking ‘This file exists?’”
Again, they are obviously free to disagree with me, which they do. I am simply arguing that we shouldn’t break syntax just because you like the way it reads better without an s. And I am asking for someone who disagrees with me (you) to explain why it is worth making this type of change.
It's important for code to be clean and free of old cruft, even if that means old code has to change. Readibility is one of the most important aspects of maintainable code. Arguably more important than testing, IMHO. This is another reason open source solutions so consistently win over corporate ones, the open source world understands this.
Are there changes I disagree with? Of course. But I'd rather live in a world that moves forward and occasionally breaks me, than one where I have perfect compatibility but am stuck on code lacking the new innovations my competitors benefit from.
The whole idea behind deprecating things is to give people time to make the changes before they become breaking.
I went and looked: exists? was marked as deprecated in 2013 and removed in 2022. That's enormously generous, my previous comparison with the disttools debacle in python was inaccurate. You had a decade!
You've moved the goal posts quite a bit from your previous post to this one. Originally, you said that "progress is often only possible by breaking things... it's the only way forward". But the Ruby example that cortesoft provided isn't necessary to make progress. The language would get by just fine without that deprecation (or even without making the change at all). Saying "but it's a trivial change" in response is irrelevant, because you weren't originally trying to justify changes on the basis that they were easy, but that they were necessary.
> Saying "but it's a trivial change" in response is irrelevant
Of course it's relevant. It's a laughably trivial example compared to the other one in this thread.
It's absolutely a choice. All software can progress while preserving backward compatibility for existing users. It's not always easy, but it's never impossible.
> All software can progress while preserving backward compatibility for existing users
That's an incredibly ignorant claim. Just run "git log" in glibc, it won't take you very long to prove yourself wrong.
glibc, which has had ABI compatibility for decades?
There have been plenty of build breaking changes over the past couple decades, generally they happen for very good reasons and only affect niche usecases.
So existing binaries will keep working?
Not necessarily if you go back far enough to have nss problems :)
Not always without a performance cost which is a no go in projects with very expensive hardware (ahem Cisco).
Actually this is the reason why Win32 is the stable ABI for Linux.
https://blog.hiler.eu/win32-the-only-stable-abi/
While I certainly empathize with the author of that article and respect the work they put into debugging it, this is a better explanation of that whole thing: https://maskray.me/blog/2022-08-21-glibc-and-dt-gnu-hash
God, would I hate to be programming now the way I had to program in 1995.
Granted, modern coroutines do bring up some nostalgic feel for the days I had to support cooperative multitasking...
Isn't that arguing for keeping the maintenance cost high and also stagnating the design?
> That would probably not trigger anyone’s midnight pager, but it would make it clear that relying on the deprecated functionality is a bug lurking in the code.
How do you know? This is a wild assertion. This idea is terrible. I thought it was common knowledge that difficult to reproduce, seemingly random bugs are much more difficult to find and fix than compiler errors.
If you're ready to break your api, break your api. Don't play games with me. If more people actually removed deprecated APIs in a timely manner, then people will start taking it more seriously.
Last paragraph of the article:
> In case the sarcasm isn’t clear, it’s better to leave the warts. But it is also worthwhile to recognise that in terms of effectiveness for driving system change, signage and warnings are on the bottom of the tier list. We should not be surprised when they don’t work.
Yeah, totally a woosh moment for me. Read all the way up to the `* * *`. That's on me :)
That last bit was added to the article after your comment, as the author realised the sarcasm had been too subtle for most people to catch.
Most HN visitors won’t read to the last paragraph, so it’s a good thing to emphasize.
I thought I had read it. :) I thought the three `* * *` at the bottom was indicating I was about to start reading suggestions for the next article. So definitely a "Woosh" moment for me :D
Yeah, I agree - this sort of intermittent failure could be incredibly hard to track down, and will absolutely fuck with people's faith in their CI systems as well - a flappy test is the absolute worst kind of test.
I agree, maintainers should just break the API if they're going to do it.
At the same time, it's crazy that urllib (the library mentioned in the article), broke their API on a minor version. Python packaging documentation[1] provides the sensible guideline that API breaks should be on major versions.
[1] https://packaging.python.org/en/latest/discussions/versionin...
This isn't a good idea regardless of why it's being deprecated.
If it's no longer being maintained then put a depreciation warning and let it break on its own. Changing a deprecated feature just means you could maintain it but don't want to.
Alternatively if you want to aggressively push people to migrate to the new version, have a clear development roadmap and force a hard error at the end of the depreciation window so you know in advance how long you can expect it to work and can document your code accordingly.
This wishy-washy half-broken behaviour doesn't help anyone
The post is a sarcasm, see the below.
Having a deprecated API just randomly return failures is an awful idea!
Better to give an actual timeline (future version & date) for when deprecated functionality / functions will be removed, and in the meantime, if the language supports it, mark those functions as deprecated (e.g. C++ [[deprecated]] attribute) so that developers see compilation warnings if they failed to read the release notes.
Yep. I'll admit I've acted faster to hard set dates than some "in the future" message. I've also seen some tools become really noisy about deprecation spanning many lines AND repeating. Please don't log the same message over and over for each instance. Color or add emoji if you must to grab attention, but once is enough. It's annoying when you can't do anything about it at that time and have to sift through this extra noise when hunting down another issue in the CI log. Add a link that goes over it in more detail and how to migrate for that specific deprecation.
I don't think that was a serious suggestion, the article says it is sarcasm near the end.
But yes, that would be the worst idea ever.
Random is bad, but scheduled windows of breakage is good.
How are you supposed to deal with recalcitrant users? I work for an organization that is ending support for several long running APIs. And by support I mean turn off the servers, you must move to an entirely new platform.
We’ve sent out industry alerts, updated documentation and emailed all user. The problem is the contact information goes stale. The developer who initially registered and set up the keys, has moved on. The service has been running in production for years without problems and we’ve maintained backwards compatibility.
So do we just turn it off? We’ve put messages in the responses. But if it’s got 200ok we know no one is looking at those. We’ve discussed doing brownouts where we fail everything for an hour with clear error messages as to what is happening.
Is there a better approach? I can’t imagine returning wrong data on purpose randomly. That seems insane.
> How are you supposed to deal with recalcitrant users?
Keep the servers running, but make the recalcitrant users pay for the costs and a then some more. It is actually a common strategy. Big, slow companies often have trouble with deprecation, but they also have deep pockets, and they will gladly pay a premium so that they can keep the API stable at least for some time.
If you ask for money, you will probably get more reactions too.
Step 1: Stop thinking of them as "recalcitrant". They're not recalcitrant. They bought (presumably for money) a product, and expect that product to keep working as long as they need it to! They don't expect the vendor to pull the rug out from under them and break it just because the API is old and icky and their software engineers are sad to keep it around.
Instead of "deprecate like you mean it" the article should be: "Release software like you mean it" and by that, I mean: Be serious. Be really, really sure that you are good with your API because users are going to want to use it for a lot longer than you might think.
> They bought (presumably for money) a product, and expect that product to keep working as long as they need it to!
This depents on the terms of the contract. Typically, termination of service is covered in a license. If the license terms are okay in the respective jurisdiction, there is no fundamental ethical obligation to run a server beyond that. There might exist specific cases where it would be inappropriate to follow the terms by the letter, but that also has its limits.
Contract terms usually define legal obligations, not ethical obligations. They create duties parties must perform or face legal consequences--they don't speak to what those parties should do ethically.
Following legal obligations is an important part of ethics. The law has also the purpose to relieving the individual of the burden of complex ethical considerations. This is the general situation, especially in a democracy under the rule of law.
There are, of course, exceptions and disagreement about specific regulations. But as long as you have the law on your side is a very strong indicator that what you are doing is also ethical more or less okay. It is very hard to say that one person is far off ethically, if two people agreed on something and the terms of their agreement are without doubt legally correct.
Most systemic evil in the modern world is done legally, IMO. There is everything legal but nothing ethical about the way John Deere screws farmers over, how big tech sells your data, how Amazon creates a consumerist race to the bottom, how United Healthcare denies you coverage capriciously, etc.
I appreciate your viewpoint, even though mine is the exact opposite!
Nothing lasts forever. The second you decide to use a new 3P API you have to understand it might disappear one hour after your production launch, and that's okay.
You also have to understand that there will be APIs that eagerly rugpull and APIs that don't, and if you're offering the former then your users will end up moving to your competitors all else being equal.
Software evolves over time, along with business needs. What seemed like (or even was!) a good idea at some point will almost surely cease to be a good idea at some point in the future. Breaking the API is totally fine if there's a good reason and it's carefully managed.
I agree, get the API right as much as possible so it doesn't need to be deprecated.
But, perfection isn't realistic. If you don't have a plan for when you get things wrong, you're failing to plan for the inevitable.
Sleep()s that increase exponentially every month seem like a good solution. When the API has a 10 second latency hopefully someone starts asking questions. If not I think brownouts are a decent idea.
> We’ve discussed doing brownouts where we fail everything for an hour with clear error messages as to what is happening.
That sounds like the best option. People are used to the idea that a service might be down, so if that happens, they’ll look at what the error is.
A technique I used on a project was to change the URL, and have the old URL return a 426 with an explanation, a new link, and a clear date when the moved API. This reliably breaks the API for clients so that they can't ignore it, while giving them an easy temporary fix.
Clients weren't happy, but ultimately they did all upgrade. Our last-to-upgrade client even paid us to keep the API open for them past the date we set--they upgraded 9 months behind schedule, but paid us $270k, so not much to complain about there.
So did you return the correct data with the 426 Code?
No--the goal was to break the API so users noticed, with an easy fix. A lot of users weren't even checking the HTTP status codes, so it was necessary to not return the data to make sure the API calls broke.
We did roll this out in our test environment a month in advance, so that users using our test environment saw the break before it went to prod, but predictably, none of the users who were ignoring the warnings for the year before were using our test environment (or if they were, they didn't email us about it until our breaking change went to prod).
That must have been one heck of a rug pull if $270k was considered more cost-effective than getting someone to work on it sooner.
The solution I've found is to make using the API a hard error with an explicitly temporary and obnoxiously-named workaround variable.
It's loud, there's an out if you need your code working right now, and when you finally act on the deprecation, if anyone complains, they don't really have legs to stand on.Of course you can layer it with warnings as a first stage, but ultimately it's either this or remove the code outright (or never remove it and put up with whatever burden that imposes).
I heard a second-hand story about some team at Google who did this, and named the variable something like:
a year before the deadline. I would be mildly amused by addingLove this idea, as it is already implemented in a $FAANG company tool. (used company wide by 80k+ SDEs). I got used to seeing these in the logs and terminal. So much that my brain now automatically ignores it from my view/seeing like it does for my nose.
Don't make it return a different result.
Instead, if you must, add a sleep within the function for 1 ms in the first release, 2 ms in the second release, and so on. But maybe just fix the tooling instead to make deprecations actually visible.
I get the intention but it's a bad idea, same with the article
if people are meant to depend on your endpoints, they need to be able to depend on all of them
you will always have ppl who don't respond to deprecation notices, the best you can do is give them reliable information on what to expect -- if they hide the warnings and forget, that's their business
but intentionally making problems without indication that its intentional results in everyone (including your own team) being frustrated and doing more work
you cannot force ppl to update their code and trying to agitate them into doing it only serves to erode confidence in the product, it doesn't make the point ppl think it makes, even if the court of public opinion sides with you
cover your bases and make a good faith effort to notify and then deal with the inevitable commentary, there will always be some who miss the call to update
this actually seems like an reasonable technical solution to the non-technical problem that causes deprecations to be ignored in the first place
Degrading performance exponentially (1ms, 2ms, 4ms, 8ms...) WILL create a 'business need', without directly breaking critical functions. Without this degradation, there is no reason to remove the deprecated code, from a business perspective.
It should slow down your compile instead of the runtime. That way the pain is felt by the developer who can fix it.
It is sometimes a tooling issue, but far more often it's a "few teams have a discipline of squashing all toolchain warnings" issue.
I get that it's nice to be able to deprecate/remove things.
But intentionally breaking my users runtime in a way that's really hard and annoying to find? Is the author OK? This reads like a madman to me.
It would help a lot if core Python libraries like urllib, NumPy, and others used SemVer. Removing a function is a breaking change. The root of this post is from urllib breaking something in a minor release. Get rid of the pseudo SemVer where you can deprecate functions and then break in minor releases. Maybe after that the rest of the community could hope their code will work long term if they don’t increment major versions.
That just encourages people to keep using old, unmaintained, insecure versions of libraries. Then, when they're still on version 2.1.1, and your maintained version is 5.7.3, and somebody finds a major security bug in 2.1, they will come whining at you to release a 2.1.2.
Code that is not being maintained is not usually suitable for use, period.
Library maintainers have no right to police how people use their open source code, period. Maintainers are also not obligated to backport security fixes. Anything else is effectively against the concept of open source.
Notably, even this policing doesn’t fix the whining. The whining will just be about what TFA is whining about. You’re just moving the whining around.
It also does nothing to actually force people to upgrade. Instead, people can just cap against the minor version you broke your package on. Instead of being user hostile, why not make the user’s job easier?
Correctly following SemVer disincentivizes unnecessary breaking changes. That’s a very good thing for users and ultimately the health of the package. If you don’t want to backport security fixes, users are free to pay, do it themselves, or stop using the library.
And then you can offer them a support contract to produce an update for an out of support version
Python itself is making breaking changes between minor versions, so I think its natural that the ecosystem will follow.
I have not had any real problems yet myself, but its worrying.
Python has never claimed to use semver, despite everyone associating the 3 in front with the 2->3 migration.
That just means its always been a problem, instead of being a new problem.
It does use major.minor.bugfix versioning, but without clarity about when to expect breaking changes.
With the pace of 3.x releases it has become more of a problem.
Numpy does use semver.
Lots of people still complained about 2.0.
No, it does not: https://numpy.org/doc/stable/dev/depending_on_numpy.html.
> It is important to know that NumPy, like Python itself and most other well known scientific Python projects, does not use semantic versioning. Instead, backwards incompatible API changes require deprecation warnings for at least two releases.
Have they made breaking changes to the ABI in other versions?
I’m not sure how that is relevant. Are you trying to suggest that NumPy in fact does follow SemVer despite their own documentation saying they don’t?
> Get rid of the pseudo SemVer where you can deprecate functions and then break in minor releases.
I agree, but I think there's a bigger, cultural root cause here. This is the result of toxicity in the community.
The Python 2 to 3 transition was done properly, with real SemVer, and real tools to aid the transition. For a few years about 25% of my work as a Python dev was transitioning projects from 2 to 3. No project took more than 2 weeks (less than 40 hours of actual work), and most took a day.
And unfortunately, the Python team received a ton of hate (including threats) for it. As a natural reaction, it seems that they have a bit of PTSD, and since 3.0 they've been trying to trickle in the breaking changes instead of holding them for a 4.0 release.
I don't blame them--it's definitely a worse experience for Python users, but it's probably a better experience for the people working on Python to have the hate and threats trickle in at a manageable rate. I think the solution is for people like us who understand that breaking changes are necessary to pile love on doing it with real SemVer, and try to balance out the hate with support and
I had a client who in 2023 still was on 2.7.x, and when I found a few huge security holes in their code and told them I couldn't ethically continue to work on their product if they wouldn't upgrade Python, Django, and a few other packages, and they declined to renew my contract. As far as I know, they're still on 2.7.x. :shrug:
That’s probably true. I do think part of the anger is that a lot of the changes didn’t clearly improve the code around it. The obvious example is the change to print from a statement to a function. It makes the language a little cleaner, but it also breaks existing code for little practical benefit. More insidious was the breaks with strings and byte strings. It was a good and necessary change that also could lead to weird quiet breakages.
At least for me, the real blocker was broad package support.
Maintainers should think carefully about whether their change induces lots of downstream work for users. Users will be mad if they perceive that maintainers didn’t take that into account.
No. Breaking other people's code is why we have this broken software world where new bugs (and security holes) are added everywhere all the time since no software is allowed to just exist without monthly rewrites. We need a culture where breaking backwards compatibility in published APIs is almost unthinkable, not something anyone would want to be caught doing.
Deprecation warnings are precisely that --- warnings --- so that people have the option not to act on them. So, it is of little surprise that users exercise one of the two available use case scenarios, which is not acting.
There are sophisticated users who care about the quality of their code and care about it breaking as infrequently as possible. Those users follow warnings. Not only warnings that happen by default; they use additional tooling to get extra warnings. The follow up on warnings.
Deprecation warnings serve those people.
As for the others, who cares. "We generously told you this would be removed, for years".
People who ignore warnings related to compatibility and have a workflow whereby your dependencies are not pinned down to specific versions, in some project configuration file, so that they are always getting the latest dependencies, are choosing to inflict breakages on themselves.
> What if we intentionally made deprecated functions return the wrong result … sometimes? Every time it intentionally returns the wrong result, it logs the deprecation warning.1
What if we found that a highway overpass construction material was suboptimal, and we want people to use superior materials, so, every now and then, we send a chunk of concrete plummeting down to the ground, to kill a motorist?
Thanks to deprecating like we mean it, they're going to replace that overpass sooner than they would otherwise. You'll thank me later.
From the https://sethmlarson.dev/deprecations-via-warnings-dont-work-... that the post opens with:
> This API was emitting warnings for over 3 years in a top-3 Python package by downloads urging libraries and users to stop using the API and that was not enough. We still received feedback from users that this removal was unexpected and was breaking dependent libraries.
Entirely predictable.
Even many of those who saw the deprecation logging, and bothered to make a conscious decision, didn't think you'd actually break the API.
> We ended up adding the APIs back and creating a hurried release to fix the issue.
Entirely predictable.
Save yourself some anguish, and don't break API unnecessarily. Treat it like a guarantee, as much as possible.
If it's a real problem for ongoing development, consider using SemVer and multiple versions, like the linked article suggests. (With the deprecated branch getting minimal maintenance: maybe only select bug fixes, or only critical security fixes, maybe with a sunset on even those, and a deprecation warning for the entire library when it's no longer supported.)
The various projects that say something is deprecated but then don't give a removal timeline or keep delaying the removal (or even explicitly say it won't be removed, just remain deprecated) are the cause of this problem.
IMO, any deprecation should go in the following steps:
1. Decide that you want to deprecate the thing. This also includes steps on how to migrate away from the thing, what to use instead, and how to keep the existing behaviour if needed. This step would also decide on the overall timeline, starting with the decision and ending with the removal.
2. Make the code give out big warnings for the deprecation. If there's a standard build system, it should have support for deprecation warnings.
3. Break the build in an easy to fix way. If there is too much red tape to take one of the recommended steps, the old API is still there, just under a `deprecated` flag or path. Importantly, this means that at this step, 'fixing' the build doesn't require any change in dependencies or (big) change in code. This should be a one line change to make it work.
4. Remove the deprecated thing. This step is NOT optional! Actually remove it. Keep it part of your compiler / library / etc in a way to give an error but still delete it. Fixing the build now requires some custom code or extra dependency. It is no longer a trivial fix (as trivial as the previous step at least).
Honestly, the build system should provide the tools for this. Being able to say that some item is deprecated and should warn or it is deprecated and should only be accessible if a flag is set or it is removed and the error message should say "function foo() was removed in v1.4.5. Refer to the following link:..." instead of just "function foo() not found"
If the build system has the option to treat warnings as errors, it should also have the option to ignore specific warnings from being treated as such (so that package updates can still happen while CI keeps getting the warning). The warning itself shouldn't be ignored.
It is wild to realize that this is the same author as https://entropicthoughts.com/you-want-technology-with-warts
There is a new last line of TFA:
> In case the sarcasm isn’t clear, it’s better to leave the warts. But it is also worthwhile to recognise that in terms of effectiveness for driving system change, signage and warnings are on the bottom of the tier list. We should not be surprised when they don’t work.
Looking at comments I guess everyone whooshed.
The sentiment is good but the solution is wrong.
The ideal flow, IMHO:
1: Emit warning, and include if possible a deadline ("will be removed in the next major version")
2: After X time, BEFORE the deadline, turn it into a compiler OR linter error, but keep the code
3: Do the appropriated notification in the channels about it of impending doom
4: Doom
If this is done correctly, we change the expectations from "warnings LOL!" to "warnings are serious business" that is missing in a lot of contexts
I thought the industry solved this by bundling breaking changes in major version updates.
V 1.0 - foo introduced
V 1.1 - foo deprecated, recommend bar
V 2.0 - foo removed, only bar
Users can stay on 1.x indefinitely, even if it never receives updates. Development continues on 2.x, eventually 3.x and so on. Users only experience breaking changes when they manually do a major version upgrade.
I had the same clever idea once. Deprecation warning, and it would (by the power of a C-Macro) auto-turn off when the relase x was reached, with louder and louder warnings before.
One day I came back from holidays. I had just broken a big go-live where the release number passed x. Date missed, next possibility in a few weeks. The team was pissed.
Yes they COULD have fixed the warnings. But breaking the go live was quite of of proportion for not doing so.
Could the clients have reasonably foreseen when the release x would occur?
Could they not have rolled back?
I have 2 better ideas;
1. Start charging money for/of deprecated APIs. Just like Oracle did it with Java 8.
- OR -
2. Add a timestamp-based RuntimeException to the very same code-path. If the date is greater than (>) the deprecation date, simply throw RuntimeException. for 100% of the requests, all the time.
Yes, it will page someone, and yes, it will get fixed immediately!
Please don’t do this. If you want to remove the function, remove it and insist on its removal when people complain. Don’t gradually break people.
For one customer I do maintenance on a piece of software.
Building it produces about two-three dozen deprecation warnings.
The whole software stack relies on a cluster of packages that stopped receiving updates 5 years ago.
The software is not facing end-users. But it does build using NPM and not via vendored packages.
To avoid those warnings, large parts of the code need to be rewritten using a different set of packages.
That doesn't get prioritised because it works.
The software sucks in many ways, but only from the perspective of an artisan.
The customer is happy to ignore warnings as long as the software does its job.
There isn't money in fixing things that work because it got old.
The incentives for the people putting the deprecation warnings in those packages don't align with the users of those packages. Their timelines and motives are different.
The article has a new note at the bottom:
> In case the sarcasm isn’t clear, it’s better to leave the warts.
Wow, that wasn't there from the start? That explains a lot of the comments here.
I don't think deprecation should come with hostile signalling like this, but if it did, it should be consistent, and escalating with subsequent releases, performance regressions on the deprecated path, starting at least one release after the deprecation warning, not wrong results.
And it should be explicitly mentioned in the deprecation warnings.
(You don't want to break systems, but you want something people who care about the system will investigate, and will quickly find and understand the source of and understand what to do.)
(Notwithstanding that this is a joke) Maybe it's just me, but I read this as a solution that would be implemented internally at a large company to distribute pain/accountability/tech-debt across time to a team which might have high turnover. i.e., a way to align incentives by punishing teams with bombs (via their metrics) in their code, before the bomb actually detonates.
Never do this on the job. Its called corporate sabotage and will result in a lot of trouble for you. Its also a generally bad idea. The whole point of deprecation is to let others choose a moment to switch over on their own moment of choosing. If you don't want to maintain it just remove it. It is far nicer behavior. Don't force others in a debugging hell. It makes you a bad person.
This is a bad idea. But in the same spirit, a bad but less bad idea: require the user to actively enable the deprecated feature via config or something, but have a known schedule to eventually expire it (with a hard error, not randomly incorrect data) and force the user to re-enable it. Causes some pain but not random, hard to track down bug. More like dealing with expiring certs - eventually, the user will want a permanent fix.
In a bit roundabout way Java 11 added `forRemoval` attribute to `@Deprecated` annotation so developers could indicate they really mean it.
I found the referenced article linked at the top interesting: https://sethmlarson.dev/deprecations-via-warnings-dont-work-...
The underlying complaint seems to be about libraries, rather than user behavior.
Occasionally returning wrong results is irresponsible, of course, but how about making deprecated APIs grow gradually slower?
> The response.getheader method in urllib has been deprecated since 2023 because the response.headers dictionary is what should be used instead.
Then response.getheader should just do that. There is no reason to expose the implementation to the user unless performance is critical.
> The response.getheader method in urllib has been deprecated since 2023 … When the method was eventually removed, lots of code broke.
Two years doesn't seem long to me for a widely used project, unless you have an LTS version that people needing more stability can use, or you are upfront that your API support is two years or less. Of course API support of less than two years is fine, especially for a project that people aren't paying for, but personally I would be quite explicit from the outset (in fact I am with some bits I have out there: “this is a personal project and may change or vanish on a whim, use it in any workflow you depend on being stable at your own risk”). Or am I expecting a bit much there?
If using semver or similar you are fine to break the API at a major release point, that is what a major release means, though it would be preferable for you to not immediately stop all support for the previous major version.
> What if we intentionally made deprecated functions return the wrong result … sometimes?
Hell no. A complete break is far preferable. Making your entire API essentially a collection of undefined (or vaguely undefined) behaviours is basically evil. You effectively render all other projects that have yours as a dependency, also just collections of vaguely defined behaviours. If your API isn't set in stone, say so then people have nothing to complain about unless they specifically ask you to keep something stable (and by “ask you to keep something stable” I mean “offer to pay for your support in that matter”).
> Users that are very sensitive to the correctness of the results…
That is: any user with any sense.
> might want to swap the wrong result for an artificial delay instead.
That is definitely more palatable. A very short delay to start with, getting longer until final deprecation. How short/long is going to be very dependent on use case and might be very difficult to judge: the shortest needs to be long enough to be noticeable to someone paying attention and testing between updating dependencies and releasing, but short enough that it doesn't completely break anything if someone updates and releases quickly, perhaps to bring in a bugfix that has begun to affect their project.
This is still evil, IMO, but a much lesser evil. I'd still prefer the complete break, but then again I'm the sort of person who would pay attention to deprecation notices, so unless you are a hidden nested dependency I'd not be affected.
This is silly and ignores how things are "deprecated" in every other practice on earth. Do we still install knob and tube electricity? Absolutely not. Is there still some out there that somebody is supporting some? Of course.
Does this mean that people and places shouldn't migrate out of older practices? No. But people have different priorities. And sure, we may treat "squeaky wheel policies" as a bad idea, but quite frankly that is far and away the most common policy out there.
To that end, please don't go out of your way to insist that your priority is everyone else's priority.
Make the deprecated method slower with a function from the deprecation release so the longer it goes on until it gets to around a day.
then remove the function.
« When the method was eventually removed, lots of code broke. »
Well, removing it earlier would just mean that lots of code would break earlier...
> The response.getheader method in urllib has been deprecated since 2023 because the response.headers dictionary is what should be used instead.
Who cares? This is such a trivial example, where there is no maintenance cost associated with leaving a dummy method in that does a dictionary lookup. I understand there are times when maintenance costs are substantial, but it's hard to take someone seriously when this is their example.
The obsession with newness, and stigmatizing something that doesn't conform to your sense of chasing newness, is really silly.. can you believe it's been deprecated since 2023 and someone still uses it???? 2023 was not some kind of dark ages of distant history.
Chaotic Evil code.
Ugh, I am finding it hard to express how much I hate this.
Software developers have enough treadmills they need to stay on. Deliberately breaking backward compatibility in the name of "deprecating something old" doesn't have to be one of them. Please don't be that platform or library that deprecates and removes things and makes me have to dust off that old software I wrote in 2005 to move over to a different set of APIs just to keep it working.
Instead randomly trigger kill self process. That's a safer way than random wrong returns.
Wow, tell me you've never shipped anything important / heavily used without telling me that.
I expected this to suggest a tick-tock deprecation cycle, where one version deprecated and the next removed, but this is definitely an idea that belongs on the domain "entropicthoughts.com"
How about never deprecate
sure - because i rewrite all my code every year!
Well written, but author is at the risk of tripping over Poe's Law, especially if readers don't read to the bottom.
Author's point is that their modest proposal (and its sibling, introducing intentional delays in resolving the deprecated API pieces) is a bad idea. Instead, author suggests that making the API change at all is begging the question "Who does this API serve?" It is, perhaps, okay actually if the old system never gets deprecated.
> especially if readers don't read to the bottom.
That note at the bottom was only added after the article was posted.
What about a sleep? Each release that the feature gets more deprecated, the program adds a couple milliseconds of just blocking. Less destructive, and gives a nice slow ramp up with real but non-destructive consequences.
Profoundly awful idea.
Filed under: trying to solve another domain's problem with only the tools in your domain (this is almost always a bad idea)
Thats madness. Just remove deprecated functions after a reasonable time period.
Another idea is to make the deprecation warning increasingly scary sounding.
"DEPRECATED: i will literally kill you"
"Continuing to use this feature is a moral failure on your part and you should be ashamed, your parents should be ashamed, and dishonor will follow you wherever you go."
"If you don't migrate I will tell your mum"
sorry, i'm not afraid of this. i'll keep using the feature.
Code churn is an evil, negative value activity. Design sanely and carefully before settling upon a published, standard contract. Churn of external-facing APIs should be extremely rare to solve essential critical problems, not for pseudo-maintenance bikeshedding. Also, the "it's a hobby" excuse doesn't fly.
tl;dr: Stop changing and breaking shit unnecessarily.
Jesus, what a terrible idea. This is such a terrible idea that I would not hire this guy based on this post alone.
What I want from code is for it to a) work, and b) if that's not possible, to fail predictably and loudly.
Returning the wrong result is neither of the above. It doesn't draw attention to the deprecation warnings as OP intended--instead, it causes a mysterious and non-deterministic error, literally the worst kind of thing to debug. The idea that this is going to work out in any way calls into question the writer's judgment in general. Why on earth would you intentionally introduce the hardest kind of bug to debug into your codebase?