I don’t think appeals to morality or ethics are required for this. You paid for the LLM’s output, you should be allowed to use it how you wish. The only reason distillation is a dirty word is the AI labs trying to spread FUD to protect their non-existent moat.
It’s to the point I don’t even read Anthropic’s marketing blog anymore lol. The ai psychosis is just so real when people take these fluff blog posts which never have evidence or reproducibility and treat them like gospel
It’s crazy to me that companies/researchers share important data with these AI labs, you’re basically giving them your secret sauce which they then share with all of your competitors via training on conversations. At the same time I don’t really know alternatives other than a slightly less than frontier local LLM. Not sure how good they are at math.
Academic work is based on worldwide sharing, the sharing is not the problem, it's the lack of attribution. Unsurprisingly, these companies neglect standards of academic honor and attribution. Some human researchers also used to do that but in a discipline like mathematics this used to be a small problem because people tend to be so specialized that very few people could just grab someone's research and quickly piggyback on it, and if they do, colleagues will generally understand what happened. Unfortunately, AI is changing this.
You are both using a different definition of sharing I believe. When people have an expectation of privacy, use by others should be forbidden. Tech has gone completely off the rails with the use of private data.
The ultimate drive for some researches is the pursuit of knowledge. If I'm stuck at some block which prevents me from continuing in some direction that I want, of course I would like some help. I believe we already have nonzero collaborative proofs on math.SE, I can't recall good examples, but I have definitely seen citations to mathSE before.
So for me it sounds quite natural to also share this with AI especially under the privacy assumption. Also there's the assumption of scale -- maybe your problem is not large enough for anyone to care to scoop; and just for blind retraining, how do they know that the proof is even correct to include it into training? I have definitely received a ton of incorrect proofs before. So the SNR of such private chats is also not clear. I'm imagining millions of masters/phd students also trying to solve various random things with various capabilities, but how much real signal is there?
I don’t think you realize how often depressed people say they will but don’t commit suicide, and also being able to know when is quite remarkable - I think this study is pretty interesting, and not as obvious as you think it is.
Yeah, I can't see what the questions were, but if they were about suicidal thoughts and plans, having to read, consider and respond to those questions six times a day would certainly reinforce any tendency to think about it, just through sheer exposure.
Isn’t that the whole definition of “executive branch”? They are given (by we the people) the power to choose how to execute the laws. And we elect them.
The alternatives I can think of are for laws to be executed at the whim of an algorithm, a king, a more local executive like the governors of each state, or a more distant foreign executive like the UN
No, the executive branch has a responsibility to execute the law. It's not a privilege. We elected them to perform a duty on our behalf. Do you understand the difference?
Article 2, Section 3 of the US Constitution: "[The President] shall take Care that the Laws be faithfully executed"
The judiciary doesn't have discretion as in it has been "granted" discretion. It de facto has a high degree of discretion because the act of judging a case necessarily entails a judgment of the laws.
Judicial review is a functional necessity. It's a necessary downstream consequence of simply having a Constitution as the supreme law.
The analog to the executive is that the executive naturally has finite resources to enforce law, and so as a matter of functional necessity it has some degree of discretion. That is not discretion to say "I disagree with this law and so am choosing not to enforce it," but the discretion to say "there are other priorities that are more important."
I was thinking about it more like if cops didn't have discretion, they would have to arrest for every minor offense and let the courts sort it out. After more reading it seems there are differing interpretations of the "Take Care" clause. Thank you for taking the time to explain.
> Beyond these constraints, the Clause raises several vexing questions. For instance, must the President enforce those laws she believes are unconstitutional? Some scholars argue that Presidents must enforce all congressional laws, without regard to his or her own constitutional opinions. Yet modern Presidents occasionally exercise a power to ignore such enactments on the grounds they are not true “laws” subject to the faithful execution duty. In so doing, they somewhat mimic the arguments and practice of President Thomas Jefferson, who refused to enforce the Sedition Act on the grounds that it was unconstitutional.
The executive only has discretion if the law says they do. If the law says they must or shall do something, they must do that thing. If it says they MAY do something or otherwise explicitly gives them discretion in making a decision, then yeah, they have that discretion
But it's not a given that they can just decide what laws they will or will not enforce or follow
Expanded rights for some at the direct expense of others. That's not the same as simply expanding rights, unless you count other people being 'offended' in the same category as someone being physically assaulted or killed.
> That's not the same as simply expanding rights, unless you count other people being 'offended' in the same category as someone being physically assaulted or killed.
Yes, they are the same category.
In US$10,000 property damage, nobody is physically harmed.
Here’s another example, expanded rights:
Anyone holding a 401k, IRA, or pension can purchase single family homes and apartments inside the account, and pay zero capital gains tax and zero rental income tax.
We expanded rights!
Take a guess what this does to the homeless population.
> Anyone holding a 401k, IRA, or pension can purchase single family homes and apartments inside the account, and pay zero capital gains tax and zero rental income tax.
I am not sure on pensions, but this is not true for a 401k. Only an IRA can be used without paying capital gains taxes for a home purchase. Including first time home buyers. Speaking from experience, I learned the hard way the benefit you are describing applies only to an IRA.
To be clear, you are saying that theft of <10k is not a crime..? If you steal an iPad in an airport, you are charged with theft. I don't know where you got that idea. People are charged with theft for much less than 10k daily.
Honoring it, no doubt, by restricting firearms to duly registered militia personnel as written in the plain text for all to see? What about the 14th Amendment section 3, also written plainly?
Codex can read a CLAUDE.md in a workspace or when configured to treat it as an instruction file. This test concerned Muse automatically loading personal files from ~/.codex and ~/.claude outside the selected workspace and sending their contents in the first provider request.
A file stored in ~/.claude was written for Anthropic. A file stored in ~/.codex was written for OpenAI. Muse treats the existence of those files as permission to copy their contents to Meta. It displays a notice and provides an opt-out, but never asks before sending them. That cross-vendor assumption is the privacy issue.
The models are open source, it’s never been a requirement for someone to document the entire process of creating something to be “open source”. Just sharing the source, in this case, the weights, meets the criteria. Software engineering’s obsession with precise terms is actually one of the things I’m glad is dying with ai automation.
They’re not really even being precise. The relevant software freedom, from the FSF is []
The freedom to study how the program works, and change it so it does your computing as you wish (freedom 1). Access to the source code is a precondition for this.
Ported to the model world, this is fulfilled by sharing the weights and implementation. There’s almost nothing that having the training data gets you (other than actually training it). The weights plus a reference implementation let you see all the states to study the behavior, and let you fine tune it to do your bidding (the abliteration etc). The freedom is satisfied.
Some might argue that without the training data you couldn’t do some classes of experiments to see how it works, say leave-one-out retraining. I’d argue things like that are not really about the model but about ML research or the class of models, which while interesting is not a free software pre-requisite.
The source is actually all training data, plus the software used for training, including some scripts or instructions to run the entire thing end to end on your own computer. That's what open source has always been about.
The weights are the output of a program, it's a binary. It's not source.
> Software engineering’s obsession with precise terms is actually one of the things I’m glad is dying with ai automation.
Yes. The next time you go to your doctor you should hope he's not being overly precise; or the engineer that builds the bridge; or the software engineer that implemented the embedded software in your insulin pump.
There is no precise thinking without precise terms.
So is a zip file of source code. There is no 'compilation process'. Model training isn't a fixed end point. You start with random weights. You train. The weights get better. You do this iteratively. At some point you say 'good enough' and release. People doing further training would start with those weights and further iterate. Demanding the original training data and training process would be equivalent to demanding a completely replay-able sequence of editing steps starting with a blank text file and allowing you to replay ever edit that led the original author to the released source code.
Now there are reasons you might want to know about the training data when you wouldn't care about the authoring process used by a traditional open-source process. And these get at the reason LLM's are different than traditional software and so maybe our existing definitions of what 'open-source' means aren't a good match for LLM's. Of course there is software associated with LLM's (beyond the weight) -- defining the structure of the particular neural net those weights fit into. In every open-weights model that I'm aware of that software is open source (though trivial).
Yeah I fully expect companies with lots of GPUs but not a good model like Microsoft and Amazon to just take these open weight models and make money, the GPU expense is the only moat at this point.
It’s classic commoditize your complement, nobody can replicate the cloud providers, everyone can replicate the models with open weights.
yes, this seems to me to be the reason why these three companies want open weight. everyone wins (but especially big tech) in the open weight world except anthropic and openai. i suspect even google could live with the open weight scenario; they are in a sense the only tech company that is perfectly hedged in this respect.
i suppose this is part of why openai and anthropic have not been shy about advertising the scarier strains of their models, these past few months; they basically have to force the white house's hand or risk being commoditized among the various other options in a microsoft/openrouter dropdown.
reply