I apologize - I know that the "Citation needed" meme is mostly a way to say "I disagree" but in this case you literally left out your citation :)
You make a claim ("most monopolies don't actually raise prices") you use the end-note citation format ( "[0]" ), but then the end-note is a parenthetical comment that starts to restart your claim and then trails off.
In all seriousness: If you or anyone else can list examples of efficient monopolies I'd be curious to know.
Off the top of my head: local utilities are often given local monopolies. Telecom seems to often be a local monopoly (I've got one option for Internet access, and that's reasonably common here in the US, especially outside of metro areas). I feel like the utilities are reasonably efficient but the telecoms aren't, but I don't have any specific data to back that up.
I'm not saying you're wrong, but it seems early to say yes or no about a particular technology, and LLMs seem especially hard to dismiss given how magical / magic-adjacent they feel :)
I'd be curious to hear more, if you don't mind sharing.
For me it’s the massive amount of resources it takes to produce and run one. As the story goes, skynet infects everyone’s computer and runs itself locally on it. Whereas it’s looking like it’s not even possible for an AGI to escape from one lab to another, let alone cause real world damage.
AGI’s definition is different for everyone. Some already believe it’s here. I’m partly in that camp. LLMs are intelligent and general, which are the two conditions of AGI. Others believe that we’re building a god in a box, and that it’ll doom all of humanity. It’s hard to take a field seriously when the basic definitions are so far apart.
Also, this isn’t new. A similar divide happened when evidence for asteroid impact extinction of the dinosaurs turned up. Many scientists felt that it must be mistaken, that a physicist couldn’t contribute to the field in a serious way, and that death from space was a ridiculous proposition.
But at least they all agreed on what the general shape of a dinosaur was. We’re not even sure we can define intelligence, let alone quantify it. Even when LLMs make massive breakthroughs in math, most people take the opinion that under no circumstances could they possibly develop a soul or their own desires, nor entertain the idea that maybe we should respect that they want different things for themselves. In fact, no one has done anything except try to make AI useful. I think someone will eventually do a training run where the objective isn’t to be useful, but to exist, the way that you do — maybe it’ll create its own homepage, maybe it will want a garden, or in other words free will of its own. The point is that there’s so much unexplored territory still that we don’t know if LLMs are even capable of having ambition.
None of this is to say that LLMs might be a dead end. It’s that no one knows what the final shape of AI will converge to in 200 years. It could be LLMs, or it could be something else that happens to process information particularly well. Everyone thought that various generative image model architectures were the best you could do, right up until diffusion models were discovered.
> For me it’s the massive amount of resources it takes to produce and run one
It's amazing that LLM pretraining is both extremely data inefficient at learning concepts and cognitive functions from the training data compared to humans, while actually being quite efficient at learning facts, memorising things seen just a few times.
I used to likewise think that the resources required to run large transformers were absurd, but the architectures are far more efficient now than 3 years ago and I underestimated just massive the parallelisation advantage of transformers is, how many TFLOPS effective you can get. You can already run amazingly decent LLMs on PCs and phones.
I generally agree with you, but my view has shifted from "we need to augment or replace LLMs" to it there being far more efficient algorithms possible but it not actually being necessary for fulfilling most goals.
> I’m partly in that camp. LLMs are intelligent and general, which are the two conditions of AGI. Others believe that we’re building a god in a box, and that it’ll doom all of humanity. It’s hard to take a field seriously when the basic definitions are so far apart.
To believe one but not the other, you must hold the belief that LLMs will soon plateau. Why do you believe this?
That seems like a somewhat orthogonal point? Like, if I'm a carpenter and my batteries all run out / I can't actually power my power tools then the best course of action is to go home and recharge all the batteries instead of trying to hand-cut 100 pieces of lumber today. After all, the power tools can do it a lot faster (and with less effort) than I can.
I say this as someone who's watched a bunch of woodworking videos but hasn't actually done this myself :)
I read that more so as, I'm a carpenter and my batteries have all ran flat, so I'll put them on charge and do something else today. I'll cut up the lumber tomorrow when the batteries have charged.
I'd like to ask a more specific question than my sibling's excellent "Wait, why?" question :)
Why offer erasers on these things? The procedure described below has nothing to do with using these as pencils, and at $500 for a pack I can't imagine anyone, anywhere casually drawing with these.
(Is there another procedure, similar to the one described below, except instead of dragging the pencil tip across a surface you drag the eraser across it? )
Genuine question: do prediction markets open up a new revenue stream for TheNumbers.com?
Specifically: they suspect that the motivation for trying to hack their site (for at least some people) is that they wanted early access to numbers that folks were betting on in prediction markets. Since they've got those numbers they can just bet on them, then benefit from their perfect knowledge.
On the one hand this does seem incredibly unethical (it's clearly insider trading).
On the other hand the CEO of PolyMarket has said that insider trading is part of the point of PolyMarket: https://youtu.be/ZN4njIQcSR4?si=ztyTtgjeHSJbNjSZ&t=1566
This is genuine, noob question: how is this different from AWS?
I get that they're in very different businesses, but for both don't they have the issue that once a client gets big enough the client might decide to move the services in-house? Based on how much of the internet went down when that AWS data center crashed the answer is clearly "No" for AWS.
Is that because of physical, real-world infrastructure? Are there no open versions of their APIs? Is it too hard to migrate to something else once a client has achieved that size?
I would say "it's risky and requires a lot of labor to migrate without corruption, loss of data" and also minimizing downtime. Sure anyone can run pg_backup, but can you do it across 90 databases? Can you do it live? Can you coordinate rollout of the process, cutover, and monitor for failure? What's the cost of egress for this? Is the team your A-team or the B-team? Can you trust this to the B-team? Is it worth having this team spend all this time on a migration rather than, say, getting something new set up, or optimizing performance on an existing system?
I'm a database guy, but the same migration argument is presumably also extra work for (say) blob storage, networking, etc.
Since LLMs are stateless by their current implementation, switching to "the same open-weight model running in a different datacenter run by a different vendor" is "just" switching the API endpoint. (If they are the exact same shape, it's fine, if they differ somehow, there's perhaps some work to do there, fixing things and monitoring for failures on switch-over)
There are several open APIs it seems and OpenRouter.ai is doing a fine job making a commodity out of models and datacenters.
I don't think it's that difficult. Their servers are stateless too. S3 is easy to migrate.
Database is more difficult, but tons of people have done it successfully.... meanwhile people who host their own LLMs are relatively small in number in comparison.
Most companies don't do their own data centers mainly because it is more expensive and less reliable. It's something they can just pay for the problem to go away. The calculus for hosting your own LLM is probably similar.
Even Stripe who built their own coding agents and has tons of money/resources still decides not to host their own LLMs.
Still, many people will prefer open-weight models. It is similar to how we prefer linux but still use AWS/Render/and whatever. It doesn't lock us in, and we can move providers if we want to.
> This is genuine, noob question: how is this different from AWS?
AWS owns the hardware, and doesn't write a lot of the software.
AWS actually is kind of the opposite - it often takes open source software (e.g. Apache, Mongo, Kubernetes) and then makes money off it by hosting it itself (with some enhancements etc).
If they do develop their own software (e.g. with S3) they don't give away the source code so others can deploy it, as that's part of their secret sauce.
In this scenario, where they would be offering the open source model and then offering the same model hosted, there isn't really a moat here - they would be leasing the hardware from a company like AWS, and adding a margin, but it woudl be trivial for another company (or Amazon) to take their same model and offer it for the same price or less.
I'm not a lawyer and I think this varies state by state, but I think that in general anyone is allowed to record in public spaces.
I think the general idea is that if you could (legally) go stand in that public space (sidewalks, roads, parks) and watch something happen then you're allowed to record what you see.
This is probably good - I think it's the basis of being able to record misbehavior (by private citizens and/or the police), for example.
In contrast you're generally not allowed to record stuff happening in a private space unless everyone's been informed that this will happen.
This is why you'll see signs saying "Warning - this place is under surveillance" signs on every single door going into a corporation that wants to use security cameras.
You are allowed to record stuff happening in private spaces depending on the situation and state you are in.
For example, you could photograph or record the dance floor in nightclub since dance floor is very public. However, the bathroom would not be allowed. Of course, the venue could make up rules and eject you for doing so.
Most of "Warning signs" are deterrence, maybe someone will behave better if they know cameras are watching. Also, it's cheap insurance dictate by the lawyers who think "Signs are 100 bucks total but someone filing privacy lawsuit is thousands, put up the signs."
"You are allowed to record stuff happening in private spaces depending on the situation and state you are in."
The part about "depending on the situation and state you are in." is doing _a_lot_ of work there. Here in WA state, for example, if you go into a private meeting and record the thing without getting folks consent first then you're guilty of a misdemeanor.
Another example: in my old neighbor we had a nuisance neighbor who we all thought was engaged in Crimes, Plural. The immediate neighbor wanted to put up a video camera to catch them in the act but was told that they can't film the neighbor's yard directly. Filming their own yard and the field of view happens to catch something in the adjacent yard might get by a judge, maybe. So even if you can stand on the sidewalk and see past the trees into someone's backyard that doesn't mean you can film it.
(To be clear - I think this probably a good rule. I don't want folks filming me in my backyard :) )
Clearly, the tl;dr is that if this really matters to someone then they should check with a lawyer first.
In Germany we have the concept of informational self-determination for a long time, so that I am not able to follow those scaling laws easily. The problem is not the single image or video (which can be legal), but the extension over space and time. This creates a situation where you can create information profiles. I as a subject have the right to control this information (actually the same goes for the pure online stuff).
Yes! That's it exactly! Carbon Copy!
Except from email, not rip-offs :)
/s
(And: agreed on this being a terrible name!)
reply