In my work (scientific or otherwise), I try to avoid dependencies if possible. That's not always possible, so a solution like Docker or NixOS is needed, but the problem can be improved a lot without a technical solution. Either feels like fighting an uphill battle though as most researchers think short term and just pick whatever is convenient in the moment.
The credulous LLM users Dan Luu is complaining about don't take the equivalent of thousands of photos of the same sunset. They take a couple photos at most, say "LGTM", and move on to taking a photo of something else, even if their couple of photos of the sunset aren't great.
As someone who will often compare different things, I think doing that usually takes time. I think LLMs can help reduce the time (especially if it's programming, not necessarily so much in other areas), but I'm not seeing many people use LLMs to try a large number of independent approaches and compare them.
I've thought exactly that "writing is thinking" before as a reason to not let a LLM write for me.
Then again, I've seen a counterargument [1] by someone who clearly heavily uses LLMs for writing (going by both their LLMy writing style and their own admission). The person I'm citing describes a process where they get a LLM to write something, they check over it and provide feedback to the LLM, the LLM rewrites, and the process repeats iteratively. So clearly he is putting thought into the process.
I think there is something valuable missing, even if it's hard to clearly express. I'll try. The threshold for what I'm willing to accept if I'm simply approving something is likely different from what I'll get if I write something myself, for instance. Saying "LGTM" is too tempting. It seems to me like he's outsourcing his selection of topics to cover as well. If you're not thinking yourself about what to cover then it would be very easy to miss a critical subject. There also an asymmetry between checking and generating something with constraints placed on it. Checks can't catch everything, and a constrained generating process can reduce the amount that needs to be checked, avoid issues that can't be checked so easily, and focus your attention on areas that you know historically have had issues with this generating process. I've thought about this quite a bit in terms of whether to write new code or use an existing library. Sometimes "the devil you know" (my code) is better than an existing library simply because I understand its flaws better.
I've tried doing it that way, and thought it was even acceptable for the reasons you said. I did learn a lot through that process and clarified my ideas. But later I rewrote the whole thing from scratch and then had the LLM review it. It made some good suggestions but no substantial changes. The difference was night and day. That final product had my voice, and I understood it better. LLMs are powerful tools and can improve quite a lot of the writing process, but using them to do all the writing leaves a lot on table along with your fly open.
This was interesting as I had wondered whether Terence Tao realized that the Navier-Stokes Millenium Prize problem was kinda useless from a fluid dynamics application perspective. Turns out that he does!
> While the equations do come from a very natural physical motivation - the study of incompressible fluids - the regularity problem is not important for its direct physical application. [...]
I disagree that the Millenium Prize problem would tend to "spur further development of the field". I'm not seeing that. The people working on the Millenium Prize problem aren't helping solve more practical problems like making better turbulence models. I wish we had more people of Tao's caliber working directly on turbulence modeling! Indirect work won't cut it here. In my view, the Millenium Prize problem has mostly been a distraction from more important issues.
I'm Ben Trettel, an experienced mechanical engineer with a PhD, specializing in computational fluid dynamics, design optimization, and verification & validation of computer simulations.
I am particularly interested in opportunities to build cutting-edge physical products where computational simulation and design optimization are key.
Please email me at the address above about only specific work opportunities that are matched to my skills and background, not generic work opportunities that anyone can apply to or non-work opportunities like advertising your job board.
I haven't seen anything like this before, but I think it's a good idea. I would suggest starting it yourself and putting it on GitHub something similar so that anyone can submit more via pull requests.
I'm not sure if building anything is worth it at this stage. I wonder if starting it as a community-editable Proton Sheet would work. If that leads to an accumulation of quality links, it would be worth building a site for it.
I would back up and instead ask whether getting a PhD is a good idea. Unfortunately, academia is a minefield. PhD students are largely cheap labor. Getting a PhD can be a valuable apprenticeship, but often it's abusive and poorly paid. You might nominally get some freedom, but the grant funding wants you to do something you might not care for. The opportunity cost of a PhD is huge. You should consider as an alternative taking a more conventional job, maybe a part time one, and doing science on the side to figure out how to start your business. A part-time engineer making $50K/year is getting a much better deal than the vast majority of PhD students.
The problem-first vs. science-first framing sounds good on paper, but it would be difficult to accurately determine whether a particular lab has either focus ahead of time. What you see from the outside is mostly marketing. Most labs are really neither and instead are publication-first. Publications are the currency valued in academia.
Bingo. OP needs to do a lot more digging into what a PhD entails and why you get one, and hopefully not from the same sources of where they learned this terminology like "hard-tech" and "technology-first"
I'm Ben Trettel, an experienced mechanical engineer with a PhD, specializing in computational fluid dynamics, design optimization, and verification & validation of computer simulations.
I am particularly interested in opportunities to build cutting-edge physical products where computational simulation and design optimization are key.
I think you are holding climate science to a far higher standard than the other physical sciences.
In the physical sciences, blinded experiments are rarely done. The need for blinding is greatly reduced compared against the medical and social sciences as the data measured is far more objective and observers usually can't influence the results. I've done experiments where I basically start recording data, turn a valve, and from that point on, the result is outside of my control as long as I'm watching from a distance of 10 feet or so. That's often not the case in the medical and social sciences. In my experience, blind experiments in the physical sciences take the form of a blind prediction challenge where a bunch of teams are asked to predict what a certain experiment will do before the experiment is run. This is a good practice that I advocate. I also am interested in blind data analysis and "fake-data simulation" as Andrew Gelman calls it.
There also are a lot of times in the physical sciences where duplicating the "full scale" case is not possible for various reasons (cost, legality for nuclear weapons testing, etc.). Instead they do experiments on scale models (which might not have similitude [0]) or parts of the full problem. This is not ideal, and I do think the researchers could do better, but the situation is unavoidable. Again, there's no reason to single out climate science on this.
I also want to strongly push back on the "computers models are basically mathematical assumptions, not physical laws" part. I'm a mechanical engineer who works in computational fluid dynamics. The models I use have a lot of overlap with climate models, but don't get as much scrutiny even when they are of similar reliability. A large computer model like a climate model has a lot of components, some of which could be regarded as very reliable (likely what you mean as "physical laws") and some of which are less reliable. But even the less reliable components are not assumptions. They always are backed up by some data, perhaps not as a comprehensive as is wanted, yes, but some data.
reply