If you've not read Fred Brooks MMM, give it a shot.
It seems like you're implying that teams of 60-80 developers should be expected to outperform teams of 12. This is simply not true. The most important feature of source code as a language is that it allows precise mindshare among close knit teams. It doesn't guarantee it, but it makes it possible for people to talk about the product at a level that is otherwise very difficult.
A convenient side-effect of the source code is that it instructs the machine what to do. But instructing the machine was never the bottleneck, the essential difficulty of software development is in understanding what are the correct instructions to achieve some objective, not typing them out.
The problem is that communication doesn't scale at all. Having just 3 developers with good alignment about mental models, best practices, and design direction is hard enough, and if you found the right three people at the right time with the right ideas, you could generate billions of dollars of value.
Large monolithic teams on the order of 80 are a product of people in control not understanding how software development works, and how to make it work well.
As a user of local models, does this mean that there are 'magic incantations' that can increase the performance of some local models?
I see some details about recovering information via whatever technique. It's interesting, but appears not generalized.
So for a specific question, yes, but this is not about techniques like adding a good embedding that just generally tends to improve open model performance on certain tasks.
There is some research suggesting that a prefix from a stronger model will tend to elicit better completions from a smaller one. I am doing some experiments to see if I can replicate this in a practically useful way, e.g. Fable + 4B Qwen, or 125B Qwen Flash Next + 4B Qwen, results TBD.
I don't think that follows from the published results. Would have been an interesting hypothesis to add though, and quite easy. Just throw the same setup at some benchmarks.
When I look at the technology and capability of frontier models for ideation, brainstorming, and research, or when I look at the open source community and open weight models, and running local LLM's... I'm convinced that this is a technology revolution the size of the internet.
When I look at businesses trying to turn this research and technology into profit, I think that this is a bubble that will burst. You are not alone, I am very doubtful of the current concoction of AI features being offered. The true winners and legitimate applications will emerge over the next 30 years. Google, Nvidia, and Microsoft could all become IBM, Xerox, and Kodak. Or maybe they find a way to pivot. Because truly, there is an amazing amount of nonsense BS in the AI hype train.
Acknowledging the outside chance that recursive self-improvement works and we just scale directly into a Kardashev 1 civilization within my children's lifetime, and K-2 in theirs'. Can't rule it out no matter how hard I try.
honestly that would still probably mean more than story points. maybe all this AI insanity has an upside that the new cargo cult washes out the old one.
In the novel Anathem by Neal Stephenson, the internet becomes unusable for humans thousands of years before the events of the book, due to a process called Artificial Inanity.
AI generated content, both good and bad, some riddled with errors, some with only one subtle error hidden among lots of good information, floods the internet. The internet becomes an unnavigable swamp of weaponized nonsense for average humans. The problem is further compounded by the fact that searching and accessing the internet will be noticed by AI agents that will generate still more swamp content in response.
Unfortunately, it seems that this fiction ended up being prophetic. The open internet will fall to entropy, not legislation or one-sided international trade agreements. I think we need more projects like Anna's Archive, where the public uses torrents and distributed infrastructure to save and organize the world's information. Google has abjectly failed in its original mission to organize the world's information and make it universally accessible and useful.
How would that be immune? It already has many copies of the same books and no way to tell which ones are erroneous or incomplete. A malicious actor could easily flood it with garbage.
It will happen eventually, especially with AI generated books published electronically. I wonder if there is a cutoff date or something to try and avoid this problem of low effort books.
So he could sell the data? It’s almost certainly not because he cares so much about certifying human versus AI. Even then the biometric data is not going to be enough. Any human could still use the biometric data to pass AI work off as their own.
Indeed and the data is worth much much more if it is tied to a biometric.
And if you assume that in the future there will be crypto banks globally then you want to be that final authority layer between the real and the computer world
Yeah but my original comment was talking about being able to reliably mark something as AI generated or human generated. Crypto, encryption etc. whatever you throw at it it’s just not possible (other than media that can be watermarked at the generation source).
America is headed towards techno zaibatsu economy.
Just a product of the dynamics of capital and power. Too much cash, Nvidia is forced to try and find ways to deploy excess capital very quickly. Most efficient way is to absorb players in emerging industries, bet on the potential for growth in new markets to help keep ROI up.
It is a great day for startups, the goal is to sell out and get rich, no?
INB4 China comes in and dismantles the American zaibatsus in 2245 like America did to Japan in 1945.
We simply need to bring back that 90% federal income tax bracket we had in the 1950s. If we had the same tax policy today, a CEO or investor would be limited to 5 million dollars a year in earnings. This is what fixed everything.
If a CEO cannot personally get richer by cutting wages, cutting staffing, and outsourcing; all of it stops. That was the secret of affordability. Don't let a thousand rich people own everythjng. Tax them so we can have millions of working millionaires instead of 900 non-working billionaire elites.
As those 1950s taxes and post 1929 financial regulations were cut in the following decades, the wealth inequality that led to the great depression has returned.
This is more a result of Clinton's failed CEO pay reform. It is what created this monster, not the tax bracket. Now, saying that, we DO need to tax these people higher. Eliminate all the ways they're able to squirm out from paying a fair share.
Marginal tax rates are meaningless because, even in the 1950s, the system was complex and can be gamed.
If you want to judge the net effect of tax policy over time, look at the percentage of GDP captured by federal receipts (the vast majority of which are taxes). In the U.S., it's averaged ~16.5%-~17.5% with occasional spikes and dips. Currently, it's very close to what it was in those 90%-marginal-rates 1950s.
I disagree on this. It's not the wealth that is the issue, it's the corruption that allows wealth to be converted into power. Citizens united is so backwards that it actually encodes the corruption as a right.
People have a right to support candidates > they can print a sign and stand on a street corner > printing signs costs money > people have a right to spend money supporting candidates.
People have a right to assemble > they can stand on a street corner with their friends > they can hold a big sign together > they can have lots of friends and form a group and print a really big sign.
We want to encourage private business > limiting liability for owners who do not participate in the business would increase access to capital > we offer corporate forms that shield investors from liability > identity of owners may be kept confidential.
People complain about the wrong thing. The problem isn't that people are allowed to support candidates with their money, buy airtime, etc.. It's that companies have secret owners...
You said "People" a lot in your post. Corporations are not "people".
Only natural persons should be able to contribute to political candidates.
Corporations can lobby, as a corporate person, just like a union or other organization, but not contribute.
They can run ads themselves, but there needs to be a much greater wall between PACs and candidate campaigns, with the previous restrictions now null and void due to lack of enforcement.
If it's okay for Corporations can run ads themselves, then your objection isn't to wealthy people supporting candidates or even them doing so secretly, but... only when the little guy pools donations and they happen to be pool donations via a corporation?
I'm against anything secret about spending on political campaigns.
Candidates should only accept funds from natural persons, that amount should be limited per person.
PACs and other candidate-adjacent activities should require 100% transparency on the source of funds and not be allowed to co-operate/co-ordinate with any candidate's campaigns. Companies can fund PACs, but they should be required to receive shareholder endorsement and not co-ordinate with their employees.
Laws impacting on the 1st amendment require strict limitation on how the government legislates and for an explicit need.
In this case, it is to ensure the integrity of the political process of selecting candidates and electing members of Congress.
No, state's cannot e.g. discriminate on the basis of race by allowing corporations only composed of certain races. States also cannot infringe on the right to assemble or speak, by prohibiting people from supporting political candidates when they are organized as such a group.
But, generally, yes. States can require that the owners / members / corporate books / etc. of a company be disclosed publicly. If owners don't want their books disclosed, then they can forego the immunity that the corporate forms provide.
> People complain about the wrong thing. The problem isn't that people are allowed to support candidates with their money, buy airtime, etc.. It's that companies have secret owners...
This was a major crux of Colbert’s “our lady of perpetual exemption” bit. People absolutely know about the shadowy machinations CU enabled. The lack of transparency is a major critique.
I wouldn't say CU enabled it. CU didn't address the secrecy of corporate forms. That wasn't at issue. CU was only about ~"Can people use money with their friends to support a candidate."
To say that repealing CU is the solution is to allow the critique to go unresolved, while arguing ~"People shouldn't be free to support political candidates."
> People have a right to support candidates > they can print a sign and stand on a street corner > printing signs costs money > people have a right to spend money supporting candidates.
Individuals printing signage is not where most of the funding is going though, is it?
> People have a right to assemble > they can stand on a street corner with their friends > they can hold a big sign together > they can have lots of friends and form a group and print a really big sign.
This isn't how most of the money is being spent either, is it?
I don't know how you seem to either believe the point of the right to free assembly was to give richer corporations more sway, or be completely oblivious to the fact that that is what's actually getting people angry, not the fact that citizens want to stand with signs by the street corner. Assuming you don't have a conflict of interest leading you to that conclusion, I... don't know what to tell you.
I think your objection is fairly clear in terms of 1) you point out the difference between e.g. standing with a sign vs. more expensive options, and 2) you say "right to free assembly was to give richer corporations more sway."
Fundamentally it sounds like your complaint is that wealthy people are presumably more able to be effective with their efforts than folks who, e.g. can't afford as big of a bullhorn.
Citizens United wasn't about that. There wasn't a law saying no person shall spend more than $1k/yr supporting a candidate.
This is a completely inhuman and unamerican take. Money buys secrecy, it always has. Money is a tool, exercising speech is a right. For the Supreme Court to equate them is ahistorical, plutocratic, and terrible. You sound like you support giving corporations the right to vote as well,
Where do you draw the line between money and speech? IE, would you "simply" ban all political advertising completely - one can scream from a soapbox and that's it?
What is a corporation but people? Companies are groups of people. They're one or more owners, some managers, possibly some staff. If you take away all the people, then there's nothing there.
Democracy is built on the idea that people are equal and have equal rights. This is only sustainable if the power differential between people isn't too large.
Corporations consist of people, yes, but the power to control where the efforts of these people are directed is concentrated on very few people, giving those people a large amount of power -- so large that without any checks on their influence on government, it undermines democracy.
It's the same argument for why super-wealthy individuals are so problematic. They're literally destroying the fabric of democracy through the power imbalance afforded by their wealth.
Assets, mind share, etc. All the things that get sold off in bankruptcy proceedings.
A railroad company also has a huge amount of steel roads with cross country right of ways. You could fire everyone, and it'd still be extremely valuable. It might even be more valuable.
Even in asset-lite software land, "Atari" the name still has a lot of value, even though none of the people are there anymore. They own the right to control the things that the neurons in millions or billions of people's brains are tied to, that their habits tie them to.
Twitter, you can fire almost everyone, make the online square less pleasant for a lot of the people, the network effects truck on. People continue to use it even as they hate using it, because most everyone else is.
A corporation is a liability and pseudonym shield.
A corporation having contribution rights multiplies the otherwise imposed contribution limits on the underlying humans.
A corporation allows tax planning opportunities not afforded to the bottom 90-ish percent of the country (such as shifting income to alternate years to avoid progressive taxation, and utilizing expenses to lower that income in ways a W2 cannot)
Corporations are granted benefits that people do not get, and CU disturbs the balance of power.
Legally, the corporation itself is similar to a person and has rights unto itself, as stupid as that sounds. Those rights are different from those the individuals who run the company have.
We have the techno-feudalism and dystopia but none of the cool aesthetics from cyber punk or arguably even the real Zaibatsu. Nvidia isn’t cool, there’s no interesting aesthetics or back story, it doesn’t appear to have zaibatsu ambitions, it’s just riding a meme stock bubble.
In a different world I can imagine SpaceX to be the only thing that could come close, but well...
I tend to agree, but the Semianalysis + Dwarkesh side of reporting still does surface interesting information. You just have to take it all with a grain of salt, as indeed it is more hype focused, and look for real information hidden in the noise. And possibly to be a bit more entertained as you do so.
I suppose access journalism does have to get something out of the bargain, however small it's not zero. If that's the way you like to spend idle time, well it takes all kinds. I don't understand competitive scrabble either.
The trouble is the article is so poorly written I don't want to look for hidden information, i want to close out. I'm guessing ai wrote it, the information is constantly repeating, and not even always consistently
It seems they had a head start but are now facing stiff competition on all fronts. Software moat, GPU's for gaming, and its distant cousin datacenter compute. They rightfully invested their insane profits into many ventures, and how many of those have turned around into profit?
They are also a robotics AI company with Omniverse. They are also an AI company with Nemotron. They are also a bleeding edge network equipment company after the Mellanox aquisition.
They stand to make a lot of money if they succeed in every venture. Good for Jensen taking risks and driving innovation, I hope they succeed in chewing even 50% of what they bit off.
> They rightfully invested their insane profits into many ventures
This is the inherent downside of such a rapid rise to being the world's most valuable company AND still being considered a growth stock. At their massive scale, the number of new adjacent businesses that have both sufficient size and potential growth is limited.
> I hope they succeed in chewing even 50% of what they bit off.
Anything approaching that is vanishingly unlikely. They're being forced to play the game more like a VC. The question is if a few unicorn winners can offset dozens of losers. The challenge is that, unlike a VC, their bets are much more correlated around AI.
That is a truly terrifying future to imagine. Fortunately I would hazard to guess the medical technology to accomplish something like this is very, very far away. Not only because of the raw technological challenge, but also the barriers to development that scientists, doctors, and engineers would face before even being allowed to conduct experiments.
I do agree though the humanoid form is a dead end for robots. Just build giant cubes that process inputs and give outputs, like a dishwasher. Why wash dishes with meat wand tentacles or try to recreate meat wand tentacles when you can accomplish the job in a wholly different way with far greater efficiency...?
Where's the clothes foldeing cube? Analogous to the clothes washer and clothes dryer.... the clothes folder...
Why stop at dishwashing...? Sell an entire integrated robotic kitchen.
Dedicated machines can obviously solve an issue more efficiently, but it’s still good to have a machine that can do most things adequately.
As an example, you can have an automated washer, dryer, and folder, sure. But what if you wanted to automate the retrieval of dirty laundry and the delivery of clean laundry ? That would need to be some sort of robot to travel throughout an environment (fit through human sized areas, open doors, walk steps) to collect and deliver things. And if I have a robot roaming around the house, I would prefer to just buy one robot that could do many things rather than have to buy it to just collect things and more expensive machines as well.
I think the only real barriers are ethical. I'm pretty sure we'll figure out to do a completely synthetic biological robots without any kind of biological brain in the next 10-20 years, and they don't even have to resemble animals or humans. This should alleviate some of the ethical issues.
It seems like you're implying that teams of 60-80 developers should be expected to outperform teams of 12. This is simply not true. The most important feature of source code as a language is that it allows precise mindshare among close knit teams. It doesn't guarantee it, but it makes it possible for people to talk about the product at a level that is otherwise very difficult.
A convenient side-effect of the source code is that it instructs the machine what to do. But instructing the machine was never the bottleneck, the essential difficulty of software development is in understanding what are the correct instructions to achieve some objective, not typing them out.
The problem is that communication doesn't scale at all. Having just 3 developers with good alignment about mental models, best practices, and design direction is hard enough, and if you found the right three people at the right time with the right ideas, you could generate billions of dollars of value.
Large monolithic teams on the order of 80 are a product of people in control not understanding how software development works, and how to make it work well.
reply