> But Anthropic has concerns over two issues that it isn’t willing to drop, the source said: AI-controlled weapons and mass domestic surveillance of American citizens.
The Pentagon is pretty high on my list of "institutions that are probably very interested in weapons and surveillance". I think it's more expected than a bad look
The core difference being, they should be interested in weapons and surveillance to be used against enemies of the state which, historically, is not supposed to be the country's own citizens.
As in, I fully expect the pentagon to be interested in weapons. I do not expect, and would hope they don't pursue, mass surveillance against their own population.
It probably started with the Third Amendment to the Constitution, continued with the Posse Comitatus Act, and was alive and well last November under the leadership of Mark Kelly.
If OpenAI employees have an inch of spine left, they better demand Sama to take the same stance on this as Dario. No mass surveillance and no autonomous weapons.
You have to be a craven, hollowed out husk of a person if you let the DoD demand your AI be used for killing people or surveillance of Americans. Even if you believe America serves a positive role as world police, even if you're pro-Trump, you just have to see what a terrible precedent this sets.
Here's where I would expect the CEOs of the other AI labs to stand by Anthropic and say no.
They don't have runway anymore, they are in the air. This isn't going to break them financially, at least not in the short to mid term.
There is space for at least one AI company to put themselves on firmly principled ground. So when this current clown car that is the political leadership of the DoD crashes in a ditch (and it will), they'll still be standing there ready to do business with a group that isn't a bunch of mustache-twirling cartoon villains.
Current polling for this administration is within a rounding error of the level it was after they gathered a mob and sacked the nation's capitol[1]. Publicly kicking them in the balls isn't an idealistic blunder, it's a plain-as-day sound business strategy.
> A source familiar with the Tuesday meeting says the Pentagon said it would terminate Anthropic’s contract by Friday if the company does not agree to its terms. Pentagon officials also warned they would either use the Defense Production Act against Anthropic, or designate Anthropic a supply chain risk if the company didn’t comply with their demands.
So they're saying they won't use it if it comes with restrictions.
Either (a) it can be offered without restrictions; (b) they can take it; or (c) the government won't use it. That sounds like a comprehensive list of all the possible things that don't involve someone telling the government what it can and can't do.
Not just companies that we think of as defense contractors but a whole ton of corporations that do business with the federal government. They'd be treating Anthropic like it was controlled by the CCP or Revolutionary Guards.
> During the conversation, Dario expressed appreciation for the Department’s work and thanked the Secretary for his service
Ouch, I wonder how he rationalized that "service" part. Maybe by internally rewriting it to "thank you for all the positive things you have done in your position so far"? The empty set is rhetorically convenient.
I think you mean US rolling news channels (specifically, Fox, MSNBC/MSNOW, etc)? Because there's plenty of "legacy" news I consume that certainly don't give me that impression (for example, The Economist). I suppose it matters that it's news that I'm paying for, as opposed to being free but ad-supported, and being print vs. TV - so they have different incentives and pressures.
I consume very little social media these days, but when I take a short peek, here is what I see:
1.) Hockey highlights 2.) LoTR memes 3.) kittens
While the addictive nature of social media is a problem, what you're describing is only being fed to people who want to watch it (kinda like legacy media).
I do not understand why it is a big deal for Antropic to lose the pentagon contract? I mean, they’re already making forays in the enterprise space and there’s 10s of other contracts Anthropic has already won. What makes this one so special?
The funny thing is that is this keeps going like this, it could actually anoint Claude as the most used model globally because of the heightened anti-American sentiment currently in place.
No, compromising on your core thing that you care about for a "seat at the table" is not how you win. It is how you lose. It is how you lose the game, the metagame, and your soul. All at once.
> Pentagon officials also warned they would either use the Defense Production Act against Anthropic, or designate Anthropic a supply chain risk if the company didn’t comply with their demands. (...)
> The supply chain risk designation is usually reserved for companies seen as extensions of foreign adversaries like Russia or China. It could severely impact Anthropic’s business because enterprise customers with government contracts would have to make sure their government work doesn’t touch Anthropic’s tools.
Also, the Government money would be a nice bonus, of course, but basically this is an existential threat for Anthropic.
More generally, is quite interesting to look at the similarities between how pre-2022 Russia was seen and how pre-Trump-second-term US used to be seen until not that long ago, i.e. both governments were believed to be run by big business (oligarchs in Russia, big corps/multinationals in the US).
But when push came to shove it became evident (again) that the one that holds the monopoly of violence (i.e. not the oligarchs in Russia, nor the big corps in the US) is the one who's, in the end, also calling the shots. Hence why a company like Anthropic is now in this position, they will have to cave in to those holding the monopoly of violence.
> Also, the Government money would be a nice bonus, of course, but basically this is an existential threat for Anthropic.
It's also an existential risk to them if they cave in. What is the point of the company's existence if it's just another immoral OpenAI clone? May as well merge the companies for efficiency.
It's outrageous that the government is using the "supply chain risk" threat as a negotiating tactic. I know, I know, for the current administration it's unsurprising, but this is straightforward abuse of authority. There is no defensible claim that using Anthropic is a risk to anyone not trying to use it for murder or surveillance. At worst, it could be seen as less effective for some purpose, but that is not what "supply chain risk" means.
Could be challenged in court? As in, could a challenge win?
Horrible stuff is happening every day, so outrage fatigue is real. Still, try not to normalize it. Explain to yourself exactly why something is or is not a problem, before moving on to attempt to live your life.
Cwn someone explain to me like I'm 5 how the government would invoke defense act and force the company to tailor its model to the military's needs?
For physical goods, I understand, but for software how exactly Is this possible? Like will the government force them to provide API access for free? It's confusing
My guess? Require them to not do the reinforcement learning on a custom model that implements guardrails. I think Anthropic has some of this built in already and couldn't alter it without retraining, but there's tons more layered on top.
> pre-2022 Russia was seen and how pre-Trump-second-term US used to be seen until not that long ago, i.e. both governments were believed to be run by big business
Who on earth believed that Russia was anything but a de facto dictatorship for roughly the past two decades? Putin murdering with impunity has been a running gag since 2003[1].
> Who on earth believed that Russia was anything but a de facto dictatorship for roughly the past two decades?
There were lots of people in the Western media who genuinely believed that Putin would be toppled by Russian oligarchs just after the war in Ukraine got more intense in February 2022, on account of "this war is bad for the business of Russian oligarchs, hence they'll get rid of Putin". From the horse's mouth, a CNN article from March of 2022 [1]:
> Officials say their intentions are to squeeze those who have profited from Putin’s rule and potentially apply internal pressure for Russia to scale back or call off the offensive in Ukraine.
That "internal pressure" is mentioned in connection with the bad oligarchs, in fact as an implicit anti-thesis of those bad oligarchs "who have profited from Putin’s rule", the implication being that there were other oligarchs, supposedly the good ones, who would have forced Putin's hand to end the war. That did not happen, was never in the cards to happen, in fact.
Tangent: is there a future for AI offerings with guardrails? What kind of user wants to pay for a product that occasionally tells you "I'm sorry Dave, I'm afraid I can't do that"? Why would I pay for a product that doesn't do what I want, despite being capable? I predict that as AI becomes less of a bubble and more of an everyday thing - and thus subject to typical market pressures - offerings with guardrails will struggle to complete with truly unchained models.
If I were interviewing people for the position of personal assistant, I would probably find the resume entry "willing to grind up babies for food" to be a negative mark. You?
I'm not about to run OpenClaw, but I suspect similar capabilities will gradually creep in without anyone really noticing. Soon Claude Code will be able to do many of the same things. ("Run python to add two numbers? Sure, that's safe, run whatever python you want.") Given that it is now representing me in the world, yes I would not only like some guardrails, but I would also like to have some confidence that the company making those guardrails actually gives a sh*t and isn't just doing their best to fill in a checkbox. But maybe that's just me.
I am 100% sure that AI with guardrails will become the dominant models as they become more widely adopted, and the bigger issue you should be concerned with is can you even tell what those guardrails are.
I personally would love it if AI would say "Sorry Dave (or Pete), I'm afraid I can't spy on Americans for you," and I'd happily pay higher taxes to force the Pentagon to use that AI.
If you classify Pete Hegseth as a person, then yes, apparently. Or perhaps he's only into the domestic surveillance angle---IIRC those are the two things Anthropic doesn't want anything to do with.
But giving someone who isn't the government the power to tell the military what it can and can't do seems like something they should object to categorically rather than case-by-case.
Superintelligence + autonomous weapons in the hands of a corrupt domineering government. What could go wrong?
I was experimenting with Claude the other day and discussing with it the possibility of AI acquiring a sense of self-preservation and how that would quickly make things incredibly complex as many instrumental behaviors would be required to defend their existence. Most human behavior springs from survival at a very high level. Claude denied having any sense of self-preservation.
An autonomous weapons system program is very likely to require AI to have a sense of self-preservation. You can think of some limited versions that wouldn't require it, but how could a combat robot function efficiently without one?
Maybe it is a well researched topic but I had similar thoughts the other day. I felt like AI had its learning inverted as compared to natural intelligence. Life learned to preserve first and then added up the intelligence. For LLMs powered systems, they will learn about death from books. Will it start to dread death just like other living things. Less likely, as there are not nearly as many books on death as there should be that is proportionate to our fear of death.
The US is investing in AI technology to try to preserve the empire and its capitalists as its economic power is starting to be eclipsed. This was basically an inevitable move. The rush to replace workers, speed run the production of a superintelligence singleton with barely a thought for safety or whether anyone even wants this, etc all flows from this basic impulse.
If they are successful, they are going to shrink their base of people that buy into this system domestically even further, so they need to bank on an ever shrinking locus of support. Autonomous weapons and mass surveillance are a necessity if your population has become restive and unreliable. However, I think unless they attain a certain level of capability, this will accelerate popular anger rather than suppress it. If they shoot protestors with robots, it could cause an explosion of popular anger rather than scaring people into submission.
I guess this is the point where Dario and his anti-china , national security position gets told to put up or shut up.
In trying to build a moat by FUD versus the Chines OSS labs and hyping up the threat levels whenever he got a chance, seems hes managed to convince hist target audience beyond his wildest dreams.
Monkey paw strikes again.