This is the absolutely horrific next stage for social media platforms:
- They're already well able to surface the most addictive short video for a specific user out of millions of real videos.
- But these millions of real videos are just darts thrown into the space of "videos that could hook the user", in the end even the best-selected of them is not perfect.
- Now, behold! AI allows to generate the perfect video to surgically hit all the switches in the viewer's brain and turn it into a zombie hooked for days on end.
Let's hope our regulations hit these "social networks" hard enough so that never dare deploy this kind of technology.
Social media is not even the worst threat vector, in my opinion. It’s Anthropic/OpenAI/Google, whom people are giving an unprecedented level of access to their innermost thoughts.
We have not yet seen* the kind of large-scale, individually targeted psychological manipulation that cloud AI products can deliver. And I have no doubt the likes of Dario Amodei and Sam Altman will show us, if we give them enough time.
* I suppose the GPT-4o sycophancy/AI psychosis crisis was a preview, but that was just blunt “engagement” tuning.
The largest social media company in the world by several orders of magnitude is Meta, who are both using data collected from their social media platforms to train their AI models and improve ad targeting. Meta also offers AI chat bots on their platforms (FB Messenger and WhatsApp), to capture those innermost thoughts.
Don’t count out Zuckerberg, he’s very good at being a villain.
We tend to forget that both, the social and the media part, input and output, are interpersonal, first.
Ad targeting doesn't need improvement. The incentives offered to people with an adverse attitude towards Ads are shit. Think of incentives as having effects and side effects. It's about intent. Free TV had that down ages ago. It was awesome. But they fucked market entry and whole population segments, more or less conservative and/or liberal as well as more or less authentic sub-cultural patchworks were left unserved.
That wasn't even nonsense! MTV, suddenly going away? Uhm, guys, I only get Peeeeeeeeeeeeeeeeeeeeee... beep boop bop beep boop bop beep... BEEEEEEEEEEEEEEEEEEEEP... (pause) ...brr-ding... brr-ding... Krrrrrrrrrrr-shhhhhhhhhhhh-eeeeeeeeeee-awwwwwwwww-eeeeeeeeeee... Bong-bong-bong... KSHHHHHHHHHHHHHHHHH (click).
Silence ...
around here, and then a lot of popups, ads, .... you know, the wild wild west of internet history ...
We talked TV on the street and in school. And Video games. Now Netflix et al and your feed, too. Who's roofs is most of that media industrial complex under?
Thinking Meta et al and their ads infrastructure and UX weighs as much as that of Free TV is falling for a rather simple Kansas City Shuffle.
"New", only now native media still doesn't interface as well with the brain and our emotional processing as TV and "The News" do. And won't until Neuralink et al get their interfaces all the way up your ... lobes.
Forget Zuckerberg et al and let them do their jobs. They don't need publicity, they need to filter for suitable test subjects, edge case pop segments whose needs & desires are yet to be fulfilled.
There's still a lot of people out there with something to put on the market but they can't because a little something is missing and nobody seems to know why ... something double long running on the news maybe?
> We have not yet seen* the kind of large-scale, individually targeted psychological manipulation that cloud AI products can deliver.
We did. Pre-WW II ..., and afterwards, the old, everlasting Cold War (vs the new one). And there are self-emerging/self-organizing buckets of people who all pledged their allegiance to ideas and plans, works forever in progress and aggregating jobs and expanding industries. People with value and virtue systems, radicalized or called to some need-to-know duty. And the personal level was always covered by applied and philosophical psychology, cultural discourses, however progressively or decadently lead by public figures while peoples' desires were and still are shaped by interactions, social AND parasocial.
Shouldn't we stop wondering whether they have enough time, now that several hundreds of thousands of hours of work already done and work yet to get done are neatly compressed into a few instants?
AI will always be "two" things:
- a search engine that misses less and less and is just waiting for you to increase your processing power
- a tool to cope with your laziness or disability by deferring or skipping intervals of learning curves and/or biomechanics
- an evergrowing compendium of abstractions and intents, formalized or not, by humans, for humans, off humans (even when it's AI)
but that's irrelevant ... because humans will always be only "two" things, as well.
So, ... the threat vector remains unchanged, however AI, accessibility and skill will amplify or change the landscape: casually, no, leisurely ignorant bystanders.
depends. I, for one, take control of what I consume on social media. Bluesky's design allows this. YouTube also, if you turn off recommendations, lose watch history, and vet your subscriptions
What makes you think they’d restrict this to logged in accounts? Not so long ago, Meta got caught building “ghost” social media profiles for people who didn’t have accounts on their platforms. They essentially constructed a “missing puzzle piece” based on people this person knew who did have profiles in publicly available information. Building a psychosocial profile on a user who has not set up an account yet using AI would be trivial.
Common misconception. We won't kill the planet even if we tried to. Wed make it too hard to sustain billions of humans, sure. But they are very different things
This is a good distinction because its a common idea but its based on a conflation: human civilizationhealth and planet health. Planet health is hard to even think about apart from humans but the planet in no way “needs” us.
Fortunately that particular extreme scenario doesn't seem likely. Radiance scales with the fourth power of temperature, and we're not filling the atmosphere with Venus levels of sulfuric acid. Our CO2 output will be self-limiting when it kills us and we stop outputting it.
you really have to catch the moment to do this, and there must be someone else who is supportive of this idea.
my mother recently got served a quite... disturbing video on Facebook. I won't get into details of the vid. what I can tell is that she was so dazed that she didn't even immediately understand me when I asked her, "delete Facebook and Instagram immediately or else they will think you like it and will serve you more." she kept replaying this video in her head. in a moment, I thought I wouldn't make her snap out.
for context, she is a damned Reels addict with war-induced traumas. (Ukrainian here.) she agreed to delete Facebook but decided to keep Instagram. we frequently pick bones about her sending stupid Reels and me hating them with passion. and every time I said "delete Instagram and your life will go up," she brought up scandals until she talked to a therapist.
EDIT: to quote a sibling comment, this is, indeed, like dealing with a drug addict.
it interleaves the end note in the text at the point where the endnote is referenced (by number), reads the endnote, and then sounds a bell to indicate that we've returned to the main text.
Sometimes endnotes reference other endnotes and you have to wait until the point in the text, e.g. endnote 304 is referenced from earlier endnotes than it occurs in the text (rather late in the book) and you just have to wait until endnote 304 is read to find out what it is. that's a bit of a disadvantage, but infinite jest is the kind of book that you really need to read multiple times in order to get it anyways.
And yes, I agree. Im on my second read through now. The first time I skipped the first ~50 end notes. I thought the creator of infinite jest was a mystery, same with Joelles deformity (“was she really mangled or not— maybe we find out at the end!”). Oh, those two things are spelled out clearly in early end notes..
Any DFW fans here want to conjecture as to what “atemporal” jazz Pemulis listens to? I was thinking maybe some albert manglesdorf or perhaps ornette coleman..
I think parent is referring to the film in the novel, also named Infinite Jest. The film is so compelling that its viewers lose all interest in anything other than repeatedly viewing it, and thus eventually die.
Carfentanyl potentcy is roughly 10,000 times morphine. The pursuit of optimized technology usually far outpaces regulation or any kind of control. As technologies are controlled, even the lessened availability of more potent types causes an enormous overall issue. Humans are obsessive-compulsive optimizers, for good and for ill.
I think you're giving this a bit too much credence. And I think you also think there's something special about sorting videos by how often they're viewed in a recent timeframe (which is all you need to recreate evil social media algorithm)
there's not gonna be a regulation and even if there is, the damage would already be done. If you want to protect yourself, then best thing is to just delete the account and apps.
Stop calling them “social” media. They have not been that since algorithmic feeds driving infinite scroll replaced updates from your friends. That was more than a decade ago.
There is nothing at all social about the latest generation like TikTok and YouTube Shorts.
If YouTube is social media, anything with a comments section is social media, even the old "guestbooks" of web 1.0 days were social media. It's a pointless term whoever uses it.
Youtube is very obviously social media. It is a platform that serves algorithmically sorted third party content based on user preferences and a network of accounts that a user follows, and it allows communication between users through comments. It has communities. It has memes. It is certainly addictive in all of the ways social media can be.
Don't forget it's not "the platforms" doing this, it's people like you and me, maybe even people reading this message. We, they, are building this, and don't care about anything as long the paycheck comes. This is the reality.
They have information barriers to keep the non-sociopaths from understanding the horror they're working on. Military forces do the same thing. Even the German train drivers didn't know what kind of facilities their trains were going to or why.
I’ll leave it to other comments to discuss the societal and moral implications of being able to do this (which, I agree, ick…). On a practical level:
We train an encoding model, a “digital twin”, that predicts how each visual region responds to any video. Now we can ask: which video would make a chosen region light up the most? NEvo searches for that video automatically, using the twin’s prediction as its reward.
I only scanned the paper, so maybe I missed it, but is there any confirmation that this ‘digital twin’ works? Like, do the generated videos actually cause the same patterns as in the ‘digital twin’ brain model in real humans in an MRI machine? My instinct is to be skeptical that it’s possible to reliably create a video -> brain activation prediction model.
Yeah, this smells very fishy to me. It's almost trivially easy to gather a small validation dataset in humans for the paper. At my institution, it's about $800 an hour to scan someone. You can probably get enough data to validate the model with a half hour scan. Surely the group has enough grant funding to pop a few healthy controls in the scanner.
I haven't looked in super close detail to the paper, but their methods section says that they fit a video model (V-JEPA2) to the fMRI dataset in a voxelwise ridge regression, meaning that the baked in assumption is that the visual response affects each voxel independently. Voxelwise models are very nice for making statistical inferences, but are less good for prediction and modelling tasks, because our brains certainly do not work as collections of independent regions.
BOLD is intensely messy data, and their design is far too simple IMO to reflect anything of reality.
All the signals that are missed by the (f)MRI are never "mirrored" in the digital twin even though on screen it will look like it. Bam, Experimenter Bias in the machine and I don't know how to phrase it but ... does this method/experiment leave any wiggle room for Falsifiability?
Even if the Hawthorne Effect does not apply to humans, it most certainly translates onto the brain and sensory (post)-processing-- live and remembered (vs in-memory, stored), real, virtual or imagined.
... I just realized how vastly different peripherals are when real, remembered, virtual or imagined sensory input is (post-)processed ...
It feels more like the machine will reproduce a superficial pattern. And at least some signal streams echoing in the brain on input from the digital twin won't emerge because the digital twin has only part of the data. And that's a premise for an unnoticeable rewiring of what once used to fire together, ... given a hell of a lot of exposure, of course, ... or not ... let's see
EDIT n: the paper says pretty much all that in like ... science, bitch!
yeah. This is one of those things where they used previously collected data, built a model that has validity on modeling validity between simulation and actual brain (encoding model for digital twin), then built a new model using the encoding model that tries to reliably activate particular brain regions. The next obvious step is to validate this back to the original person, but they will need to collect new data, and so likely, will need to get research funding. One step at a time.
The digital twin types of models will become more and more useful, but I don't think we will be at the point where all research will be done entirely within these models.
A. Unless you think that brain science is immoral science, then I do not see any problem with this kind of research. As a neuroscientist, I strongly object to the insinuation as unfounded.
B. "Digital twins" e.g. [1] are a growing class of brain simulations that can successfully approximate brain activity patterns at large scale. I think these can be very useful, but we shouldn't think that they are at the level of actually simulating a brain. They are usually made of model neuronal approximative simulations (e.g. integrate and fire, balancing excitatory and inhibitatory neural populations within units), then using diffusion imaging to estimate white matter axonal wiring between those populations from the subject to increase the accuracy of the simulation. These are increasingly being used to, for example, model how a surgical intervention would effect seizure propagation prior to actual surgery.
Here is a nice episode of Theoretical Neuroscience podcast [2] on the Virtual Brain [3], one of the available models for this kind of work.
C. In terms of validation. Only partly. From my quick read, this NEVO model optimized neural response only in the digital twin encoding model. While the digital twin model reportedly has solid predictive validity [4], which by the way was not the Virtual Brain model I mentioned in point B. Moreover, the outputs looked neurobiologically plausible, but at this point, there is no independent model or new fMRI showing the optimized stimuli actually drive the target regions. This was performed using previously collected fMRI data, and full validation of this model *IS* the obvious next step, but the money to collect such data does not come from nowhere: funding will be needed, and such a paper as this can help them get it.
D. A final point I'd make. We have long been able to create static stimuli that we can be fairly certain will activate above baseline certain brain regions, on average. Certain stimuli-region pairs ar emore homogenous between people, others e.g. the fusiform face area (FFA), are small enough that individual differences prevent a simple ROI approach, and identification depends on using face stimuli to identify at the individual level, but for the most part, it is reliably locatable. Brain activations are very coarse things. In fMRI, you are talking about ~3x3x3mm voxels (27mm^3) where the hemodynamic responses have a ton of spatial autocorrelation, or in EEG, where the surface spatial area of the reeptive fields are very large(~400 mm^2). These virtual twin models already do a decent job of modeling dynamics of the brain there parameters are tuned to *at this scale*..but this scale does not have a ton of information content. Automating this with video content is not that much a reach.
This is very similar to last week with that mind reading startup thing. Please read the paper before commenting.
This is a tool to help researchers in figuring out what different parts of the brain are actually for with less experimenter bias contamination of “well we think maybe it’s about this so let’s show it video of x to see”.
The essence runs on having someone sit in a scanner for a couple hours watching all sorts of things, and then feeding that to a model that will then build its own representation of said data and try different things on it until it’s found what makes a certain part sing in the model.
The purpose is a generalized understanding of brain function, more or less the same way we’ve been doing it all these years. Expose brain to something, record it somehow, see if brains reaction in the recording helps you understand more about who we are and what cognition is.
What is a "purpose"? Something people wish something would only be used for, right? How does it relate to, what influence does it have on what something will end up being used for?
> The synthesized clips line up with what each region is known to care about, faces for FFA, places for PPA, bodies for EBA, motion for MT, patterns for V1 / V3A, and lively social scenes for pSTS / aSTS
Explain the potential to exploit strong stimulation of specific visual regions for evil. "Oh, I very much detect a face/place/body/motion/pattern/human", says the subject. What are you going to do with that, startle them?
Apply the same technology several layers deeper. Stimulate the novelty or sexual attraction or general reward neurons. Stimulate the thirst center in a Pepsi ad.
Oh you are right. Let's just end neuroscience altogether. Also computer science, aerospace, and biology in general. In fact, let's go full Amish. Wait, no. Someone might use a buggy cart to run people over.
This specific research may not be morally corrupt in itself, nor may his intentions be bad, but it could absolutely lead to something horrific. Nuclear technology was also initially developed with good intentions and provided much good for humanity. But of course it ended with biggest bad actors having a lot of nuclear weapons.
>Expose brain to something, record it somehow, see if brains reaction in the recording helps you understand more about who we are and what cognition is.
It also helps companies like Moonbug Entertainment (Candle Media) understand how to build better Distractatrons.
It’s a small TV screen, placed a few feet from the larger one, that plays a continuous loop of banal, real-world scenes — a guy pouring a cup of coffee, someone getting a haircut — each lasting about 20 seconds. Whenever a youngster looks away from the Moonbug show to glimpse the Distractatron, a note is jotted down.
“It’s not mega-interesting, what’s on the Distractatron,” said Maurice Wheeler, who runs the research group. “But if they aren’t fully focused, they might go, ‘Oh, what’s that?’ and kind of drift over. We can see what they’re looking at and the exact moment when they got distracted.”
> p8: "A child watching television under normal conditions is subject to frequent interruptions and distractions. The TV must vie for his attention. In order to simulate this condition, we decided to program distractions into the laboratory situation... Slides could be used to fill the slide tray and they could be projected automatically, at regular intervals, onto a screen similar to that of the television set. The carousel projector allows the viewer to choose three exposure times. The 7.45 second interval proved most satisfactory with the preschool children.
And nowadays we think of Sesame Street as wholesome compared to today's content. They may have optimised attention but they were delivering positive-value content. That's not the case any more.
the same year as Baudrillard's Simulacra and Simulation, featuring computer generated characters, brainwashing television commercials, a light pulse gun that causes absence seizures, gun battles inside a plastic surgery clinic and an AR simulation environment, a sadistic computer, a physician who doubles as an action hero, and James Coburn giving a lecture explaining the enemy's evil plans in the opposite role that he played in the (excellent) The President's Analyst.
This time I was not so dazzled and saw it for as atrocious everyone else things it is. The minions of "Digital Matrix, Inc." manage several assassinations with the light-pulse L.O.O.K.E.R. gun but when they use real firearms they outdo Vader's stormtroopers by shooting each other. (Want to see the scene where somebody from E.Y. tells them to stick to the L.O.O.K.E.R. gun) The bad guys explain the penultimate secret to the protagonist early on but the ultimate secret is revealed in the L.O.O.K.E.R. lab which doesn't feel like a lab at all but rather a rather good room in a theme park experience where you're supposed to uncover the secret. (Contrast that to the lab Doug Trumbull outfitted in the Brainstorms movie a few years later which is packed with real surplus equipment... I've been to that lab!)
I can see it now.. The Distractatrons: a new chapter of protagonists in Transformers! The modern equivalent of evil in this day and age of ADHD and low attention span!
An anecdote, but some reason to think that overworking part of your brain is a bad idea:
I had an Aunt who had dementia. This is obviously a terrible outlook. But her and my Uncle seemed to be doing okay. My Uncle was an ultra-competent guy, highly stable, the person you could rely on; and had been his whole adult life. So it was shocking when he had a mental breakdown and became manic. There was probably something physical going on, but what was also going on was that he had wanted my aunt to be able to live normally for as long as possible, so had been covering everything - for a year or more he'd had to be alert 7 days a week in case my aunt tried to cook on the gas stove or something like that, at which she was no longer safe. So the risk-alert part of his brain had been constantly overworked.
I appreciate that this is scientific research, but there are definitely companies out there that will try to row-hammer everyone's brain if this sort of thing is not heavily controlled.
I'm not sure it's about overworking as much as stress - stress is a known potential trigger for manic episodes. Typically in people who already have some biological potential for experiencing it. And the stress of having your partner come down with dementia and suddenly becoming their full-time caregiver must be absolutely massive.
But re the underlying physical thing: You might look back and be able to spot some episodes of hypomania in your uncle's life. Perhaps just reduced sleep, or twinges of paranoia, or periods of on-again/off-again hyperproductivity, etc.
Hope that if this story is recent he's doing well. The recent generations of medications to prevent and treat mania are a huge improvement on the ones available in the past, so fingers crossed.
A completely off-topic anecdote, then a completely unfounded fear based on a clear misunderstanding of this research. If companies ever start making us live in a MRI machine so they can reliably activate brain areas with certain stimuli, I think there is a bigger problem we are having.
Here is how I see it. This type of research helps us understand the brain, helps us do things like model potential surgery sites better (e.g. for seizure activity interventions). What it does not do is become the basis for mind reading.
Just so that you know that I am not totally unsympathetic to your apparent worldview, let me tell you how I think your concern might actually play out:
AI continues to get more powerful, and computer brain interfaces begin to move beyond EEG scalp electrodes, but begins to take the form or brain augmentation via integrating networked AI compute into implanted chips with electrode neural interfaces, where our brain's neurons and the neural interfaces learn to speak each other's language (i.e. integrate). I can actually see this happening within 20 years. At first, it would be our brain using this augmentation for greater intelligence, etc. However, remote manipulation of the interface could reverse control of brain activity, leading exactly to your fear.
But this research? This is light years from that,,,so much so that it is not even relevant to bring up unless you are just against all technology.
> I appreciate that this is scientific research, but there are definitely companies out there that will try to row-hammer everyone's brain if this sort of thing is not heavily controlled.
I will state plainly that I do not think you are asking this maliciously, and that instead you are asking from curiosity. From that perspective, though, I am a bit flabbergasted.
Are you truly curious as to how it might not be A Good Thing that those engaging in high-finance capitalistic money making programs would have access to forcefully engaging a human being's brain into activity, with or without their consent, to such an extent as to drive motivations and bias neuronal activation that leads to what they think is a choice?
Said another way: Are you really curious as to how allowing a company additional tools to bias human interactions may not be A Good Thing?
If you need an example as to how this is clearly not A Good Thing, and that at least a portion of the world population agrees with that characteristic, that there are currently companies that have tried to actively do this already, and attempts are being made at least to fine them for their behavior as an attempt to curb it. Whether or not it succeeds is a question of governance at the moment since the US is currently under plutocratic control.
However, I would really urge you to explore this topic and really engage with what's happening due to the plutocratic class' capabilities over technologic change and societally enforced or biased adoption, and how they, in many extreme cases that are continuing to become more extreme, are not A Good Thing.
> Are you truly curious as to how it might not be A Good Thing that those engaging in high-finance capitalistic money making programs would have access to forcefully engaging a human being's brain into activity, with or without their consent, to such an extent as to drive motivations and bias neuronal activation that leads to what they think is a choice?
I guess I just don't think this is the topic at hand. What you're proposing is mind control. That's not what's at issue. What's at issue is scientifically optimized superstimuli that can predictably activate certain brain regions. So when I read "rowhammer" I'm thinking about it in this context, the ability of a video to consistently activate a targeted section of your brain.
As you should know, and IIRC, cognition does not cleanly or consistently map geographically in the brain. To go from the ability to activate a subset of brain regions to influencing behavior is not a neglible step. The only real potential I imagine is in emotional activation, which is fairly well-localized.
So really what we're talking about is being able to evoke ideas and feelings with more consistency and precision than before. This doesn't seem new. And it can't be that intense either—I have the sense that your hypothetical mind control video would be extremely annoying or unpleasant before becoming influential. And for now we have the freedom to choose what we watch and turn the video off.
And it might be reasonable to say that you were referring to future possibilities, and that I'm focusing too much on what's currently available. But I am not just reacting to the first quoted clause of parent, but to their [vomits] policy recommendations. "This sort of thing", if not heavily controlled, etc. Maybe you see no value in this research, maybe there isn't any. Nonetheless the paranoid and the scifi-as-nonfiction readers will throw up the Bat signal for the idiots and moral panic orchestrators in power to wrap us all in comforting red tape for our own good.
I exaggerate. And I am talking about many more things than you were, as if the volume of text strengthens any rebuttal. All I'm saying is, this tech has little power over human behavior, in my opinion, and we should be cautious before bringing the hammer down before we know if we've got a bumblebee or a wasp.
Its like we don't already show faces in fMRI experiments because we know that it will often activate the fusiform face area. Its just light years away from anything dangerous, that I am compelled, as a neuroscientist, to try to shake some sense into people. This is HN, but it feels like twitter on some topics.
This just strikes me as the false dichotomy that there are two option; complete freedom of choice, and complete mind control that you are physically incapable of resisting. That as long as you are not physically incapable of, say turning off a video, that there is no problem. Just because you can stop using crack doesn't mean it isn't dangerously addictive. Just because you can say no to someone holding a gun to your head does not mean it isn't incredibly coercive.
Also this seems to ignore that people can be influenced or manipulated without being consciously aware of it. Like the people that say that marketing doesn’t work on them, failing to see that every interaction with the products and services they buy and use are curated in some way to influence and manipulate them.
I'm not certain I took that dichotomy as a premise, but regardless I agree there is a middle ground of many patches occupied by real events characterized by mixtures of intentionality and submission. That's the easy part. What I think is the harder question posed by my original parent is when is it worth wielding the state at the issue.
For example, I'm in theory opposed to the prohibition on crack cocaine, despite its strong consistent effect on personal agency, because of two things: basic moral principles (my body my choice), and the massive (hopefully unintended) side effects of prohibition. Namely, mass incarceration, black markets, decreased drug quality, gang violence, exhaustion of policing resources, the public perception of systemic racism, and tax losses. My guess is the downsides of a hypothetical ban on algorithmically optimized advertisements would be of a very different kind but similar quantity. And again, the other issue is basic moral principles. I'm not comfortable with bracketing free expression with numerous exceptions, rate-limits, inspectors, or whatever apparatchiks would be employed to enforce such a restriction.
You just reminded me why I unsubscribed from ground news: you can't share a link without them adding their own paywall to it, even if it's a free article. Fuck that
I bet it wasn't overwork, it was the fight or flight response being constantly activated for your uncle when he's caring for your aunt who could do anything unpredictably.
Also: one of the V3A animations reminds me loosely of things I saw when I was a kid, at night, shortly before I slept (though my experience then was more circular).
How fried does someone's brain have to be to say "others on telegram", as if everyone is in the same chatrooms together? If they're on HN, they should at least know why that doesn't make sense, so I have to assume they're in a deteriorated mental state.
Friendly reminder to touch grass on a regular basis... Even virtual grass in a video game is a good pressure valve for excessive social media use.
Isn't there that one Harry Potter warning. I think it was the potion guy who said too much luck is dangerous. I guess that is somewhat of a parallel to this. Too much positive visual stimuli is dangerous or bad.
My understanding is that those who work with the mentally handicapped use bright lights and other stimuli to soothe and control them. It is also my understanding that the autistic are stimulated by vibrant colors (coughcoughMy Little Ponycoughcough).
Who is to say that the rest of us are not also vulnerable to such controlling stimuli?
There are undoubtedly 'adversarial images' that will induce certain effects in particular biological minds. It seems like this is neither good or bad inherently, just another tool.
I can imagine truly horrifying images coming out of this. Many people who have spent a lot of time online have come across unfortunate gore images that they may have seen for only a fraction of a second that they wish they could unsee. One can imagine the brain is able to see much worse.
June 9, 2032: ReMind, a new biotech startup, has opened to a recordbreaking IPO after a demo and a series of small-scale trials that stunned the world. The company is pioneering a new way to ingrain memories into the human brain - each ReMind screen is equipped with a special camera that can quickly recognize anyone who walks up to it, and ReMind's secret sauce algorithm is able to use past data of each user to generate imagery that will have the most profound impact on their brains. They claim that the algorithm is highly effective, being able to burn in imagery into the viewer's brains with an average accuracy of 83% per viewing. The CEO has promised to launch two services to start: ReMind Long-term, which focuses on creating imagery that simulates a 'traumatic' impression for creating long-term memories, which will be immensely useful for advertisers; as well as ReMind Short-term, for milder experiences that resemble a catchy song in a visual format for a quick boost in sales for a limited-time promotion. Advertising companies around the world have already placed tens of thousands of sales and deployment of ReMind-equipped screens is planned to begin immediately. Look forward to spotting one in your grocery store, in an elevator, on gas stations, on in-flight entertainment systems and in place of billboards on public spaces. The future is looking bright for ReMind!
I’m always skeptical of these papers that claim to read minds, because my understanding is current brain reading technology is very coarse (Neuralink is SOTA, very invasive, and only 1-3 thousand electrodes for 55-70 billion of neurons; fMRI is much worse) and neural networks generate plausible images and videos from noise. Indeed, the example videos don’t seem particularly stimulating, just random.
Books are designed to be addictive. People said the same things about books as they say about screens nowadays. I think it's more important to understand the emotional hysteria people get themselves into
My favorite addiction epidemic was when apparently the "catch the ball on the string in the cup" took over parts of Europe to the point that people were ignoring their work.
This is already happening at scale by the social media feed algorithms. We don't need generated content to accomplish this. In a sea of user created content, plenty of it is already at peak activation.
The title reads a lot like the lab logs you'd find in a horror game.
But for the paper itself, it seems they're using genetic optimization over predefined keywords. Wonder what would happen if they did gradient descent on the latent space directly. Is brain stimulation just not a good domain for GD?
This does indeed seem comically evil. While surely this may provide somewhat interesting insights in how our brain processes things, this seems squarely past the "should" part of "you scientists were so obsessed with whether you could you failed to consider whether you should"
And I have yet to see a single paper like this where a researcher bails out and publicly says they refuse to work on such projects. Not one.
The most benign interpretation of this observation is that science is filled with spineless opportunists who don’t care who they hurt with what they create. A slightly less benign interpretation might be that many of these people are doing this deliberately, and getting off on the sense of power it gives them.
I was asked to train a neural network do detect how much pain a mouse was in- our partner company would be responsible for hurting and filming the mice. I refused and subsequently quit- this produced no paper and I don’t know if they got someone else to do it. I probably should have done something stronger.
When skip level bosses on my last job wanted to do boneheaded things in automotive design they usually had to keep asking different engineers until they got a yay.
When it is pushed from the top it is hard to stop at ground level.
In their defence, don't shoot the messenger. Just because they published it doesn't mean that others haven't already discovered it. Better to know its possible than be completely ignorant.
> Just because they published it doesn't mean that others haven't already discovered it.
That’s irrelevant for the moral evaluation. Everyone is responsible for their own actions. If you choose not to kill anyone it won’t stop others from killing. But the fact that this is so doesn’t give you a license to kill.
I'd say we're already well past that point. Short-form "content" already exists and is messing with people's brains, this is the same thing just taken a few steps further. By the time the tech companies start using it, it will already be too late and we'll be left discussing whether the next man-made nightmare they come up with is the point where the tech industry must be stopped.
The problem is the incentive structure. We could be doing good things with tech. The incentive is to use it to addict and destroy. Addiction and war is where all the money is, not education or health care or positive forms of entertainment and art. No money in those. Why?
I’m sure there was an early hominid version of this discourse. “Maybe bad to make sharp rock and sharp stick if this what we do with it…” “Mmm yes someday we make sharp rock big enough smash world.”
Advertisements actually run the world. Imagine a world without ads. The economy would grind to a halt. It is exploitation and manipulation, but that enables creation of large capital, which leads to great things. In the past it was pyramids and temples, and in modern times it is space exploration, scientific research and other things that require huge capital. Even the current advances in AI/LLMs are made possible by mass exploitation.
Without ads and exploitation of the masses, none of these would not be possible.
Slavery (or getting poorer people to do the work for you) is the central concept of civilization. It cannot be done without it. It's not capital or advertising that makes all this work, it's forcing the poor to work and pay tax that makes the world go round d
The pyramids were not built by slaves, and while we can waffle around about "all labor is exploitative to some extent", it doesn't take exploitation-maxxing to drive great achievements. Most great modern achievements are driven by a desire for self-actualization or recognition, not survival.
This reads like the mental hoops someone who works for or invests into Meta comes up with to resolve their cognitive dissonance.
You might as well have said that without advertising, we wouldn't have charities, freedom of thought, and literature: the argument is as strong and might make you feel even better about yourself.
Oh, turns out that you did say that advertising solved slavery in another comment. Carry on, mate.
Even if that is true, what you are saying is, essentially, that the ads-driven exploitative economy is a very expensive innovation engine.
It is then useful to ask: if innovation is what we want, do there exist engines that aren't quite as expensive?
If the market economy has more of a human face than outright slavery, there's no reason to believe that it's impossible to do better: a Copernican position would say that we'd be no more likely to be correct than an ancient Egyptian who claims it's impossible to do better than state slavery.
Please tell me how ads help science research or why should I care about space exploration or LLMs that threaten me and hundreds of thousands of other peoples jobs?
Can’t tell if this is satirical or serious. Those ancient wonders were enabled by slave labor, treating people as “capital” is the ultimate climax of capitalism.
I actually had a similar idea that I cam across in my own research in brain-to-video applications. You could theoretically train the model in real-time by using brain data as a reward signal, such that the model would find the image or video that most strongly produces the desired response (e.g. an increase in certain brainwave bands associated with pleasure, or alertness, or fear).
I can imagine this has some very undesirable usecases if used in the wrong way.
Apart from ethically bad and evil use cases of this application, can we use it to massage the parts of brain like we do it to our bones and muscles with the help of physiotherapists?
reason I am asking it could be some relief to our brains after tedious working day, especially after heavy AI usage
Sure. The relief is actually free ; though once every 10 uses we'll spike your dopamine-related brain centers to 1,100% activation for a quick second while flashing a peptide sponsor ad, then back to normal.
You can of course, also not get the spike ads, by subscribing to Premium. Just have a quick look at this animated QR code, it will explain better than I can in writing.
Edit: there used to be an OSS app you could use where you actually pick what you want activated in your brain and when, but it's been banned after one of the commercial producers's investors convinced the government that this technology was too powerful, so only government-approved apps are allowed now, and an emergency vote passed the BrainControl law authorizing logging of all visual stimuli you look at, in order to protect your children.
I'm pretty sure by blindly following this path to commercial success in marketing; we're just gonna end up with content that is as implicitly adjacent to porn as possible without being explicit.
Very interesting. We have an organic experiment converging to maximum stimulation in short form videos (which will become the majority of training data for future video gen models) Already approaching the capabilities of a “mood organ” from blade runner. Except usually most people don’t even make the choice to change their mood anymore.
https://en.wikipedia.org/wiki/Do_Androids_Dream_of_Electric_...
For a video to _literally_ maximally drive a target brain region there would be fatal consequences - e.g. the plot of David Foster Wallace's Infinite Jest (1996).
I wonder if the end-game of this field of research will be to run these simulations at scale using neuron-on-a-chip services such as [0] Cortical Cloud.
The sci-fi hand wringing is embarrassing. A huge chunk of the population is afflicted with mental health issues and the best psychitrists can seem to do is "Oh bro, wanna try some ketamine? Anti-psychotics?? Maybe some SSRIs, how about psychdelics?!" - this is a joke. Im all for more neuroscience research in every form because our contemporary understanding is embarassingly poor. This is a research tool for understanding what visual brain regions encode, not a mind-control device. Better models of brain function may eventually improve diagnosis and treatment of mental illness. Calling every neuroscience advance "Black Mirror" is not ethical insight. It saddens me the reaction so many commentors have here, it is just lazy technological illiteracy imo.
To flip the question on it's head, what exactly do you see as the "evil" potential? The most prominent comments to this effect are just scifi handwaving.
For one, it's bound to teach us something about our brains. And while I'm not usually an optimist, I actually find it more likely that we'll take active steps to stop the evil applications of this if research like this is out in the open.
Fuck this and whoever is driving this type of research. We’re increasingly living in a world designed by sociopaths. What do these people think this research will be used for?
I wish we had the Hippocratic oath for STEM, or at least that they would take ethics seriously rather than an afterthought against the god of Progress at all costs.
There are ethical use cases for this research. The first that comes to mind is mapping the brain before a surgery. Right now some surgeries are made with the patient concious to verify that relevant brain damages are not being done. Having a deeper knowledge of the patient's brain before the surgery may decrease risks further. The bad scenario (using targeted content to hook the user to a screen) already exists, it's not like it can get that worse (or at least so I hope)
Will be interesting to see how strong the controlling forces can be - enough to make you miss things in direct perception like in the book, or only softer effects further up the cognition layer stack
My current theory is that these are similar to cigarettes. Nobody likes the first draft, it burns your lungs, your entire body wants to reject it. But the nicotine stimulates just the right receptors so that if you keep at it for just long enough, you'll be hooked and start disregarding the terrible taste, smell, tar in your lungs, and yellowing of your teeth.
All of this to say, if you subjected yourself to just enough TikTok scrolling on just the right topic, you might find yourself using it occasionally after that initial hump, then slightly more frequently, then daily.
You might still not "like" it, but the habit is what matters.
As a teenager, I tried to get addicted to cigarettes so I could stand by the cool kids and smoke with them. I started smoking 4-6 cigarettes a day but hated it so much, I couldn't continue after a week or two...
I hope my brain is also different. I also have never spent hours scrolling through short-form videos on Instagram, TikTok, Facebok, etc. I never ever walk outside with my phone in my hand, instead enjoying the view.
I do enjoy watching YouTube videos at home, on the living-room flatscreen, on a variety of topics, but I select them manually, one at a time, from the vast selection The Algorithm(TM) offers me, plus my own searches.
"Prime Intellect, I would like you to begin stimulating the neurons of the pleasure center of my brain, one at a time, and remember the ones I report to you as being favorable."
Really, I don't think you need to worry. A prevailing latent attitude in the comments is that the brain is somehow super malleable and subject to external control through some incantations. Like subliminal messaging, it will at most be moral panic that fades, and then people will act like they never believed in it.
If you read the techbique it reads like something far less remarkable being PR’d to sound like a big deal.
The fact it’s bucketing by making images of lighting and facial expressions, the fact it doesn’t natively do the video it does an image then video generates from it.
The results look really bad and samey. Doubt this would work for the actual thing they’re pitching it for.
I am not sure if we should want to understand how our brain works. This is why I don't like libertarian free will people. We need to accept that we don't have free will. Otherwise we will study the brain thinking that we are above it. No. We are soon going to find out exactly how the brain works and I am not sure people understand what that means.
Customized propaganda made for the individual is the only thing I'm afraid of. Right now its easy to avoid the farm trolls that harves boomers minds on facebook, but given the costs go down and tech improves, we each of us have a trollfarm working specifically for us and that would make it extremely hard to not get brainwashed.
