Not quite the same thing but some non-negligable percentage of ads I see on Facebook are outright scams which purport to be selling musical instruments at a 'markdown'. First guitars supposedly from the Sam Ash bankruptcy sales linking to an obvious fake site and more lately 'free' giveaways of high end Gibson acoustic guitars. When I've reported them I got the feedback that it didn't violate community standards, but my insta account got perma-banned when I posted the original of a song on youtube from 1928 on a thread which started with a cover from 30 years ago. That was considered spam.

I had that reaction as well, but consider: clickbait is such because it takes more work (emotional or logical) to reject it than an ad which is merely not relevant to you. Thus, your (and my) recall of ads is probably biased towards clickbait, and we overestimate its prevalence.

In the last 6 months, I've had to buy a few things that 'normal people' tend to buy (a coffee machine, fuel, ...), for which we didn't already have trusted sellers, and so checked Google.

For fuel, Google results were 90% scams, for coffee machines closer to 75% The scams are fairly elaborate: they clone some legitimate looking sites, then offer prices that are very competitive -- between 50% and 75% of market prices -- that put them on top of SEO. It's only by looking in details at contact information that there are some things that look off (one common thing is that they may encourage bank transfers since there's no buyer protection there, but it's not always the case).

A 75% market rate is not crazy "too good to be true" thing, it's in the realm of what a legitimate business can do, and with the prices of the items being in the 1000s, that means any hooked victim is a good catch. A particular example was a website copying the one for a massive discount appliance store chain in the Netherlands. They had a close domain name, even though the website looked different, so any Google search linked it towards the legitimate business.

You really have to apply a high level of scrutiny, or understand that Google is basically a scam registry.

That usually means you tend to visit trash sites. Higher quality sites have higher quality ads. In fact, for the highest quality media, people actually PAY for ads. See things like Vogue September issue or technical shopping magazines, which earn value for being 90% ads. People used to buy local newspapers because of the ads as well.

Ad company says ads are good, water is wet, news at 11.

In my (fortunately uncommon) experience, all ads served by Google are clickbait, or even blatant scam (like fake interviews of celebrities that explain how they earn lots of money without working, magical health enhancers, etc).

> it does not align with my anecdotal experience

Given I'll often see the same fraudulent ad repeated I think anecdotal experience is there are not many of them.

I can even talk to friends about the most boring fraudulent ads and they know them. i.e. Elon doubling your bitcoin scams.

For normal ads unless they are viral, there are millions out there that are never repeated or not even seen.

Because fraud ads have short lifetimes pulled out of 'production traffic' you can collect many for the training data

I assume 'clickbait' is the safety word for 'fraud'

We successfully built an AL pipeline by building models that quantify both aleatoric and epistemic uncertainties and use those quantities to drive our labeling efforts.

Specifically, post training you measure those on an holdout set and then you slice the results based on features. While these models tend to be more complex and potentially less understandable we feel the pros out-weight the cons.

Additionally, giving access to a confidence score to your end users is really useful to have them trust the predictions and in case that there is a non-0 cost for acting due to false positives/negatives you can try to come up with a strategy that minimize the expected costs.

The information you're seeking appears to be left out of the post. My best guess is that a separate embedding model, specifically tuned for document similarly, is used to generate the vectors and then a clustering algorithm is chosen to create the clusters. They may also use PCA to reduce the embedded vector dimensions before clustering.

> How can you get overlapping clusters if the two sets of labelled examples are disjoint?

What's disjoint are the training labels and the classifier's output - not the values in high-dimension space. For classification tasks, there can be neighboring items in the same cluster but separated by the hyperplane - and therefore placed in different classes despite the proximity.

If the diagram is representative of what is happening, it would seem that each cluster is represented as a hypersphere, possibly using the cluster centroid and max distance from the centroid to any cluster member as radius. Those hyperspheres can then overlap. Not sure if that is what is actually happening though.

they cluster the examples with their model and then check the predictions against the labels.

The methodology helps focus the attention of human labelers on the controversial/borderline cases, making them more effective in how they spend thier time.