I wanted to know whether you could train a model to write YouTube comments people actually like. The obvious training signal is the like count, and there’s recent work doing exactly that: fine-tune on the top 10% most-liked comments, report roughly 3× the engagement of a model trained on random ones.

Then I looked at what a like count actually measures. A comment posted in a video’s first hour is seen by orders of magnitude more people than one posted a week later, and YouTube’s “Top comments” ranking compounds it, because being ranked high causes likes and likes cause being ranked high. Two identical comments can differ by three orders of magnitude on arrival time alone.

So the question stopped being “can a model write good comments” and became “does a like count contain anything about the comment at all.”

You can’t answer this with public data

The two large public YouTube comment corpora, YT-30M and YTCommentVerse, hold about 32 million comments between them. Both give you upvotes. Neither gives you a timestamp.

Without post times you can’t separate what a comment said from when it showed up. Every model trained on those corpora is fitting the head start with no way to see it.

So I collected a corpus that has them: 1,345,353 top-level comments from 633 videos across 34 channels, with post time, fetch time, reply count, video statistics, and the rank YouTube’s own algorithm gave each comment. Three days, about 14,900 API units.

One decision mattered more than the rest. commentThreads.list returns comments newest-first, so any per-video page cap drops the oldest comments, which are exactly the high-exposure ones the study is about. Every video is enumerated completely. Videos too big to finish inside the daily quota got excluded up front and counted in the manifest instead of quietly truncated.

Exposure is 83.5% of it

Predict each comment’s rank of likes within its own video, which takes the video effect out of the training signal. Then add one thing at a time: exposure features, then length and shape, then 200,000 TF-IDF features, then MiniLM embeddings. Split by channel, never by comment, so the question is whether it transfers to an audience the model never saw.

component	share of above-chance ordering skill
exposure (post time, video scale)	83.5%
length and shape	8.4%
what the comment actually says	8.2%

Comment length is arguably content, not exposure. Count it that way and content goes to 16.5%. Exposure is 83.5% either way, so the finding doesn’t depend on where you draw that line.

The number that closed it for me: a 384-dimensional sentence encoder buys 0.0015 AUC over a bag of words. The standard objection to a lexical null is that the text model was too weak, and a completely different representation family lands in the same place.

Absolute predictability is low throughout. The best model gets AUC 0.671 against 0.5 for chance. Exposure beats content by a wide margin, and both lose to whatever else decides which comments get liked.

The trap in this dataset

Both panels, same comments, same y-axis. Position within a video and absolute age both show the same decline.

I got this wrong first, and you should know it before you touch the data.

Bucket the comments by how late they were posted and take the mean number of likes in each bucket:

age at posting	<1h	1-6h	6-24h	1-7d	1-30d	>30d
mean likes	19.8	21.7	19.5	19.8	25.8	20.4

Flat. No arrival effect at all. I wrote that up as a finding.

Now take the mean of log1p(likes) on the identical buckets:

age at posting	<1h	1-6h	6-24h	1-7d	1-30d	>30d
mean log1p(likes)	0.764	0.387	0.230	0.181	0.163	0.129

A 5.9× decline. The effect was there the whole time.

82.1% of comments have zero likes, and the top 1% hold 94.9% of all of them. A group mean on a distribution like that is a statistic about the group’s outliers, not about the group. One viral comment sets the average for its whole bucket.

Any summary of this corpus that averages raw like counts will report no effect where a large one exists.

The metric that certified 3×

The prior work scores a generated comment by embedding it, pulling its K nearest neighbours out of a reference corpus, and averaging their like counts. I expected that to be gameable by resemblance: train on top-liked text, land near top-liked text, score high whether or not a human would have liked it.

I was wrong, twice. Selecting real top-liked comments doesn’t reproduce a 3× ratio (I get 0.88 to 1.21). Neither does generating 960 comments with half of them few-shot conditioned on top-liked examples.

What I found instead is that the statistic won’t hold still. At K=5, the ratio of means says conditioning made it 3× worse, with an interval excluding 1.0. The ratio of medians, on the same 480 samples, says 2× better. Across three runs the same number came out 138, then 0.09, then 0.25 by mean and 2.00 by median.

Same heavy tail, same failure mode as my own mistake above. A metric whose direction flips depending on how you summarise it can’t carry a 3× claim, whatever produced any particular published number.

It also barely tracks the thing it estimates. Against real like outcomes it orders comments at Spearman 0.050. A model that knows only when a comment was posted and how big the video is scores 0.19 on the same comments.

Where that leaves it

Optimizing a comment generator against like counts is mostly optimizing against a clock. A model trained on top-liked comments learns what early comments on big videos look like, and the part attributable to what the comment says is around 8%.

That doesn’t make comment quality unmeasurable. It means the measurement has to condition on exposure first, and until now no public dataset let you do that. The thing I’d build next is the piece I couldn’t: human ratings of comments with exposure held fixed, which the corpus now makes possible and doesn’t contain.

The dataset is on Hugging Face under CC BY 4.0, and the paper has the full ladder, the splits, and the parts I got wrong.