Why platform data looks so convincing
Social platforms record enormous amounts of activity: posts, replies, views, follows, shares and the times at which they happen. Compared with a survey of a thousand people, a dataset of millions of posts feels more solid. It captures behaviour as it happens, not as people recall it, and it can be searched for specific words or topics.
Size is not the same as coverage, though. A very large dataset can still describe only a narrow slice of people, and it may be shaped by the product that produced it. Several distinct gaps tend to recur.
Who is in the data
Platform data covers people who use that service. This is a smaller and different group from the general public. Users of a given platform often skew by age, location, education, language and interest, and the skew differs from one service to the next. Findings from one network therefore may not transfer to another, let alone to people who use none.
Within the user base, activity is typically uneven. A relatively small share of accounts often produces a large share of the posts, while many users mostly read, watch or scroll without posting. Those quiet users are real participants in a culture, yet they leave few visible traces. A study that counts posts about a topic is mostly measuring the habits of frequent posters.
Who is invisible
- Non-users. People who have never joined, or who left, do not appear at all.
- Silent users. Readers and viewers who do not post, like or comment are often absent from public datasets.
- Private and restricted accounts. Content shared only with friends or in closed groups is usually out of reach for outside analysis. This may be where much of the personal conversation happens.
- Deleted and removed content. Posts taken down by users or moderators disappear, so data collected later may be missing precisely the material that was most controversial.
- Other languages and dialects. Collection methods based on keywords or particular languages can overlook communities who discuss the same idea in different words, slang or scripts. Spelling variants matter too, as regional dialects and internet slang show.
- People using several accounts or none under their own name. One person may appear many times, and many people may appear as one account.
Accounts are not people, and activity is not opinion
Counts of accounts, posts or reactions are easy to produce, but each unit hides complications. Fake accounts and automated activity can inflate numbers, an issue discussed in fake accounts. Organised groups can coordinate posting to make a topic look bigger than it is. Meanwhile an engagement figure may measure a reaction as varied as agreement, mockery or simple curiosity, which is part of why engagement metrics are tricky to interpret.
What users see is also shaped by the platform. Recommendation algorithms decide which posts are promoted, so what spreads may reflect ranking choices as much as audience taste. A trend visible in platform data can thus be partly an artefact of how the service distributes attention. Likewise, public lists of what is popular, such as trending lists, are produced by formulas that platforms can change.
Rules and definitions change over time
Platforms alter features, moderation rules, character limits, what counts as a “view” and what data they record. Any of these can create a jump or a fall in a time series without a matching change in human behaviour. Comparing figures before and after such an update can mislead, a problem explored under platform rule changes. Historic data may also be reprocessed or removed, so earlier analyses cannot always be reproduced.
Access rules and why checking is hard
Much platform data is controlled by the company that holds it. Outside researchers typically obtain it in one of a few ways: through an official interface that offers limited data, through a formal agreement, by collecting publicly visible pages, or from a sample the company supplies. Each route has constraints.
- Official access may be restricted in volume, scope or time period, and terms can change or be withdrawn.
- Agreements with companies may limit what can be shared or reproduced.
- Collecting public pages can be limited by the platform’s terms and by technical barriers, and it misses everything private.
- Data supplied by the company may be a sample whose selection method is only partly described.
Because of these limits, other people often cannot repeat an analysis on the same data to check it. When a company publishes its own figures, readers must usually take the definitions and methods largely on trust. This does not imply that such figures are wrong, only that independent verification is typically harder than for a public dataset.
There are also privacy and legal reasons for restrictions, which are legitimate. The effect for the reader is the same: fewer chances to confirm what a headline claims.
Reading a platform-based trend claim
Useful questions include:
- Which platform, and what kind of people use it?
- Were posts, users or reactions counted, and how were fake or duplicated accounts handled?
- Does the measure reflect what people think, or what the platform promoted?
- Could a platform change explain a sudden shift?
- Can the result be checked against a different source, such as a survey or sales records?
Platform data is often excellent for describing activity on the platform itself, such as how a hashtag spread or when discussion peaked. It is much weaker as a stand-in for what society as a whole thinks or does. Stating the claim at the right level, “among users of this service who posted publicly”, often keeps a finding honest.