API cursor-paginated entry count vs equivalent profile counts should be documented

[JamesKoenig]

(this is not a bug report on the api's behavior, just reporting that the differences should be documented)

The cursor paginated functions:

• app.bsky.graph.getFollowers
• app.bsky.graph.getFollows

Currently return a number of entries less than (extremely rarely equal to) the corresponding app.bsky.actor.getProfile entries.

I wrote this to showcase the consistency of this behavior, its output when run today (Sat. Oct 26th) at 2:41:11UTC was:

checking cpsc.gov
┌───────────┬─────────┬───────┐
│ (index)   │ profile │ api   │
├───────────┼─────────┼───────┤
│ followers │ 17527   │ 17367 │
│ following │ 12      │ 12    │
└───────────┴─────────┴───────┘
---
checking pfrazee.com
┌───────────┬─────────┬────────┐
│ (index)   │ profile │ api    │
├───────────┼─────────┼────────┤
│ followers │ 169413  │ 167346 │
│ following │ 613     │ 592    │
└───────────┴─────────┴────────┘
---
checking haileyok.com
┌───────────┬─────────┬──────┐
│ (index)   │ profile │ api  │
├───────────┼─────────┼──────┤
│ followers │ 8308    │ 8060 │
│ following │ 1677    │ 1655 │
└───────────┴─────────┴──────┘
---
done!

N.B. this script takes about a half hour to finish due to extremely conservative 250ms pauses between paginated requests.

So the api documentation for get-followers (docs/api/app-bsky-graph-get-followers.api.mdx) and for get-follows (docs/api/app-bsky-graph-get-follows.api.mdx) should be updated to reflect this.

Other cursor-paginated api endpoints likely have the same differences to their profile counts, but were not checked. If they have the same issue, they should also be updated.

Any developer starting to use the API, when finding their follows/followed counts are completely different than the result, is likely to think they have a bug in their code or that they're not exhaustively paginating the results. Documenting this behavior, regardless of cause, would be a prophylactic against that notion, and as a result should help foster a community of developers and tinkerers to develop features with the api or even contribute to bluesky as a project.

[JamesKoenig]

Note: in my conversion of a date from my timezone (19:41:11PDT Fri Oct 25.th) to UTC, I neglected to advance the date. This has been corrected in editing.

[bnewbold]

I don't know that we'll be able to document a complete list, but here are some reasons accounts might not show up:

• account-controlled account state: deactivated, deleted
• infra-level account state: takedown, suspension
• label-based moderation (!takedown)
• blocks: either direction, direct or via mod list
• mutes (possibly?)

We made an implementation decision to not have basic profile aggregations reflect all these counts. Permanent account deletion or takedown should get reflected eventually. For things like account deactivation, it is a lot of compute churn at scale to do thousands of counter updates every time somebody toggles visibility: we want deactivation/reactivation to be simple and immediate. The trade-off is that, as you have discovered, the aggregate counts don't match API responses.

[notque]

I am having the same problem and came here. The only answer that would match is mutes.

I guess I'd have to do a full test case, but it wouldn't be all that surprising if that was what it was. But it makes it very hard to tell.