4 min read

Exploring the AppView; knowing when to admit defeat

Thanks to some helpful guidance from Bailey Townsend I discovered that the getProfile endpoint on my own PDS is just proxying the Bluesky API, which means that the point of failure is the Bluesky AppView. This is backed up by another post linked in the thread that directly states that once a did:web is deleted Bluesky's AppView treats that deletion as permanent.

Funnily enough I had theorized that this might be the case yesterday but then figured I was being dumb since I could run the request against my own PDS's API. AppViews are one of the more opaque parts of the Bluesky ecosystem in general right now and the one folks are least likely to self-host (although Blacksky is working on spinning one up right now and zeppelin.social previously ran one as a proof of concept).

I don't have a strong desire to try and get an AppView (which takes up a fair amount of storage and consumes the entire network firehouse) running when I really just want to debug a single document. But unfortunately my curiosity has been piqued enough that I'd really like to find out exactly why the AppView is behaving this way - it seems like a reasonable expected behavior, but where is it happening? This is a good enough reason to dive a bit deeper into things, so I'm going to set up the atproto dev environment and see what I can find out.

Cloning their repo and running the make commands mostly worked for me, although I had to manually build the dev-env:

cd packages/dev-env
npm run build

I started the dev-env container with make run-dev-env-logged and waited for all the test data to propagate, until it finished:

Time for some code spelunking! I cannot emphasize enough that I have never worked in Typescript before this week and haven't done server-side JS work in years, so I apologize in advance for my inevitable dumb mistakes.

Assuming the default PDS docker container API is based on their client library, we can find the offending getProfile code in packages/pds/src/api/app/bsky/actor/getProfile.ts. 

export default function (server: Server, ctx: AppContext) {
  if (!ctx.bskyAppView) return

  server.app.bsky.actor.getProfile({
    auth: ctx.authVerifier.authorization({
      authorize: (permissions, { req }) => {
        const lxm = ids.AppBskyActorGetProfile
        const aud = computeProxyTo(ctx, req, lxm)
        permissions.assertRpc({ aud, lxm })
      },
    }),
    handler: async (reqCtx) => {
      return pipethroughReadAfterWrite(ctx, reqCtx, getProfileMunge)
    },
  })
}

A quick glance at the source code certainly seems to imply it's computing a proxy to the Bluesky API when we run getProfile, which backs up my current understanding of the failure. But what's happening when I delete an account to make the AppView treat it as perma-deleted?

I used pdsadmin to delete my account so I'll check the source code for that.

elif [[ "${SUBCOMMAND}" == "delete" ]]; then
  DID="${2:-}"

  if [[ "${DID}" == "" ]]; then
    echo "ERROR: missing DID parameter." >/dev/stderr
    echo "Usage: $0 ${SUBCOMMAND} <DID>" >/dev/stderr
    exit 1
  fi

  if [[ "${DID}" != did:* ]]; then
    echo "ERROR: DID parameter must start with \"did:\"." >/dev/stderr
    echo "Usage: $0 ${SUBCOMMAND} <DID>" >/dev/stderr
    exit 1
  fi

  echo "This action is permanent."
  read -r -p "Are you sure you'd like to delete ${DID}? [y/N] " response
  if [[ ! "${response}" =~ ^([yY][eE][sS]|[yY])$ ]]; then
    exit 0
  fi

  curl_cmd_post \
    --user "admin:${PDS_ADMIN_PASSWORD}" \
    --data "{\"did\": \"${DID}\"}" \
    "https://${PDS_HOSTNAME}/xrpc/com.atproto.admin.deleteAccount" >/dev/null

  echo "${DID} deleted"

We can see pdsadmin account delete posts to my PDS host with com.atproto.admin.deleteAccount. If we check the Typescript implementation, that runs this function:


export default function (server: Server, ctx: AppContext) {
  server.com.atproto.admin.deleteAccount({
    auth: ctx.authVerifier.adminToken,
    handler: async ({ input }) => {
      const { did } = input.body
      await ctx.actorStore.destroy(did)
      await ctx.accountManager.deleteAccount(did)
      const accountSeq = await ctx.sequencer.sequenceAccountEvt(
        did,
        AccountStatus.Deleted,
      )
      await ctx.sequencer.deleteAllForUser(did, [accountSeq])
    },
  })
}

Most of the relevant work is being doing by the AppContext object. This is about the point where my personal ability to maintain different layers of context falls apart. I spent more time than I'd like to admit trying to figure out how to trace this more specifically (and figure out why, e.g., a deleted did:plc could hypothetically be recovered when a did:web couldn't). As far as I can tell account deletion results in a full de-indexing but the delete event itself is tracked, which could result in a conflict when trying to get or create a new profile. I spent a while poking around in the pds and dataplane (which I think is an old name for AppView) folders trying to understand AppContext better but didn't come away with any smoking guns - no "this specific rule about did:web is being applied" that I could find. Without re-implementing my entire PDS to log everything that's happening I don't think I'll be able to get much further, since you can't do much with did:web locally.

I think I'm ready to admit defeat on understanding exactly what's happening here for the time being (or until I feel like I can explain it well enough to want to post it in a forum or something) and move to something that isn't miles out of my comfort zone, but at the very least this process has forced me to really dive in to the code and protocol in a way I might not have otherwise. And it makes me feel a bit better to have had a few other devs weigh in with basically "idk but it's probably the AppView." Leave it to me to find the most bizarre edge case possible within a week of trying something out. 'Til next time!

Published by:

Brent