Network Administrator for Pawb.Social, furry, and a programmer

Mastodon: @crashdoom@furry.engineer

  • 55 Posts
  • 47 Comments
Joined 1 year ago
Cake day: June 2nd, 2023
  • Crashdoom@pawb.socialOPMtoPawb.Social Feedback@pawb.social•[RFC] Use of Automated Moderation ToolsEnglish
    2·
    9 months ago

    So, the issue lays in that there’s no technical way to notify the remote user (someone not on furry.engineer or pawb.fun) that they’ve been suspended on our end, without sending a message to them directly. If we suspend them on our end, that doesn’t per se suspend them on their end and they wouldn’t know that their messages were no longer reaching our users; They would still be able to message other users on their instance, and users on other instances, but not to our users.

    We’re apprehensive about notifying remote accounts specifically because we don’t often know the moderation practices of the remote instance (to know if they’ll deal with it, or if they have open-registration allowing anyone to join without approval) and it may encourage further abusive behavior through ban evasions (creating new accounts on that instance or elsewhere to continue messaging) from the user being made aware that we’re no longer receiving their messages.

  • Crashdoom@pawb.socialOPMtoPawb.Social Feedback@pawb.social•[RFC] Use of Automated Moderation ToolsEnglish
    4·
    9 months ago

    Appreciate the feedback so far, let me try to see if I can answer most / many of the questions:

    What are the risks of #4?

    Many users are worried about the risk of automated actions going wrong and not knowing what we mean with “pattern of spammy behavior.”

    For how we would identify the pattern of behavior that would allow for automated actions, we would review any major spam wave, such as the one we’ve been experiencing over the past few days:

    We would then identify any indicators we could use that are indicative of the known spam, and create a heuristic ruleset that would limit or suspend those accounts while targeting only those accounts actively engaging in the spam, not just referring to it. There are additional safeguards we can add, such as preventing rules being applied to users where the user is followed by someone on our instances.

    For the risk of automated actions going wrong, if we were using a limit (not a suspend) then the account would be hidden from public view but could still be viewed if specifically searched by name, it would also suppress all notifications from that user unless they are followed by you. (e.g. if they messaged you out of the blue, you wouldn’t see it if you weren’t following them.)

    If a suspend was used, the account would be marked for deletion from our instances but all follower relationships would immediately break (e.g. if you were following them, the system would automatically unfollow when they are suspended). Typically, we can restore data within 30 days, but follower relationships are typically unrecoverable. So long as rules are appropriately limited in scope to only target those with a lot of spam indicators, no false-positives should occur.

    What about appeals?

    For local users (anyone registered on furry.engineer and pawb.fun), all actions against your account (except reports) can be appealed. If you have a post removed or are suspended, all actions can be appealed directly to the admin team.

    For remote users, we can remove restrictions on remote accounts if we receive an appeal from any of our users, or by the affected account directly. This can be done via email, or just through a DM to one of the admins who can pass it to the team.

    Would the AI model have oversight?

    Yes. Where the team believe the filter has flagged sufficient content appropriately and maintains no false-positives, we may promote a model or ruleset to allowing automated actions (limit / suspend).

    We’ll keep an eye on the actions of each ruleset by reviewing the daily / weekly actions taken to ensure they meet the criteria and have not misidentified any users or content, and we’ll also start publicly tracking the statistics of the models / rulesets we create and use, including a count of false-positives or reversed decisions.

    Will you notify users?

    Due to limitations in Mastodon, we can only notify local users (users on furry.engineer or pawb.fun) when actions are taken against their account; This process happens automatically when your post is removed, or your account is warned, limited, or suspended.

    There’s no easy way to notify remote users other than sending them a DM, but doing so could be seen as spammy or lead to inciting further abusive behavior by informing them of our activity. While we can have transparency with our users due to having an invite-only platform, other instances are frequently open-registration which can allow the abusive user to re-create an account to continue to harass our users. BUT, I’m open to suggestions on this.

Moderates