Reindex speed done by groups

My problem was easy: when giving user rights using a ReferenceField to a top folder containing 40k objects I could get some coffee and lunch before my action was completed. This due to the fact Plone was reindexing allowedUsersAndRoles index on every single object in the folder and its sub folders. This would take up to 15 minutes and the action was fired several times a day.

Working on Plone Intranet we came across collective.workspace, a simple implementation based on putting users in groups and giving groups rights on objects. This means all reindexing is done creating a single object, setting rights based on a group. When a user enters or leaves the group no objects are reindexed, meaning updating your security model is super quick! 

Normal:
- Folder
  - Subfolder
    - Subsubfolder  

Adding user X to folder will include folder, subfolder and subsubfolder to be reindexed. Total time will be: (amount of time per object * num. objects).

New:
- Folder  (groupX)
  - Subfolder  (groupX)
     - Subsubfolder    (groupX) 

class Folder(Workspace):
    implements(IWorkspace)
    adapts(interfaces.IFolder)

    @property
    def available_groups(self):
        groups = {
            'groupX': ['GroupXRights', ],}
       return groups

No (re-)indexing is done when a user is added to groupX, because the group's permissions are already indexed. Group membership is global, so this works everywhere.

The only thing we had to do was remove all reindexobject calls in my events.py and create an event based on my security model, when to add or remove a member from the groups. 

# Events
def set_groups_and_members(context, event):
    # Edit event for all IHasWorkspace
 # Sets the members for the groups from the referencefields in context with
 # the same name. ie group 'groupX' contains members
 # from context.getGroupX()

 group_memberships = defaultdict(list)

    # Get members for the groups
 workspace = IWorkspace(context)
    logger.debug('%s' % workspace)
    for fieldname in workspace.available_groups:
        # Get the accessor and call to get the members
        field = context.schema.get(fieldname)
        for item in set(field.getAccessor(context)()):
            group_memberships[item.getId()].append(field.getName())

    # add or update members
 for memberid, groups in group_memberships.items():
        if memberid not in workspace.members:
            workspace.add_to_team(memberid, groups)
        else:
            membership = workspace[memberid]
            membership.update(dict(groups=groups))

    # Remove old members
 for memberid in set(workspace.members) - set(group_memberships):
        workspace.remove_from_team(memberid)