Avoid scanning the client list when a client exits