Performance issues

ambidex

Fri, 02/10/2017 - 10:04

Forums

We're experiencing major performance issues, even timing out on some actions. To put things in perspective: Our website has a total of ~4500 users, ~100 classes, ~42 courses. It might be called pretty big, I'm not sure if there are other users that have a similar setup in size. Timeouts At this moment we're experiencing blocking issues while implementing a new user registration service. When it creates a new user, it will automatically assign the user to the correct Class. Of course, Opigno default rules will kick in and will (among other things) assign the corresponding class-courses to the user. It looks like - when debugging - that the call stack jumps from ~0.7 seconds to ~38.3 seconds between RulesLoop->evaluate() and RulesActionContainer->evaluate(). Memberships The current Opigno setup seems to create a MASSIVE amount of OG Memberships. I think this is because every Student, that has been assigned to a Class, will automatically be assigned to all Class Courses. This causes a major overhead on possible unnecessary correlations, and all of these will be loaded when some of the rules are being triggered. I'm really wondering why all these memberships have to be created between the user and the class courses. Isn't it possible to relate class courses through the classes? Workarounds? Any suggestions how I would go about to reduce the load? Update 2017-02-10 10:45 +1 GMT I've pinned down the rule which is causing the massive load, by debugging inside RulesActionContainer->evaluate(). I've recursively checked for the action's root parent, which made it clear that og_user_was_granted_role has a massive amount of actions that are being evaluated. Disabling this rule makes the page load within a second, in contrast to a 30+ second timeout. I'm now about to dig into this rule, find out what it actually does and try to find a alternative / fix. Update 2017-02-10 11:12 +1 GMT WRONG: Also pinned down the rules_og_member_unsubscribe_class rule as a really hefty rule when deleting users. I'll be keeping track of the rules I'm encountering and try to describe each one's goal. RIGHT: Had a small bug in my debugging logic. I think it's actually the og_user_delete rule, not yet entirely sure.

James Aparicio

Fri, 02/10/2017 - 10:39

Hi ambidex,

If you are want maximum performance i suggest you replace the rules by custom code. A rule acting on a specific even is a hook so you can replace them by your custom code.

For sure it will be better performance wise.

http://www.drupalcontrib.org/api/drupal/contributions!og!og.api.php/7

Best regards

ambidex

Fri, 02/10/2017 - 10:48

Hi James, thanks for the

Hi James, thanks for the suggestion. I'd rather prevent that I have to rewrite all the rules to custom code since our client does not have have a budget big enough to make that happen. I have made some progress pinning down the culprit though, I'll update my original post in a bit, to keep it all aggregated.

James Aparicio

Fri, 02/10/2017 - 11:13

Hi ambidex,

I took a look at that particular rule and in fact there seems a way of optimizing it.

After it loops over the class courses, it loops over all the memberships. So at that point we can load the membership of the course and check if it exists instead of doing the loop as it is.

Thank you for pointing this out. We will update this rule.

Best regards

ambidex

Fri, 02/10/2017 - 11:25

Thanks James for keeping

Thanks James for keeping track of my updates. I'll try your suggestion in a bit. I also tracked down another heavy rule, see latest update, possibly the same setup as the other with a slightly different outcome.

James Aparicio

Fri, 02/10/2017 - 11:31

Hi ambidex,

Same logic in that rule, looping over the memberships instead of getting the specific one. We will change this logic in the rules.

Best regards

ambidex

Fri, 02/10/2017 - 11:41

Hi James,

Hi James, Could you possibly make an export of those rules as you would suggest to optimise them? Since I do not directly understand your suggestion in #4. I'll report back what the outcome of will be.

James Aparicio

Fri, 02/10/2017 - 11:56

Hi ambidex.

Hi ambidex. Here is the class granted role rule, loading the membership instead of looping over it. For the other ones i will keep you updated once we push the code. Let me know if this change made a big difference when adding users to classes performance wise. { "rules_class_user_was_granted_a_role" : { "LABEL" : "Class user was granted a role", "PLUGIN" : "reaction rule", "OWNER" : "rules", "TAGS" : [ "class", "og", "opigno", "role" ], "REQUIRES" : [ "rules", "rules_conditional", "opigno", "opigno_class_app" ], "ON" : { "og_user_was_granted_role" : [] }, "DO" : [ { "entity_fetch" : { "USING" : { "type" : "user", "id" : [ "user-id" ] }, "PROVIDE" : { "entity_fetched" : { "user" : "user" } } } }, { "entity_fetch" : { "USING" : { "type" : "node", "id" : [ "node-id" ] }, "PROVIDE" : { "entity_fetched" : { "node" : "node" } } } }, { "CONDITIONAL" : [ { "IF" : { "node_is_of_type" : { "node" : [ "node" ], "type" : { "value" : { "class" : "class" } } } }, "DO" : [ { "LOOP" : { "USING" : { "list" : [ "node:opigno-class-courses" ] }, "ITEM" : { "course" : "course" }, "DO" : [ { "CONDITIONAL" : [ { "IF" : { "node_is_of_type" : { "node" : [ "course" ], "type" : { "value" : { "course" : "course" } } } }, "DO" : [ { "opigno_load_og_membership" : { "USING" : { "user" : [ "user" ], "group" : [ "course" ] }, "PROVIDE" : { "og_membership" : { "og_course_membership" : "OG Course Membership" } } } }, { "CONDITIONAL" : [ { "IF" : { "NOT data_is_empty" : { "data" : [ "og-course-membership" ] } }, "DO" : [ { "class_map_role_to_course" : { "USING" : { "class_role_id" : [ "role-id" ], "clgroup_id" : [ "node-id" ], "cogroup_id" : [ "course:nid" ] }, "PROVIDE" : { "course_role_id" : { "course_role_id" : "Course role id" } } } }, { "CONDITIONAL" : [ { "IF" : { "NOT data_is_empty" : { "data" : [ "course-role-id" ] } }, "DO" : [ { "og_give_og_role_to_user" : { "role" : [ "course_role_id" ], "user" : [ "user" ], "group" : [ "course" ] } } ] } ] } ] } ] } ] } ] } ] } } ] } ] } ] } }

ambidex

Fri, 02/10/2017 - 14:06

It looks like that had a

It looks like that had a dramatic impact! I'm going to verify this a few times to be sure that this was not a one timer, but the new rule config made it execute in ~80ms (rule evaluation time) in contrast to the +30s originally. I'll be back to confirm.

ambidex

Fri, 02/10/2017 - 14:37

I've just tried to optimise

I've just tried to optimise `rules_og_member_unsubscribe_class` based on the other fixed rule, but this visual programming interface in Rules is killing me. :(

ambidex

Fri, 02/10/2017 - 14:44

Hi James, would you be so

Hi James, would you be so kind to check following exported rule? It's a "fixed" version for `rules_og_member_unsubscribe_class`, although I'm not sure if this is covering everything the original version did. https://gist.github.com/nvaken/fb578b7f50a77e024635d3d25c43a997

ambidex

Fri, 02/10/2017 - 15:01

Well, I am able to verify

Well, I _am_ able to verify that the exported rule by James is a dramatic performance winner. Also, my above exported gist rule seems to work just fine. Although, I would still like to hear if my version covers everything it should, still not really sure. Edit: Well, I'm pretty sure my rule does not work as it should I'm reading errors from the rule debug log: 63.482 ms Unable to get variable course_member_membership, it is not defined. 70.349 ms Unable to evaluate condition data_is.

James Aparicio

Fri, 02/10/2017 - 15:52

Hi ambidex,

As i said, once we update the all the rules with this change i will update this post.

Regarding your specific rule, i took a look and it should "work" correctly. Its just missing a check if the membership actually exists after you try to load it, thats why you get those message errors. It should work, but then again creating those error messages does not help performance.

Best regards

James Aparicio

Mon, 02/13/2017 - 10:18

Hi ambidex,

I have tested and pushed to the opigno_class_app both updated rules with the performance tweaks. "Class user was granted role" and "Class og member unsubscribe".

Best regards

ambidex

Wed, 02/22/2017 - 16:44

Hi James!

Hi James! Thanks for the feedback, we'll add the new rule to our instance! Thanks!

Hi ambidex,

Hi James, thanks for the

Hi ambidex,

Thanks James for keeping

Hi ambidex,

Hi James,

Hi ambidex.

It looks like that had a

I&#039;ve just tried to optimise

Hi James, would you be so

Well, I am able to verify

Hi ambidex,

Hi ambidex,

Hi James!

I've just tried to optimise