So you installed Munin and created a couple of custom plugins to graph your various business metrics. For instance, maybe you graphed the number of orders made in the last 24 hours, and you thought you were SO SMART when you broke it out by status. You can see 'new' and 'done' and 'pending'. Well done you. 

But you forgot one thing: when an order is processed, it changes from 'new' to 'done'. So those two lines are inverses of each other. You've not really got any visibility into how many orders you're processing a day. 

Munin will produce a total for you automatically if you add the graph_total directive to your plugin config. You can either do this directly in your plugin, or in your munin-node.conf. 

[Examples here]. 

For some cases, this will work fine. If you're plotting the total orders by status, for instance. But if any of your values are ever unreported, the total fails. Every single field must always be present for this to work. 

A better solution is to use cdefs to plot your own total line. A cdef is an rrdtool feature that lets you calculate a value for a given field. In the example above, you might add this to your plugin config: 

# Bad - don't copy and paste this one
total.label Total
total.info Total assessments
total.cdef done,new,pending,+,+

But this suffers from the same problem: if any of the fields are empty, then your total will also be empty. We've gotta use the 'undefined' operator and the 'if' operator to set it to 0. 

done,UN,0,done,IF

Let's take this step by step: done,UN determines whether done is undefined. Let's say it is: this statement then becomes true,0,done,IF. Or, more familiarly: 
if (true) { return 0; } else { return done; }. 

Thanks to the way RPN works you can chain as many of these together as you like, and then add the right number of plus signs at the end to sum them. So your new munin config looks like this: 

total.label Total
total.info Total assessments
total.cdef done,UN,0,done,IF,new,UN,0,new,IF,pending,UN,0,pending,IF,+,+

I like to add a total.colour 000000 to make my lines stand out. 

As a side note, I've only had success with this approach with counter/gauge graphs. Trying to sum the values of derived graphs hasn't worked.