Sync dependencies #347
Comments
|
Another idea: don't sync dependencies to the relational database at all but instead only keep them in Redis. Whenever Web wants to display something about dependencies, the required graph algorithms are implemented as Lua scripts and executed on the Redis server. Only downside with this: you couldn't use the dependency information in filters then (trying to support this would probably require some nasty work merging result sets from Redis and SQL), but things like showing all (reverse) dependencies of a checkable or rendering nice dependency graphs starting from a checkable should work nicely. There could also be some combined solution where we think about what filtering on dependencies should be supported, write information specifically for these queries to the relational database and use Lua for everything else. |
|
My proposal, which fits nicely with what is planned in Web, uses primarily recursive common table expressions to query connected nodes in either direction. Required database versions:
The related tables consist of the following structures:
Glossary
Schema AdditionsTableshost
host_state
service
service_state
redundancy_group
redundancy_group_state
dependency_node
dependency_edge
dependency_edge_state
RelationserDiagram
dependency_node ||--|| host : is
dependency_node ||--|| service : is
dependency_node ||--|| redundancy_group : is
dependency_edge }|--|{ dependency_node : "refers to / referred by"
dependency_edge }|--|| dependency_edge_state : "has one / belongs to many"
redundancy_group ||--|| redundancy_group_state : has
Example QueriesRoot ProblemsWITH RECURSIVE UnreachableNodes AS (
SELECT id as id,
0 as level,
dependency_node.host_id,
cast('' as binary(20)) as service_id,
cast('' as binary(20)) as redundancy_group_id,
0 as is_group_member
FROM dependency_node
WHERE dependency_node.host_id = '<host-id>'
and dependency_node.service_id is null
UNION ALL
SELECT e.to_node_id,
r.level + 1,
tn.host_id,
tn.service_id,
tn.redundancy_group_id,
r.redundancy_group_id IS NOT NULL AND r.level > 0 as is_group_member
FROM dependency_edge e
INNER JOIN UnreachableNodes r ON e.from_node_id = r.id
INNER JOIN dependency_node tn ON e.to_node_id = tn.id
LEFT JOIN dependency_edge_state es ON e.dependency_edge_state_id = es.id
LEFT JOIN redundancy_group_state rgs
on rgs.redundancy_group_id = tn.redundancy_group_id
WHERE es.failed = 'y'
or rgs.is_reachable = 'n')
SELECT rn.id,
rn.level,
rn.is_group_member,
h.name as host_name,
s.name as service_name,
rg.name as redundancy_group_name
FROM UnreachableNodes rn
LEFT JOIN host h on rn.host_id = h.id
LEFT JOIN host_state hs on rn.host_id = hs.host_id
LEFT JOIN service s on rn.service_id = s.id
LEFT JOIN service_state ss on rn.service_id = ss.service_id
LEFT JOIN redundancy_group rg on rn.redundancy_group_id = rg.id
LEFT JOIN redundancy_group_state rgs on rn.redundancy_group_id = rgs.redundancy_group_id
WHERE rn.level > 0
and rn.is_group_member = 0
and (
ss.affects_children = 'y'
or (rn.service_id IS NULL and hs.affects_children = 'y')
or (rgs.failed = 'y' and rgs.is_reachable = 'y')
);Direct ParentsSELECT e.to_node_id,
h.name as host_name,
s.name as service_name,
e.display_name as dependency_name,
rg.name as redundancy_group_name
FROM dependency_edge e
INNER JOIN dependency_node tn ON e.to_node_id = tn.id
INNER JOIN dependency_node fn ON e.from_node_id = fn.id
LEFT JOIN host h on tn.host_id = h.id
LEFT JOIN service s on tn.service_id = s.id
LEFT JOIN redundancy_group rg on tn.redundancy_group_id = rg.id
WHERE fn.host_id = '<host-id>'
and fn.service_id is null;Direct ChildrenSELECT e.from_node_id,
h.name as host_name,
s.name as service_name,
e.display_name as dependency_name,
rg.name as redundancy_group_name
FROM dependency_edge e
INNER JOIN dependency_node tn ON e.to_node_id = tn.id
INNER JOIN dependency_node fn ON e.from_node_id = fn.id
LEFT JOIN host h on fn.host_id = h.id
LEFT JOIN service s on fn.service_id = s.id
LEFT JOIN redundancy_group rg on fn.redundancy_group_id = rg.id
WHERE tn.host_id = '<host-id>'
and tn.service_id is null; |
That's redundant with the host/service state information and I believe it also doesn't provide the information you hope for. I think you might want a Similarly for redundancy groups. After all, they are just a set of dependencies that counts as failed if all individual dependencies are failed.
This table can become huge. Imagine a dependency structure like the following (boxes are hosts, arrows are dependencies), scale up the top and bottom level and the size of this table will grow quadratically in the size of the Icinga 2 config. graph TD;
A1-->B;
A2-->B;
A3-->B;
A4-->B;
B-->C1;
B-->C2;
B-->C3;
B-->C4;
So if this table is necessary, I see three options:
Why not simply stick to "child" and "parent"?
How do these map to the tables given? Also, can you please give an example how you'd expect these additional virtual nodes to be generated? At least with redundancy groups, that's not obvious. |
Yeah, I thought of this, but figured this would be noted sooner or later, as it happend now ;)
Before this becomes a problem (for the database or our queries), doesn't Icinga already suffer from this anyway? I'd go with 3 for now.
Hah, I knew this question comes up. Because the table is called _edge, not _relation, and left or right were not neutral enough to me. 😅
Eh. There is no simple closure table. Added the names to the other two.
Done. |
To be honest, I don't know what would happen exactly, it's quite possible that things will get slower with lots of dependencies. But so far, it does not expand all indirect dependencies, so it's not obvious that this problem already exists.
Just to be sure: we're still talking about a directed graph here (otherwise, I don't know how this should work)? And the edges are aligned in the direction between a child and a parent, the edge just doesn't necessarily directly connect to something that's a host/service, i.e. the parent/child of a dependency in Icinga 2.
That's not what I meant. More like which of these options do you have in mind? Or something totally different? graph LR;
ChildHost-->RedundancyGroup;
RedundancyGroup-->ParentHostA;
RedundancyGroup-->ParentHostB;
ChildHost-->Dependency;
Dependency-->ParentHostC;
graph LR;
ChildHost-->RedundancyGroup;
RedundancyGroup-->DependencyA;
DependencyA-->ParentHostA;
RedundancyGroup-->DependencyB;
DependencyB-->ParentHostB;
ChildHost-->Dependency;
Dependency-->ParentHostC;
|
|
As Julian have already pointed out, the
I'm not sure at the moment why this is going to be useful, but technically redundancy groups only define a way to construct alternate dependencies and don't provide any actual states by themselves. So, why not just link the
Looking at this where clause, I'm wondering why the nodes are being expanded to themselves in the first place! Isn't that an indication of a cyclic graph (dependency), which Icinga 2 clearly prohibits, since Icinga 2 Footnotes
|
It is not required to re-construct any relationships, yes. But it stores information crucial for a very specific use-case more efficiently. At least compared to work required to fetch the same information by traversing the edges each time. This table is only there to easily aggregate the number of affected children, as in the example. Without this table, this information cannot be shown in the UI in lists. (e.g. host and service list)
It's this alternation, which is abstracted away by this table. I don't want to have to express this in a query. Icinga knows about this, and can evaluate/update it in case it changes.
This doesn't check for cyclic dependencies. It's used to know in advance that the recursion will stop with this row. |
|
I've updated the proposal now with what I've discussed with @julianbrost yesterday. This is mainly:
|
The total number of affected children, which Web would like to show in a host's or service's list item, is better be calculated by Icinga (DB) in advance and stored in table The list of affected children will either be not shown, be a simple link to the zoomed in map or only be visible in case the parent has a problem while the list then only includes unreachable nodes. This is because, traversing the hierarchy top down (with a CTE) is considered to be very expensive, compared to a bottom up approach which is done when querying root problems. |
Models for the new schema additions described here: Icinga/icingadb#347 (comment)
|
The Responsible Node:
Use case:In the redundancy group details view, we want to display the plugin output of the member that affected the Problem:In case, a member (other than the responsible one) becomes If the group status is also |
UpdateWe have dropped the idea of displaying the plugin output for redundancy groups. We therefore no longer need the |
|
I've updated the proposal again and made sure the shown schema matches what I've pushed in #795. In addition, I updated the root problem query and removed the affected children query. |
|
As we've discussed yesterday, it seems that At the moment the only information stored in them and used by Web, are Problems arise, as @yhabteab noticed, once we take a look where erDiagram
dependency_edge ||--|| dependency : "declared by one / declares one"
dependency ||--|| dependency_state : "has one"
Initially this looked fine:
But once the dependency object in question also defines a redundancy group, things get interesting: erDiagram
dependency_edge ||--|| dependency : "declared by one / declares one"
dependency ||--|| dependency_state : "has one"
dependency }|--|| redundancy_group : "belongs to"
Normally, redundancy groups are established in case multiple parents are cross linked with multiple children. (i.e. each child has the same parents as other children) graph TD;
A-->C;
A-->D;
B-->C;
B-->D;
subgraph AB [ redundancy group ]
A~~~B
end
But redundancy groups are also linked by erDiagram
dependency_edge ||--|| dependency : "has one"
dependency ||--|| dependency_state : "has one"
dependency }|--|| redundancy_group : "belongs to (many)"
dependency_node ||--|| redundancy_group : "has one"
dependency_edge }|--|{ dependency_node : "has many"
To be expected, since redundancy groups are actual nodes in our representation of dependency hierarchies in Web. But to avoid showing too complex structures and ease understanding, redundancy groups are supposed to be unique with their associated members. (The parents of the original dependency objects) graph TD;
A-->G[redundancy group];
G-->D;
B-->G;
G-->C;
With this uniqueness in mind, the edges (which connect a redundancy group with their members) are halved. But now, we end up with only two edges, which still need to be linked to a given dependency object. But there are four dependency objects involved, which will be used? -- To solve this, @julianbrost suggested to not store dependency states in the database, but edge states. This means that As mentioned earlier, only
erDiagram
dependency_edge }|--|| dependency_edge_state : "has one / belongs to many"
The reason for this should hopefully be obvious, since we also save state of hosts and services in separate tables. But the relationship of |
|
Now onto why Remember, redundancy groups are supposed to be separate nodes in their hierarchies. Connected edges pointing to their children are a result of the deduplication. graph TD;
A-->C;
A-->D;
B-->C;
B-->D;
subgraph AB [ redundancy group ]
A~~~B
end
graph TD;
A-->G[redundancy group];
G-->D;
B-->G;
G-->C;
Such edges, where the parent is a redundancy group, have no distinct state. Instead, they all share the same, so why shouldn't this be mirrored in the database? Because of different dependency configurations you might point out. That's true. But exactly for this reason, Differences only come up, once such dependency objects are not a result of an apply rule. But if so, are we really talking of the same redundancy group? To me, this is a design flaw in Icinga's redundancy group implementation. -- But I have another reason, for sharing state between multiple edges. graph TD;
c((A cluster of children));
p[Parent]-->c;
The dependency map will group children above a given number (e.g. >10) and connect them with a single line to the parent. This line resembles a single edge. To be able to do this, correctly, the map needs to distinguish relations based their definition. (i.e. dependency configuration) Say, there are two distinct definitions part of 20 edges which all have the same parent. The map should render the following: graph TD;
c1((A cluster of 10 children));
c2((A cluster of 10 children));
p[Parent]-->c1;
p-->c2;
The lines connecting them, resemble the two definitions in question. Phrased differently, two distinct dependency states. So, in the database are 20 edges stored. But they reference only two states. |
That's not enforced at all. Nothing stops you from setting config options to different values depending on the specific child object the dependency object is instantiated for. So apply rules are completely irrelevant here (like you can use them to create dependency objects but it doesn't matter if you did). |
|
Okay, but that only makes things worse. To me. Parents in a redundancy group should be equal, or not? Why else would they be in said group? |
|
Yes, in order order to merge/deduplicate redundancy groups between two children, both redundancy groups have to contain the same set of parents with the same configuration ( All I'm saying is that apply rules are unsuitable to obtain that grouping. |
|
Okay, I just used apply rules as example, since I imagine most users of Icinga use them (backed by config we got shared) and then don't use different settings depending on where they apply. Anyway, they're only an abstraction layer, the result are multiple dependency objects with the, probably, same configuration. My reason for sharing states between edges. |
|
I've updated the schema definition and example queries above accordingly now. Main changes: removed: dependency added: dependency_edge_state |
According to my understanding (from my observation and discussion with @nilmerg), the graph TD;
host-A .-> service-B;
host-A .-> service-C;
service-B --> service-C;
Also, IMO if the root is unreachable because of its implicit dependency, then it should not affect the state of the dependencies it is part of. @nilmerg correct me if I am wrong. |
At the moment dependencies are not synchronized in Redis and the database and there is no idea for the storage scheme yet.
Since dependencies represent trees (or even graphs?), They can be difficult to store in the database.
At the moment I can think of the following ways to store the dependencies in the database:
I'm sure there's more.
All schemes must be compared in terms of complexity and performance when writing and reading.
I think it also makes sense to store the parent dependencies directly for the objects. This would allow us to display these in the detail areas in Web without much effort.
The text was updated successfully, but these errors were encountered: