Commits

Jozsef Bakosi authored 92893a426cd
Replace graph aggregation with scalable algorithm Using a graph-based partitioner with zoltan requires a unique graph. Since the mesh is read in a distributed fashion, this requires communication. So far this was done simply aggregating the full graph on each compute node which does not scale well, evidenced by out of memory during this step even with moderate-size, e.g., nelem=O(1e+6), meshes. This commit replaces the graph aggregation, preceding the call to zoltan's phg, by constructing a distributed table, storing the aggregated graph, assembled by a query-response algorithm. This algorithm assigns compute nodes as owners whose task is to aggregate points surrounding points (the graph for a given global node id). If multiple compute nodes contribute to a single global mesh node, the compute node with a lower id is the owner. Compared to the previous one, this algorithm requires only point-to-point communication among a limited set of compute nodes, that share part of the graph adjacent to the global id owned. As a result, this scales much, better than the previous one, using less memory. The result changes slightly with a chance of increasing randomness in the final result (as this data is handed to zoltan for partitioning). This required some minor adjustments to a couple of regression tests.