Scotch domain decomposition. More...

#include <scotchDecomp.H>

Inheritance diagram for scotchDecomp:

Collaboration diagram for scotchDecomp:

Public Member Functions
	scotchDecomp (const scotchDecomp &)=delete
	No copy construct.
void	operator= (const scotchDecomp &)=delete
	No copy assignment.
	TypeName ("scotch")
	Runtime type information.
	scotchDecomp (const label numDomains)
	Construct with number of domains (no coefficients or constraints).
	scotchDecomp (const dictionary &decompDict, const word &regionName="")
	Construct given decomposition dictionary and optional region name.
virtual	~scotchDecomp ()=default
	Destructor.
virtual bool	parallelAware () const
	Knows about coupled boundaries.
virtual labelList	decompose (const polyMesh &mesh, const pointField &points=pointField::null(), const scalarField &pointWeights=scalarField::null()) const
	Return for every coordinate the wanted processor number.
virtual labelList	decompose (const polyMesh &mesh, const labelList &agglom, const pointField &agglomPoints, const scalarField &agglomWeights=scalarField::null()) const
	Return for every coordinate the wanted processor number.
virtual labelList	decompose (const CompactListList< label > &globalCellCells, const pointField &cc=pointField::null(), const scalarField &cWeights=scalarField::null()) const
	Return the wanted processor number for every cell.
virtual labelList	decompose (const labelListList &globalCellCells, const pointField &cc=pointField::null(), const scalarField &cWeights=scalarField::null()) const
	Return the wanted processor number for every cell.
virtual labelList	decompose (const pointField &points, const scalarField &pointWeights=scalarField::null()) const
	Inherit all decompose methods.
virtual labelList	decompose (const polyMesh &mesh, const scalarField &cellWeights, const boolList &blockedFace, const PtrList< labelList > &specifiedProcessorFaces, const labelList &specifiedProcessor, const List< labelPair > &explicitConnections) const
	Inherit all decompose methods.
labelList	decompose (const polyMesh &mesh, const scalarField &cWeights) const
	Inherit all decompose methods.
Public Member Functions inherited from metisLikeDecomp
virtual	~metisLikeDecomp ()=default
	Destructor.
labelList	decompose (const polyMesh &mesh, const scalarField &cWeights) const
	Inherit decompose from decompositionMethod.
Public Member Functions inherited from decompositionMethod
	decompositionMethod (const decompositionMethod &)=delete
	No copy construct.
void	operator= (const decompositionMethod &)=delete
	No copy assignment.
	TypeName ("decompositionMethod")
	Runtime type information.
	declareRunTimeSelectionTable (autoPtr, decompositionMethod, dictionary,(const dictionary &decompDict, const word &regionName),(decompDict, regionName))
	decompositionMethod (const dictionary &decompDict, const word &regionName="")
	Construct given the decomposition dictionary, optionally region-specific.
virtual	~decompositionMethod ()=default
	Destructor.
label	nDomains () const noexcept
	Number of domains.
label	nDomains (const label n)
	Set number of domains. Return old value.
virtual bool	geometric () const
	True if the method is purely geometric, often using cell centre points.
void	setConstraints (const polyMesh &mesh, boolList &blockedFace, PtrList< labelList > &specifiedProcessorFaces, labelList &specifiedProcessor, List< labelPair > &explicitConnections) const
	Helper: extract constraints:
void	applyConstraints (const polyMesh &mesh, const boolList &blockedFace, const PtrList< labelList > &specifiedProcessorFaces, const labelList &specifiedProcessor, const List< labelPair > &explicitConnections, labelList &finalDecomp) const
	Helper: apply constraints to a decomposition.
labelList	decompose (const polyMesh &mesh, const scalarField &cWeights) const
	Decompose a mesh.

Protected Member Functions
virtual label	decomposeSerial (const labelList &adjncy, const labelList &xadj, const List< scalar > &cWeights, labelList &decomp) const
	Decompose non-parallel.
Protected Member Functions inherited from metisLikeDecomp
virtual label	decomposeGeneral (const labelList &adjncy, const labelList &xadj, const List< scalar > &cellWeights, labelList &decomp) const
	Serial and/or collect/distribute for parallel operation.
	metisLikeDecomp (const metisLikeDecomp &)=delete
	No copy construct.
void	operator= (const metisLikeDecomp &)=delete
	No copy assignment.
	metisLikeDecomp (const label numDomains)
	Construct with number of domains (no coefficients or constraints).
	metisLikeDecomp (const word &derivedType, const dictionary &decompDict, int select=selectionType::NULL_DICT)
	Construct for derived type name and decomposition dictionary.
	metisLikeDecomp (const word &derivedType, const dictionary &decompDict, const word &regionName, int select=selectionType::NULL_DICT)
	Construct for derived type name, decomposition dictionary and (optional) region name.
Protected Member Functions inherited from decompositionMethod
FOAM_NO_DANGLING_REFERENCE const dictionary &	findCoeffsDict (const word &coeffsName, int select=selectionType::DEFAULT) const
	Locate coeffsName dictionary or the fallback "coeffs" dictionary.
	decompositionMethod (const label numDomains)
	Construct with specified number of domains, no coefficients or constraints.

Additional Inherited Members
Static Public Member Functions inherited from decompositionMethod
static label	nDomains (const dictionary &decompDict, const word &regionName="")
	Return region-specific or top-level `numberOfSubdomains` entry.
static const dictionary &	optionalRegionDict (const dictionary &decompDict, const word &regionName)
	Return an optional region-specific dictionary from "regions" sub-dictionary, or dictionary::null on failure.
static autoPtr< decompositionMethod >	New (const dictionary &decompDict, const word &regionName="")
	Return a reference to the selected decomposition method, optionally region-specific.
static void	calcCellCells (const polyMesh &mesh, const labelList &agglom, const label nLocalCoarse, const bool parallel, CompactListList< label > &cellCells)
	Determine (local or global) cellCells from mesh agglomeration.
static void	calcCellCells (const polyMesh &mesh, const labelList &agglom, const label nLocalCoarse, const bool parallel, CompactListList< label > &cellCells, CompactListList< scalar > &cellCellWeights)
	Determine (local or global) cellCells and face weights from mesh agglomeration.
Protected Types inherited from decompositionMethod
enum	selectionType { DEFAULT = 0 , EXACT = 1 , MANDATORY = 2 , NULL_DICT = 4 }
	Selection type when handling the coefficients dictionary. More...
Static Protected Member Functions inherited from decompositionMethod
static FOAM_NO_DANGLING_REFERENCE const dictionary &	findCoeffsDict (const dictionary &dict, const word &coeffsName, int select=selectionType::DEFAULT)
	Locate coeffsName dictionary or the fallback "coeffs" dictionary within an enclosing dictionary.
Protected Attributes inherited from metisLikeDecomp
const dictionary &	coeffsDict_
	Coefficients for all derived methods.
Protected Attributes inherited from decompositionMethod
const dictionary &	decompDict_
	Top-level decomposition dictionary (eg, decomposeParDict).
const dictionary &	decompRegionDict_
	Region-specific decomposition dictionary information.
label	nDomains_
	Number of domains for the decomposition.
PtrList< decompositionConstraint >	constraints_
	Optional constraints.

Detailed Description

Scotch domain decomposition.

When run in parallel will collect the whole graph on to the master, decompose and send back. Use ptscotchDecomp for proper distributed decomposition.

Coefficients dictionary: scotchCoeffs, coeffs.

Quoting from the Scotch forum, on the 2008-08-22 10:09, Francois PELLEGRINI posted the following details:

RE: Graph mapping 'strategy' string

Strategy handling in Scotch is a bit tricky. In order
not to be confused, you must have a clear view of how they are built.
Here are some rules:

1- Strategies are made up of "methods" which are combined by means of
"operators".

2- A method is of the form "m{param=value,param=value,...}", where "m"
is a single character (this is your first error: "f" is a method name,
not a parameter name).

3- There exist different sort of strategies : bipartitioning strategies,
mapping strategies, ordering strategies, which cannot be mixed. For
instance, you cannot build a bipartitioning strategy and feed it to a
mapping method (this is your second error).

To use the "mapCompute" routine, you must create a mapping strategy, not
a bipartitioning one, and so use stratGraphMap() and not
stratGraphBipart(). Your mapping strategy should however be based on the
"recursive bipartitioning" method ("b"). For instance, a simple (and
hence not very efficient) mapping strategy can be :

"b{sep=f}"

which computes mappings with the recursive bipartitioning method "b",
this latter using the Fiduccia-Mattheyses method "f" to compute its
separators.

If you want an exact partition (see your previous post), try
"b{sep=fx}".

However, these strategies are not the most efficient, as they do not
make use of the multi-level framework.

To use the multi-level framework, try for instance:

"b{sep=m{vert=100,low=h,asc=f}x}"

The current default mapping strategy in Scotch can be seen by using the
"-vs" option of program gmap. It is, to date:

r
{
    job=t,
    map=t,
    poli=S,
    sep=
    (
        m
        {
            asc=b
            {
                bnd=
                (
                    d{pass=40,dif=1,rem=1}
                 |
                )
                f{move=80,pass=-1,bal=0.002491},
                org=f{move=80,pass=-1,bal=0.002491},
                width=3
            },
            low=h{pass=10}
            f{move=80,pass=-1,bal=0.002491},
            type=h,
            vert=80,
            rat=0.8
        }
      | m
        {
            asc=b
            {
                bnd=
                (
                    d{pass=40,dif=1,rem=1}
                  |
                )
                f{move=80,pass=-1,bal=0.002491},
                org=f{move=80,pass=-1,bal=0.002491},
                width=3
            },
            low=h{pass=10}
            f{move=80,pass=-1,bal=0.002491},
            type=h,
            vert=80,
            rat=0.8
        }
    )
}

Given that this information was written in 2008, this example strategy will unlikely work as-is with the more recent Scotch versions. Therefore, the steps for getting the current default strategy from within Scotch, is to do the following steps:

Edit the file system/decomposeParDict and use the following settings:
```
method          scotch;

scotchCoeffs
{
    writeGraph true;
}
```
Run decomposePar. For example, it will write a file named region0.grf.
Now, instead of using gmap, run gpart with the following command structure to get the default strategy:
```
gpart \<nProcs\> -vs \<grfFile\>
```
where:
- <grfFile> is the file that was obtained with the option writeGraph=true, namely region0.grf.
- <nProcs> is the numberOfSubdomains defined in the dictionary file.
At the end of the execution will be shown a long string, similar to the following example (complete line was cropped at [...]):
```
S    Strat=m{asc=b{width=3,bnd=d{pass=40,dif=1,rem=0}[...],type=h}
```

Edit the file system/decomposeParDict once again and add the strategy entry as exemplified:

method          scotch;

scotchCoeffs
{
    //writeGraph true;
    strategy "m{asc=b{width=3,bnd=d{pass=40,dif=1,rem=0}[...],type=h}";
}

Finally, run decomposePar once again, to at least test if it works as intended.

Also support for multi-level decomposition by specifying inter-level communication weights:

numberOfSubdomains  2048;
coeffs
{
    // Divide into 64 clusters, each of 32 cores
    domains (64 32);
    // Inside a cluster the communication weight is 1% of that inbetween
    // clusters
    domainWeights (1 0.01);
}

Alternatively the first-level decomposition can be left out by assuming weights are 1 for that level:

numberOfSubdomains  2048;
coeffs
{
    // Divide into 2048/32 clusters
    domains (32);
    // Inside a cluster the communication weight is 1% of that inbetween
    // clusters
    domainWeights (0.01);
}

Note: gpart can be found in the current search path by adding the respective bin folder from the Scotch installation, namely by running the following commands:

source $(foamEtcFile config.sh/scotch)
export PATH=$PATH:$SCOTCH_ARCH_PATH/bin

Source files

Definition at line 259 of file scotchDecomp.H.

Constructor & Destructor Documentation

◆ scotchDecomp() [1/3]

scotchDecomp ( const scotchDecomp & )

delete

No copy construct.

References scotchDecomp().

Referenced by operator=(), and scotchDecomp().

Here is the call graph for this function:

Here is the caller graph for this function:

◆ scotchDecomp() [2/3]

scotchDecomp ( const label numDomains )

explicit

Construct with number of domains (no coefficients or constraints).

Definition at line 520 of file scotchDecomp.C.

References metisLikeDecomp::metisLikeDecomp().

Here is the call graph for this function:

◆ scotchDecomp() [3/3]

scotchDecomp	(	const dictionary &	decompDict,
		const word &	regionName = "" )

explicit

Construct given decomposition dictionary and optional region name.

Definition at line 526 of file scotchDecomp.C.

References Foam::GlobalIOList< Tuple2< scalar, vector > >::typeName, metisLikeDecomp::metisLikeDecomp(), decompositionMethod::NULL_DICT, and regionName.

Here is the call graph for this function:

◆ ~scotchDecomp()

virtual ~scotchDecomp ( )

virtualdefault

Destructor.

Member Function Documentation

◆ decomposeSerial()

Foam::label decomposeSerial	(	const labelList &	adjncy,
		const labelList &	xadj,
		const List< scalar > &	cWeights,
		labelList &	decomp ) const

protectedvirtual

Decompose non-parallel.

Hack to test clustering. Note that decomp is non-compact

numbers!

//- Set up variable sizes architecture check ( SCOTCH_archVcmplt(&archdat), "SCOTCH_archVcmplt" );

//- Stategy flags: go for quality or load balance (or leave default) SCOTCH_Num straval = 0; //straval |= SCOTCH_STRATQUALITY; //straval |= SCOTCH_STRATQUALITY;

//- Number of cells per agglomeration //SCOTCH_Num agglomSize = SCOTCH_archSize(&archdat); SCOTCH_Num agglomSize = 3;

//- Build strategy for agglomeration check ( SCOTCH_stratGraphClusterBuild ( &stradat, // strategy to build straval, // strategy flags agglomSize, // cells per cluster 1.0, // weight? 0.01 // max load imbalance ), "SCOTCH_stratGraphClusterBuild" );

Implements metisLikeDecomp.

Definition at line 101 of file scotchDecomp.C.

References UList< T >::cdata(), Foam::check(), refPtr< T >::clear(), metisLikeDecomp::coeffsDict_, DebugInfo, UList< T >::empty(), Foam::endl(), Foam::exit(), Foam::FatalError, FatalErrorInFunction, Foam::FatalIOError, FatalIOErrorInFunction, forAll, Foam::Info, keyType::LITERAL, Foam::max(), Foam::min(), n, OFstream::name(), decompositionMethod::nDomains(), decompositionMethod::nDomains_, Foam::nl, refPtr< T >::ref(), List< T >::resize(), List< T >::resize_nocopy(), ConstPrecisionAdaptor< Type, InputType, Container >::set(), PrecisionAdaptor< Type, InputType, Container >::set(), List< T >::setSize(), UList< T >::size(), Foam::sum(), and WarningInFunction.

Here is the call graph for this function:

◆ operator=()

void operator= ( const scotchDecomp & )

delete

No copy assignment.

References scotchDecomp().

Here is the call graph for this function:

◆ TypeName()

TypeName ( "scotch" )

Runtime type information.

References regionName.

◆ parallelAware()

virtual bool parallelAware ( ) const

inlinevirtual

Knows about coupled boundaries.

Implements decompositionMethod.

Definition at line 335 of file scotchDecomp.H.

◆ decompose() [1/7]

Foam::labelList decompose	(	const polyMesh &	mesh,
		const pointField &	points = pointField::null(),
		const scalarField &	pointWeights = scalarField::null() ) const

virtual

Return for every coordinate the wanted processor number.

Reimplemented from metisLikeDecomp.

Definition at line 538 of file scotchDecomp.C.

References metisLikeDecomp::decompose(), Foam::getGraphPathBase(), mesh, and points.

Here is the call graph for this function:

◆ decompose() [2/7]

Foam::labelList decompose	(	const polyMesh &	mesh,
		const labelList &	agglom,
		const pointField &	agglomPoints,
		const scalarField &	agglomWeights = scalarField::null() ) const

virtual

Return for every coordinate the wanted processor number.

Parameters

agglom agglomeration: fine-to-coarse

Reimplemented from metisLikeDecomp.

Definition at line 557 of file scotchDecomp.C.

References metisLikeDecomp::decompose(), Foam::getGraphPathBase(), and mesh.

Here is the call graph for this function:

◆ decompose() [3/7]

Foam::labelList decompose	(	const CompactListList< label > &	globalCellCells,
		const pointField &	cc = pointField::null(),
		const scalarField &	cWeights = scalarField::null() ) const

virtual

Return the wanted processor number for every cell.

Reimplemented from metisLikeDecomp.

Definition at line 578 of file scotchDecomp.C.

References metisLikeDecomp::decompose().

Here is the call graph for this function:

◆ decompose() [4/7]

Foam::labelList decompose	(	const labelListList &	globalCellCells,
		const pointField &	cc = pointField::null(),
		const scalarField &	cWeights = scalarField::null() ) const

virtual

Return the wanted processor number for every cell.

Reimplemented from metisLikeDecomp.

Definition at line 597 of file scotchDecomp.C.

References metisLikeDecomp::decompose().

Here is the call graph for this function:

◆ decompose() [5/7]

Foam::labelList decompose	(	const pointField &	points,
		const scalarField &	pointWeights = scalarField::null() ) const

virtual

Inherit all decompose methods.

Reimplemented from metisLikeDecomp.

Definition at line 298 of file decompositionMethod.C.

◆ decompose() [6/7]

Foam::labelList decompose	(	const polyMesh &	mesh,
		const scalarField &	cellWeights,
		const boolList &	blockedFace,
		const PtrList< labelList > &	specifiedProcessorFaces,
		const labelList &	specifiedProcessor,
		const List< labelPair > &	explicitConnections ) const

virtual

Inherit all decompose methods.

Parameters

blockedFace Whether owner and neighbour should be on same processor

(takes priority over explicitConnections)

Parameters

specifiedProcessorFaces Whether whole sets of faces (and point neighbours) need to be kept

on single processor

Parameters

explicitConnections Additional connections between boundary faces

Reimplemented from metisLikeDecomp.

Definition at line 422 of file decompositionMethod.C.

◆ decompose() [7/7]

Foam::labelList decompose	(	const polyMesh &	mesh,
		const scalarField &	cWeights ) const

Inherit all decompose methods.

Definition at line 445 of file decompositionMethod.C.

The documentation for this class was generated from the following files:

src/parallel/decompose/scotchDecomp/scotchDecomp.H
src/parallel/decompose/scotchDecomp/scotchDecomp.C

Public Member Functions

Protected Member Functions

Additional Inherited Members

Detailed Description

Constructor & Destructor Documentation

◆ scotchDecomp() [1/3]

◆ scotchDecomp() [2/3]

◆ scotchDecomp() [3/3]

◆ ~scotchDecomp()

Member Function Documentation

◆ decomposeSerial()

◆ operator=()

◆ TypeName()

◆ parallelAware()

◆ decompose() [1/7]

◆ decompose() [2/7]

◆ decompose() [3/7]

◆ decompose() [4/7]

◆ decompose() [5/7]

◆ decompose() [6/7]

◆ decompose() [7/7]