Skip to content

Introduce the concept of a matrix worker. #5

Description

@robertmaynard

A standard deployment file ( .cdep ) is written to target a single compiler, configuration and library type. This is great for testing projects with very specific setups, but isn't great when you want to verify a project against all combinations.

This is why we should introduce the concept of the matrix to deployment files. Instead of having
a worker target a single element of the compiler, configuration, and library matrix, why not have a worker that is meant to verify all elements.

For the initial pass we should limit the matrix to 3 dimensions, which are compiler, configuration type, and library type. I envision that we could easily extend the .cdep file to support a matrix
by extending the syntax for compiler, configuration, and library type as follows:

{
"hostname" : "bigboard",
"user" : "hiro",
"compilers" : [  
   {"name" : "legacy_gcc",  "c" : "/usr/bin/gcc-4.6", "cpp" : "/usr/bin/g++-4.6"},
   {"name" : "gcc4.8",  "c" : "/usr/bin/gcc-4.8", "cpp" : "/usr/bin/g++-4.8"},
   {"name" : "clang",   "c" : "/usr/bin/clang",   "cpp" : "/usr/bin/clang++"}
   ]
"src_location" : "/home/hiro/Work/bigboard/src",
"build_location" : "/home/hiro/Work/bigboard/build",
"build_generator" : "Ninja",
"build_flags" : "-j8",
"build_configuration" : [ "debug", "release" ]
"library_type" : [ "static", "shared" ]
}

Now what this means is that cfarm will create multiple build directories under the `build_location``, each labelled to identify the compiler, configuration and library type it represents. For the above example we would generate the following folders

/home/hiro/Work/bigboard/build/
 - legacy_gcc_debug_static/
 - legacy_gcc_debug_shared/
 - legacy_gcc_release_static/
 - legacy_gcc_release_shared/
 - gcc4.8_debug_static/
 - gcc4.8_debug_shared/
 - gcc4.8_release_static/
 - gcc4.8_release_shared/
 - clang_debug_static/
 - clang_debug_shared/
 - clang_release_static/
 - clang_release_shared/

Because of the explosion of build directories we need to add 3 quality of life improvements.

  1. Give each worker a pretty_name this will be a combination of the deployment file name plus the build
    director for the worker so for example a pretty_name could be linux_clang_debug_static.
  2. Override fabric.utils.puts to use a workers pretty_name instead of the current hostname
  3. Add support to cfarm commands to invoke workers based on their pretty_name and on their deployment name. This will allow a user to test a single component of the matrix.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions