Nat! bio photo

Nat!

Senior Mull

Twitter Github Twitch

mulle-match, the evolution of mulle-xcode-to-cmake

I have a small utility called mulle-xcode-to-cmake that is modestly popular. It converts Xcode projects into cmake projects. The spiritual successor to it and the program I actually use is mulle-match-to-cmake, which is quite different and which I like to introduce here.

The quickest way to witness the power of this fully armed and operational battle station is to check out these two repositories mulle-match and mulle-bashfunctions, which is a bash library.

$ git clone https://github.com/mulle-nat/mulle-bashfunctions.git
$ git clone https://github.com/mulle-sde/mulle-match.git
$ PATH="${PWD}/mulle-bashfunctions:${PWD}/mulle-match:${PATH}"

For the sake of this presentation, also clone the mulle-xcode-to-cmake repository, which we will convert to use with mulle-match-to-cmake.

$ git clone https://github.com/mulle-nat/mulle-xcode-to-cmake.git

mulle-match

mulle-match is the base for mulle-match-to-cmake. With mulle-xcode-to-cmake cloned, let’s see what mulle-match will do:

$ cd mulle-xcode-to-cmake
$ mulle-match list
src/PBXReading/MullePBXUnarchiver.h
src/PBXReading/MullePBXUnarchiver.m
...
src/mulle-xcode-to-cmake/main.m

mulle-match will have listed the source files in this project. That is because mulle-match has some - tweakable - preconceived notions, where sources reside and where they don’t (like in a build folder). But listing sources alone is not that useful. In a CMakeLists.txt we need to keep sources and headers seperate. It is also helpful to distinguish public and private headers, when installing headers.

mulle-match can categorize filenames with so called patternfiles. A few useful patternfiles can be had with minimal effort with the init command. These get installed in .mulle/etc/match/match.d:

$ mulle-match init
$ mulle-match patternfile list --cat

-----------------------------------------
match.d/50-source--private-headers
-----------------------------------------
*[_-]private.h
*[_-]private.inc
private.h
private.inc

-----------------------------------------
match.d/60-source--public-headers
-----------------------------------------
*.h

-----------------------------------------
match.d/70-source--sources
-----------------------------------------
*.[cm]

So there are three patternfiles now : 50-source--private-headers, 60-source--public-headers, 70-source--sources.

The identifier after the number and the ‘-‘ is called the type and the identifier after the ‘–’ is called the category. Patternfiles are evaluated by mulle-match in numeric order. The contents of a patternfile are like a .gitignore file.

We now can get typed and categorized output from mulle-match, for instance in a JSONish output style:

$ mulle-match list --format '{ "name": "%f", "type": "%t", "category": "%c" },\n'

{ "name": "src/PBXReading/MullePBXUnarchiver.h", "type": "source", "category": "public-headers" },
{ "name": "src/PBXReading/MullePBXUnarchiver.m", "type": "source", "category": "sources" },
...
{ "name": "src/mulle-xcode-to-cmake/main.m", "type": "source", "category": "public-headers" },

mulle-match-to-cmake

mulle-match-to-cmake is part of the mulle-match package.

mulle-match-to-cmake uses a properly setup mulle-match project to transform the typed and categorized list of files into cmake statements. So be sure to have done the above steps already before you execute:

$ mulle-match-to-cmake --stdout

set( INCLUDE_DIRS
src/PBXReading
src/PBXWriting
src/mulle-xcode-to-cmake
)

set( PUBLIC_HEADERS
src/PBXReading/MullePBXUnarchiver.h
src/PBXReading/NSObject+DecodeWithObjectStorage.h
..
src/mulle-xcode-to-cmake/PBXPathObject+HierarchyAndPaths.h
)

set( SOURCES
src/PBXReading/MullePBXUnarchiver.m
src/PBXReading/NSObject+DecodeWithObjectStorage.m
...
src/mulle-xcode-to-cmake/main.m
)

These definitions are obviously not sufficient to compile the project, but they are the most variable parts of a projects. If you include the generated output as a file into CMakeLists.txt, the CMakeLists.txt can remain hand-written and unchanged.

Let’s clobber the original CMakeLists.txt for demonstration purposes, with a much simpler one:

$ cat <<EOF > CMakeLists.txt
cmake_minimum_required( VERSION 3.0)

project( mulle-xcode-to-cmake C)

include( Headers.cmake)
include( Sources.cmake)

include_directories( \${INCLUDE_DIRS})

add_executable( mulle-xcode-to-cmake
\${SOURCES}
\${PUBLIC_HEADERS}
\${PRIVATE_HEADERS}
)

find_library( FOUNDATION_LIB NAMES Foundation)

target_link_libraries( mulle-xcode-to-cmake
\${FOUNDATION_LIB}
)
EOF

Use mulle-match-to-cmake to generate Headers.cmake and Sources.cmake:

$ mulle-match-to-cmake --headers-file Headers.cmake --sources-file Sources.cmake

And check that it builds with:

$ mkdir build
$ cd build
$ cmake -G "Unix Makefiles" ..
$ make

Now you only need to periodically run mulle-match-to-cmake to sync the cmake project with changes made in the filesystem.

This is easy to accomplish with cmake, since it can include other cmake files. It is a bit more complicated to support more toy-like build-systems such as meson that lack an include facility.

Adding the library target back in

The original project builds a library and and executable, whereas the CMakeLists.txt, we just created, just does an executable. This modified CMakeLists.txt adds a library target. All the headers are made part of this library target. The sources will have to be separated into EXECUTABLE_SOURCES and SOURCES. The executable will then link against the library:

$ cat <<EOF > CMakeLists.txt
cmake_minimum_required( VERSION 3.0)

project( mulle-xcode-to-cmake C)

include( Headers.cmake)
include( Sources.cmake)

include_directories( \${INCLUDE_DIRS})

add_library( mullepbx STATIC
\${SOURCES}
\${PUBLIC_HEADERS}
\${PRIVATE_HEADERS}
)

add_executable( mulle-xcode-to-cmake
\${EXECUTABLE_SOURCES}
)

find_library( FOUNDATION_LIB NAMES Foundation)

target_link_libraries( mulle-xcode-to-cmake
\${FOUNDATION_LIB}
mullepbx
)
EOF

To separate the source files we need a new patternfile. We will keep it as simple as possible, since there is only one executable source file:

$ echo "main.m" | mulle-match patternfile add -p 65 -c executable-sources source -

Now run

$ mulle-match-to-cmake --headers-file Headers.cmake --sources-file Sources.cmake

again, and that’s it!

Why mulle-match is 5K lines of code

It’s not easy to get good performance out of a shell script that has to interpret multiple patternfiles, then create regular expressions and then match these regular expressions against each line that a find spits out. What is acceptable for a few dozen lines, gets unbearably slow when you’re hitting 1000 files or so.

Though mulle-match is highly parallel in execution, the main “trick” of mulle-match is that it compiles the patternfiles into bash functions.

For example the patternfile

*[_-]private.h
*[_-]private.inc
private.h
private.inc

gets compiled into

__p__m_50_source__private_headers()
{

   local rval=1

   case $1 in
      *([^/])[_-]private.h|*/*([^/])[_-]private.h)
         rval=0
      ;;
   esac
   case $1 in
      *([^/])[_-]private.inc|*/*([^/])[_-]private.inc)
         rval=0
      ;;
   esac
   case $1 in
      private.h|*/private.h)
         rval=0
      ;;
   esac
   case $1 in
      private.inc|*/private.inc)
         rval=0
      ;;
   esac
   return ${rval}
}

History

I wrote the utility at a time, when my main focus was Xcode and I wanted to maintain an Xcode project and a CMakeLists.txt side-by-side for my MulleFoundation. Though useful, it was still too much work. Then I shifted the focus to generate the Xcode project from cmake. But then I couldn’t maintain the project structure with Xcode anymore and Xcode wasn’t that useful anymore.


Post a comment

All comments are held for moderation; basic HTML formatting accepted.

Name:
E-mail: (not published)
Website: