Audience: You are expected to be an advanced GCC user (fluent with command-line), and to have successfully downloaded and installed GCC MELT on your computer (probably under GNU/Linux running a recent version -4.7 at least- of GCC).
Notice that GCC MELT may use environment variables starting with GCCMELT or MELTGCC (e.g. GCCMELT_SOURCE_PATH or GCCMELT_MODULE_PATH etc, etc...). Don't use them unless you want to change the behavior of the GCC MELT plugin.
It is assumed that you have already installed a recent GCC MELT plugin on your Gnu/Linux system. See the download and build GCC MELT page for more. You may of course want to use gcc-4.7 or /usr/bin/gcc-4.7 or gcc.real (to bypass ccache on some systems) or whatver appropriate instead of gcc, and g++-4.7 instead of g++ etc, etc...
The GCC MELT plugin is actually loaded,
like every GCC plugin, by some
compilation (by gcc
, g++
etc...) of some
source code. Since there is no way to run GCC
MELT otherwise, it is sometimes convenient to have
some empty source file, for example empty4melt.c
(read "empty for MELT dot
see"). You can create such an empty file with touch
empty4melt.c
command in a terminal (or later, in
your Makefile, when relevant). The name of that empty
file does not matter much, but it should have a .c
extension (to be handled like a C file
by gcc). That empty file could contain some comment, so
you could also get it with e.g. date +"/* empty file at
%c */" > empty4melt.c
or likewise. If you are more
used to C++ (or some other language
supported by GCC) use
an empty4melt.cc file for g++, etc. Of course
when you use MELT on some of your own
source files, you give them to gcc or g++
etc...
Compiling an empty file is often done
for important GCC MELT side
effects, such as translating a *.melt file
(in MELT) to
a *.so module. This is why it is so important
that gcc -fplugin=melt
don't go
thru ccache
etc.
Alternatively, instead of
compiling some empty file, you could use the -x
c
option
of GCC (specifying explicitly
the C language) and compile
the /dev/null file.
You should give, preferably as its first program
argument, the -fplugin=melt
option
to gcc (or
to g++, gfortran, gccgo etc....). If you
pass only this argument, the GCC MELT
plugin does nothing more. So running just the command g++
-fplugin=melt -c yourexample.cc has exactly the same
effect as g++ -c yourexample.cc (except that
the cc1plus compiler proper is dlopen-ing
the GCC MELT plugin when you
invoke the compiler as g++ -fplugin=melt ).
Several parameters or settings of GCC
MELT are wired inside the plugin. You can get the builtin
settings with gcc -fplugin=melt
-fplugin-arg-melt-print-settings -c empty4melt.c -o /dev/null
which outputs on stdout the settings, in a shell-friendly format which could
be source-d by /bin/sh (or any Posix friendly
shell e.g. bash or zsh).
When really using GCC MELT to customize
some GCC compilation, you'll want to give
a work-directory with
the -fplugin-arg-melt-workdir=work-directory
program argument. This work-directory gets filled
by GCC MELT with internal files (which may
be re-used from one invocation to the next, so the work directory is
somehow also a cache directory). The work directory is never cleared
by MELT, and may grow indefinitely if you
run MELT many times ; you may want to
clear it from time to time (e.g. with your make clean or
with a crontab entry).
The GCC MELT plugin does some real work
only if given some mode
with -fplugin-arg-melt-mode=mode
. Without
any mode, nothing special happens. Try to run
gcc -fplugin=melt -fplugin-arg-melt-mode=help -c empty4melt.c -o /dev/null
to get a list of available modes. Your MELT code usually should install [your] new modes.
These exploring modes are new in MELT 1.0 and should interest a large number of GCC power users, even those not wishing to customize themselves GCC by coding their own MELT extensions.
The counting mode justcountipa
(for just counting in inter-procedural analysis) can be used to give some simple statistics about Gimple instructions, Basic blocks, and emitted functions (after some inlining) by the GCC compiler. For example, to get such statistics when compiling the file general.c (with 1162 source lines) of GNU bash shell (version 4.2), just run: make general.o CFLAGS='-O2 -fplugin=melt -fplugin-arg-melt-mode=justcountipa'
to get the following notices:
general.c:1132:1: note: MELT inform: just counted 19 basic blocks and 39 gimple-sAfter more than 30 such notices, MELT enhanced GCC finally says↵
in function <function_decl 0x7f7261dcdb00 get_group_array> get_group_array (ngp) ^ general.c:1099:1: note: MELT inform: just counted 19 basic blocks and 38 gimple-s↵
in function <function_decl 0x7f7261dcda00 get_group_list> get_group_list (ngp) ^ In file included from general.c:30:0: /usr/include/unistd.h:715:12: note: MELT inform: just counted 16 basic blocks and 24 gimple-s↵
in function <function_decl 0x7f7261c6d700 group_member> extern int group_member (__gid_t __gid) __THROW; ^ general.c:1013:1: note: MELT inform: just counted 24 basic blocks and 85 gimple-s↵
in function <function_decl 0x7f7261a7be00 initialize_group_array> initialize_group_array () ^
general.c:65:1: note: MELT inform: just counted 7 basic blocks and 12 gimple-sNotice that the source locations are given from bottom of source file to top, because (strangely) inter-procedural analysis is done in that order by GCC. MELT has inserted its own melt_justcountipa simple inter-procedural analysis pass into GCC workflow, so you better give some optimization level to↵
in function <function_decl 0x7f7261dbd800 posix_initialize> posix_initialize (on) ^ In file included from /usr/include/features.h:371:0, from /usr/include/wchar.h:27, from config-bot.h:140, from config.h:1129, from general.c:21: /usr/include/x86_64-linux-gnu/sys/stat.h:454:1: note: MELT inform: just↵
counted 3 basic blocks and 2 gimple-s↵
in function <function_decl 0x7f7262016f00 stat> __NTH (stat (const char *__path, struct stat *__statbuf)) ^ /usr/include/inttypes.h:331:1: note: MELT inform: just counted 3 basic blocks and 2 gimple-s↵
in function <function_decl 0x7f7261fd8000 strtoimax> __NTH (strtoimax (const char *__restrict nptr, char **__restrict endptr, ^ cc1: note: MELT inform: just counted 41 functions, cumulating 517 basic blocks and 1046 gimples
gcc
. Also, function declarations are verbosely
printed as
GCC trees.
The findgimple
mode (and
also gofindgimple) is useful to find some gimples
inside the internal representations
of GCC. This is a quite powerful mode, and
it requires
two patterns
(given in MELT syntax), the
first πgimple
being a pattern
on Gimple-s, the second being a
pattern πfun
on function
declaration Tree-s. It then shows within all functions of the
compiled translated unit matching the πfun
pattern all the gimples instructions matching
the πgimple
pattern. So
our findgimple
mode is a sort of
super- grep working on internal GCC repreesentations
of GCC; it does not use
textual regexps but
more "semantic" patterns. The simplest pattern is the
catch-all wildcard (or joker)
pattern ?_
, it is always matching, so the
match never fails. The wildcard is the default for both patterns. The
gimple pattern πgimple
is
passed
thru -fplugin-arg-melt-gimple-pattern=πgimple
and the function pattern πfun
is passed
thru -fplugin-arg-melt-function-pattern=πfun
. Optional
action expressions (evaluated inside the match
) can be passed
with -fplugin-arg-melt-action=action
and could use pattern variables bound in πgimple
.
So to just find every gimple instruction in your translation unit, compile it with
-fplugin=melt -fplugin-arg-melt-mode=findgimple
-fplugin-arg-melt-gimple-pattern='?_'
(we
need to quote the argument for the shell because of the ?
character, and we could even
forget -fplugin-arg-melt-gimple-pattern='?_'
since the wildcard is the default).
Such a compilation might be slow, because MELT would generate some C++ code, then compile and dynamically load (using dlopen) it. We'll see below how to make that faster (using the gofindgimple mode), by avoiding useless recompilation of generated C++ code.
Here are some more examples of findgimple modes uses (we
omit the always needed -fplugin=melt
-fplugin-arg-melt-mode=findgimple for brevity). Notice that
patterns (all starting with ?
) may be nested:
-fplugin-arg-melt-gimple-pattern='?(gimple_call_1 ?_ ?(tree_function_decl_of_name "xmalloc" ?_ ?_) ?(tree_integer_cst ?(some_integer_greater_than 30)))'but notice that you won't be able to use textual grep to find the same (e.g. because often xmalloc is passed some sizeof or some constant-folded expression).
-fplugin-arg-melt-gimple-pattern='?(gimple_call_1_more ?_ ?(tree_function_decl_of_name "printf" ?_ ?_) ?(tree_string_cst ?(some_string_value_containing "current")) ?(some_integer_greater_than 6))' \ -fplugin-arg-melt-function-pattern='?(tree_function_decl_named ?(cstring_prefixed "foo") ?_)'Here ?(some_integer_greater_than 6) matches the arity (number of arguments) of the call, and ?(tree_function_decl_named ?(cstring_prefixed "foo") ?_) is the
πfun
pattern on the
function.Notice that the findgimple mode is always generating and
compiling some C++ file (for the
particular search you are interested in), and that usually takes
several seconds (then all the generated files are deleted). If the
same search is to be done one many files (e.g. all the translation
units of your software) it is worthwhile to keep the generated
files. For that, use
the -fplugin-arg-melt-output=file-prefix
argument of the findgimple mode. Then the
generated C++ files
(named file-prefix*.cc for real generated code
or file-prefix*.[ch] for meta-data code) and
the file-prefix*.so module are kept. To reuse
them again, use the gofindgimple
mode
like -fplugin-arg-melt-extra=file-prefix
-fplugin-arg-melt-mode=gofindgimple
(without
giving any -fplugin-arg-melt-arg, of course).
Use the justshowpasses
mode to show the
passes executed by GCC. Of course, you
could also (and instead) pass -fdump-all-passes
to gcc
.
Another interesting mode is the runtime evaluator; you
can evaluate some non-empty sequence of one or
more MELT expressions with
the eval mode, giving the sequence of expressions to
evaluate thru a string passed
with -fplugin-arg-melt-arg=MELT-expressions
. For
example, gcc -fplugin=melt
-fplugin-arg-melt-mode=eval \
is
evaluating a sequence made of one single
expression (melt_version_str) -we have to add quotes to keep the bash shell happy- and printing the result (of
the last evaluated expression); you may get some warnings and an
output having lines similar to:
-fplugin-arg-melt-arg='(melt_version_str)' -c empty4melt.c
cc1: note: MELT generated new file /tmp/fileRi68N5-GccMeltTmp-101885fb/melt_p17576_run0001_eXt.cc cc1: note: MELT generated C++ code of module /tmp/fileRi68N5-GccMeltTmp-101885fb/melt_p17576_run0001_eXt with 0 secondary files in 0 CPU millisec. MELT is building binary /tmp/fileRi68N5-GccMeltTmp-101885fb/melt_p17576_run0001_eXt from source /tmp/fileRi68N5-GccMeltTmp-101885fb/melt_p17576_run0001_eXt with flavor runextend cc1: note: MELT plugin has built module /tmp/fileRi68N5-GccMeltTmp-101885fb/melt_p17576_run0001_eXt flavor runextend cc1: warning: MELT debug depth -f[plugin-arg-]melt-debug-depth= defaulted to 9 [enabled by default] ;;; result of eval mode of 1 expression[s] is ... "1.0-rc3 [melt-branch_revision_204086]" ;;; *** end of evaluated result *** cc1: note: MELT removed 7 temporary files from /tmp/fileRi68N5-GccMeltTmp-101885fb
The above (simplified) output shows that MELT has made a temporary directory, has filled it with some temporary C++ files, has compiled them into a module (some *.so file inside), has dlopen-ed that module and executed it, and finally has cleaned that temporary directory.
The MELT
expression (melt_version_str)
illustrates the Lisp-like
syntax of the MELT language: almost every MELT
expression is enclosed in parenthesis: after a left paren ( you get the operator (or functions), followed by optional operands (or arguments), ending with a right paren ) so parenthesis are highly signficant.
As an exercise, try to evaluate melt_version_str
without the parenthesis. The result is the value for the melt_version_str primitive itself, it is not a string.
You probably will want to use many times the MELT expression evaluator. To avoid typing many times similar long commands, you may want to define the following bash function:
# for interactive bash shell function melt_eval { touch empty4melt.c; gcc -fplugin=melt -fplugin-arg-melt-mode=eval \ -fplugin-arg-melt-arg="$*" -c empty4melt.c -o /dev/null }
Once you have such a shell function (perhaps even in
your $HOME/.bashrc if you wish) you could simply
type melt_eval '(melt_version_str)'
. However, your shell
requires you to backslash-escape some characters, notably
the '
(quote character), like melt_eval '\'1'
or
simply melt_eval \'1
You can also evaluate a file
containing MELT expressions with
the evalfile
mode, giving
with -fplugin-arg-melt-arg=melt-file
the name
of the MELT source file. If no name, or
if -
is given, the standard input is read till
end of file, all the expressions are evaluated, and the result of the
last one is printed (by translating them to
some C or C++
temporary file, compiling it, and dlopen-ing the temporary
shared object).
You can run a MELT file with
the runfile
mode. The file name should be passed
with -fplugin-arg-melt-arg=melt-file
, it is
run (also by generating C
or C++ code, compiling it,
then dlopen-ing it) for its side effect without implicitly
displaying any result.
There is an experimental read-eval-print loop provided by
the repl
mode. Expressions are grouped in sequences,
terminated by a double newline, and read from stdin. So you
can type MELT expressions, and when you
enter an empty line (separating two sequences), they are
evaluated.
Modes to translate your MELT source to generated C or C++ are explained below.
An interesting mode (removed in MELT 1.1),
was the probe (if you have installed the optional probe GUI
and have a melt-probe script). It shows you, in a graphical
interface window, the source code files, and enable you to get
intersting GCC internal representations
(notably Gimple-s and Tree-s) at a particular source
location. Try something like
gcc -fplugin=melt -fplugin-arg-melt-mode=probe -O -Iyour-include
-c source-file.c
on a
relevant source-file.c (with your-include
directory) of your application. Future versions of MELT might provide a Web interface (then the probe would be deprecated).
Most of the modes above are happier with an empty4melt.c source input to gcc.
The GCC MELT plugin handles various files and directories:
;
up to end of line. Expressions are
parenthesised. Often, a ;; -*- Lisp -*-
comment
near the start of the file is making emacs
happier. MELT source files are the
only ones hand-typed.-fplugin-arg-melt-debugging
is given.
Conceptually, GCC MELT does not load a module, it loads a descriptive generated file (and all the files mentionned in it), or a module list, and may recompile the module[s] (of some flavor) if it is missing.
The sources path is a colon separated path of source directories. The modules path is a colon separated path of module directories.
Don't forget to give a work directory to GCC MELT. Files created there are never removed automatically, it is up to you to clean it.
MELT may create a temporary directory which will be cleaned and removed by the plugin after the compilation.
You often want to translate your *.melt
files to MELT modules. This involves a
(quick) translation of your MELT code
into big generated C
or C++ code, then a (slow) compilation of
these generated files into a MELT
module. The actual bottleneck is the compilation (with
a make
process started by
the MELT plugin) of the
generated C
or C++ code into .so
modules, because the emitted C
or C++ code is much larger than the
original MELT file (an expansion
factor of more than 10x is not uncommon).
Advice: avoid giving a file name to your MELT source file similar to the files on which your MELT modules will be used. So don't have your foo.melt used to extend the compilation of your foo.c.
The MELT translator is not very friendly
on MELT syntax errors. It should locate
the first few errors in your *.melt
file,
but don't expect it to continue reliably the translation. Since
the MELT
→ C
or C++ translation runs quickly,
reliable syntax error recovery don't matter much in practice.
The following MELT modes are relevant to translation of *.melt files:
translatequickly
mode is often use when
developping your MELT extension. It
translates your MELT source file to a
module (forking a make using -O0)
of quicklybuilt flavor with debugging facilities
(like (debug ...)
expressions in
your MELT code) enabled. For example:gcc -fplugin=melt -fplugin-arg-melt-mode=translatequickly -fplugin-arg-melt-arg=yourfile.melt -c empty4melt.c
translateoptimized
mode should be useful for
making "production" ready modules. Since it produce a module
of optimized flavor, the internal make of the
module invokes the compiler with -O2 to build the module
(with disabled debugging), so that internal make takes
much more time. For example:gcc -fplugin=melt -fplugin-arg-melt-mode=translateoptimized \
-fplugin-arg-melt-arg=yourfile.melt \
-fplugin-arg-melt-workdir=work-directory/ \
-c -x c /dev/null -o /dev/null
translatedebug
mode is rarely useful, it
produce a module of debugnoline flavor, only useful to run
the gdb debugger on the cc1 or cc1plus
process running that module. You don't like to need to go at such a
low level of debugging, and you may need to have
a gdb-debugable variant of your cc1 program from
your GCC compiler.translatetomodule
mode combines the effects of
all above modes: it translates your MELT
source file into modules
of optimized, quicklybuilt
and debugnoline flavors.translatefile
mode translates
your MELT source file
into C or C++
files without bothering to make any modules from them.All the translating modes above require
the MELT source
file yourfile.melt
to be given with
the -fplugin-arg-melt-arg=yourfile.melt
option to gcc (or g++, etc). You certainly will
give an empty4melt.c input source to gcc or pass -c -x c /dev/null -o /dev/null,
etc. You may give the output module with
the -fplugin-arg-melt-output=module
option. You usually want to give some work directory -fplugin-arg-melt-workdir=work-directory/
(because MELT may notice that it is
generating an already existing C
or C++ emitted file there, and won't
overwrite it, so the internal make won't need to
recompile it again.).
.melt
filesWhen using GNU make in your development process, you may want to add the following fragments to your Makefile:
.SUFFIXES: .melt .quicklybuilt.so .optimized.so
line near the beginning.MELTGCC= gcc -fplugin=melt
and/or MELTGXX= g++ -fplugin=melt
#the MELT work directory meltworkdir: [ -d $@ ] || mkdir $@ #the empty file for MELT empty4melt.c: touch $@
#build MELT quicklybuilt modules: %.quicklybuilt.so: %.melt | empty4melt.c meltworkdir $(MELTGCC) -fplugin-arg-melt-mode=translatequickly -fplugin-arg-melt-workdir=meltworkdir \ -fplugin-arg-melt-arg=$< -fplugin-arg-melt-output=$@ -c empty4melt.c -o /dev/null(you might use, instead of empty4melt.c the -x c -c /dev/null -o /dev/null trick)
#build MELT optimized modules: %.optimized.so: %.melt | empty4melt.c meltworkdir $(MELTGCC) -fplugin-arg-melt-mode=translateoptimized -fplugin-arg-melt-workdir=meltworkdir \ -fplugin-arg-melt-arg=$< -fplugin-arg-melt-output=$@ -c empty4melt.c -o /dev/null
Hint: if you are an Emacs user, compile inside emacs with M-x compile
Your extensions usually need to define their
own MELT modes. You can load an
extra module with
the -fplugin-arg-melt-extra=your-modules
argument to gcc or g++ etc. So if you have
a yourmeltextension.melt file defining some yourmode
mode, you'll probably
pass -fplugin-arg-melt-extra=yourmeltextension
-fplugin-arg-melt-mode=yourmode
to
your GCC compiler. For example, you
could add in your Makefile a specific rule to use yourmeltextension when compiling some-file.cc, like:
#when compiling some-file.cc use MELT with optimized module from yourmeltextension.melt and yourmode
some-file.o: some-file.cc yourmeltextension.optimized.so | meltworkdir
$(MELTGXX) -fplugin-arg-melt-extra=yourmeltextension \
-fplugin-arg-melt-mode=yourmode \
$(CXXFLAGS) -c $< -o $@
During development of your MELT
extension, you may want to run some tests. Then you probably want to
enable MELT to spill numerous debug
messages, in particular when you code
some (debug "Here x=" x)
debugging
expressions. This is enabled
with -fplugin-arg-melt-debugging=mode
(or -fplugin-arg-melt-debug
) when using quicklybuilt flavored modules. So for debugging your MELT
extension you probably want to explicitly pass -fplugin-arg-melt-extra=yourmeltextension.quicklybuilt
to gcc or g++ (i.e. your $(MELTGXX)
in your Makefile), since the default flavor of your extension module is optimized. Notice
that optimized flavor of modules won't give any debugging
output (nor check your assert_msg assertions), but quicklybuilt module flavor enable such debugging output (which you should explicitly activate at run-time with -fplugin-arg-melt-debugging=mode
or -fplugin-arg-melt-debugging=all
).