Skip to content

Commit 7837ade

Browse files
SimonDanischSimonDanisch
authored
Merge pull request #18 from JuliaLang/sd/combine
combine PackageCompiler with static-julia
2 parents d25c0a5 + e32fd1f commit 7837ade

File tree

13 files changed

+803
-226
lines changed

13 files changed

+803
-226
lines changed

‎.gitignore‎

Lines changed: 7 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -5,9 +5,12 @@ deps.jl
55
*.csv
66
snoopy.jl
77
precompile.jl
8-
*.so
8+
sysimg
99
*.ji
10-
*.dll
11-
*.dylib
1210
*.o
13-
sysimg
11+
*.so
12+
*.so.*
13+
*.dylib
14+
*.dll
15+
hello
16+
hello.exe

‎.mailmap‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
Luca Trevisani <lucatrv@users.noreply.github.com> <lucatrv@users.noreply.github.com>
2+
Luca Trevisani <lucatrv@users.noreply.github.com> <lucatrv@hotmail.com>
3+
4+
Simon Danisch <simondanisch@users.noreply.github.com> <sdanisch@gmail.com>
5+
6+
Viral B. Shah <viral@mayin.org> <viral@mayin.org>
7+
Viral B. Shah <viral@mayin.org> <viral@juliacomputing.com>
8+
Viral B. Shah <viral@mayin.org> <viralbshah@anubis.juliacomputing.io>

‎README.md‎

Lines changed: 124 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,4 @@
11
# PackageCompiler
2-
32
[![Build Status](https://travis-ci.org/SimonDanisch/PackageCompiler.jl.svg?branch=master)](https://travis-ci.org/SimonDanisch/PackageCompiler.jl)
43

54
[![Coverage Status](https://coveralls.io/repos/SimonDanisch/PackageCompiler.jl/badge.svg?branch=master&service=github)](https://coveralls.io/github/SimonDanisch/PackageCompiler.jl?branch=master)
@@ -15,7 +14,7 @@ using PackageCompiler
1514

1615
# This command will use the runtest.jl of Matcha + UnicodeFun to find out what functions to precompile!
1716
# force = false to not force overwriting julia's current system image
18-
compile_package("Matcha", "UnicodeFun", force = false, reuse = false)
17+
compile_package("Matcha", "UnicodeFun", force = false, reuse = false)
1918

2019
# build again, reusing the snoop file
2120
compile_package("Matcha", "UnicodeFun", force = false, reuse = true)
@@ -28,7 +27,7 @@ compile_package("Matcha", "relative/path/for_snooping.jl")
2827
revert()
2928

3029
# Or if you simply want to get a native system image e.g. when you have downloaded the generic Julia install:
31-
build_clean_image()
30+
build_clean_image()
3231
```
3332

3433

@@ -46,3 +45,125 @@ build_clean_image()
4645
- I used julia-debug to uncover most bugs, but actually, the last errors I was trying to uncover where due to using julia-debug!
4746

4847
- you’re pretty much on your own and need to use gdb to find the issues and I still don’t know what the underlying julia issues are and when they will get fixed :wink: See: https://github.com/JuliaLang/julia/issues/24533. Hopefully we look at a better story with Julia 1.0!
48+
49+
50+
# Static Julia Compiler
51+
52+
Building shared libraries and executables from Julia code.
53+
54+
Run `juliac.jl -h` for help:
55+
56+
```
57+
usage: juliac.jl [-v] [-q] [-c] [-J <file>]
58+
[--compile {yes|no|all|min}] [-C <target>]
59+
[-O {0,1,2,3}] [-g {0,1,2}] [--inline {yes|no}]
60+
[--check-bounds {yes|no}] [--math-mode {ieee,fast}]
61+
[--depwarn {yes|no|error}] [-a] [-o] [-s] [-e] [-j]
62+
[--version] [-h] juliaprog [cprog] [builddir]
63+
64+
Static Julia Compiler
65+
66+
positional arguments:
67+
juliaprog Julia program to compile
68+
cprog C program to compile (required only when
69+
building an executable; if not provided a
70+
minimal standard program is used)
71+
builddir build directory, either absolute or relative
72+
to the Julia program directory (default:
73+
"builddir")
74+
75+
optional arguments:
76+
-v, --verbose increase verbosity
77+
-q, --quiet suppress non-error messages
78+
-c, --clean delete builddir
79+
-J, --sysimage <file>
80+
start up with the given system image file
81+
--compile {yes|no|all|min}
82+
enable or disable JIT compiler, or request
83+
exhaustive compilation
84+
-C, --cpu-target <target>
85+
limit usage of CPU features up to <target>
86+
-O, --optimize {0,1,2,3}
87+
set optimization level (type: Int64)
88+
-g {0,1,2} set debugging information level (type: Int64)
89+
--inline {yes|no} control whether inlining is permitted
90+
--check-bounds {yes|no}
91+
emit bounds checks always or never
92+
--math-mode {ieee,fast}
93+
set floating point optimizations
94+
--depwarn {yes|no|error}
95+
set syntax and method deprecation warnings
96+
-a, --autodeps automatically build required dependencies
97+
-o, --object build object file
98+
-s, --shared build shared library
99+
-e, --executable build executable file
100+
-j, --julialibs sync Julia libraries to builddir
101+
--version show version information and exit
102+
-h, --help show this help message and exit
103+
104+
examples:
105+
juliac.jl -vae hello.jl # verbose, build executable and deps
106+
juliac.jl -vae hello.jl prog.c # embed into user defined C program
107+
juliac.jl -qo hello.jl # quiet, build object file only
108+
juliac.jl -vosej hello.jl # build all and sync Julia libs
109+
```
110+
111+
### Notes
112+
113+
1. The `juliac.jl` script uses the `ArgParse` package, make sure it is installed.
114+
115+
3. On Windows install `Cygwin` and the `mingw64-x86_64-gcc-core` package, see:\
116+
https://github.com/JuliaLang/julia/blob/master/README.windows.md
117+
118+
2. A shared library containing the system image `libhello.so`, and a
119+
driver binary `hello` are created in the `builddir` directory.
120+
Running `hello` produces the following output:
121+
122+
```
123+
$ ./hello
124+
hello, world
125+
f() = -0.37549581296986956
126+
┌─────────────────────────────────────────────────┐
127+
100 │⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⢀⠎│
128+
│⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⡰⠁⠀│
129+
│⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⣠⠊⠀⠀⠀│
130+
│⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⢀⠔⠁⠀⠀⠀⠀│
131+
│⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⢀⠔⠁⠀⠀⠀⠀⠀⠀│
132+
│⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⡠⠊⠀⠀⠀⠀⠀⠀⠀⠀│
133+
│⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⡠⠊⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀│
134+
│⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⡠⠊⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀│
135+
│⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⢀⠤⠊⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀│
136+
│⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⣀⠔⠁⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀│
137+
│⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⢀⠤⠊⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀│
138+
│⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⣀⠤⠒⠁⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀│
139+
│⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⢀⡠⠔⠊⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀│
140+
│⠀⠀⠀⠀⠀⠀⢀⣀⠤⠔⠊⠁⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀│
141+
0 │⣀⠤⠤⠔⠒⠉⠁⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀│
142+
└─────────────────────────────────────────────────┘
143+
1 10
144+
```
145+
146+
## Under the hood
147+
148+
The `juliac.jl` script uses the `--output-o` switch to compile the user
149+
script into object code, and then builds it into the system image
150+
specified by the `-J` switch. This prepares an object file, which is
151+
then linked into a shared library containing the system image and user
152+
code. A driver script such as the one in `program.c` can then be used to
153+
build a binary that runs the julia code.
154+
155+
Instead of a driver script, the generated system image can be embedded
156+
into a larger program following the embedding examples and relevant
157+
sections in the Julia manual. Note that the name of the generated system
158+
image (`"libhello"` for `hello.jl`) is accessible from C in the
159+
preprocessor macro `JULIAC_PROGRAM_LIBNAME`.
160+
161+
With Julia 0.7, a single large binary can be created, which does not
162+
require the driver program to load the shared library. An example of
163+
that is in `program2.c`, where the image file is the binary itself.
164+
165+
For more information on static Julia compilation see:\
166+
https://juliacomputing.com/blog/2016/02/09/static-julia.html
167+
168+
For more information on embedding Julia see:\
169+
https://github.com/JuliaLang/julia/blob/master/doc/src/manual/embedding.md

‎REQUIRE‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,4 @@
11
julia 0.6
22
SnoopCompile
33
@windows WinRPM
4+
ArgParse

‎examples/hello.jl‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
module Hello
2+
3+
using UnicodePlots
4+
using Distributions
5+
f() = rand(Normal())
6+
7+
Base.@ccallable function julia_main(ARGS::Vector{String})::Cint
8+
println("hello, world")
9+
@show f()
10+
println(lineplot(1:10, (1:10).^2))
11+
return 0
12+
end
13+
14+
end

‎examples/program.c‎

Lines changed: 63 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,63 @@
1+
// This file is a part of Julia. License is MIT: http://julialang.org/license
2+
3+
// Standard headers
4+
#include <string.h>
5+
#include <stdint.h>
6+
7+
// Julia headers (for initialization and gc commands)
8+
#include "uv.h"
9+
#include "julia.h"
10+
11+
#ifdef JULIA_DEFINE_FAST_TLS // only available in Julia v0.7 and above
12+
JULIA_DEFINE_FAST_TLS()
13+
#endif
14+
15+
// Declare C prototype of a function defined in Julia
16+
extern int julia_main(jl_array_t*);
17+
18+
// main function (windows UTF16 -> UTF8 argument conversion code copied from julia's ui/repl.c)
19+
#ifndef _OS_WINDOWS_
20+
int main(int argc, char *argv[])
21+
{
22+
int retcode;
23+
int i;
24+
uv_setup_args(argc, argv); // no-op on Windows
25+
#else
26+
int wmain(int argc, wchar_t *wargv[], wchar_t *envp[])
27+
{
28+
int retcode;
29+
int i;
30+
char **argv;
31+
for (i = 0; i < argc; i++) { // convert the command line to UTF8
32+
wchar_t *warg = argv[i];
33+
size_t len = WideCharToMultiByte(CP_UTF8, 0, warg, -1, NULL, 0, NULL, NULL);
34+
if (!len) return 1;
35+
char *arg = (char*)alloca(len);
36+
if (!WideCharToMultiByte(CP_UTF8, 0, warg, -1, arg, len, NULL, NULL)) return 1;
37+
argv[i] = arg;
38+
}
39+
#endif
40+
41+
// initialization
42+
libsupport_init();
43+
// jl_options.compile_enabled = JL_OPTIONS_COMPILE_OFF;
44+
// JULIAC_PROGRAM_LIBNAME defined on command-line for compilation
45+
jl_options.image_file = JULIAC_PROGRAM_LIBNAME;
46+
julia_init(JL_IMAGE_JULIA_HOME);
47+
48+
// build arguments array: `String[ unsafe_string(argv[i]) for i in 1:argc ]`
49+
jl_array_t *ARGS = jl_alloc_array_1d(jl_apply_array_type(jl_string_type, 1), 0);
50+
JL_GC_PUSH1(&ARGS);
51+
jl_array_grow_end(ARGS, argc - 1);
52+
for (i = 1; i < argc; i++) {
53+
jl_value_t *s = (jl_value_t*)jl_cstr_to_string(argv[i]);
54+
jl_arrayset(ARGS, s, i - 1);
55+
}
56+
// call the work function, and get back a value
57+
retcode = julia_main(ARGS);
58+
JL_GC_POP();
59+
60+
// Cleanup and gracefully exit
61+
jl_atexit_hook(retcode);
62+
return retcode;
63+
}

‎juliac.jl‎

Lines changed: 131 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,131 @@
1+
## Assumptions:
2+
## 1. gcc / x86_64-w64-mingw32-gcc is available and is in path
3+
## 2. Package ArgParse is installed
4+
5+
using PackageCompiler, ArgParse
6+
7+
function main(args)
8+
9+
s = ArgParseSettings("Static Julia Compiler",
10+
version = "$(basename(@__FILE__)) version 0.7-DEV",
11+
add_version = true)
12+
13+
@add_arg_table s begin
14+
"juliaprog"
15+
arg_type = String
16+
required = true
17+
help = "Julia program to compile"
18+
"cprog"
19+
arg_type = String
20+
default = nothing
21+
help = "C program to compile (required only when building an executable; if not provided a minimal driver program is used)"
22+
"builddir"
23+
arg_type = String
24+
default = "builddir"
25+
help = "build directory, either absolute or relative to the Julia program directory"
26+
"--verbose", "-v"
27+
action = :store_true
28+
help = "increase verbosity"
29+
"--quiet", "-q"
30+
action = :store_true
31+
help = "suppress non-error messages"
32+
"--clean", "-c"
33+
action = :store_true
34+
help = "delete builddir"
35+
"--sysimage", "-J"
36+
arg_type = String
37+
default = nothing
38+
metavar = "<file>"
39+
help = "start up with the given system image file"
40+
"--compile"
41+
arg_type = String
42+
default = nothing
43+
metavar = "{yes|no|all|min}"
44+
range_tester = (x -> x == "yes" || x == "no" || x == "all" || x == "min")
45+
help = "enable or disable JIT compiler, or request exhaustive compilation"
46+
"--cpu-target", "-C"
47+
arg_type = String
48+
default = nothing
49+
metavar = "<target>"
50+
help = "limit usage of CPU features up to <target>"
51+
"--optimize", "-O"
52+
arg_type = Int
53+
default = nothing
54+
metavar = "{0,1,2,3}"
55+
range_tester = (x -> 0 <= x <= 3)
56+
help = "set optimization level"
57+
"-g"
58+
arg_type = Int
59+
default = nothing
60+
dest_name = "debug"
61+
metavar = "{0,1,2}"
62+
range_tester = (x -> 0 <= x <= 2)
63+
help = "set debugging information level"
64+
"--inline"
65+
arg_type = String
66+
default = nothing
67+
metavar = "{yes|no}"
68+
range_tester = (x -> x == "yes" || x == "no")
69+
help = "control whether inlining is permitted"
70+
"--check-bounds"
71+
arg_type = String
72+
default = nothing
73+
metavar = "{yes|no}"
74+
range_tester = (x -> x == "yes" || x == "no")
75+
help = "emit bounds checks always or never"
76+
"--math-mode"
77+
arg_type = String
78+
default = nothing
79+
metavar = "{ieee,fast}"
80+
range_tester = (x -> x == "ieee" || x == "fast")
81+
help = "set floating point optimizations"
82+
"--depwarn"
83+
arg_type = String
84+
default = nothing
85+
metavar = "{yes|no|error}"
86+
range_tester = (x -> x == "yes" || x == "no" || x == "error")
87+
help = "set syntax and method deprecation warnings"
88+
"--autodeps", "-a"
89+
action = :store_true
90+
help = "automatically build required dependencies"
91+
"--object", "-o"
92+
action = :store_true
93+
help = "build object file"
94+
"--shared", "-s"
95+
action = :store_true
96+
help = "build shared library"
97+
"--executable", "-e"
98+
action = :store_true
99+
help = "build executable file"
100+
"--julialibs", "-j"
101+
action = :store_true
102+
help = "sync Julia libraries to builddir"
103+
end
104+
105+
s.epilog = """
106+
examples:\n
107+
\ua0\ua0juliac.jl -vae hello.jl # verbose, build executable and deps\n
108+
\ua0\ua0juliac.jl -vae hello.jl prog.c # embed into user defined C program\n
109+
\ua0\ua0juliac.jl -qo hello.jl # quiet, build object file only\n
110+
\ua0\ua0juliac.jl -vosej hello.jl # build all and sync Julia libs\n
111+
"""
112+
113+
parsed_args = parse_args(args, s)
114+
115+
# TODO: in future it may be possible to broadcast dictionary indexing, see: https://discourse.julialang.org/t/accessing-multiple-values-of-a-dictionary/8648
116+
if !any(getindex.(parsed_args, ["clean", "object", "shared", "executable", "julialibs"]))
117+
parsed_args["quiet"] || println("Nothing to do, exiting\nTry \"$(basename(@__FILE__)) -h\" for more information")
118+
exit(0)
119+
end
120+
121+
kw_args = map(parsed_args) do kv
122+
Symbol(replace(kv[1], "-", "_")) => kv[2]
123+
end
124+
kw_args = filter((k, v)-> k != :juliaprog, kw_args)
125+
PackageCompiler.julia_compile(
126+
parsed_args["juliaprog"];
127+
kw_args...
128+
)
129+
end
130+
131+
main(ARGS)

0 commit comments

Comments
 (0)