Setting Up an Erlang Cluster on EC2
Generating a working release to deploy to EC2 required a fair bit of
trial and error. This post assumes that you have a set of working OTP
applications that you would like to deploy on multiple machines. Let’s
call these applications
I like to organize my files and folders as follows:
1 2 3 4 5 6 7 8
All applications go in the
apps folder. Release specific files,
configuration, and the bundled release itself go in the
rebar.config at the top level looks like the following:
1 2 3 4 5 6 7 8
With this config file, we can run
rebar compile at the root level of
our project and it will automatically compile every application.
Generating the release
The release will bundle together your applications along with any dependencies necessary to run them on a similar system. It will also bundle in the Erlang RunTime System (ERTS) so that the release executables can be run without Erlang installed on the target machine. Make sure, however, that the build machine and the target machine are of similar (if not identical) architectures.
First, you’ll want to use
to generate a set of files, among which is the all important
reltool.config file. You can do this by running in the
rebar create-node nodeid=example, substituting
for your project name or whatever you want. Then modify it to look
like the following:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42
As you can see, we’ve added a line that looks like
for every application we want included and we’ve also included the app
name in the
rel option. These applications will all be
executed on start.
Configure App Config
By default the node name is set to
email@example.com but we probably
want to strip the attached ip address. To change this edit the
flag passed in
Any application specific configuration should happen in
rel/files/sys.config. This file also gets copied into the release
bundle (you can see all the file and directory manipulation in the
overlay option of the
reltool.config file). For now, make the
sys.config file look like
1 2 3 4 5 6
kernel options will restrict the port range for inter-node
communication. This is important if you plan on deploying on instances
that will be part of a security group or behind a firewall. You can
put other application configuration variables in this file as well.
Generate the release and deploy.
Now, generate the release by running
Note that you may need to run
rebar get-deps first if your
applications have any dependencies. This will bundle your
applications, ERTS, dependencies, configuration files, and arguments
You can test it out by running
./rel/example/bin/example start and a
ps aux | grep example should show all the processes now
running attached to your project.
To deploy to EC2, you need to make sure that all instances in the cluster have the following ports open:
Port 4369 is used by the Erlang Port Mapper Daemon (epmd). Finally, bundle up your release and run it on all your instances. To connect your instances:
You should get a
pong as a response if you did everything correctly.
Further down the road
To get this example configuration working, many things were done manually. Steps that can (and should) be automated are the generation, distribution, and deployment of each release. In addition, we can automate adding nodes to a cluster by including a ping as part of our bundled application or by making a custom boot script (or by running a separate script after the deploy).
I struggled a lot to get a working configuration as the use of rebar and reltool is not obvious (at least not to me). Hope this helps somebody.