How to Set Up a Chainlink Node on AWS (WIP)

February 8, 2020 - Work in Progress

I’m writing this guide so that you can set up a Chainlink node. This guide is not complete. It’s a good idea to read this guide and other references too in order to understand what you are doing. Always check the official Chainlink documentation.

“Solidity development tools have reached a very high level of quality, and now you have great data. So, if before there was something stopping you from building a decentralized financial product that did something useful, I think the reality is that people who have realized that that limitation is gone or going away will build the products that do the defi things that come to revolutionize this space.” — Sergey Nazarov, Chainlink CEO

If you’ve already completed my guide on setting up a test node, then much of this will be familiar to you. I recommend you complete that guide for the background information.

Here are some useful resources that might help you:

My guide on making a Chainlink Test Node with AWS, Fiews Ethereum Service Provider, and the official Chainlink Docker.

How to Create a Testnet Chainlink Node (pdf)

Raymond Mogg’s node tutorial may be useful to you if you want to use Google Cloud and Kubernetes.

DevlinTrace’s ETH and LINK node guides

Video: Creating a TestNet Chainlink Node on Google Cloud

Video: Chainlink Node & Database Setup

If you have never used a terminal then it’s very important that you first complete a test node guide linked above. If you are comfortable with these concepts then start.

The ideal thing to do is to first create a testnet node and identify possible problems by running this testnet node for some time. Then, after testing, create the mainnet node. Mainnet nodes risk real tokens worth real money, and mainnet nodes require great reputation and uptime.

You don’t have to use AWS (Amazon Web Services). There are many ways to create a VM instance that will work. If you can get a node up and running on AWS, you can probably (with minor changes) also get one on another cloud service provider.

Start and setup two ec2 instances with AWS. On each instance, set up a node.

Node 1

First create a VM instance on AWS. Log in to AWS (create an account if you haven’t already, but remember that it may take some time for your account to initialize.) Go to the EC2 dashboard, and click launch instance. You can choose which data center your instance is located from at the top right of the page.

Choose the Amazon Linux 2 AMI (HVM), SSD Volume Type -64-bit x86

You need to choose how your node gets the Ethereum blockchain. We are going to use EaaS in this guide.

You can store the blockchain yourself (geth, parity) or use EaaS (Ethereum as a service) to access blockchain information. If you store it yourself, you have to manage it and pay for the storage. If you access it on a service, you have to pay for the service. Fiews is an EaaS which is already very Chainlink friendly. There’s also EaaS providers Infura and the well-known LinkPool.

If you want to have your own copy of the ethereum blockchain, then you have to factor that into your VM setup! It’s best to make a VM separately for the ethereum blockchain. For an eth testnet, it’s recommended that you use a t3a.large or t3.large AWS instance with at least 200GB EBS gp2 volume. For eth mainnet, the recommendations are c5.xlarge or m5.xlarge and at least 500GB EBS volume.

For running Chainlink only while using EaaS, the VM doesn’t have to be quite as large. Chainlink is fairly light usage by itself. You could use t2.small. This guide uses a t3.micro with some added storage.

On the next page, configure instance details. These are left alone unless you decide to opt for more expensive options.

On the next page, increase the storage size (20 GB).

On the page after that, add a tag that you remember.

Add a security rule so that only you can SSH into your instance. Since you’re the only one who is changing your node setup and settings anyway, the security rule is set up to allow only your IP address to protect your node. (Note: If you launch your SSH window and get no response, you can edit the security group inbound rules to allow different or all IPs.)

After that, review, and then launch your VM instance. The key pair window is next. The key pair lets you SSH into your instance from your desktop. Name and create a new key pair and it downloads to your desktop.

Access your VM with SSH. In this guide I’ll use PuTTY.

Once PuTTY is installed, search for PuTTYgen, a separate app. Open PuTTYgen and load in the key file that you just downloaded. You have to browse for all files, not just the default PuTTY key files. A warning window might appear about old key formats. Read and close that, then save the private key with RSA selected. You can set a passphrase if you want for more security.

Open PuTTY and find the Auth menu under SSH. Browse for the PuTTY key you saved. Go back to the Session menu and enter the host name of the instance. Find the host name in the VM description area on AWS, and paste that into the host name field in PuTTY. Since you have loaded the keyfile already, you can save this PuTTY setup to save time connecting in the future. Enter a name in the saved sessions field, and click save. Next time you want to connect, you can click on the name you saved in the list, and then load it.

Click open and start a connection with the VM. Check for any update messages or password messages, complete those, and then continue. Logging in as ec2-user is usually what is required for AWS.

Make a new directory for Chainlink.

mkdir ~/.chainlink

Make an environment file.

echo "ROOT=/chainlink
LOG_LEVEL=debug
ETH_CHAIN_ID=1
CHAINLINK_TLS_PORT=0
SECURE_COOKIES=false
ALLOW_ORIGINS=*" > ~/.chainlink/.env

The log level is set to “debug”. This is a good time to point out that you don’t want logs filling up your storage, so you should add a log rotate process or something to take care of the logs that will, over time, take storage space.

Assuming you use EaaS:

echo "ETH_URL=Put your ethereum url here" >> ~/.chainlink/.env

In this guide I’m using Fiews EaaS. Signing up for a Fiews account is very easy and even offers free trials for Chainlink nodes on testnet. To use Fiews on mainnet, sign up for a plan. The low plan at this time is $5/month. You could also choose LinkPool’s free mainnet RPC which comes with it’s own limitations.

Set Up a Separate Database

This section and the following sections are work-in-progress. There could be errors.

Now as noted in Chainlink Docs, your node uses ̶s̶q̶l̶i̶t̶e̶3̶ database by default. (Actually, after version 0.8.0, postgreSQL is the default, sqlite3 is no longer used.) You need to use a dedicated database instead. You can use a postgreSQL db.

Start one on AWS and get your username, password, server, port, and database name information. There are multiple ways to do this. You could create another EC2 instance for just the database, or you could use AWS RDS to make a postgresql db, or use AWS RDS managed db Aurora with postgresql settings. RDS is going to give you a very capable database, which may be overkill for your project.

For a postgresql database on an EC2:

sudo amazon-linux-extras install postgresql10 vim epel -y
sudo yum install -y postgresql-server postgresql-devel
sudo /usr/bin/postgresql-setup --initdb

Start up postgres.

sudo systemctl enable postgresql
sudo systemctl start postgresql

Log in to postgres, and postgres console.

su - postgres

To set the password for the user:postgres (enter an incorrect password, or press ctrl-z):

sudo -u postgres psql postgres # \password postgres

Once you have set your password, you are inside postgres (ctrl-z to stop and leave).

su - postgres

sudo -u postgres createuser -s ec2-user
sudo -u postgres createdb ec2-user
psql

In psql, type

\l

to view databases.

Change the password for ec2-user. Exit postgres if you haven’t already with \q.

psql ec2-user
ec2-user-# \password

Leave the postgres area by typing \q and hit enter.

To easily edit the environment file to add our database information, check that nano editor is installed and use it. Press ctrl-z first to leave psql.

sudo yum install nano

then do

nano ~/.chainlink/.env

You should see the new ETH_URL (or you can use nano to edit it).

Add the following lines to the environment file.

DATABASE_URL=postgresql://$USERNAME:$PASSWORD@$SERVER:$PORT/$DATABASE
DATABASE_TIMEOUT=0

So your file should look like (watch out for extra spaces):

ROOT=/chainlink
LOG_LEVEL=debug
ETH_CHAIN_ID=1
CHAINLINK_TLS_PORT=0
SECURE_COOKIES=false
ALLOW_ORIGINS=*
ETH_URL=wss://cl-main.fiews.io/v1/lL571Hld1...-YOUR ETH URL
DATABASE_URL=postgresql://ec2-user:YOURPASSWORD@YOUREC2PUBLICIP ADDR:5432/YOURDATABASENAME
DATABASE_TIMEOUT=0

Default port for postgresql is 5432. Your database name is probably ec2-user if you followed the steps above.

Add a security rule for postgresql, port 5432.

Control + O = Save File (hit enter), Control + X = Exit nano editor

Install Chainlink

Close and then reopen the server session. Then install Chainlink’s Docker.

sudo yum update -y
sudo amazon-linux-extras install docker
sudo service docker start
sudo usermod -a -G docker ec2-user
cd ~/.chainlink
sudo docker run --name chainlink -p 6688:6688 -v ~/.chainlink:/chainlink -it --env-file=.env smartcontract/chainlink local n

Now run the following lines in order to setup automatic restarting. If your node stops right now, it requires you to log in again which ruins uptime.

echo "user@example.com" > ~/.chainlink/.api
 
echo "password" >> ~/.chainlink/.api
 
echo "my_wallet_password" > ~/.chainlink/.password

"user@example.com" is the email you used for the chainlink dashboard when you created the node earlier.

replace "password" with the dashboard password you set

"my_wallet_password" is the wallet password of your node

The following deletes the chainlink docker so that we can replace it with one that always restarts by using the user/password information you just entered.

docker rm chainlink
cd ~/.chainlink && docker run --restart always --name chainlink -p 6688:6688 -v ~/.chainlink:/chainlink -it --env-file=.env smartcontract/chainlink local n -p /chainlink/.password -a /chainlink/.api

Check it with

docker container ls

Node 2

Redo this process and make another node instance. If your main node goes down it needs a backup.

It’s recommended to make this node in a different zone. For instance, if node 1 is in AWS Oregon (us-west-2) then make node 2 in a different zone like AWS Ohio (us-east-2).

The next part is about how to make it switch over to the backup in a failure case.

Testing Failover Between Nodes

At the end of making node 2, when you run the chainlink docker with restart options, is when you set up the backup stuff.

Go back to node 1 and typedocker attach chainlink

and nothing happens, because node 2 should still be running. Stop node 2 and node 1 will start. Test some more to make sure that each node will start when the other one stops.

Access the Chainlink http://localhost:6688 gui. Create an ssh tunnel:

ssh -i $KEY $USER@$REMOTE-IP -L 6688:localhost:6688 -N

where the values above refer to the key, username, and IP for the AWS instance. This is a reminder to use security rules to only allow your IP to login with terminal. If this is unfamiliar to you refer to my Test Node article.

Running Chainlink Jobs

Todo: In the next part, I’ll discuss running Chainlink jobs.

Running Your Node

Todo: Add this section.

Right now, the Chainlink team has a large amount of requests to add nodes. Instead, list your mainnet node on third party services like market.link, which is LinkPool market for nodes/jobs.