NGINX rewrites on Amazon Elastic Beanstalk

We are running most of our backend code at Dishero on Amazon Elastic Beanstalk and it has been great so far. With Elastic Beanstalk we don’t need to worry about provisoning new instances and auto-scaling, in most cases it just allows us to upload our app and ELB takes care of the rest, however in some rare cases when we actually want to tinker with the internals and do some changes to the actual container it appears to be non-trivial and sometimes takes time to figure out.

Typical ELB setup

In a typical auto-scaled ELB setup, we have one elastic load balancer and multiple instances that execute our code. The setup is as follows:

  • Elastic Load Balancer is the one that is terminating the SSL connection, traffic to our instances is pure HTTP over port 80. You can find out more about setting up SSL certificates on Elastic Beanstalk here.
  • The instances are configured to forward all traffic from port 80 to 8080 using iptables.
  • Each instance has an NGINX running, which is listening on port 80880 and forwarding the traffic to our actual Node applicaiton.

The problem

We want to configure nginx to redirect all non-https traffic to https, and while we are at it to redirect all non-www traffic to www (i.e always push users to https://www.example.com/…).

  • It would be nice also to serve these as one redirect, taking a naive approach and writing two different rules (one for www and one for https) might result in two sequential redirects.
  • Since we are running the same configuration on multiple configurations which have different base URLs, we do not want to hardcode the actual URLs in the actual rulesm but rather keep them generic.

The solution

After several unsuccessfull iterations we arrived at the following set of rules:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
set $fixedWWW '';
set $needRedir 0;
# nginx does not allow nested if statements
# check and decide on adding www prefix
if ($host !~* ^www(.*)) {
set $fixedWWW 'www.';
set $needRedir 1;
}
# what about that https? the traffic is all http right now
# but elastic load balancer tells us about the original scheme
# using $http_x_forwarded_proto variable
if ($http_x_forwarded_proto != 'https') {
set $needRedir 1;
}
# ok, so whats the verdict, do we need to redirect?
if ($needRedir = 1) {
rewrite ^(.*) https://$fixedWWW$host$1 redirect;
}

So the question is where should we put it?

The file that configures nginx to proxy traffic from 8080 to the application in the Elastic Beanstalk environment is located at /etc/nginx/conf.d/00_elastic_beanstalk_proxy.conf

Obviously, SSH’ing to all the instances, modifying the file and restarting nginx manually is of no use, it will get overwritten next time the app is deployed and newly deployed instances won’t have the changes either.
Luckily for us Beanstalk allows us to customize the EC2 environment it provisions using configuration files, that system is pretty flexible and allows not only to install yum packages, write and overwrite files in the system, but to run commands and shell scripts during app deployments as well.

We may be tempted to use the config files to overwrite the 00_elastic_beanstalk_proxy.conf file in /etc/nginx/conf.d directly, and then wonder where are our changes and why they are nowhere to be seen in the system. Actually it might work well if all we want is add new nginx configuration files, but the issue is with existing nginx files, during the deployment process, the customization stage happens before nginx default files are being installed into their by the Elastic Beanstalk system, so even if we set up our copy of 00_elastic_beanstalk_proxy.conf, moments later it will still be overwritten with the default one. We need to overwrite that default one instead, and the source location of these is /tmp/deployment/config/, the one we are mostly interested in is suprisingly named #etc#nginx#conf.d#00_elastic_beanstalk_proxy.conf

So eventually, after all the trial and error, the solution appears to be quite simple, the one thing that needs to be added to our project is the following nginx.config file inside our .ebextensions folder:

.ebextensions/nginx.config
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
files:
"/tmp/deployment/config/#etc#nginx#conf.d#00_elastic_beanstalk_proxy.conf" :
mode: "000755"
owner: root
group: root
content: |
upstream nodejs {
server 127.0.0.1:8081;
keepalive 256;
}
server {
listen 8080;
set $fixedWWW '';
set $needRedir 0;
# nginx does not allow nested if statements
# check and decide on adding www prefix
if ($host !~* ^www(.*)) {
set $fixedWWW 'www.';
set $needRedir 1;
}
# what about that https? the traffic is all http right now
# but elastic load balancer tells us about the original scheme
# using $http_x_forwarded_proto variable
if ($http_x_forwarded_proto != 'https') {
set $needRedir 1;
}
# ok, so whats the verdict, do we need to redirect?
if ($needRedir = 1) {
rewrite ^(.*) https://$fixedWWW$host$1 redirect;
}
location / {
proxy_pass http://nodejs;
proxy_set_header Connection "";
proxy_http_version 1.1;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
gzip on;
}