Installation on Server

Published: Monday, December 9th, 2013 by Chris Zieba


The following installation is for a Ubuntu 12.04.3 LTS (64 bit) VPS. All configuration files attached are examples.

This installation will also work for a 32 bit OS. The only change is in the Install LogicPull section, where node_modules needs to be rebuilt using the command npm install.

There are many ways in which LogicPull can be installed on a server. Nginx now supports web sockets (HTTP 1.1) so another setup is to just install nginx to handle SSL decryption and any web socket traffic. Stunnel is a great tool but does not have the ability to add a X-FORWARD- header, so we are stuck with localhost as the client IP address in our app. This is not a big deal, unless you plan to use the client IP for brute checks, etc.

Getting Started

For the purposes of this guide, we'll assume that you'll be using the following packages and operating system:

Install Packages

Graphviz is used to tidy the graph in the editor, and to display a progress during the interview (e.g. 4 of 212). If you plan on disabling these, you do not need to install Graphviz. Git is another package that is not absolutely necessary, but makes keeping LogicPull up-to-date really easy.

		sudo apt-get update
sudo apt-get upgrade
sudo apt-get install make build-essential libssl-dev g++ git nginx varnish mongodb-server stunnel4 graphviz pdftk openjdk-7-jdk

Install NodeJS

The version of NodeJS available in the Official Ubuntu Package Repository does not meet the minimum version required to run all the features of LogicPull, so it is built from source. More information on how to install nodejs.

		sudo su
cd /opt
wget http://nodejs.org/dist/v0.8.8/node-v0.8.8.tar.gz
tar -zxf node-v0.8.8.tar.gz
cd node-v0.8.8
./configure && make && sudo make install
exit

Install Forever

Forever is a CLI tool that ensures the LogicPull instance is run continuously (forever). This is installed as a global node module. If you prefer, you can install forever as part of the LogicPull project. More information is available here.

		sudo npm install forever -g
	

Install Apache FOP

Apache Fop does not have package in the Ubuntu Repo, so its binary is downloaded directly. More information on how to install Apache FOP.

sudo su
wget http://apache.mirror.nexicom.net/xmlgraphics/fop/binaries/fop-1.1-bin.zip
unzip fop-1.1-bin.zip -d /opt/
exit

Make sure that Apache FOP has the correct permissions to execute.

sudo chmod a+x /opt/fop-1.1/fop

Install LogicPull

The source code for LogicPull is located on github. You can install LogicPull in any directory you want, but for the purpose of this guide, we will install it in /srv/www/LogicPull/. If you choose another directory, you will need to make sure you use the correct directory in the varnish, and nginx configuration. Also, make sure that the folder /srv/www/ exists!

Download the source using github. You can also download the zip file directly.

cd /srv/www
git clone https://github.com/ChrisZieba/LogicPull.git

Create the needed directories.

cd /srv/www/LogicPull
mkdir generated uploads logs generated/output generated/answers generated/tmp uploads/deliverables logs/forever public/javascripts/preload

Make sure the directories have the correct permissions.

sudo chmod -R 755 public/

Initialize the log files.

touch logs/forever/outfile logs/forever/logfile logs/forever/errfile
chmod a+w logs/forever/errfile logs/forever/logfile logs/forever/outfile

Configuration

The configuration files provided below are examples only. You will need to make sure you use the correct ports, domains, SSL certificates, etc.

MongoDB

Start the mongodb instance

sudo service mongodb start

Stunnel

Stunnel is optional, and only needed if you plan on using SSL in your app. You can skip over this step if have opted to leave Stunnel out of the installation, but note that you will have to make adjustments to your default varnish configuration.

Stunnel listens on port 443 and sends decrypted traffic to Varnish. Any files with the extension .conf located in /etc/stunnel/ are used by stunnel, so you can use the below example, changing the location of the .pem file, and the IP, and save the file as /etc/stunnel/https.conf. If you are not using SSL, then you can skip over this step. A full example of the config file is included.

Be sure to change the cert and accept settings. For more documentation on stunnel, please see here.

File: /etc/stunnel/https.conf

debug = 5 
output = /var/log/stunnel4/stunnel.log 

[logicpull] 
cert = /etc/ssl/stunnel.pem 
accept  = 66.135.211.57:443 
connect = 80 

Make sure the tunnels start up automatically on system boot.

File excerpt: /etc/default/stunnel4

ENABLED 1

You can generate a self signed certificate if you plan to run LogicPull on a private network, or don't mind the warnings browsers give for unverified certificates.

cd /etc/ssl/
openssl genrsa -out stunnel.pem
openssl req -new -x509 -key stunnel.pem -out cert.pem -days 1095
cat cert.pem >> stunnel.pem

Make sure the pem file has the correct permissions.

chmod 600 stunnel.pem

Varnish

Varnish will serve the content on port 80, so change line 43 in /etc/default/varnish to port 80, or to your own specifications. If using a port other than 80, make sure to change the Stunnel configuration also. More documentation on Varnish.

File excerpt: /etc/default/varnish

DAEMON_OPTS="-a :80 \ 
	-T localhost:6082 \ 
	-f /etc/varnish/default.vcl \ 
	-S /etc/varnish/secret \ 
	-s malloc,256m" 

Set the default configuration settings for Varnish. Below is an example of a full config file.

Be sure to change the occurrences of example.com to the domain you are using for you application.

File: /etc/varnish/default.vcl

# This is a basic VCL configuration file for varnish.  See the vcl(7)
# man page for details on VCL syntax and semantics.

# Default backend definition.  Set this to point to nginx
backend default {
    .host = "127.0.0.1";
    .port = "81";
    .connect_timeout = 5s;
    .first_byte_timeout = 30s;
    .between_bytes_timeout = 60s;
    .max_connections = 800;
}

backend nodejs {
    .host = "127.0.0.1";
    .port = "3000";
    .connect_timeout = 1s;
    .first_byte_timeout = 2s;
    .between_bytes_timeout = 60s;
    .max_connections = 800;
}

sub vcl_recv {
    set req.backend = default;
    set req.grace = 120s;

    # Redirect to SSL
    if (req.http.Host == "example.com" && client.ip != "127.0.0.1") {
       set req.http.redirect-location = "https://example.com" + req.url;
       error 750 req.http.redirect-location;
    }

    # Set the correct IP so my backends don’t log all requests as coming from Varnish
    if (req.restarts == 0) {
        if (req.http.x-forwarded-for) {
            set req.http.X-Forwarded-For = req.http.X-Forwarded-For + ", " + client.ip;
        } else {
            set req.http.X-Forwarded-For = client.ip;
        }
    }
    
    # remove port, so that hostname is normalized
    set req.http.Host = regsub(req.http.Host, ":[0-9]+", "");
    
    #part of Varnish’s default config
    if (req.request != "GET" &&
        req.request != "HEAD" &&
        req.request != "PUT" &&
        req.request != "POST" &&
        req.request != "TRACE" &&
        req.request != "OPTIONS" &&
        req.request != "DELETE") {
        /* Non-RFC2616 or CONNECT which is weird. */
        return (pipe);
    }

    if (req.request != "GET" && req.request != "HEAD") {
        return (pass);
    }
    
    # pipe websocket connections directly to Node.js
    if (req.http.Upgrade ~ "(?i)websocket") {
        set req.backend = nodejs;
        return (pipe);
    }
    
    # do not cache large static files
    if (req.url ~ "\.(avi|flv|mp(e?)g|mp4|mp3|gz|tgz|bz2|tbz|ogg)$") {
        return(pass);
    }
    
    if (req.http.Host ~"^(www\.)?example.com"){
        # Requests made to this path, relate to websockets - pass does not seem to work (even for XHR polling)
        if (req.url ~ "^/socket.io/") {
            set req.backend = nodejs;
            return (pipe);
        }
    }

    # part of Varnish’s default config
    if (req.http.Authorization || req.http.Cookie) {
        /* Not cacheable by default */
        return (pass);
    }
    return (lookup);
}

sub vcl_pipe {
    # we need to copy the upgrade header
    if (req.http.upgrade) {
        set bereq.http.upgrade = req.http.upgrade;
    }
    #closing the connection is necessary for some applications – I haven’t had any issues with websockets keeping the line below uncommented
    #set bereq.http.Connection = "close";
    return (pipe);
}

sub vcl_pass {
    return (pass);
}
 
sub vcl_hash {
    hash_data(req.url);
    if (req.http.host) {
        hash_data(req.http.host);
    } else {
        hash_data(server.ip);
    }
    return (hash);
}
 
sub vcl_hit {
    return (deliver);
}
 
sub vcl_miss {
    return (fetch);
}
 
sub vcl_fetch {
    if (beresp.ttl <= 0s ||
        beresp.http.Set-Cookie ||
        beresp.http.Vary == "*") {
		/*
		 * Mark as "Hit-For-Pass" for the next 2 minutes
		 */
		set beresp.ttl = 120 s;
		return (hit_for_pass);
    }
    return (deliver);
}
 
sub vcl_deliver {
    return (deliver);
}
 
sub vcl_error {
    if (obj.status == 750) {
        set obj.http.Location = obj.response;
        set obj.status = 302;
        return (deliver);
    }

    set obj.http.Content-Type = "text/html; charset=utf-8";
    set obj.http.Retry-After = "5";
    synthetic {""};
    return (deliver);
}
 
sub vcl_init {
 	return (ok);
}

sub vcl_fini {
	return (ok);
}

NginX

Nginx will run on port 81, instead of the more commonly used port 80 because Varnish is running as the reverse proxy on port 80. A full copy of the example configuration file is below. There is plenty of documentation available at nginx.

Create the virtual host file for the web application.

Be sure to change the occurrences of example.com to the domain you are using for you application. Also note that root of our application was installed in /srv/www/LogicPull/. If you installed LogicPull in a different directory then you must change the configuration accordingly.

File: /etc/nginx/sites-available/logicpull.com

#Sample config file for nginx 
#Varnish handles any SSL redirection

upstream nodejs {
    server 127.0.0.1:3000;
    server 127.0.0.1:3000;
}
 
server {
    listen 127.0.0.1:81;
    charset utf-8;
    server_name example.com;
    port_in_redirect off;

    access_log /srv/www/LogicPull/logs/access.log;
    error_log /srv/www/LogicPull/logs/error.log info;
    root /srv/www/LogicPull/public;

    gzip on;
    gzip_vary on;
    gzip_comp_level 6;
    gzip_proxied any;
    gzip_disable "msie6";
    gzip_types text/plain text/css application/json application/x-javascript text/xml application/xml application/xml+rss text/javascript text/x-javascript application/javascript;

    expires -1;

    location = / {
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header Host $http_host;
        proxy_set_header X-NginX-Proxy true;
        proxy_pass http://nodejs;
        proxy_redirect off;
    }

    location / {
        location ~* \.(jpg|jpeg|rtf|json|png|gif|css|js|swf|flv|ico)$ {
            expires max;
            access_log off;
            log_not_found off;
        }
        try_files $uri $uri/ @proxy;     
    }   

    location @proxy {
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header Host $http_host;
        proxy_set_header X-NginX-Proxy true;
        proxy_pass http://nodejs;
        proxy_redirect off;
    } 
}

The site needs to be enabled.

sudo ln -sf /etc/nginx/sites-available/logicpull.com /etc/nginx/sites-enabled/logicpull.com

LogicPull

The mongo database needs to be initialized. This will add an admin user, and set up all the tables in the LogicPull database.

mongo LogicPull /srv/www/LogicPull/bin/db/init.js

To avoid any conflict when pulling updates, make sure git does not track files that are likely to be modified.

cd /srv/www/LogicPull
git update-index --assume-unchanged public/robots.txt
git update-index --assume-unchanged public/sitemap.xml
git update-index --assume-unchanged config.js
git update-index --assume-unchanged Gruntfile.js
cd node_modules/
git update-index --assume-unchanged $(git ls-files | tr '\n' ' ')

If you are installing on a 32 bit system you will need to run npm install to rebuild the node modules.

cd /srv/www/LogicPull
rm -rf node_modules/
npm install

LogicPull comes with a configuration file located in config.js. Each line is commented to provide a description of its purpose. Make sure you set the correct settings based on the environment you plan to run in. The following settings must be set to run LogicPull:.

The LogicPull application is now ready to run, but before we start the app, we need to reload some services

sudo service nginx restart
sudo service varnish restart
sudo /etc/init.d/stunnel4 restart

Now we can start LogicPull with the following command. Make sure that the log files referenced below exist and are writable. More information on how to use forever is available here.

NODE_ENV=production forever -a -l /srv/www/LogicPull/logs/forever/logfile -o /srv/www/LogicPull/logs/forever/outfile -e /srv/www/LogicPull/logs/forever/errfile start /srv/www/LogicPull/LogicPull.js