In the previous exercise, you got your very own server that could serve static resources up and running. This is cool because you now have a stable, accessible, pretty legit chunk of computation that you can refer to and access whenever you want through the internet. You'll be able to use it as a relay point between multiple devices too since it is always going to live at a known location on the internet. What a time to be alive.

What we'd now like to do is set things up so that we essentially deploy code on this machine and call it and use it as needed. While any programming language could be used to do this, we'll go with Python since that is a language to which all of you should have some exposure. To be even more specific, we'll be using the Flask framework, which is widely used, stable, and well-documented¹.

Get Started

Assuming your server is set up in the server closet you should be able to ssh into it. Go ahead and first thing you'll need to do is install some dependencies:

sudo apt install python3-pip python3-dev build-essential libssl-dev libffi-dev python3-setuptools

Then let's do this to add python virtual environments to the machine:

sudo apt install python3-venv

Now make a new folder in your home directory. We're going to put all our Flask app stuff in here (don't worry in the future you can make other apps):

mkdir efi_test

Move into that directory with the cd command. In this directory we are going to make a Python Virtual Environment called efi_venv. A Virtual Environment is a cloned separate instance of your system-wide Python. The command is this:

cd efi_test
python3 -m venv efi_venv

Now in order to "activate" that Python virtual environment we use this command:

source efi_venv/bin/activate

If you did that right, you'll see (efi_venv) prepending your terminal lines. That means this particular environment of Python is active. If you run deactivate. It'll exit you out of here.

With this virtual environment activated, any Python-related task...such as running Python or installing libraries will affect this version of Python and not the big system-wide one.²

Let's install the Python libraries we need for Flask. First do this which helps us get more efficient installation packages:

pip3 install wheel

Then run this:

pip3 install uwsgi flask

Ok you should be good. If, from the terminal you run python, it'll bring up a Python shell. Inside there do : import flask. If nothing pukes back at you, you're golden. Exit Python with exit().

Let's write a basic Flask application now:

Basic Application

Make a new file inside this efi_test directory and call it efi_test.py. This will be your "main" Flask application for this assignment which would eventually contain a lot of code. For now it'll be short and sweet. Anything that interfaces with your application will just respond with "hey there!"

from flask import Flask
app = Flask(__name__)
app.debug=True #enable some debugging
from werkzeug.debug import DebuggedApplication #bring in debugging library
app.wsgi_app = DebuggedApplication(app.wsgi_app, True) #make this a debuggable application

@app.route("/") #anything coming in at the root of your application
def hello(): #unconditionally return "hey there!"
    return "hey there!"

if __name__ == "__main__":
    app.run(host='0.0.0.0')

Now we need some other files. Make another file called wsgi.py (in same directory of course). In it put this:

from efi_test import app

if __name__ == "__main__":
    app.run()

The wsgi.py file will be responsible for "starting" and linking your flask application to nginx via wsgi which stands for Web Server Gateway Interface, a standardized protocol for linking Python scripts to in/out bound network traffic.

Next we need to make an wsgi initilization file. Create a file called efi_test.ini and inside of it put the following:

[uwsgi]
module = wsgi:app
master = true
processes = 5
socket = efi_test.sock
chmod-socket = 660
vacuum = true
die-on-term = true

This file has some basic specifications that define the wsgi process that will be interfacing our Flask application to the other parts of the computer. Some of these could be futzed with and things will be about the same. Others are pretty critical. Don't worry about them too much for now. These are good starter values.

Now we could, if we wanted to, run our Flask application right now. However there is no way for nginx, which is our reverse proxy, to know to direct/link incoming/outgoing requests to that application. In order to fix that we need to update the nginx configuration file. To do this, using sudo, edit the file shown:

sudo vim /etc/nginx/sites-available/default

Inside of the server block do two things:

First change the one line so that it says server_name ERROR.

Then secondly, add in the set of lines below. What this will do is take any connections to the sub-domain ERROR/efi_test/ and pass them onto the flask application using uwsgi:

    location /efi_test {
        include uwsgi_params;
        uwsgi_pass unix:/home/None/efi_test/efi_test.sock;
        rewrite /efi_test/?(.*) /$1 break;
        include uwsgi_params;
    }

When done, save your file and then run:

sudo nginx -t

If nothing spit back that means you have no syntax errors in your file. Good job copy-pasting or typing. I'm proud of you. Then restart your nginx service.

sudo systemctl restart nginx.service

And that's that. If you then went to http://ERROR/efi_test you should see a 502 bad Gateway. This is actually good because it means Nginx is directing inbound traffic to an endpoint but there's nothing "there" yet. If you were to instead visit http://ERROR/mit you'd get a 404 Error (File not found) since Nginx is just assuming that under the /var/www/html/ file structure is an mit directory and there obviously isn't (since you didn't put one there). 502 is telling us that Nginx is actually routing correctly but that there's no other task present on the server to interface with the "gateway" to that entry point of /efi_test. We need to turn that on/fix this.

Turning it on can be as simple as running a Python command, however because of all the uwsgi interfacing we need to do as well as the setting of some environment variables, the command to do so is pretty long and annoying.

Also in reality we'd really like to just have this up and running always and not jamming up our terminal with a running Python process. So a smarter thing to do is put this in a systemd file and then turn it on and let it run forever "in the background". Then if we need to stop/restart we can just use simple commands rather than retyping or copy-pasting some initial run that has a bunch of annoying things.

So in order to do this, we need to make a systemd file that will allow us to auto-run our Flask application. This will essentially wrap all the annoying stuff into one file that we can call using simple commands.

Go and create this file (don't forget the sudo):

sudo vim /etc/systemd/system/efi_test.service

Into it place the following. Note this is basically just entering all the necessary startup lines of commands into one place.

[Unit]
Description=uWSGI instance to serve efi_test
After=network.target

[Service]
User=None
Group=www-data
WorkingDirectory=/home/None/efi_test
Environment="PATH=/home/None/efi_test/efi_venv/bin"
ExecStart=/home/None/efi_test/efi_venv/bin/uwsgi --ini efi_test.ini

[Install]
WantedBy=multi-user.target

Next do this which will start your flask application:

sudo systemctl start efi_test.service

and then, before we forget, do this so that the process will automatically start up on reboots.

sudo systemctl enable efi_test.service

Now if you go to http://ERROR/efi_test you should see some text that just says "hey there!"

Making Changes

Ok let's make small change to our Flask file. Go into efi_test.py and change the text that is returned from the hello function to be hello from None.

Save the file. Now in order to have the changes actually take place you'll need to restart your efi service. Do that with:

sudo systemctl restart efi_test.service

If you revisit http://ERROR/efi_test you should now see your good, new, altered text.

If you ran into an issue, you can always check if things are running by doing:

sudo systemctl status efi_test.service

Excellent! Run the checker below to verify that you have this working. Don't worry about what text you put in the box.

Debugging tips

Always restart your efi service after any change to the code

sudo systemctl restart efi_test.service

You can also run on your current application.

sudo systemctl status efi_test.service

This will actually provide you any information on any major startup-python bugs (like a missing library, a bad indent, etc...)

You can view access logs, which show the request that came in:

sudo cat /var/log/nginx/access.log

For run-time bugs with your application, which will hopefully be the bulk of your bugs, the lines in the example file:

app.debug=True #enable some debugging
from werkzeug.debug import DebuggedApplication #bring in debugging library
app.wsgi_app = DebuggedApplication(app.wsgi_app, True) #make this a debuggable application

Actually allow you to get a debug traceback in the browser. For example, the traceback below is happening in the next part when I'm trying to get query arguments and none are provided so Python is trying to convert a '' to an int and freaking out. Bugs like this won't "crash" the server but do result in errors. Setting up our application this way will allow you to immediately get this information back so you're not just wondering why you're getting a 502 error!

Note this is also really really unsecure and the fact we're doing this on live servers is really only because there's nothing secure on them. (think about it...broadcasting debug message to the outside world is like revealing your deepest weaknesses to anybody...not a good look). In a deployment server where it actually matters we should remember to turn debugging off!!

Loading...

Doing More With Flask

Flask is a full-fledged web framework. It is well-documented and widely used, so there's little reason for us to spend a bunch of time rewriting documentation...instead we should just go to the source: Flask Documentation and numerous other websites will go over effectively how to do things with it.

As a starter assignment, let's respond to incoming query arguments in our HTTP request. These are the things that show up after the ? in your URL. They generally show up as key-value pairs like http://internet.com?name=None&school=mit with individual pairs separated by ampersands.

You can use them to provide inputs a web service and that's exactly how we'll do that here. Dig through the Flask documentation and find a way to have a sub-url called heat_index peform an operation where it looks for two incoming query arguments named rh and t that should be numbers (ints or floats) and return the corresponding heat index (formula can be found here), rounded to closest integer.

For example going to: http://ERROR/efi_test/heat_index?rh=80&t=100 should yield 158 in the response. For other rh&t pairings, you can verify the appropriate heat index using this chart, but make sure t is at least 80 for the heat index to be valid.

Feel free to ask for help on Piazza and/or office hours! the solution to this should not be more than about ten lines of additional code...but if you can end up doing this...it will be very useful and you understand a good portion of Flask then!

When ready, submit the button below which will ping your site with some values to see if it works.

Loading...