Frontend¶

The frontend is provided by Python code using the Flask microframework. This displays the web interface, allowing a user to upload their input files, start a job, display a list of all jobs in the system, and get back job results. This Python code uses utility classes and functions in the saliweb.frontend module, together with functionality provided by Flask.

Initialization¶

The first step is to create a Python module that creates the web application itself, using the saliweb.frontend.make_application() function (which in turn creates a Flask application and configures it). For a web service ‘ModFoo’ this should be done in the Python module frontend/modfoo/__init__.py. This is fairly standard boilerplate:

import flask
import saliweb.frontend


parameters = [...]
app = saliweb.frontend.make_application(__name__, parameters)

The ‘parameters’ object here is a list of parameters the job requires at submission time, and will be described later.

Web page layout¶

Each web page has a similar layout (header, footer, links, and so on). This is provided by a system-wide Jinja2 template called saliweb/layout.html, which can be seen at GitHub. This system-wide template should be overriden for each web service, by providing a file frontend/modfoo/templates/layout.html that ‘extends’ the template:

{% extends "saliweb/layout.html" %}

{%- block css %}
<link rel="stylesheet" type="text/css"
 href="{{ url_for("static", filename='modfoo.css') }}" />
{%- endblock %}

{% block navigation %}
{{ get_navigation_links(
       [(url_for("index"), "ModFoo Home"),
        (url_for("job"), "Current ModFoo queue"),
        (url_for("help"), "Help"),
        (url_for("contact"), "Contact")])
}}
{% endblock %}

{% block sidebar %}
<p><i>Version <a href="https://github.com/salilab/modfoo">{{ config.VERSION }}</a></i></p>
{% endblock %}

{% block footer %}
<p>
Please cite Bob et al., JMB 2008.
</p>
{% endblock %}

This example demonstrates a few Jinja2 and Flask features:

Parts of the base template can be overriden using the block directive. Here, a custom stylesheet is added (by overriding the css block), links are added to all pages to the navigation bar at the top of the page (navigation block), and the sidebar and footer (at the left and bottom of the page, which are blank in the system-wide template) are filled in.

Links to other parts of the web service can be provided using Flask’s url_for function. This takes either the name of the Python function that renders the page (such as “index”; see below) or “static” to point to static files (such as stylesheets or images) in the web service’s html subdirectory.

A number of global variables are available which can be substituted in using Jinja2’s {{ }} syntax, most notably config which stores web service configuration, such as the name and version in config.SERVICE_NAME and config.VERSION respectively.

Some helper functions are available. Here the get_navigation_links function is used, which takes a list of (URL, description) pairs.

See the Jinja2 manual for more information on Jinja2 templates.

Displaying standard pages¶

The bulk of the functionality of the frontend is implemented by providing Python functions for each page using Flask routes.

For a typical web service, the index, submission, results and results file pages need to be implemented by providing index, job, results, and results_file functions, respectively. These pages will be considered in turn.

Note

Additional pages (such as page to download the software, or a help page) can be simply implemented by adding more Python functions with appropriate routes (and, if appropriate, adding links to these pages to layout.html). See below.

Index page¶

The index page is the first page seen when using the web service, and typically displays a form allowing the user to set parameters and upload input files. (In more complex web services this first form can lead to further forms for advanced options, etc.) The Python code for this is straightforward - it simply defines an index function which uses the Flask render_template function to render a Jinja2 template, and is then decorated using app.route to tell Flask to use this function to service the ‘/’ URL:

@app.route('/')
def index():
    return flask.render_template('index.html')

The Jinja2 template in turn looks like:

{% extends "layout.html" %}

{% block body %}
<form method="post" action="{{ url_for("job") }}"
 enctype="multipart/form-data" name="modfooform">

  <p>Job name (optional)</p>
  <input type="text" name="job_name" />

  <p>Email address (optional)</p>
  <input type="text" name="email" value="{{ g.user.email }}" />

  <p>Upload PDB file</p>
  <input type="file" name="input_pdb" />
</form>
{% endblock %}

The template extends the previously-defined layout.html so will get the sidebar, footer, etc. defined there.

The form is set up to submit to the submission page (url_for("job")). It allows the user to upload a single PDB file, as well as pick an optional name for their job and an optional email address to be notified when the job completes.

Note that the email address is filled in using g.user.email. (g is used by Flask to store global data.) If the user is logged in to the webserver, g.user will be a LoggedInUser object, and their email address will be available for use in this fashion (otherwise, the user will simply have to input a suitable address if they want to be notified). Similarly, g.user.modeller_key provides the MODELLER license key of the logged-in user.

The names of the form parameters above (job_name, input_pdb) should also be described in the Python code frontend/modloop/__init__.py for automated use of the service (see Automated use), by passing suitable Parameter and/or FileParameter objects when the application is created. Note that email is omitted because automated usage typically does not use email notification:

import flask
import saliweb.frontend
from saliweb.frontend import Parameter, FileParameter


parameters = [Parameter("job_name", "Job name", optional=True),
              FileParameter("input_pdb", "PDB file to be refined")]
app = saliweb.frontend.make_application(__name__, parameters)

Submission page¶

The submission page is called when the user has input all the information needed for the job. It is implemented by defining the job function which handles the /job URL. This serves double duty - an HTTP POST to this URL will submit a new job, while a GET will show all jobs in the system (the queue).

Showing the queue is handled by the saliweb.frontend.render_queue_page() function. Job submission should validate the provided information, then actually submit the job to the backend. If validation fails, it should throw an InputValidationError exception; this will be handled by the web framework as a message to the user asking them to fix the problem and resubmit. (If some kind of internal error occurs, such as a file write failure, in this or any other method, the user will see an error page and the server admin will be notified by email to fix the problem.)

To submit the job, first create a saliweb.frontend.IncomingJob object, which also creates a new directory for the job files. Put all necessary input files in that directory, for example using IncomingJob.get_path(), then actually run the job by calling IncomingJob.submit().

Note

‘Input files’ include PDB files, parameter files, etc. but not shell scripts, Python scripts, or executables. These should never be generated by the frontend, but instead should be generated by the backend. If these files are generated by the frontend, it is very easy for an unscrupulous end user to hack into the cluster by compromising the web service. The backend should always check inputs provided by the frontend (e.g. in the preprocess() method) for sanity before running anything.

Note

When taking files as input, you can simply write them into the job directory using their save method. But never trust the filename provided by the user! Ideally, save with a fixed generic name (e.g. input.pdb). If this is not possible, use the secure_filename function to get a safe version of the filename.

Finally, the submission page should inform the user of the results URL (IncomingJob.results_url), so that they can obtain the results when the job finishes. This uses the function saliweb.frontend.render_submit_template(), which will either display an HTML page (similarly to Flask’s render_template, as before) or XML in the case of automated usage.

The example below reads in the PDB file provided by the user on the index page, checks to make sure it contains at least one ATOM record, then writes it into the job directory and finally submits the job:

@app.route('/job', methods=['GET', 'POST'])
def job():
    if flask.request.method == 'GET':
        return saliweb.frontend.render_queue_page()
    else:
        return submit_new_job()


def submit_new_job():
    # Get form parameters
    input_pdb = flask.request.files.get('input_pdb')
    job_name = flask.request.form.get('job_name') or 'job'
    email = flask.request.form.get('email')

    # Validate input
    file_contents = input_pdb.readlines()
    atoms = 0
    for line in file_contents:
        if line.startswith('ATOM  '):
            atoms += 1
    if atoms == 0:
        raise saliweb.frontend.InputValidationError(
                   "PDB file contains no ATOM records!")

    # Create job directory, add input files, then submit the job
    job = saliweb.frontend.IncomingJob(job_name)

    with open(job.get_path('input.pdb'), 'w') as fh:
        fh.writelines(file_contents)

    job.submit(email)

    # Inform the user of the job name and results URL
    return saliweb.frontend.render_submit_template(
        'submit.html', email=email, job=job)

It uses the following template as submit.html, providing the job and email variables, to notify the user:

{% extends "layout.html" %}

{% block body %}
<p>Your job {{ job.name }} has been submitted.</p>

<p>Results will be found at <a href="{{ job.results_url }}">this link</a>.</p>

{%- if email %}
<p>You will be notified at {{ email }} when job results are available.</p>
{%- endif %}
{% endblock %}

Results page¶

The results page is used to display the results of a job, and is implemented by providing a results function. The function can either display the job results directly, or it can display links to allow output files to be downloaded. In the latter case, URLs to these files can be generated by calling CompletedJob.get_results_file_url().

The example below assumes the backend generates a single output file on success, output.pdb, and a log file, log, on failure. It uses the utility function saliweb.frontend.get_completed_job() to get a CompletedJob object from the URL (this will show an error message if the job has not completed, or the password is invalid) and then looks for files in the job directory using the CompletedJob.get_path() method:

@app.route('/job/<name>')
def results(name):
    job = saliweb.frontend.get_completed_job(name,
                                             flask.request.args.get('passwd'))
    # Determine whether the job completed successfully
    if os.path.exists(job.get_path('output.pdb')):
        template = 'results_ok.html'
    else:
        template = 'results_failed.html'
    return saliweb.frontend.render_results_template(template, job=job)

This also uses the function saliweb.frontend.render_results_template(), which as before will either display an HTML page (similarly to Flask’s render_template), or XML in the case of automated usage.

On successful job completion, it shows the results_ok.html Jinja template, which uses the CompletedJob.get_results_file_url() method to show a link to download output.pdb and the CompletedJob.get_results_available_time() method to tell the user how long the results page will be available for:

{% extends "layout.html" %}

{% block body %}
<p>Job '<b>{{ job.name }}</b>' has completed.</p>

<p><a href="{{ job.get_results_file_url('output.pdb') }}">Download output PDB</a>.</p>

{{ job.get_results_available_time() }}
{% endblock %}

On failure it shows a similar page that links to the log file:

{% extends "layout.html" %}

{% block body %}
<p>Your job '<b>{{ job.name }}</b>' failed to produce any output models.</p>

<p>For more information, you can
<a href="{{ job.get_results_file_url('log') }}">download the log file</a>.
</p>
{% endblock %}

Results files¶

If individual results files can be downloaded, a results_file function should be provided. Similar to the results page, this looks up the job information using the URL, then sends it to the user using Flask’s send_from_directory function. The user is prevented from downloading other files that may be present in the job directory, getting an HTTP 404 (file not found) error instead, using the Flask abort function:

@app.route('/job/<name>/<path:fp>')
def results_file(name, fp):
    job = saliweb.frontend.get_completed_job(name,
                                             flask.request.args.get('passwd'))
    if fp in ('output.pdb', 'log'):
        return flask.send_from_directory(job.directory, fp)
    else:
        flask.abort(404)

Note

The “results files” don’t have to actually exist as real files in the job directory. Files can also be constructed on the fly and their contents returned to the user in a custom Flask Response object.

Alternative submit/results page for short jobs¶

For short jobs, it may not be desirable for job submission to pop up a page containing a ‘results’ link that the user then needs to click on (as the job may be complete by that point). In this case, the job submission page can redirect straight to the job results page using saliweb.frontend.redirect_to_results_page(). To avoid the user seeing an uninformative ‘job is still running’ page, this page should be overridden using the still_running_template argument to saliweb.frontend.get_completed_job(). (Normally this would display something very similar to the submit page, but can auto-refresh if desired using saliweb.frontend.StillRunningJob.get_refresh_time().) The resulting logic would look similar to:

@app.route('/job', methods=['GET', 'POST'])
def job():
    if flask.request.method == 'GET':
        return saliweb.frontend.render_queue_page()
    else:
        return submit_new_job()


@app.route('/job/<name>')
def results(name):
    job = get_completed_job(name, request.args.get('passwd'),
                            still_running_template='running.html')
    ...  # as for the previous results page, above


def submit_new_job():
    ...  # as for the previous submit page, above

    job.submit(email)

    # Go straight to the results page
    return saliweb.frontend.redirect_to_results_page(job)

It uses the following template as running.html, providing the job variable (which is a saliweb.frontend.StillRunningJob object), to notify the user and auto-refresh:

{% extends "layout.html" %}

{% block meta %}
<meta http-equiv="refresh" content="{{ job.get_refresh_time(10) }}" />
{% endblock %}

{% block body %}
<p>Your job {{ job.name }} has been submitted.</p>

<p>The job is currently running. When it is complete, results will be found
here - simply refresh or bookmark this page.</p>

{%- if job.email %}
<p>You will be notified at {{ job.email }} when the job has finished.</p>
{%- endif %}
{% endblock %}

See LigScore at https://github.com/salilab/ligscore/ for a web service that uses this submit/results logic.

Additional pages¶

Additional pages can be added if desired, simply by adding more Python functions with appropriate URLs (and adding the names of the functions to the get_navigation_links function in the layout.html Jinja template). Typically these just use render_template to show some content:

@app.route('/contact')
def contact():
    return flask.render_template('contact.html')


@app.route('/help')
def help():
    return flask.render_template('help.html')

Controlling page access¶

By default, all pages can be viewed by both anonymous and logged-in users. This can be modified, for example to restrict access only to named users, by checking the value of flask.g.user in any function, which is either a LoggedInUser object or None.

If access should be denied, either raise an AccessDeniedError exception (which will return an error page to the user with a message) or call flask.abort with a suitable HTTP error code, e.g. 401, “unauthorized” (which will return a generic error page without a message).

Frontend¶

Initialization¶

Web page layout¶

Displaying standard pages¶

Index page¶

Submission page¶

Results page¶

Results files¶

Alternative submit/results page for short jobs¶

Additional pages¶

Controlling page access¶

Table of Contents

Previous topic

Next topic

This Page