.. currentmodule:: saliweb.frontend .. _frontend: Frontend ******** The frontend is provided by Python code using the `Flask microframework `_. This displays the web interface, allowing a user to upload their input files, start a job, display a list of all jobs in the system, and get back job results. This Python code uses utility classes and functions in the :mod:`saliweb.frontend` module, together with functionality provided by Flask. Initialization ============== The first step is to create a Python module that creates the web application itself, using the :func:`saliweb.frontend.make_application` function (which in turn creates a Flask application and configures it). For a web service 'ModFoo' this should be done in the Python module ``frontend/modfoo/__init__.py``. This is fairly standard boilerplate: .. literalinclude:: ../examples/frontend-new.py :language: python The 'parameters' object here is a list of parameters the job requires at submission time, and will be :ref:`described later `. Web page layout =============== Each web page has a similar layout (header, footer, links, and so on). This is provided by a system-wide `Jinja2 template `_ called ``saliweb/layout.html``, which can be seen `at GitHub `_. This system-wide template should be overriden for each web service, by providing a file ``frontend/modfoo/templates/layout.html`` that 'extends' the template: .. literalinclude:: ../examples/layout.html :language: html+jinja This example demonstrates a few Jinja2 and Flask features: - Parts of the base template can be overriden using the ``block`` directive. Here, a custom stylesheet is added (by overriding the ``css`` block), links are added to all pages to the navigation bar at the top of the page (``navigation`` block), and the sidebar and footer (at the left and bottom of the page, which are blank in the system-wide template) are filled in. - Links to other parts of the web service can be provided using Flask's `url_for `_ function. This takes either the name of the Python function that renders the page (such as "index"; :ref:`see below `) or "static" to point to static files (such as stylesheets or images) in the web service's ``html`` subdirectory. - A number of global variables are available which can be substituted in using Jinja2's ``{{ }}`` syntax, most notably ``config`` which stores web service configuration, such as the name and version in ``config.SERVICE_NAME`` and ``config.VERSION`` respectively. - Some helper functions are available. Here the ``get_navigation_links`` function is used, which takes a list of (URL, description) pairs. See the `Jinja2 manual `_ for more information on Jinja2 templates. .. _std_pages: Displaying standard pages ========================= The bulk of the functionality of the frontend is implemented by providing Python functions for each page using `Flask routes `_. For a typical web service, the index, submission, results and results file pages need to be implemented by providing ``index``, ``job``, ``results``, and ``results_file`` functions, respectively. These pages will be considered in turn. .. note:: Additional pages (such as page to download the software, or a help page) can be simply implemented by adding more Python functions with appropriate routes (and, if appropriate, adding links to these pages to ``layout.html``). :ref:`See below `. Index page ---------- The index page is the first page seen when using the web service, and typically displays a form allowing the user to set parameters and upload input files. (In more complex web services this first form can lead to further forms for advanced options, etc.) The Python code for this is straightforward - it simply defines an ``index`` function which uses the Flask `render_template `_ function to render a Jinja2 template, and is then `decorated using app.route `_ to tell Flask to use this function to service the '/' URL: .. literalinclude:: ../examples/frontend-index.py :language: python The Jinja2 template in turn looks like: .. literalinclude:: ../examples/index.html :language: html+jinja The template extends the previously-defined ``layout.html`` so will get the sidebar, footer, etc. defined there. The form is set up to submit to the submission page (``url_for("job")``). It allows the user to upload a single PDB file, as well as pick an optional name for their job and an optional email address to be notified when the job completes. Note that the email address is filled in using ``g.user.email``. (``g`` is used by Flask to store `global data `_.) If the user is logged in to the webserver, ``g.user`` will be a :class:`LoggedInUser` object, and their email address will be available for use in this fashion (otherwise, the user will simply have to input a suitable address if they want to be notified). Similarly, ``g.user.modeller_key`` provides the MODELLER license key of the logged-in user. .. _parameters: The names of the form parameters above (``job_name``, ``input_pdb``) should also be described in the Python code ``frontend/modloop/__init__.py`` for automated use of the service (see :ref:`automated`), by passing suitable :class:`Parameter` and/or :class:`FileParameter` objects when the application is created. Note that ``email`` is omitted because automated usage typically does not use email notification: .. literalinclude:: ../examples/frontend-new-params.py :language: python Submission page --------------- The submission page is called when the user has input all the information needed for the job. It is implemented by defining the ``job`` function which handles the ``/job`` URL. This serves double duty - an HTTP POST to this URL will submit a new job, while a GET will show all jobs in the system (the queue). Showing the queue is handled by the :func:`saliweb.frontend.render_queue_page` function. Job submission should validate the provided information, then actually submit the job to the backend. If validation fails, it should throw an :exc:`InputValidationError` exception; this will be handled by the web framework as a message to the user asking them to fix the problem and resubmit. (If some kind of internal error occurs, such as a file write failure, in this or any other method, the user will see an error page and the server admin will be notified by email to fix the problem.) To submit the job, first create a :class:`saliweb.frontend.IncomingJob` object, which also creates a new directory for the job files. Put all necessary input files in that directory, for example using :meth:`IncomingJob.get_path`, then actually run the job by calling :meth:`IncomingJob.submit`. .. note:: 'Input files' include PDB files, parameter files, etc. but *not* shell scripts, Python scripts, or executables. These should never be generated by the frontend, but instead should be generated by the backend. If these files are generated by the frontend, it is very easy for an unscrupulous end user to hack into the cluster by compromising the web service. The backend should always check inputs provided by the frontend (e.g. in the :meth:`~saliweb.backend.Job.preprocess` method) for sanity before running anything. .. note:: When taking files as input, you can simply write them into the job directory using their `save method `_. But never trust the filename provided by the user! Ideally, save with a fixed generic name (e.g. ``input.pdb``). If this is not possible, use the `secure_filename `_ function to get a safe version of the filename. Finally, the submission page should inform the user of the results URL (:attr:`IncomingJob.results_url`), so that they can obtain the results when the job finishes. This uses the function :func:`saliweb.frontend.render_submit_template`, which will either display an HTML page (similarly to Flask's `render_template `_, as before) or XML in the case of automated usage. The example below reads in the PDB file provided by the user on the index page, checks to make sure it contains at least one ATOM record, then writes it into the job directory and finally submits the job: .. literalinclude:: ../examples/frontend-submit.py :language: python It uses the following template as ``submit.html``, providing the ``job`` and ``email`` variables, to notify the user: .. literalinclude:: ../examples/submit.html :language: html+jinja Results page ------------ The results page is used to display the results of a job, and is implemented by providing a ``results`` function. The function can either display the job results directly, or it can display links to allow output files to be downloaded. In the latter case, URLs to these files can be generated by calling :meth:`CompletedJob.get_results_file_url`. The example below assumes the backend generates a single output file on success, ``output.pdb``, and a log file, ``log``, on failure. It uses the utility function :func:`saliweb.frontend.get_completed_job` to get a :class:`CompletedJob` object from the URL (this will show an error message if the job has not completed, or the password is invalid) and then looks for files in the job directory using the :meth:`CompletedJob.get_path` method: .. literalinclude:: ../examples/frontend-results.py :language: python This also uses the function :func:`saliweb.frontend.render_results_template`, which as before will either display an HTML page (similarly to Flask's `render_template `_), or XML in the case of automated usage. On successful job completion, it shows the ``results_ok.html`` Jinja template, which uses the :meth:`CompletedJob.get_results_file_url` method to show a link to download ``output.pdb`` and the :meth:`CompletedJob.get_results_available_time` method to tell the user how long the results page will be available for: .. literalinclude:: ../examples/results_ok.html :language: html+jinja On failure it shows a similar page that links to the log file: .. literalinclude:: ../examples/results_failed.html :language: html+jinja Results files ------------- If individual results files can be downloaded, a ``results_file`` function should be provided. Similar to the results page, this looks up the job information using the URL, then sends it to the user using Flask's `send_from_directory `_ function. The user is prevented from downloading other files that may be present in the job directory, getting an HTTP 404 (file not found) error instead, using the Flask `abort `_ function: .. literalinclude:: ../examples/frontend-results-file.py :language: python .. note:: The "results files" don't have to actually exist as real files in the job directory. Files can also be constructed on the fly and their contents returned to the user in a custom Flask `Response `_ object. Alternative submit/results page for short jobs ---------------------------------------------- For short jobs, it may not be desirable for job submission to pop up a page containing a 'results' link that the user then needs to click on (as the job may be complete by that point). In this case, the job submission page can redirect straight to the job results page using :func:`saliweb.frontend.redirect_to_results_page`. To avoid the user seeing an uninformative 'job is still running' page, this page should be overridden using the ``still_running_template`` argument to :func:`saliweb.frontend.get_completed_job`. (Normally this would display something very similar to the submit page, but can auto-refresh if desired using :meth:`saliweb.frontend.StillRunningJob.get_refresh_time`.) The resulting logic would look similar to: .. literalinclude:: ../examples/frontend-submit-redirect.py :language: python It uses the following template as ``running.html``, providing the ``job`` variable (which is a :class:`saliweb.frontend.StillRunningJob` object), to notify the user and auto-refresh: .. literalinclude:: ../examples/running.html :language: html+jinja See LigScore at https://github.com/salilab/ligscore/ for a web service that uses this submit/results logic. .. _add_pages: Additional pages ---------------- Additional pages can be added if desired, simply by adding more Python functions with appropriate URLs (and adding the names of the functions to the ``get_navigation_links`` function in the ``layout.html`` Jinja template). Typically these just use ``render_template`` to show some content: .. literalinclude:: ../examples/frontend-additional.py :language: python Controlling page access ======================= By default, all pages can be viewed by both anonymous and logged-in users. This can be modified, for example to restrict access only to named users, by checking the value of ``flask.g.user`` in any function, which is either a :class:`LoggedInUser` object or ``None``. If access should be denied, either raise an :exc:`AccessDeniedError` exception (which will return an error page to the user with a message) or call `flask.abort `_ with a suitable HTTP error code, e.g. 401, "unauthorized" (which will return a generic error page without a message).