.. _advanced: Advanced features ================= Tasks ----- When developing a remote laboratory, there are tasks that typically take a long time. For example, you might need to compile some code submitted by the user. Or you might run a task of a control laboratory that takes 20 seconds. In general, if a task takes 0-2 seconds is fine to be run in a view. But other than that, it might be too long. For this reason, in **weblablib** you can define tasks, which will be run in other threads or processes. Defining a task is quite easy. In ``hardware.py`` you can add something like: .. code-block:: python @weblab.task() def program_device(contents): """ Programs a device. Typically takes 5-10 seconds """ if weblab_user.time_left < 10: raise Exception("Error: user does not have " "enough time to run it!") time.sleep(10) # In this case return len(contents) From this point, you can run the synchronously: .. code-block:: python code = "my-own-code" # This runs it in this thread, # exactly as if it was not a task result = program_device(code) Or asynchronously (common when it's tasks) and play with the ``WebLabTask`` object: .. code-block:: python # This other code runs it in a different # process task = program_device.delay(code) # The following is a string that you can store in # Flask session or in weblab_user.data task.task_id # a string 'submitted', 'running' or 'failed'/'done' if finished. task.status task.submitted # bool: not yet started by a worker task.running # bool: started by a worker, not yet finished task.done # bool: finished successfully task.failed # bool: finished with error task.finished # task.failed or task.done # These two attributes are None while 'submitted' or 'running' task.result # the result of the function task.error # the exception data, if an exception happened If you store the ``task.task_id``, you can retrieve the task in other views or later on: .. code-block:: python # With the task_id, you can later obtain it in the same view in # the future: task = weblab.get_task(task_id) # and then ask if it is still running or not, and obtain # the result. You can also run: if task.done: print(task.result) At any point (including ``on_dispose``), you can see what tasks are still running: .. code-block:: python for task in weblab.running_tasks: print(task.status) Or all the tasks assigned in this session (finished or not): .. code-block:: python for task in weblab.tasks: print(task.name, task.result) Also, inside the task, you can get information and change information about the task: .. code-block:: python from weblablib import current_task @weblab.task() def program_device(path): # ... current_task.task_id print(current_task.data) current_task.update_data({ 'a': 'b' }) And obtain this information from outside: .. code-block:: python task = weblab.get_task(task_id) print(task.data['a']) When WebLab-Deusto calls to clean resources to your laboratory, **weblablib** will report of whether all the tasks assigned to the current session have finished or not, and no user will be assigned until the task is finished. So make sure that your task ends in time so as to not consume time of other users, and avoid starting tasks when the ``weblab_user.time_left`` is too short. Multiple laboratories in the same server ---------------------------------------- If you are running multiple laboratories in the same server, you should configure a different ``WEBLAB_REDIS_BASE`` value and/or ``WEBLAB_REDIS_URL``. **weblablib** relies on Redis to store the current status of the users and the laboratory, so if you run both in the default database with the default redis base name, there might be conflicts. To avoid this, either you use a different database (by default in Redis there are 16 databases, so you can use ``redis://localhost:6379/1`` or ``redis://localhost:6379/2``), or you can use the same one but using ``WEBLAB_REDIS_BASE`` different (e.g., ``lab1`` and ``lab2`` ). This would be recommended so later if you need to debug what is in Redis you can clearly see that there are values starting by ``lab1:`` or by ``lab2:`` refering to one or the other. Multiple laboratories through the same server --------------------------------------------- If you have 3 Raspberry Pi with different laboratories running, and, at the same time, you have a single server that proxies requests to all, you may face session problems. To avoid this, please rely on the Flask session configuration variables, such as: .. tabularcolumns:: |p{6.5cm}|p{8.5cm}| ================================= ========================================= ``SESSION_COOKIE_NAME`` The name of the cookie. By default it's ``session``, so it's better to change it in each laboratory to ``lab1sess`` and ``lab2sess`` or similar. ``SESSION_COOKIE_PATH`` The path of the cookie. By default the session cookie is stored in ``/``, but this way you can make sure that if you put ``/lab1``, when the user goes to ``/lab2``, no problem will arise. ``SECRET_KEY`` It is also recommendable that each lab have a different key. If everything else fails, at least the session created by other laboratory will not affect to the present one. ================================= ========================================= Using database users -------------------- In some cases, you might want to have a local database in your laboratory, and users represented there. For example, sometimes you might want to create a ``folder``, or a ``secret`` for that user, randomly generated and stored somewhere so the next time the user comes in, he sees the same thing. Also, there is a function called ``create_token`` in the weblab object to create random secrets in a secure way and URL-friendly (so you can put them in a query or similar, or even as a folder name or similar). To do this, in the ``on_start`` method you can create the user if it doesn't exist. This example uses `Flask-SQLAlchemy `_: .. code-block:: python # Using Flask-SQLAlchemy ( http://flask-sqlalchemy.pocoo.org/ ) from .models import LabUser from mylab import db @weblab.on_start def start(client_data, server_data): user = LabUser.query.filter_by(username_unique=username_unique).first() if user is None: # first time, assign a folder folder_name = weblab.create_token() # Lab configuration programs_folder = current_app.config['PROGRAMS_FOLDER'] os.mkdir(programs_name) # Add the user user = LabUser(username=weblab_user.username, username_unique=weblab_user.username_unique, folder=folder_name) db.session.add(user) db.session.commit() And then there is a ``user_loader`` function for loading the user, as well as a ``weblab_user.user`` object which internally uses that load_user: .. code-block:: python # Using Flask-SQLAlchemy ( http://flask-sqlalchemy.pocoo.org/ ) from .models import LabUser @weblab.user_loader def load_user(username_unique): return LabUser.query.filter_by(username_unique=username_unique).first() @app.route('/files') @requires_active def files(): user_folder = weblab_user.user.folder return jsonify(files=os.listdir(user_folder)) You can use this in different ways: you can create your own class and use it relying on a database, or you can use Redis or similar. Forbidden page -------------- By default, if a new user comes to your laboratory, he will see a simple ``Access forbidden`` message. However, you can do two other things: #. Forward the user to a link by adding ``WEBLAB_UNAUTHORIZED_LINK`` to ``config``. For example, typically here you will put a link to your public WebLab-Deusto system. If a user bookmarks the laboratory, he will be redirected to your WebLab-Deusto so he authenticates. In other scenarios, you might point to LabsLand, to your LMS (e.g., Moodle) or similar. #. Display another website. You can create a template in the ``templates`` folder and use it by adding the ``WEBLAB_UNAUTHORIZED_TEMPLATE`` variable. If you set it to ``forbidden.html``, you will see it in WebLab-Deusto. Timeout management ------------------ By default, if the user does not contact the laboratory in 15 seconds, it is assumed that the user left. You can configure this by managing the ``WEBLAB_TIMEOUT`` variable. Also, an ``ExpiredUser`` exists only for an hour by default. If you want to extend this time, use the ``WEBLAB_EXPIRED_USERS_TIMEOUT`` variable. Similarly, if you want to delete from memory users as soon as possible, you can configure it to ``240`` seconds (3 minutes) or similar. It is not recommended to use smaller values or the users might not have the chance to return to the previuos system. https ----- If you want to force https, sometimes you may find that the URL returned does not use it because of a misconfiguration in the web server (e.g., nginx, apache). An easy way to fix it is by setting the configuration of ``WEBLAB_SCHEME`` to ``https``. Processes vs. threads --------------------- By default, weblablib creates a set of threads per process run, which are running tasks and cleaning threads. By default, 3 threads are dedicated to tasks, and 1 to cleaning expired sessions. So if you run: .. code-block:: shell gunicorn --bind 127.0.0.1:8080 -w 10 wsgi_app:application For example, you'll be running 10 processes, and each of them 3 threads for tasks (30) and 1 thread for cleaning expired sessions. You can reduce the number of threads per process by changing ``WEBLAB_TASK_THREADS_PROCESS``. Another approach (which is indeed cleaner) is to run no thread, and run the tasks, etc. outside. To do this, you can configure ``WEBLAB_TASK_THREADS_PROCESS`` to ``0``, ``WEBLAB_AUTOCLEAN_THREAD`` to ``False``, and then run in parallel: .. code-block:: shell $ export FLASK_APP=laboratory.py $ flask loop or: .. code-block:: shell $ export FLASK_APP=laboratory.py $ flask loop --threads 10 This way, you'll have a process running 10 threads the ``run-tasks`` and ``clean-expired-threads`` tasks continuously. Another alternative is to run each process separately and per task: .. code-block:: shell $ export FLASK_APP=laboratory.py $ flask clean-expired-users And in another process: .. code-block:: shell $ export FLASK_APP=laboratory.py $ flask run-tasks These two processes end immediately. You can run them in a loop outside in a shell, use ``cron`` or similar tools or so. This way, the ``gunicorn`` processes will only manage web requests, and the external processes will run the tasks and clean expired users. Base URL -------- By default, everything is running in ``/``, and **weblablib** automatically generate ``/weblab/sessions/`` URLs. If you have more than one lab in a public server (quite common if you have a single public IP for several laboratories), then you may have to play with ``SCRIPT_NAME``. For example: .. code-block:: shell SCRIPT_NAME=/lab1 gunicorn --bind 127.0.0.1:8080 -w 10 \ wsgi_app:application And then in nginx or Apache configuring that https://yourserver/lab1 goes to http://localhost:8080/lab1 will work. In this case, you have to configure ``http_experiment_url`` to ``http://localhost:8080/lab1``. In some circumstances, you may also want to provide a base URL for weblab alone. In that case, you can use the ``WEBLAB_BASE`` url. .. _internationalization: Internationalization (i18n) --------------------------- The object ``weblab_user`` has a ``locale`` parameter; which is ``None`` in the Anonymous user, but it's ``en``, ``es``... depending on what WebLab-Deusto said (which may come from the previous system, such as the LMS or Moodle). Therefore, if you are using ``Flask-Babel`` or ``Flask-BabelEx``, the script for selecting locale should be similar to: .. code-block:: python @babel.localeselector def get_locale(): locale = request.args.get('locale', None) if locale is None: locale = weblab_user.locale if locale is None: locale = session.get('locale') if locale is None: locale = request.accept_languages.best_match(SUPPORTED_LANGUAGES) if locale is None: locale = 'en' session['locale'] = locale return locale *New in weblablib 0.3*