Installation¶
Warning
To run WAVES-core, it’s strongly recommended that you setup a dedicated user, because WAVES-core run with saga-python, and this module need to create some directories you might not be able to create (.radical and .saga) with another user (such as www-data)
Warning
WAVES-core was initially developed with Python 2.7 and Django 1.11 and is currently not tested on latest version (Python 3 and Django 2.0).
You can install WAVES-core either as a stand alone application, or inside any existing Django project
0. Prerequisites¶
Note
- In order to install WAVES-core you will need:
WAVES is developed with Django. You may need to know a little about it.
1. Install WAVES-core as stand alone application¶
1.1. Clone repository:
Current version is master (master)
user@host:$ git clone https://github.com/lirmm/waves-core.git [your_app] user@host:$ cd [your_app]1.2. Install dependencies:
user@host:~your_app$ virtualenv .venv user@host:~your_app$ source .venv/bin/activate (.venv) user@host:~your_app$ pip install -r requirements.txtYou might need other dependencies if working with other DB layout than sqlite.
1.3. Install database:
(.venv) user@host:~your_app$ ./manage.py check (.venv) user@host:~your_app$ ./manage.py makemigrations wcore (may only display "No changes detected in app 'wcore'") (.venv) user@host:~your_app$ ./manage.py migrate (.venv) user@host:~your_app$ ./manage.py createsuperuser (then follow instructions)1.4. If everything is ok:
You can start your test server and job queue like this:
(.venv) user@host:~your_app$ ./manage.py runserver (.venv) user@host:~your_app$ ./manage.py crontab addGo to http://127.0.0.1:8000/admin to setup your services WAVES-core comes with default front pages visible at http://127.0.0.1:8000
See also
Django crontab for other crontab setup
Note
From previous release, a known bug occured while using wqueue command. This bug block you from using this daemon queue. Please use “crontab” instead, and contact us if you experience issues.
1.5.(optional) Use celery and redis as job queue
Using the combination of waves job queue and crontab can have some limitations. For example, with this configuration, the minimum refresh interval is one minute. If you need smaller refresh interval you can use the combination of celery and redis as an asynchronous job queue.
1.5.1. Install the redis server
Redis is an in-memory database that allows to store the tasks.
user@host:~your_app$ sudo apt-get install redis-server1.5.2. Install dependencies
(.venv) user@host:~your_app$ pip install django-celery-beat==1.5.0 (.venv) user@host:~your_app$ pip install celery==4.3.0 (.venv) user@host:~your_app$ pip install redis==3.2.11.5.3. Launch job queue
(.venv) user@host:~your_app$ celery -A waves_core worker -l INFO (.venv) user@host:~your_app$ celery -A waves_core beat -l INFO --scheduler django_celery_beat.schedulers:DatabaseSchedulerIf the command crontab add was launched before, it is necessary to remove the crontab task:
(.venv) user@host:~your_app$ ./manage.py crontab remove1.5.4. Add “django celery beat” to the project
Thanks to “django celery beat” library, you can configure the recurrent tasks directly in the administrator area. According to add this application to the project, you need to add the application in the configuration file (~your_app/waves_core/settings.py).
INSTALLED_APPS = [ 'polymorphic', 'django.contrib.admin', 'django.contrib.auth', 'django.contrib.contenttypes', 'django.contrib.sessions', 'django.contrib.messages', 'django.contrib.staticfiles', 'waves.wcore', 'waves.authentication', 'crispy_forms', 'rest_framework', 'corsheaders', 'adminsortable2', 'django_crontab', 'django_celery_beat', ]1.5.5. Add periodic tasks to the database
Apply Django database migrations so that the necessary tables are created. Then, add some entries in the database with default configuration.
(.venv) user@host:~your_app$ ./manage.py migrate (.venv) user@host:~your_app$ ./manage.py loaddata fixtures_celery_beat.jsonVisit the Django-Admin interface to set up some periodic tasks.
In the periodic tasks pannel, the available tasks are job_queue and purge_jobs, two functions present in the tasks.py file of wcore application. Configure each of them with the desired intervals and saved.
2. Install WAVES-core inside existing Django project¶
To create a Django project, have a look at Django tutorial
See also
WAVES-core is a reusable app see: https://docs.djangoproject.com/en/1.11/intro/reusable-apps/#your-project-and-your-reusable-app
2.0. Setup a virtualenv for your project:
virtualenv ~/.venv/[waves_env]
2.1. Install waves package:
Use pip to install waves-core as third party package.
pip install waves-core
- If you want to install the latest development version (at your own risk :-))
pip install -e git+https://github.com/lirmm/waves-core.git#egg=waves-core
2.2. Activate WAVES-core in settings:
WAVES-core application has minimum dependencies to:
Optionally, WAVES-core can use:
Add required dependencies to your INSTALLED_APPS, you should at least find these in your project:
INSTALLED_APPS = [ 'polymorphic', # mandatory ... 'waves.wcore', # mandatory 'waves.authentication', # mandatory if API token access needed 'crispy_forms', # mandatory 'rest_framework', # mandatory ... 'rest_framework.authtoken', # optional see http://www.django-rest-framework.org/api-guide/authentication/#tokenauthentication 'corsheaders', # optional see https://github.com/ottoyiu/django-cors-headers 'adminsortable2', # optional see https://django-admin-sortable2.readthedocs.io ... ]2.3. Include the services urls in your project urls.py:
url(r'^waves/', include('waves.wcore.urls', namespace='wcore')) url(r'^waves/api/', include('waves.wcore.api.urls', namespace='wapi'))2.4. Create your database:
python manage.py makemigrations wcore python manage.py migrate wcore python manage.py check2.5. Extra configuration:
Depending on your needs, you might want to expose WAVES API to any registered user, if so have a look at: Corsheader to allow cross-origin Resource Sharing
Some WAVES-core API services requires authentication, see DRF authentication for authenticating methods API POST calls
Note
WAVES-core allows simple “api_key” authentication with standard Token Authentication processes, to use it simply add ‘waves.authentication’ in INSTALLED_APPS.
- This then allow to call WAVES API services with a api_key:
- with Authorization token header
- with GET / POST parameter with api_key value.
Each authenticated api service need a valid Authorization header as explained here: http://www.django-rest-framework.org/api-guide/authentication/#tokenauthentication
To use this service with apache in mod_wsgi: please mind to enable “WSGIPassAuthorization On” parameter in conf
3. Use other than SqlLite default DB layer¶
You may need to install the Python and MySQL development headers and libraries like so:
- sudo apt-get install python-dev default-libmysqlclient-dev # Debian / Ubuntu
- sudo yum install python-devel mysql-devel # Red Hat / CentOS
- brew install mysql-connector-c # macOS (Homebrew) (Currently, it has bug. See below)
On Windows, there are binary wheels you can install without MySQLConnector/C or MSVC.
Then install pip mysql package in your virtualenv:
pip install mysqlclient