Getting started

This page describes how to setup a working Job-Runner environment, including the Job-Runner WebSocket Server and the Job-Runner Worker. After a successful installation, you’ll have four processes running:

  1. ./manage.py runserver, webserver for admin, REST api and dashboard (part of Job-Runner)
  2. ./manage.py broadcast_queue, queue broadcaster (part of Job-Runner)
  3. ./scripts/job_runner_ws_server, WebSocket server (part of Job-Runner WebSocket Server)
  4. ./scripts/job_runner_worker --config-path job_runner_worker.ini, the worker executing our jobs

Install all components

  1. Create three virtualenv environments called job-runner, job-runner-ws-server and job-runner-worker. If you are on Ubuntu, you might want to install the virtualenvwrapper package first:

    # if you need to install virtualenvwrapper first
    $ sudo apt-get install virtualenvwrapper
    
    $ mkvirtualenv job-runner && deactivate
    $ mkvirtualenv job-runner-ws-server && deactivate
    $ mkvirtualenv job-runner-worker && deactivate

    See also

    http://virtualenvwrapper.readthedocs.org/en/latest/ if you are not familiar with virtualenvwrapper and virtualenv

  2. Install the Job-Runner in the job-runner virtualenv. To activate this virtualenv, execute workon job-runner. See Setup Job-Runner for installation details.

  3. Install the Job-Runner WebSocket Server in the job-runner-ws-server virtualenv. See the documentation in the job-runner-ws-server repository for installation instructions.

  4. Install the Job-Runner Worker in the job-runner-worker virtualenv. See the documentation in the job-runner-worker repository for installation instructions.

Get all components up and running

Job-Runner

  1. Open two console tabs and execute in the first one:

    $ workon job-runner
    $ manage.py runserver

    Execute in the second tab:

    $ workon job-runner
    $ manage.py broadcast_queue
  2. Open a browser and point it to http://localhost:8000/admin/. This will open the admin interface of the Job-Runner. While executing manage.py syncdb you were asked to create superuser credentials, use these to login. If you did not create any, open a new console and execute:

    $ workon job-runner
    $ manage.py createsuperuser
  3. In the admin interface, assign yourself to a group:

    1. Go to Auth - Users
    2. Click your username
    3. Under Permissions - Groups click the + icon to add yourself to a new group (call it Test Group for now)
    4. Save the user
  4. Create a project:

    1. Go to Job-Runner - Projects
    2. Click the Add project button
      • Title: Test Project
      • Groups: select Test Group
    3. Click the save button
  5. Create a worker for this project:

    1. Go to Job Runner - Workers
    2. Click on the Add Worker button
      • Title: Test Worker
      • Api key: testworker
      • Secret: verysecret
      • Project: Select Test Project
    3. Click the save button
  6. You now have created a group, project and worker :) Leave both processes you started in the first step running!

Job-Runner WebSocket Server

  1. Open a new console tab and execute:

    $ workon job-runner-ws-server
    $ job_runner_ws_server
  2. That’s it! Leave this process running :)

Job-Runner Worker

  1. Open a new console tab and execute:

    $ workon job-runner-worker
  2. Create a file named job-runner-worker.ini with the following content:

    [job_runner_worker]
    api_base_url=http://localhost:8000/
    api_key=testworker
    secret=verysecret
    concurrent_jobs=4
    log_level=debug
    script_temp_path=/tmp
    ws_server_hostname=localhost
    ws_server_port=5555
    broadcaster_server_hostname=localhost
    broadcaster_server_port=5556

    Please refer to the documentation in the job-runner-worker repository for the meaning of these variables.

  3. Now start the worker by executing:

    $ job_runner_worker --config-path job-runner-worker.ini

Congratulations! You now have all components up and running. If you point your browser to http://localhost:8000/, you will see an empty dashboard, with top-right a label Dashboard is live, meaning that the dashboard is connected to the WebSocket server. If this is red with a warning, please make sure the job_runner_ws_server process is still running!

Your first job

In this part, you’ll setup and schedule your first job! This will be a simple Python script, printing “Hello world!” and then sleeping between 3 - 15 sec. This script will be re-scheduled every 1 minute after the schedule dts of the previous run.

  1. Point your browser to http://localhost:8000/admin/

  2. First create a Python template which will form the base for all future Python jobs:

    1. Go to Job-Runner - Job templates

    2. Click the Add job template button

    3. Enter the following:

      • Title: Python

      • Body:

        #!/usr/bin/env python
        
        {{ content|safe }}
        
      • Worker: Select Test Worker

      • Auth groups: Select Test Group

    4. Click the save button

  3. Now create the actual job:

    1. Go to Job-Runner - Jobs

    2. Click the Add job button

    3. Enter the following:

      • Title: Hello world!

      • Job template: Python

      • Script content:

        import random
        import time
        
        print "Hello world!"
        time.sleep(random.randint(3, 15))
        
      • Reschedule interval: 1

      • Reschedule interval type: select Every x minutes

      • Reschedule type: select Increment schedule dts by interval

    4. Under Runs, select the current date and type by clicking on the date-picker and time-picker icons.

    5. Save the job.

  4. Now go to http://localhost:8000/. If all components are set-up correctly, you should see the job you just created moving from scheduled > in queue > started > completed!

Project Versions

Table Of Contents

Previous topic

Setup Job-Runner

Next topic

Project settings

This Page