Makefile

build environment

Before looking deeper at the targets, first read about Python environment.

To install system requirements follow Buildhosts.

All relevant build tasks are implemented in git://manage.sh and for CI or IDE integration a small Makefile wrapper is available. If you are not familiar with Makefiles, we recommend to read gnu-make introduction.

The usage is simple, just type make {target-name} to build a target. Calling the help target gives a first overview (make help):

buildenv
  rebuild ./utils/brand.env
babel.compile
  pybabel compile ./searx/translations
data.*
  all       : update searx/languages.py and ./data/*
  languages : update searx/data/engines_languages.json & searx/languages.py
  useragents: update searx/data/useragents.json with the most recent versions of Firefox.
docs.*
  html      : build HTML documentation
  live      : autobuild HTML documentation while editing
  gh-pages  : deploy on gh-pages branch
  prebuild  : build reST include files (./build/docs/includes)
  clean     : clean documentation build
docker
  build     : build docker image
  push      : build and push docker image
gecko.driver
  download & install geckodriver if not already installed (required for
  robot_tests)
node.*
  env       : download & install npm dependencies locally
  clean     : drop npm installations
py.*
  build     : Build python packages at ./dist
  clean     : delete virtualenv and intermediate py files
pyenv.* :
  install   : developer install of searx into virtualenv
  uninstall : uninstall developer installation
  cmd ...   : run command ... in virtualenv
  OK        : test if virtualenv is OK
pypi.upload:
  Upload python packages to PyPi (to test use pypi.upload.test)
test.* :
  pylint    : lint PYLINT_FILES, searx/engines, searx & tests
  pep8      : pycodestyle (pep8) for all files except PYLINT_FILES
  unit      : run unit tests
  coverage  : run unit tests with coverage
  robot     : run robot test
  clean     : clean intermediate test stuff
themes.* :
  all       : build all themes
  oscar     : build oscar theme
  simple    : build simple theme
----
run            - run developer instance
install        - developer install of searx into virtualenv
uninstall      - uninstall developer installation
clean          - clean up working tree
search.checker - check search engines
test           - run shell & CI tests
test.shell     - test shell scripts
ci.test        - run CI tests

Python environment

activate environment

source ./local/py3/bin/activate

We do no longer need to build up the virtualenv manually. Jump into your git working tree and release a make install to get a virtualenv with a developer install of searx (git://setup.py).

$ cd ~/searx-clone
$ make install
PYENV     [virtualenv] installing ./requirements*.txt into local/py3
...
PYENV     OK
PYENV     [install] pip install -e 'searx[test]'
...
Successfully installed argparse-1.4.0 searx
BUILDENV  INFO:searx:load the default settings from ./searx/settings.yml
BUILDENV  INFO:searx:Initialisation done
BUILDENV  build utils/brand.env

If you release make install multiple times the installation will only rebuild if the sha256 sum of the requirement files fails. With other words: the check fails if you edit the requirements listed in git://requirements-dev.txt and git://requirements.txt).

$ make install
PYENV     OK
PYENV     [virtualenv] requirements.sha256 failed
          [virtualenv] - 6cea6eb6def9e14a18bf32f8a3e...  ./requirements-dev.txt
          [virtualenv] - 471efef6c73558e391c3adb35f4...  ./requirements.txt
...
PYENV     [virtualenv] installing ./requirements*.txt into local/py3
...
PYENV     OK
PYENV     [install] pip install -e 'searx[test]'
...
Successfully installed argparse-1.4.0 searx
BUILDENV  INFO:searx:load the default settings from ./searx/settings.yml
BUILDENV  INFO:searx:Initialisation done
BUILDENV  build utils/brand.env

drop environment

To get rid of the existing environment before re-build use clean target first.

If you think, something goes wrong with your ./local environment or you change the git://setup.py file, you have to call make clean.

make run

To get up a running a developer instance simply call make run. This enables debug option in git://searx/settings.yml, starts a ./searx/webapp.py instance, disables debug option again and opens the URL in your favorite WEB browser (xdg-open):

$ make run
PYENV     OK
SEARX_DEBUG=1 ./manage.sh pyenv.cmd python ./searx/webapp.py
...
INFO:werkzeug: * Running on http://127.0.0.1:8888/ (Press CTRL+C to quit)

make clean

Drop all intermediate files, all builds, but keep sources untouched. Before calling make clean stop all processes using Python environment.

$ make clean
CLEAN     pyenv
PYENV     [virtualenv] drop ./local/py3
CLEAN     docs -- ./build/docs ./dist/docs
CLEAN     locally installed npm dependencies
CLEAN     test stuff
CLEAN     common files

make docs docs.autobuild docs.clean

We describe the usage of the doc.* targets in the How to contribute / Documentation section. If you want to edit the documentation read our live build section. If you are working in your own brand, adjust your Global Settings.

make docs.gh-pages

To deploy on github.io first adjust your Global Settings. For any further read deploy on github.io.

make test

Runs a series of tests: make test.pylint, test.pep8, test.unit and test.robot. You can run tests selective, e.g.:

$ make test.pep8 test.unit test.sh
TEST      test.pep8 OK
...
TEST      test.unit OK
...
TEST      test.sh OK

make test.sh

Lint shell scripts / if you have changed some bash scripting run this test before commit.

make test.pylint

Pylint is known as one of the best source-code, bug and quality checker for the Python programming language. The pylint profile we use at searx project is found in project’s root folder git://.pylintrc.

search.checker.{engine name}

To check all engines:

make search.checker

To check a engine with whitespace in the name like google news replace space by underline:

make search.checker.google_news

To see HTTP requests and more use SEARX_DEBUG:

make SEARX_DEBUG=1 search.checker.google_news

To filter out HTTP redirects (3xx):

make SEARX_DEBUG=1 search.checker.google_news | grep -A1 "HTTP/1.1\" 3[0-9][0-9]"
...
Engine google news                   Checking
https://news.google.com:443 "GET /search?q=life&hl=en&lr=lang_en&ie=utf8&oe=utf8&ceid=US%3Aen&gl=US HTTP/1.1" 302 0
https://news.google.com:443 "GET /search?q=life&hl=en-US&lr=lang_en&ie=utf8&oe=utf8&ceid=US:en&gl=US HTTP/1.1" 200 None
--
https://news.google.com:443 "GET /search?q=computer&hl=en&lr=lang_en&ie=utf8&oe=utf8&ceid=US%3Aen&gl=US HTTP/1.1" 302 0
https://news.google.com:443 "GET /search?q=computer&hl=en-US&lr=lang_en&ie=utf8&oe=utf8&ceid=US:en&gl=US HTTP/1.1" 200 None
--

make pybuild

Build Python packages in ./dist/py:

$ make pybuild
...
BUILD     pybuild
running sdist
running egg_info
...
running bdist_wheel

$ ls  ./dist
searx-0.18.0-py3-none-any.whl  searx-0.18.0.tar.gz

To upload packages to PyPi, there is also a pypi.upload target (to test use pypi.upload.test). Since you are not the owner of PyPi: searx you will never need to upload.