Makefile¶

All relevant build tasks are implemented in git://manage and for CI or IDE integration a small Makefile wrapper is available. If you are not familiar with Makefiles, we recommend to read gnu-make introduction.

The usage is simple, just type make {target-name} to build a target. Calling the help target gives a first overview (make help):

INFO:  sourced NVM environment from /home/runner/.nvm
run         : run developer instance
format.:
  python    : format Python code source using black
test.:
  yamllint  : lint YAML files (YAMLLINT_FILES)
  pylint    : lint PYLINT_FILES, searx/engines, searx & tests
  pyright   : static type check of python sources
  black     : check black code format
  unit      : run unit tests
  coverage  : run unit tests with coverage
  robot     : run robot test
  rst       : test .rst files incl. README.rst
  clean     : clean intermediate test stuff
gecko.driver:
  download & install geckodriver if not already installed (required for robot_tests)
py.:
  build     : Build python packages at ./dist
  clean     : delete virtualenv and intermediate py files
pyenv.:
  install   : developer install of SearXNG into virtualenv
  uninstall : uninstall developer installation
  cmd ...   : run command ... in virtualenv
  OK        : test if virtualenv is OK
pypi.upload:
  Upload python packages to PyPi (to test use pypi.upload.test)
docs.:
  html      : build HTML documentation
  live      : autobuild HTML documentation while editing
  gh-pages  : deploy on gh-pages branch
  prebuild  : build reST include files (./build/docs/includes)
  clean     : clean documentation build
docker.:
  build     : build docker image
  push      : build and push docker image
node.:
  env       : download & install SearXNG's npm dependencies locally
  env.dev   : download & install developer and CI tools
  clean     : drop locally npm installations
themes.:
  all       : build all themes
  simple    : build simple theme
pygments.:
  less      : build LESS files for pygments
static.build.:  [build] /static
  commit    : build & commit /static folder
  drop      : drop last commit if it was previously done by static.build.commit
  restore   : git restore of the /static folder (after themes.all)
nvm.: use nvm (without dot) to execute nvm commands directly
  install   : install NVM locally at /home/runner/work/searxng/searxng/.nvm
  clean     : remove NVM installation
  status    : prompt some status informations about nvm & node
  nodejs    : install Node.js latest LTS
  cmd ...   : run command ... in NVM environment
  bash      : start bash interpreter with NVM environment sourced
data.:
  all       : update searx/languages.py and ./data/*
  languages : update searx/data/engines_languages.json & searx/languages.py
  useragents: update searx/data/useragents.json with the most recent versions of Firefox.
weblate.:
  push.translations: push translation changes from SearXNG to Weblate's counterpart
  to.translations: Update 'translations' branch with last additions from Weblate.
environment ...
  SEARXNG_REDIS_URL : 
----
install        - developer install of SearxNG into virtualenv
uninstall      - uninstall developer installation
clean          - clean up working tree
search.checker - check search engines
test           - run shell & CI tests
test.shell     - test shell scripts
ci.test        - run CI tests

Python environment (`make install`)¶

We do no longer need to build up the virtualenv manually. Jump into your git working tree and release a make install to get a virtualenv with a developer install of SearXNG (git://setup.py).

$ cd ~/searxng-clone
$ make install
PYENV     [virtualenv] installing ./requirements*.txt into local/py3
...
PYENV     OK
PYENV     [install] pip install -e 'searx[test]'
...
Successfully installed argparse-1.4.0 searx
BUILDENV  INFO:searx:load the default settings from ./searx/settings.yml
BUILDENV  INFO:searx:Initialisation done
BUILDENV  build utils/brand.env

If you release make install multiple times the installation will only rebuild if the sha256 sum of the requirement files fails. With other words: the check fails if you edit the requirements listed in git://requirements-dev.txt and git://requirements.txt).

$ make install
PYENV     OK
PYENV     [virtualenv] requirements.sha256 failed
          [virtualenv] - 6cea6eb6def9e14a18bf32f8a3e...  ./requirements-dev.txt
          [virtualenv] - 471efef6c73558e391c3adb35f4...  ./requirements.txt
...
PYENV     [virtualenv] installing ./requirements*.txt into local/py3
...
PYENV     OK
PYENV     [install] pip install -e 'searx[test]'
...
Successfully installed argparse-1.4.0 searx
BUILDENV  INFO:searx:load the default settings from ./searx/settings.yml
BUILDENV  INFO:searx:Initialisation done
BUILDENV  build utils/brand.env

If you think, something goes wrong with your ./local environment or you change the git://setup.py file, you have to call make clean.

`make buildenv`¶

Rebuild instance’s environment with the modified settings from the brand: and server: section of your settings.yml.

What is the git://utils/brand.env needed for and why do you need to rebuild it if necessary?

Short answer: installation and maintenance scripts are running outside of instance’s runtime environment and need some values defined in the runtime environment.

All the SearXNG setups are centralized in the settings.yml file. This setup is available as long we are in a installed instance. E.g. the installed instance on the server or the installed developer instance at ./local (the later one is created by a make install or make run).

Tasks running outside of an installed instance, especially installation and maintenance tasks running at (pre-) installation time do not have access to the SearXNG setup (from a installed instance). Those tasks need a build environment.

The make buildenv target will update the build environment in:

git://utils/brand.env

Tasks running outside of an installed instance, need the following settings from the YAML configuration:

SEARXNG_URL from server.base_url (aka PUBLIC_URL)
SEARXNG_BIND_ADDRESS from server.bind_address
SEARXNG_PORT from server.port

The GIT_URL and GIT_BRANCH in the origin:utils/brand.env file, are readed from the git VCS and the branch that is checked out when make buildenv command runs.

I would like to create my own brand, how should I proceed?

Create a remote branch (example.org), checkout the remote branch (on your local developer desktop) and in the git://searx/settings.yml file in the server: section set base_url. Run make buildenv and create a commit for your brand.

On your server you clone the branch (example.org) into your HOME folder ~ from where you run the installation and maintenance task.

To upgrade you brand, rebase on SearXNG’s master branch (on your local developer desktop), force push it to your remote branch. Go to your server, do a force pull and run sudo -H ./utils/searxng.sh instance update.

Node.js environment (`make node.env`)¶

Node.js version 16.13.0 or higher is required to build the themes. If the requirement is not met, the build chain uses nvm (Node Version Manager) to install latest LTS of Node.js locally: there is no need to install nvm or npm on your system.

Use make nvm.status to get the current status of you Node.js and nvm setup.

Here is the output you will typically get on a Ubuntu 20.04 system which serves only a no longer active Release Node.js v10.19.0.

$ make nvm.status
INFO:  Node.js is installed at /usr/bin/node
INFO:  Node.js is version v10.19.0
WARN:  minimal Node.js version is 16.13.0
INFO:  npm is installed at /usr/bin/npm
INFO:  npm is version 6.14.4
WARN:  NVM is not installed
INFO:  to install NVM and Node.js (LTS) use: manage nvm install --lts

To install you can also use make nvm.nodejs

`make nvm.nodejs`¶

Install latest Node.js LTS locally (uses nvm):

$ make nvm.nodejs
INFO:  install (update) NVM at /share/searxng/.nvm
INFO:  clone: https://github.com/nvm-sh/nvm.git
...
Downloading and installing node v16.13.0...
...
INFO:  Node.js is installed at searxng/.nvm/versions/node/v16.13.0/bin/node
INFO:  Node.js is version v16.13.0
INFO:  npm is installed at searxng/.nvm/versions/node/v16.13.0/bin/npm
INFO:  npm is version 8.1.0
INFO:  NVM is installed at searxng/.nvm

`make run`¶

To get up a running a developer instance simply call make run. This enables debug option in git://searx/settings.yml, starts a ./searx/webapp.py instance and opens the URL in your favorite WEB browser (xdg-open):

$ make run

Changes to theme’s HTML templates (jinja2) are instant. Changes to the CSS & JS sources of the theme need to be rebuild. You can do that by running:

$ make themes.all

Alternatively to themes.all you can run live builds of the theme you are modify:

$ LIVE_THEME=simple make run

`make clean`¶

Drops all intermediate files, all builds, but keep sources untouched. Before calling make clean stop all processes using the Python environment (make install) or Node.js environment (make node.env).

$ make clean
CLEAN     pyenv
PYENV     [virtualenv] drop local/py3
CLEAN     docs -- build/docs dist/docs
CLEAN     themes -- locally installed npm dependencies
...
CLEAN     test stuff
CLEAN     common files

`make docs docs.autobuild docs.clean`¶

We describe the usage of the doc.* targets in the How to contribute / Documentation section. If you want to edit the documentation read our live build section. If you are working in your own brand, adjust your Global Settings.

`make docs.gh-pages`¶

To deploy on github.io first adjust your Global Settings. For any further read deploy on github.io.

`make test`¶

Runs a series of tests: make test.pylint, test.pep8, test.unit and test.robot. You can run tests selective, e.g.:

$ make test.pep8 test.unit test.sh
TEST      test.pep8 OK
...
TEST      test.unit OK
...
TEST      test.sh OK

make search.checker

To check a engine with whitespace in the name like google news replace space by underline:

make search.checker.google_news

To see HTTP requests and more use SEARXNG_DEBUG:

make SEARXNG_DEBUG=1 search.checker.google_news

To filter out HTTP redirects (3xx):

make SEARXNG_DEBUG=1 search.checker.google_news | grep -A1 "HTTP/1.1\" 3[0-9][0-9]"
...
Engine google news                   Checking
https://news.google.com:443 "GET /search?q=life&hl=en&lr=lang_en&ie=utf8&oe=utf8&ceid=US%3Aen&gl=US HTTP/1.1" 302 0
https://news.google.com:443 "GET /search?q=life&hl=en-US&lr=lang_en&ie=utf8&oe=utf8&ceid=US:en&gl=US HTTP/1.1" 200 None
--
https://news.google.com:443 "GET /search?q=computer&hl=en&lr=lang_en&ie=utf8&oe=utf8&ceid=US%3Aen&gl=US HTTP/1.1" 302 0
https://news.google.com:443 "GET /search?q=computer&hl=en-US&lr=lang_en&ie=utf8&oe=utf8&ceid=US:en&gl=US HTTP/1.1" 200 None
--

Makefile¶

Python environment (`make install`)¶

`make buildenv`¶

Node.js environment (`make node.env`)¶

`make nvm.nodejs`¶

`make run`¶

`make clean`¶

`make docs docs.autobuild docs.clean`¶

`make docs.gh-pages`¶

`make test`¶

`make test.shell`¶

`make test.pylint`¶

`search.checker.{engine name}`¶

Table of Contents

Project Links

Navigation

This Page