The ops project Tool

After installation of devapps you have the ops project command available¹. The tool allows to install and maintain projects and resources.

Project

To use the tool, first activate your environment via poetry shell in case of a development installation (virtual environ activation in case of a pip(x) based install).

Verification:

$ poetry debug info

Poetry
Version: 1.1.15
Python:  3.7.12

Virtualenv
Python:         3.7.12
Implementation: CPython
Path:           /home/runner/miniconda3/envs/devapps_py3.7
Valid:          True

System
Platform: linux
OS:       posix
Python:   /home/runner/miniconda3/envs/devapps_py3.7


Let's now create a (new) directory for the project:

$ mkdir $HOME/myproject && cd $HOME/myproject && ls -lta
total 8
drwxr-xr-x  2 runner docker 4096 Jul 25 16:06 .
drwxr-x--- 17 runner docker 4096 Jul 25 16:06 ..

Resources

devapps (and derived packages) contain resource defining python modules named resources.py.

Resources of devapps

Here are the resources defined in devapps. They are required for running the tests but also provide some tools:

CmdOutput

$ ops project --list_resources_files

$ ops project --list_resources_files
07-25 16:06:21 [info     ] Directories                    [project] conda_prefix=/home/runner/miniconda3 fs_dir=/home/runner/miniconda3/fs(default)
07-25 16:06:21 [info     ] Listing Defined Resources      [project] 
07-25 16:06:21 [info     ] details                        [project] json=[
  {
    "bin_name": "lc_tools",
    "conda_chan": "conda-forge",
    "conda_pkg": "git fzf jq ripgrep fd-find httpie htop tmux",
    "disabled": false,
    "doc": "",
    "host_conf_dir": "$PROJECT_ROOT/conf/${host:-$HOSTNAME}/lc_tools",
    "installed": true,
    "module": "devapp",
    "module_dir": "/home/runner/work/devapps/devapps/src/devapp",
    "name": "lc_tools",
    "path": "/home/runner/miniconda3/envs/lc_tools/bin",
    "provides": [
      "git",
      "fzf",
      "jq",
      "rg",
      "fd",
      "http",
      "htop",
      "tmux"
    ],
    "run": "lc_tools",
    "verify_present": "verify_tools"
  },
  {
    "bin_name": "lc_tools_kf",
    "conda_chan": "kalefranz",
    "conda_pkg": "httpd",
    "disabled": false,
    "doc": "",
    "host_conf_dir": "$PROJECT_ROOT/conf/${host:-$HOSTNAME}/lc_tools_kf",
    "installed": true,
    "module": "devapp",
    "module_dir": "/home/runner/work/devapps/devapps/src/devapp",
    "name": "lc_tools_kf",
    "path": "/home/runner/miniconda3/envs/lc_tools_kf/bin",
    "provides": [
      "rotatelogs"
    ],
    "verify_present": "verify_tools"
  },
  {
    "bin_name": "redis_server",
    "cmd": "redis-server",
    "disabled": false,
    "doc": "",
    "host_conf_dir": "$PROJECT_ROOT/conf/${host:-$HOSTNAME}/redis_server",
    "installed": true,
    "module": "devapp",
    "module_dir": "/home/runner/work/devapps/devapps/src/devapp",
    "name": "redis_server",
    "path": "/home/runner/miniconda3/envs/redis_server/bin",
    "pkg": "redis-server",
    "port": 6379,
    "provides": [
      "redis-server",
      "redis-cli"
    ],
    "run": "redis_server",
    "systemd": "redis-server"
  }
]

[
    "  i   lc_tools ['git', 'fzf', 'jq', 'rg', 'fd', 'http', 'htop', 'tmux']",
    "  i   lc_tools_kf ['rotatelogs']",
    "s i   redis-server ['redis-server', 'redis-cli']"
]

More Tools

devapps based applications usually define more, e.g. databases, more tools or log targets.

Batteries Included - but Replaceable

DevApps' resource management is only meant as a convenience machinery to quickly get up projects or dev setups up and running. In production you'll have more distributed setups anyway, installed e.g. via Ansible and/or Container Orchestrators.

Means: You do not need to have those resources managed as shown below - we install "normal" versions, packaged as Conda packages and use standard config options (see previous chapter why).

Project Init

Via the --init_at flag you set up a new project, within the given directory, plus its resources:

CmdOutput

$ ops project --init_at . --port_offset 2000 --force
$ tree -L 2

$ ops project --init_at . --port_offset 2000 --force
07-25 16:06:22 [info     ] chdir                          [project] args=/home/runner/myproject store_log=None
07-25 16:06:22 [info     ] Directories                    [project] conda_prefix=/home/runner/miniconda3 fs_dir=/home/runner/miniconda3/fs(default)
07-25 16:06:22 [warning  ] Installing resources           [project] resources=[{'bin_name': 'lc_tools', 'conda_chan': 'conda-forge', 'conda_pkg': 'git fzf jq ripgrep fd-find httpie htop tmux', 'disabled': False, 'doc': '', 'host_conf_dir': '$PROJECT_ROOT/conf/${host:-$HOSTNAME}/lc_tools', 'installed': True, 'module': 'devapp', 'module_dir': '/home/runner/work/devapps/devapps/src/devapp', 'name': 'lc_tools', 'path': '/home/runner/miniconda3/envs/lc_tools/bin', 'provides': ['git', 'fzf', 'jq', 'rg', 'fd', 'http', 'htop', 'tmux'], 'run': 'lc_tools', 'verify_present': 'verify_tools'}, {'bin_name': 'lc_tools_kf', 'conda_chan': 'kalefranz', 'conda_pkg': 'httpd', 'disabled': False, 'doc': '', 'host_conf_dir': '$PROJECT_ROOT/conf/${host:-$HOSTNAME}/lc_tools_kf', 'installed': True, 'module': 'devapp', 'module_dir': '/home/runner/work/devapps/devapps/src/devapp', 'name': 'lc_tools_kf', 'path': '/home/runner/miniconda3/envs/lc_tools_kf/bin', 'provides': ['rotatelogs'], 'verify_present': 'verify_tools'}, {'bin_name': 'redis_server', 'cmd': 'redis-server', 'disabled': False, 'doc': '', 'host_conf_dir': '$PROJECT_ROOT/conf/${host:-$HOSTNAME}/redis_server', 'installed': True, 'module': 'devapp', 'module_dir': '/home/runner/work/devapps/devapps/src/devapp', 'name': 'redis_server', 'path': '/home/runner/miniconda3/envs/redis_server/bin', 'pkg': 'redis-server', 'port': 6379, 'provides': ['redis-server', 'redis-cli'], 'run': 'redis_server', 'systemd': 'redis-server'}]
07-25 16:06:22 [info     ] create_project_dirs            [project] store_log=None
07-25 16:06:22 [info     ] creating                       [project] dir=/home/runner/myproject/bin
07-25 16:06:22 [info     ] creating                       [project] dir=/home/runner/myproject/data
07-25 16:06:22 [info     ] creating                       [project] dir=/home/runner/myproject/log
07-25 16:06:22 [info     ] creating                       [project] dir=/home/runner/myproject/work
07-25 16:06:22 [info     ] creating                       [project] dir=/home/runner/myproject/conf
07-25 16:06:22 [info     ] creating                       [project] dir=/home/runner/myproject/tmp
07-25 16:06:22 [info     ] creating                       [project] dir=/home/runner/myproject/build
07-25 16:06:22 [info     ] chdir                          [project] args=/home/runner/myproject store_log=None
07-25 16:06:22 [info     ] git_init                       [project] store_log=None
-ne 
bin/git: line 34: MC: command not found
hint: Using 'master' as the name for the initial branch. This default branch name
hint: is subject to change. To configure the initial branch name to use in all
hint: of your new repositories, which will suppress this warning, call:
hint: 
hint:   git config --global init.defaultBranch <name>
hint: 
hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and
hint: 'development'. The just-created branch can be renamed via this command:
hint: 
hint:   git branch -m <name>
Initialized empty Git repository in /home/runner/myproject/.git/
-ne
$ tree -L 2
.
├── bin
│   ├── fd
│   ├── fzf
│   ├── git
│   ├── htop
│   ├── http
│   ├── jq
│   ├── redis-cli
│   ├── redis-server
│   ├── rg
│   ├── rotatelogs
│   └── tmux
├── build
├── conf
├── data
├── log
├── tmp
│   └── tmux
└── work

9 directories, 11 files

As you can see, we created start files in the bin subdirectory of the project directory, pointing to where the actual binaries had been installed. We did set a global port_offset, which affects any port of listening resources started.

Controlling Resources Installation

There are few options regarding which resources are to be installed, where. See the output of the -h regarding this:

CmdOutput

$ ops p -h

$ ops p -h

Creating A Project With Resources
────────────────────────────────────────────────────────────────────────────────

This plugin is helper for creating a project directory, incl. required local
resources. Your system remains unchanged, except <project_dir> and
<conda_prefix>.

It provides an install action (implicitely by providing the
--init_resource_match or --init_at switch)

Default action is: list (show installable resources, -m <match> filters).

At install we will (re-)initialize a "project_dir", at location given with
--init_at (default: '.'), incl:

❖ Installing available resources, like databases and tools within a given
  directory (conda_prefix)

❖ Creating resource start wrappers in <project_dir>/bin

❖ Generating default config when required

❖ Optionally generating systemd unit files (e.g. via: --init_create_all_units)

❖ Instances support: export <name>_instances=x before running and you'll get x
  systemd units created, for startable commands.

  Example: export client_instances=10; ops p -irm client -icau
  (Name of a resource: ops p [-m <match>])

❖ Any other parametrization: Via environ variables Check key environ vars in
  list output and also doc text.

Privilege escalation is not required for any of these steps.

Main command line flags [matching ops_devapp.project]:
appc    add_post_process_cmd     ''
Add this to all commands which have systemd service units. Intended for output redirection. Not applied when stdout is a tty.
Example: -appc='2>&1 | rotatelogs -e -n1 "$logfile" 1M' ($logfile defined in wrapper -> use single quotes).
Tip: Use rotatelogs only on powers of 10 - spotted problems with 200M. Use 100M or 1G in that case. 
cp      conda_prefix             /home/runner/miniconda3

    Resources install location, except filesystem based ones. Env vars resolved.

    Aliases:
    - local|l: <project_dir>/conda
    - default|d: $HOME/miniconda3 (default path of conda)
    - current|c: Any current conda_prefix set when running.

    Note: Installing resources outside the project keeps the project relocatable and resources reusable for other products.

damsu   delete_all_matching_service_unit_files ''
This removes all matching unit files calling devapp service wrappers. Say "service" to match all 
di      dev_install              False
Set the project up in developer mode - incl. make and poetry file machinery 
emrf    edit_matching_resource_file ''
Open resource files in $EDITOR, matching given string in their content 
f       force                    False
Assume y on all questions. Required when started w/o a tty 
fr      force_reinstall          False
Do not only install resources detected uninstalled but reinstall all 
fd      fs_dir                   default

    Filesystem based resource location. Env vars resolved.
    Aliases:
    - local|l: <project_dir>/fs
    - default|d: $HOME/miniconda3/fs (default path of conda)
    - conda|c: Within conda_prefix/fs

ia      init_at                  ''
Set up project in given directory. env vars / relative dirs supported. Sets install action implicitly 
icau    init_create_all_units    False
If set we create unit files for ALL service type resources 
icuf    init_create_unit_files   ''
List service unit files you want to have created for systemctl --user.  
Valid: Entries in rsc.provides, rsc.cmd or rsc.exe (i.e. the filename of the wrapper in bin dir). Not: rsc.name.
irm     init_resource_match      ''
Like resource match but implies install action 
i       install                  ACTION
Install 
is      install_state            False
show install state infos 
l       list                     ACTION*
Show available definition files. 
lrf     list_resources_files     ACTION
Alias for list action 
        log_resources_fully      False
Always output all settings of resources when logging 
m       resource_match           ''
Provide a match string for actions. Examples: -irm "redis, hub" or -irm '!mysql, !redis' (! negates). 
s       system                   ''
Set to a server for system mode API (e.g. lc hub(s))

-hf [match string]: List ALL (matching) flags. E.g. -hf or -hf log.

More CLI flags

More control flags are only accessible via --helpfull <match> (-hf): Try ops p -hf log_level, ops p -hf port (...)

Project initialization flags

In the example above, a --port_offset=2000 flag was given, determining the start parameters written into the redis wrapper:

$ bin/redis-server
bin/redis-server: line 35: MC: command not found
4103:C 25 Jul 2023 16:06:25.162 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
4103:C 25 Jul 2023 16:06:25.162 # Redis version=7.0.12, bits=64, commit=22906652, modified=1, pid=4103, just started
4103:C 25 Jul 2023 16:06:25.162 # Configuration loaded
4103:M 25 Jul 2023 16:06:25.163 * monotonic clock: POSIX clock_gettime
                _._                                                  
           _.-``__ ''-._                                             
      _.-``    `.  `_.  ''-._           Redis 7.0.12 (22906652/1) 64 bit
  .-`` .-```.  ```\/    _.,_ ''-._                                  
 (    '      ,       .-`  | `,    )     Running in standalone mode
 |`-._`-...-` __...-.``-._|'` _.-'|     Port: 8379
 |    `-._   `._    /     _.-'    |     PID: 4103
  `-._    `-._  `-./  _.-'    _.-'                                   
 |`-._`-._    `-.__.-'    _.-'_.-'|                                  
 |    `-._`-._        _.-'_.-'    |           https://redis.io       
  `-._    `-._`-.__.-'_.-'    _.-'                                   
 |`-._`-._    `-.__.-'    _.-'_.-'|                                  
 |    `-._`-._        _.-'_.-'    |                                  
  `-._    `-._`-.__.-'_.-'    _.-'                                   
      `-._    `-.__.-'    _.-'                                       
          `-._        _.-'                                           
              `-.__.-'                                               

4103:M 25 Jul 2023 16:06:25.163 # Server initialized
4103:M 25 Jul 2023 16:06:25.163 # WARNING Memory overcommit must be enabled! Without it, a background save or replication may fail under low memory condition. Being disabled, it can can also cause failures without low memory condition, see https://github.com/jemalloc/jemalloc/issues/1328. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
4103:M 25 Jul 2023 16:06:25.164 # WARNING You have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo madvise > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled (set to 'madvise' or 'never').
4103:M 25 Jul 2023 16:06:25.164 * Ready to accept connections

Idempotency

If you need to re-parametrize the project (e.g. set different port offsets) then run ops project --init_at again and have new start wrappers created.

Unit Files

We do not try to manage the live cycle of services but leave that to systemd (available on all Linux major distributions).

The --init_create_unit_files=<name of daemon resource> will create a unit file after installing the resource itself:

Creating a resource incl. unit file

CmdOutput

$ ops project --init_at=. --init_create_unit_files=redis-server --force

$ ops project --init_at=. --init_create_unit_files=redis-server --force
07-25 16:06:26 [info     ] chdir                          [project] args=/home/runner/myproject store_log=None
07-25 16:06:26 [info     ] Directories                    [project] conda_prefix=/home/runner/miniconda3 fs_dir=/home/runner/miniconda3/fs(default)
07-25 16:06:26 [info     ] verify_systemctl_availability  [project] store_log=None
systemctl is /usr/bin/systemctl
07-25 16:06:26 [warning  ] Installing resources           [project] resources=[{'bin_name': 'lc_tools', 'conda_chan': 'conda-forge', 'conda_pkg': 'git fzf jq ripgrep fd-find httpie htop tmux', 'disabled': False, 'doc': '', 'host_conf_dir': '$PROJECT_ROOT/conf/${host:-$HOSTNAME}/lc_tools', 'installed': True, 'module': 'devapp', 'module_dir': '/home/runner/work/devapps/devapps/src/devapp', 'name': 'lc_tools', 'path': '/home/runner/miniconda3/envs/lc_tools/bin', 'provides': ['git', 'fzf', 'jq', 'rg', 'fd', 'http', 'htop', 'tmux'], 'run': 'lc_tools', 'verify_present': 'verify_tools'}, {'bin_name': 'lc_tools_kf', 'conda_chan': 'kalefranz', 'conda_pkg': 'httpd', 'disabled': False, 'doc': '', 'host_conf_dir': '$PROJECT_ROOT/conf/${host:-$HOSTNAME}/lc_tools_kf', 'installed': True, 'module': 'devapp', 'module_dir': '/home/runner/work/devapps/devapps/src/devapp', 'name': 'lc_tools_kf', 'path': '/home/runner/miniconda3/envs/lc_tools_kf/bin', 'provides': ['rotatelogs'], 'verify_present': 'verify_tools'}, {'bin_name': 'redis_server', 'cmd': 'redis-server', 'disabled': False, 'doc': '', 'host_conf_dir': '$PROJECT_ROOT/conf/${host:-$HOSTNAME}/redis_server', 'installed': True, 'module': 'devapp', 'module_dir': '/home/runner/work/devapps/devapps/src/devapp', 'name': 'redis_server', 'path': '/home/runner/miniconda3/envs/redis_server/bin', 'pkg': 'redis-server', 'port': 6379, 'provides': ['redis-server', 'redis-cli'], 'run': 'redis_server', 'systemd': 'redis-server'}]
07-25 16:06:26 [info     ] create_project_dirs            [project] store_log=None
07-25 16:06:26 [info     ] chdir                          [project] args=/home/runner/myproject store_log=None
07-25 16:06:26 [info     ] have written unit file         [project] fn=/home/runner/.config/systemd/user/redis-server-myproject.service
07-25 16:06:26 [info     ] have written unit file         [project] fn=/home/runner/.config/systemd/user/redis-server-myproject.service
07-25 16:06:26 [info     ] git_init                       [project] store_log=None
07-25 16:06:26 [info     ] All project file created.      [project] 
07-25 16:06:26 [info     ] Enabling systemd user service units. [project] 
07-25 16:06:26 [info     ] systemd --user available, calling daemon-reload [project] 
07-25 16:06:26 [info     ] Done init.                     [project]

You control the service using systemctl --user:

CmdOutput

$ systemctl --user --no-pager start  redis-server-myproject
$ systemctl --user --no-pager status redis-server-myproject
$ systemctl --user --no-pager stop   redis-server-myproject

$ systemctl --user --no-pager start  redis-server-myproject
$ systemctl --user --no-pager status redis-server-myproject
● redis-server-myproject.service - redis-server myproject  redis-server
     Loaded: loaded (/home/runner/.config/systemd/user/redis-server-myproject.service; disabled; vendor preset: enabled)
     Active: active (running) since Tue 2023-07-25 16:06:26 UTC; 36ms ago
   Main PID: 4150 (bash)
      Tasks: 6 (limit: 8307)
     Memory: 7.9M
        CPU: 9ms
     CGroup: /user.slice/user-1001.slice/user@1001.service/app.slice/redis-server-myproject.service
             ├─4150 bash /home/runner/myproject/bin/redis-server
             └─4152 "redis-server *:6379" "" "" "" "" ""

Jul 25 16:06:26 fv-az1030-910 redis-server[4152]: 4152:M 25 Jul 2023 16:06:2…79.
Jul 25 16:06:26 fv-az1030-910 redis-server[4152]: 4152:M 25 Jul 2023 16:06:2…zed
Jul 25 16:06:26 fv-az1030-910 redis-server[4152]: 4152:M 25 Jul 2023 16:06:26.4…
Jul 25 16:06:26 fv-az1030-910 redis-server[4152]: 4152:M 25 Jul 2023 16:06:26.4…
Jul 25 16:06:26 fv-az1030-910 redis-server[4152]: 4152:M 25 Jul 2023 16:06:2….12
Jul 25 16:06:26 fv-az1030-910 redis-server[4152]: 4152:M 25 Jul 2023 16:06:2…nds
Jul 25 16:06:26 fv-az1030-910 redis-server[4152]: 4152:M 25 Jul 2023 16:06:2… Mb
Jul 25 16:06:26 fv-az1030-910 redis-server[4152]: 4152:M 25 Jul 2023 16:06:2… 0.
Jul 25 16:06:26 fv-az1030-910 redis-server[4152]: 4152:M 25 Jul 2023 16:06:2…nds
Jul 25 16:06:26 fv-az1030-910 redis-server[4152]: 4152:M 25 Jul 2023 16:06:2…ons
Hint: Some lines were ellipsized, use -l to show in full.
$ systemctl --user --no-pager stop   redis-server-myproject

Hint

In order to install unit files for ALL service type resources, you can supply --init_create_all_units, alternatively.

---

1. Technically project is implemented as a plugin of the ops tool. ↩