Adapter Build Instructions (pEpEngine 2.1.x)

ATTENTION: This document cover Release_2.1 branch only
For the WIP/DRAFT version for 3.2, please see Adapter Build Instructions (pEpEngine_3.x) DRAFT

Intended Purpose Of This Document

The original and still valid intent of these instructions is to provide and support ONE working way to build and use the stack from dev to dev. Whenever there is a change in the build process relevant for the user of the component, please reflect that in these instructions. The aim is that they are being kept up to date at all times. Its is perfectly ok of course that dev’s all have their own preference regarding dev setup. But if you have a different setup than the ‘reference build’ resulting from these instructions, you will have to translate these changes yourself, and you cant expect your particular way of doing things to be supported by the respective component maintainer.

Last Verified

The definition of a ‘verification’ is to complete all of the instructions up to at least one adapter, starting from scratch with an equivalent of a fresh, vanilla OS installation.

On 23.8.2022 by heck, on macOS 12.5. On 18.8.2021 by heck, on vanilla Debian 11 (with one minor tweak in pEpPythonAdapter setup.py).

Changes to be incorporated for Debian 11:

  • Rust can be installed with apt install rustc
  • libboost-locale1.67-dev gone, now libboost-locale1.74-dev
  • setup.py, use libboost-python39 not libboost-python3

Adapter Build Instructions

This document covers the whole download/build/install process for the whole pEp software stack on platforms:

  • Debian Linux (10/stable)
  • MacOS (>=10.10)

Adapters included in this build are:

The prerequisite for this manual is a machine running Debian Linux 10 with standard system utilities installed.

About this document

This is work in progress, if you have any problems please get in touch with me via:

  • IRC pEp Intern: heck
  • Mail: heck@pep.foundation

I depend on your feedback to improve this document and i will help you build your adapter. How to contribute:

  • Use the suggestions section in the appendix of this document
  • Change the document BUT mark your change, using “TODO:”
  • Ask me if you want to become a co-maintainer

As i am the maintainer of this document, please do not change this document in any other way.
If you are not working at pEp, please contact irc://irc.freenode.net/#prettyeasyprivacy (please highlight sva)

Limitations

Component Versions

Currently, if you follow these instructions, what will be built is just an exemplary version of each component. Or put differently: This is one of the prossible, working conbinations of versions of components. Whenever there is the requirement to use a new version of a dependency, then this document will be updated, and you see what engine/adapter versions are the ones that start using this recently updated component. So, its is supported to use any engine/adapter versions >= the one specified (in the git checkout command) in combination with the versions of all other components in here.
As a developer what you would want to do, is to use the latest 2.1.x versions of engine/adapter.

Dir Layout

Currently, the instructions will only work using the following directory layout. These folder will be created while you cary out the instructions in this manual.

~/src               - source src main dir
~/local/include     - target install dir for header files
~/local/lib         - target install dir for libraries
~/local/bin         - target install dir for binaries
~/local/share        - target install dir for shared files

Installing System Dependencies

Debian Linux

As verified to work on Debian 10 (64bit) - minimal installation

Debian Packages

Prepare your linux system, by installing all these packages. Login as root, and execute the following line.

apt install sudo curl git build-essential python3 clang pkg-config nettle-dev capnproto libssl-dev python3-lxml libtool autoconf uuid-dev sqlite3 libsqlite3-dev

This will install all the dependencies listed below.

Toolchain

system: sudo curl
build: git build-essential python3 clang pkg-config

Component Dependencies

sequoia: nettle-dev capnproto libssl-dev
yml2: python3-lxml
libetpan: libtool autoconf
pEpEngine: uuid-dev sqlite3 libsqlite3-dev

Rust

Currently, in Debian10/stable the version of rust is too old. Please install rust Rust from www.rust-lang.org (1.46 or later) After the rust installation has finished, logout and back in to refresh your shell environment.

Sudo

We need to configure sudo, so your normal user has sudo rights.
Login as root, and execute the following line, where has to be replaced with the username of your normal user.
Make sure to logout/login any session with the respective user, for the changes to become effective.

usermod -a -G sudo <USERNAME>

MacOS

As verified to work on MacOS 10.14

sudo port install gnupg2 capnproto nettle pkgconfig coreutils libtool autoconf automake

Building Dependencies

Sequoia

The pEp engine is based on Sequoia as encryption backend, written in Rust.
Login as your normal user and execute the following lines to build and install Sequoia.

please note: this build might take some time, 1, 2, or 3 coffees, depending on your system

mkdir -p ~/src/sequoia
cd ~/src/sequoia
git clone https://gitlab.com/sequoia-pgp/sequoia.git .
git checkout openpgp/v1.3.1
cargo update -p nettle-sys --precise 2.0.8
cargo build -p sequoia-openpgp-ffi --release
make -Copenpgp-ffi install PREFIX=$HOME/local

YML2

Build and install YML2 (more info: https://fdik.org/yml/index)

mkdir -p ~/src/yml2
cd ~/src/yml2
echo 'export PATH=$PATH:$HOME/src/yml2' >> ~/.bash_profile
source ~/.bash_profile
git clone https://gitea.pep.foundation/fdik/yml2 .
git checkout 2.7.4
make install

libetpan

pEp Engine requires libetpan with a set of patches that have not been upstreamed yet.

mkdir -p ~/src/libetpan
cd ~/src/libetpan
git clone https://gitea.pep.foundation/pEp.foundation/libetpan .
mkdir build
./autogen.sh --prefix=$HOME/local
make install

ASN1c

Build and install ASN1c interface description language.

mkdir -p ~/src/asn1c
cd ~/src/asn1c
git clone https://github.com/vlm/asn1c.git .
git checkout tags/v0.9.28
autoreconf -iv
./configure --prefix=$HOME/local
make install

Building The pEpEngine

Versions

The pEpEngine releases are tagged and have major, minor and patch numbers. The format of the release tags is: Release_<major>.<minor>.<patch>. Use git tag after the the git clone command to get a list of all tags/releases. Of any given minor release, please always use the latest patch release, that is, where <patch> component has the highest number.

e.g. git checkout Release_2.1.33

Build

mkdir -p ~/src/pEpEngine
cd ~/src/pEpEngine
git clone https://gitea.pep.foundation/pEp.foundation/pEpEngine .
git checkout Release_2.1.33

Create the build config file “local.conf” (paste the whole following code block into your shell)

echo '
PREFIX=$(HOME)/local
PER_MACHINE_DIRECTORY=$(PREFIX)/share/pEp

YML2_PATH=$(HOME)/src/yml2

ETPAN_LIB=-L$(PREFIX)/lib
ETPAN_INC=-I$(PREFIX)/include

ASN1C=$(PREFIX)/bin/asn1c
ASN1C_INC=-I$(PREFIX)/share/asn1c

OPENPGP=SEQUOIA
SEQUOIA_LIB=-L$(PREFIX)/lib
SEQUOIA_INC=-I$(PREFIX)/include

export PKG_CONFIG_PATH=$(PREFIX)/share/pkgconfig/
' > local.conf

Build the engine

make install
make dbinstall

Test

Currently, we can’t really “test” easily enough that the engine works, without installing external dependencies to run the test suite. But lets check that the freshly built library is existing.

ls -l ~/local/lib/libpEpEngine*
-rwxr-xr-x 1 heck heck 1857744 Feb 24 23:59 /home/heck/lib/libpEpEngine.so (or .dylib on mac)

That looks good.

Building The Adapters

The build instructions for the various adapters follow here.
Login as normal user to execute those instructions.

Versions

Adapter releases are tagged and have major, minor and patch numbers. The format of the release tags is: Release_<major>.<minor>.<patch>. Use git tag after the the git clone command to get a list of all tags/releases. Of any given minor release, please always use the latest patch release, that is, where <patch> component has the highest number.

e.g. git checkout Release_2.1.33

libpEpAdapter

libpEpAdapter is a static library containing common functionality that can be shared across adapters. Most adapters require libpEpAdapter.

mkdir -p ~/src/libpEpAdapter/
cd ~/src/libpEpAdapter/
git clone https://gitea.pep.foundation/pEp.foundation/libpEpAdapter .
git checkout Release_2.1.21
make install PREFIX=$HOME/local

pEpPythonAdapter

This adapter requires libpEpAdapter, please see build instructions above.

Dependencies Debian Linux

sudo apt install python3-setuptools libboost-python-dev libboost-locale1.67-dev python3-venv

Dependencies MacOS

sudo port install boost

Build

mkdir -p ~/src/pEpPythonAdapter/
cd ~/src/pEpPythonAdapter/
git clone https://gitea.pep.foundation/pEp.foundation/pEpPythonAdapter .
git checkout 2.1.3

Create the build config file “local.conf” (paste the whole following code block into your shell)

echo 'PREFIX=$(HOME)/local' > local.conf

Build the adapter

make venv
make

Test

make venv 
make envtest
make[1]: Entering directory '/home/heck/src/pEpPythonAdapter'
python3 -c 'import pEp'
make[1]: Leaving directory '/home/heck/src/pEpPythonAdapter'

That looks good

pEpJNIAdapter

This adapter requires libpEpAdapter, please see build instructions above.

Dependencies Debian Linux

sudo apt install default-jdk

Dependencies MacOS

sudo port install gsed

Build

mkdir -p ~/src/pEpJNIAdapter/
cd ~/src/pEpJNIAdapter/
git clone https://gitea.pep.foundation/pEp.foundation/pEpJNIAdapter .
git checkout Release_2.1.25

Create the build config file “local.conf” (paste the whole following code block into your shell)

echo '
PREFIX=$(HOME)/local
YML2_PATH=$(HOME)/src/yml2
' > local.conf

Build the adapter

make

Test

cd test/java/foundation/pEp/jniadapter/test/basic
make

.
.
.
main      : ======== TEST: 'startSync()' ================================ CTX: 'BasicTestContext' ========= [0.100 sec] === SUCCESS ======
main      : ======== TEST: 'Keygen2' ==================================== CTX: 'BasicTestContext' ========= [0.106 sec] === SUCCESS ======
Adapter.cc::shutdown - called
Adapter.cc::shutdown - sync_is_running - injecting null event

*** sync_protocol thread shutdown pEp sync protocol (null) (null)
Adapter.hxx::sync_thread - sync protocol loop ended
main      : ======== TEST: 'stopSync()' ================================= CTX: 'BasicTestContext' ========= [0.002 sec] === SUCCESS ======
main      : SUCCESS: 12
main      : FAILED : 0
main      : TOTAL  : 12

That looks good.

pEpJSONServerAdapter

The p≡p JSON Server Adapter (or short: JSON Adapter) also needs the libpEpAdapter; see above.

Because documentation shall be as near as the code it belongs to the most accurate (and least outdated) documentation is in the README.md in the root of JSON Adapter’s source repo. If that README.md contains errors, is outdated or omitting important informations, please file a bug in the “JSON” project of Gitea.

git clone https://gitea.pep.foundation/pEp.foundation/pEpJSONServerAdapter
cd pEpJSONServerAdapter

Requirements / Dependencies

  • C++ compiler: tested with g++ 4.8, 4.9, 8.3 and clang++ 2.8. Newer versions should work, too. If not -> File a bug!
  • GNU make
  • libboost-thread-dev (tested with 1.58, 1.62, 1.67, 1.70 and 1.74)
  • libboost-program-options-dev
  • libboost-filesystem-dev
  • p≡p Engine (which needs sequoia, a patched libetpan, libboost-system-dev)
  • libpEpAdapter
  • webserver
  • OSSP libuuid

The directory structure of the JSON Adapter does not – yet – fulfill the requirements described above; that is fixed soon.

For the time being, just go into the server subdirectory and type make.

Running the JSON Adapter

The stand-alone JSON Adapter (a.k.a. pEp-mini-json-adapter) can be started directly on command line, e.g. like this:

./pEp-mini-json-adapter -d 1 -D -l stderr -H ../html/

If it starts up without errors you should got a file ~/.pEp/json-token which contains the address and port the JSON Adapter is listening on, and a “security-token” which mus be given in every function call to the JSON Adapter to prevent other users on the same machine from using your JSON Adapter.

You can now open http://127.0.0.1:4223/ (it is the default port, if available. Consult the json-token file for the actually used port) in your webbrowser and see a demo page where you can emit function calls via the JSON Adapter to the pEpEngine.

Appendix

Suggestions

Please add your suggestions after this line. Thanks a lot for your contribution.