Adapter Build Instructions (pEpEngine 3.x) DRAFT

ATTENTION: This document covers Release_3.x branch only. (Not Release_2.1.x)
For the normative version for 2.1, please see Adapter Build Instructions (pEpEngine 2.1)

Intended Purpose Of This Document

The original and still valid intent of these instructions is to provide and support ONE working way to build and use the stack from dev to dev.
IMPORTANT: Whenever there is a change in the build process relevant for the user of the component, please reflect that in these instructions.
The aim is that they are being kept up to date at all times.
Its is perfectly ok of course that dev’s all have their own preference regarding dev setup. But if you have a different setup than the ‘reference build’ resulting from these instructions, you will have to translate these changes yourself, and you cant expect your particular way of doing things to be supported by the respective component maintainer.

Last Verified

The definition of a ‘verification’ is to complete all of the instructions up to at least one adapter, starting from scratch with an equivalent of a fresh, vanilla OS installation.

On 23.8.2022 by heck, on macOS 12.5.

Changes to be incorporated for Debian 11:

Rust can be installed with apt install rustc
libboost-locale1.67-dev gone, now libboost-locale1.74-dev
setup.py, use libboost-python39 not libboost-python3

Adapter Build Instructions

This document covers the whole download/build/install process for the whole pEp software stack on platforms:

Debian Linux (10/stable)
MacOS (>=12.0)

Adapters included in this build are:

pEpJNIAdapter
pEpPythonAdapter
pEpJSONServerAdapter

The prerequisite for this manual is a machine running Debian Linux 10 with standard system utilities installed.

Contribution

IMPORTANT: If you are the maintainer of any of the components used in this document, please update the build instructions of your component whenever the build of your component changes.

If you are a user of these instructions, please treat it like software. Send a bug-report, or even better a fix (PR).

Build Support

If you have any problems please get in touch with me via:

IRC pEp Intern: heck
Mail: heck@pep.foundation

If you are not working at pEp, please contact irc://irc.freenode.net/#prettyeasyprivacy (please highlight sva)

Limitations

Component Versions

Currently, if you follow these instructions, what will be built is just the very latest commit from each software repository. Please note that we use a “master/HEAD is unstable” policy. That means if you like to build a stable version of an adapter and its dependencies, you would need to build a released version of the adapter and its dependencies. But as of now, these instructions dont describe how to build specific versions of adapters/engine/etc.

To build a specific release or release candidate of an adapter/enigne, follow the DEPENDENCIES file starting from the adapter repo.

Dir Layout

Currently, the instructions assume a certain directory layout. These dirs will be created while you carry out the instructions.

~/src               - source src main dir
~/local/include     - target install dir for header files
~/local/lib         - target install dir for libraries
~/local/bin         - target install dir for binaries
~/local/share        - target install dir for shared files

Installing System Dependencies

Debian Linux

As verified to work on Debian 10 (64bit) - minimal installation

Debian Packages

Prepare your linux system, by installing all these packages. Login as root, and execute the following line.

apt install sudo curl git build-essential python3 python3-pip clang pkg-config nettle-dev capnproto libssl-dev python3-lxml libtool autoconf uuid-dev sqlite3 libsqlite3-dev

This will install all the dependencies listed below.

Toolchain

system: sudo curl
build: git build-essential python3 clang pkg-config

Systems Dependencies Per Component

Sequoia: nettle-dev capnproto libssl-dev
yml2: python3-lxml
libetpan: libtool autoconf
pEpEngine: uuid-dev sqlite3 libsqlite3-dev

Rust

Currently, in Debian10/stable the version of rust is too old. Please install rust Rust from www.rust-lang.org (1.46 or later) After the rust installation has finished, logout and back in to refresh your shell environment.

Sudo

We need to configure sudo, so your normal user has sudo rights.
Login as root, and execute the following line, where has to be replaced with the username of your normal user.
Make sure to logout/login any session with the respective user, for the changes to become effective.

usermod -a -G sudo <USERNAME>

MacOS

As verified to work on MacOS 10.14

Disable SIP
Install xcode
Install xcode command line tools
Install Python from www.python.org (version 3.8.5 or later)
Install Rust from www.rust-lang.org (1.46 or later)
Install MacPorts from www.macports.org (latest version)

sudo port install gnupg2 capnproto nettle pkgconfig coreutils libtool autoconf automake

Building Dependencies

SequoiaBackend

The pEp engine is based on Sequoia as encryption backend, written in Rust.
Login as your normal user and execute the following lines to build and install Sequoia.

please note: this build might take 1, 2, or 3 coffees, depending on your system.

mkdir -p ~/src/pEpEngineSequoiaBackend
cd ~/src/pEpEngineSequoiaBackend
git clone https://gitea.pep.foundation/pEp.foundation/pEpEngineSequoiaBackend.git .
git checkout v1.0.0
echo '
PREFIX="$(HOME)"/local
DEBUG=release
' > local.conf
make install

YML2

Build and install YML2 (more info: https://fdik.org/yml/index)

mkdir -p ~/src/yml2
cd ~/src/yml2
git clone https://gitea.pep.foundation/fdik/yml2 .
git checkout 2.7.5
make install

echo 'export PATH=$PATH:$HOME/src/yml2' >> ~/.bash_profile
source ~/.bash_profile

libetpan

pEp Engine requires libetpan with a set of patches that have not been upstreamed yet.

mkdir -p ~/src/libetpan
cd ~/src/libetpan
git clone https://gitea.pep.foundation/pEp.foundation/libetpan .
mkdir build
./autogen.sh --prefix=$HOME/local
make install

ASN1c

Build and install ASN1c interface description language.

mkdir -p ~/src/asn1c
cd ~/src/asn1c
git clone https://github.com/vlm/asn1c.git .
git checkout tags/v0.9.28 -b pep-engine
autoreconf -iv
./configure --prefix=$HOME/local
make install

libpEpTransport

mkdir -p ~/src/libpEpTransport
cd ~/src/libpEpTransport
git clone https://gitea.pep.foundation/pEp.foundation/libpEpTransport .
echo '
PREFIX=$(HOME)/local
YML2_PATH=$(HOME)/src/yml2
YML2_PROC=$(YML2_PATH)/yml2proc $(YML2_OPTS)
YML2_OPTS=--encoding=utf8
' > local.conf
make 
make install

pEpEngine

Build

mkdir -p ~/src/pEpEngine
cd ~/src/pEpEngine
git clone https://gitea.pep.foundation/pEp.foundation/pEpEngine .

Create the build config file “local.conf” (paste the whole following code block into your shell)

echo '
PREFIX=$(HOME)/local 
YML2_PATH=$(HOME)/src/yml2
ASN1C=$(PREFIX)/bin/asn1c
' > local.conf

Build the engine

make install
make dbinstall

Test

Currently, we can’t really “test” easily enough that the engine works, without installing external dependencies to run the test suite. But lets check that the freshly built library is existing.

ls -l ~/local/lib/libpEpEngine*
-rwxr-xr-x 1 heck heck 1857744 Feb 24 23:59 /home/heck/lib/libpEpEngine.so (or .dylib on mac)

That looks good.

Building The Adapters

The build instructions for the various adapters follow here.
Login as normal user to execute those instructions.

libpEpCxx11

mkdir -p ~/src/libpEpCxx11/
cd ~/src/libpEpCxx11/
git clone https://gitea.pep.foundation/pEp.foundation/libpEpCxx11.git .
echo 'PREFIX=$(HOME)/local' > local.conf
make install

libpEpAdapter

libpEpAdapter is a static library containing common functionality that can be shared across adapters. Most adapters require libpEpAdapter.

mkdir -p ~/src/libpEpAdapter/
cd ~/src/libpEpAdapter/
git clone https://gitea.pep.foundation/pEp.foundation/libpEpAdapter .
echo 'PREFIX=$(HOME)/local' > local.conf
make install

pEpPythonAdapter

This adapter requires libpEpAdapter, please see build instructions above.

Dependencies Debian Linux

sudo apt install python3-setuptools libboost-python-dev libboost-locale1.67-dev python3-venv

Dependencies MacOS

sudo port install boost

Build

mkdir -p ~/src/pEpPythonAdapter/
cd ~/src/pEpPythonAdapter/
git clone https://gitea.pep.foundation/pEp.foundation/pEpPythonAdapter .

Create the build config file “local.conf” (paste the whole following code block into your shell)

echo 'PREFIX=$(HOME)/local' > local.conf

Build the adapter

make venv
make

Test

make venv 
make envtest
make[1]: Entering directory '/home/heck/src/pEpPythonAdapter'
python3 -c 'import pEp'
make[1]: Leaving directory '/home/heck/src/pEpPythonAdapter'

That looks good

pEpJNIAdapter

This adapter requires libpEpAdapter, please see build instructions above.

Dependencies Debian Linux

sudo apt install default-jdk

Dependencies MacOS

Install Java JDK from www.oracle.com

sudo port install gsed

Build

mkdir -p ~/src/pEpJNIAdapter/
cd ~/src/pEpJNIAdapter/
git clone https://gitea.pep.foundation/pEp.foundation/pEpJNIAdapter .

Create the build config file “local.conf” (paste the whole following code block into your shell)

echo '
PREFIX=$(HOME)/local
YML2_PATH=$(HOME)/src/yml2
' > local.conf

Build the adapter

make

Test

cd test/java/foundation/pEp/jniadapter/test/basic
make

.
.
.
main      : ======== TEST: 'startSync()' ================================ CTX: 'BasicTestContext' ========= [0.100 sec] === SUCCESS ======
main      : ======== TEST: 'Keygen2' ==================================== CTX: 'BasicTestContext' ========= [0.106 sec] === SUCCESS ======
Adapter.cc::shutdown - called
Adapter.cc::shutdown - sync_is_running - injecting null event

*** sync_protocol thread shutdown pEp sync protocol (null) (null)
Adapter.hxx::sync_thread - sync protocol loop ended
main      : ======== TEST: 'stopSync()' ================================= CTX: 'BasicTestContext' ========= [0.002 sec] === SUCCESS ======
main      : SUCCESS: 12
main      : FAILED : 0
main      : TOTAL  : 12

That looks good.

pEpJSONServerAdapter

The p≡p JSON Server Adapter (or short: JSON Adapter) also needs the libpEpAdapter; see above.

Because documentation shall be as near as the code it belongs to the most accurate (and least outdated) documentation is in the README.md in the root of JSON Adapter’s source repo. If that README.md contains errors, is outdated or omitting important informations, please file a bug in the “JSON” project of Gitea.

git clone https://gitea.pep.foundation/pEp.foundation/pEpJSONServerAdapter
cd pEpJSONServerAdapter

Requirements / Dependencies

C++ compiler: tested with g++ 4.8, 4.9, 8.3, 9.4.0, 11.2.0 and clang++ 2.8. Newer versions should work, too. If not -> File a bug!
GNU make
libboost-thread-dev (tested with 1.58, 1.62, 1.67, 1.70 and 1.74)
libboost-program-options-dev
libboost-filesystem-dev
p≡p Engine (which needs pepenginesequoiabackend, a patched libetpan, libboost-system-dev)
libpEpAdapter
webserver
OSSP libuuid

The directory structure of the JSON Adapter does not – yet – fulfill the requirements described above; that is fixed soon.

For the time being, just go into the server subdirectory and type make.

Running the JSON Adapter

The stand-alone JSON Adapter (a.k.a. pEp-mini-json-adapter) can be started directly on command line, e.g. like this:

./pEp-mini-json-adapter -d 1 -D -l stderr -H ../html/

If it starts up without errors you should got a file ~/.pEp/json-token which contains the address and port the JSON Adapter is listening on, and a “security-token” which mus be given in every function call to the JSON Adapter to prevent other users on the same machine from using your JSON Adapter.

You can now open http://127.0.0.1:4223/ (it is the default port, if available. Consult the json-token file for the actually used port) in your webbrowser and see a demo page where you can emit function calls via the JSON Adapter to the pEpEngine.

Appendix

Suggestions

Please add your suggestions after this line. Thanks a lot for your contribution.