Engine/GroupEncryption
p≡p Group Encryption Protocol family
Front matter
| Title: | Managed Group Encryption Specification |
| Author: | Volker Birk |
| Team: | Engine Team |
| Reviewer(s): | Krista Bennett (incomplete during engine implementation) |
| Created: | 2020-11-13 |
| Last updated: | 2021-03-09 |
| Ticket reference: | ENGINE-606, ENGINE-822 |
Introduction
p≡p secures message based communication, which can bebilateral or multilateral. The p≡p Group Encryption Protocol family defines how to protect multilateral communication between a group of people (or machines in M2M case). There will be different protocols for managed groups (which are created and dissolved by a Group Manager) and unmanaged groups (which will be created and dissolved on-the-fly).
Goals
This document describes the Managed Group Encryption Protocol and the API for the Application Programmers. It will be later extended by description of the Unmanaged Group Encryption Protocol.
Glossary
- Personal Identity: An Identity, which is used exclusively by one User. The Address of a Personal Identity is called Personal Address.
- Group: Set of Personal Identities.
- Group Manager: Identity, who manages a Group.
- Managed Group: Group with a Group Manager.
- Group Identity: An Identity, which is used by a Group. In email this can be a mailing list address. In pEp the Address of a Group Identity is Called Group Address.
- Group Member: Personal Identity, which is element of the Group. A Group Member has a rating of
PEP_rating_reliableor better. Identities with other ratings cannot be Group Member. - Group Key: Secret Key. Default Key for the Group Identity.
- Group Rating: the Group Rating is the Minimum Rating of the Ratings of all Group Members and the straight Rating of the Group Identity.
- Group Message: Message to the Group Identity from a Group Member with a Rating of at least
PEP_rating_reliable. - Group Management Message: Message with Management data for a Managed Group.
Background
The Managed Group Encryption Protocol is derived from mailing list encryption and is a generalization of that problem. A mailing list can be encrypted by creating a list key and providing this secret key to all list members. Then list members can use that key to encrypt messages to the list, which can be read by all list members.
Abstract
Analogous to mailing list encryption, in Managed Group Encryption Protocol there is a Group Key. It is generated by the p≡p engine of the Group Manager and sent to all Group Members. When a member identity is added by the Group Manager then the Group Key is sent to this new member, too. When a Group Member is excluded from the Group by the Group Manager then a Key Reset is executed for the Group Key. This Group Key Reset is part of p≡p Key Reset Protocol. When the Group is dissolved, the management data is deleted. No further action happens to the Group Key in this case.
The Managed Group Encryption Protocol may be supported by p≡p Applications by adding User Interface components for the managment tasks for:
- creating a group,
- inviting communication partners,
- excluding communication partners, and
- dissolving a group.
These functions support a user who wants to become a Group Manager. The p≡p engine implements the protocol and provides an API as part of p≡p API for the management tasks so that application programmers can use it for connecting the UI.
Use cases
This specification defines the design of the Managed Group Encryption Protocol. The following use cases illustrate the reasons why this design was chosen.
Use case: User defines a closed group of people who need to communicate in a secure way, becoming Group Manager
The user wants to name people who form a closed group. Inside this group confidentiality must be preserved. The user is defining a Group Identity for addressing the Group.
The user needs a way to name people as communication partners. In p≡p this is done by listing identites, each person being represented by one identity in the list.
To keep group communication confidential the communication in this group must be protected. It must be protected differently than the communication between the user and a single person from this group, so communication with one single person cannot be accessed by other group members, while communication with the group can.
All Group Members must be informed about their membership.
Sample:
Alice wants to create the group dev@pep-project.org and inviting Bob and Claire. Therefore, she is using her identity Alice Cooper alice@pep.null for being Group Manager. She’s listing Bob with Bob Hope bob@pep.null and Claire Robinson claire@pep.null.
Use case: Group Manager adds a person to the group, who is becoming Group Member
The user needs a way to extend the group by adding an identity to the list. When a person is added then access to the group communication must be granted to this person.
When a Group Member is added the Group Member must be informed about the membership.
Sample:
Alice is adding Devon Parker devon@pep.null to the Group.
Use case: Group Manager removes a Group Member from the group
The user needs a way to remove a person from the group by removing the person’s identity from the list. When a person is removed then a Key Reset is to be executed to guarantee that the removed person cannot access further communication.
When a Group Member is removed the Group Member must be informed about losing membership status.
Sample:
Alice is removing Bob Hope bob@pep.null from the group.
Use case: Membership in a Managed Group granted
This information must be managed by p≡p. There is no user notification.
Use case: Membership revoked
This information must be managed by p≡p. There is no user notification.
Technical Requirements
Key Reset
The Managed Group Encryption Protocol is dependent on the implementation of the Group Key Reset protocol as part of the Key Reset protocol. The Group Key Reset protocol description is still pending.
Group management storage for the Group Manager
The Group Manager needs to store the list of identities representing the Group Members and information about whether that group is active as well as information for each invited member about whether or not they have accepted the group invitation and joined the group.
Group management storage per Group Member
A Group Member needs to store her own status in the group for each group where she is member of.
Solution
Status quo
The current implementation of p≡p does not support Group Encryption. But because Key Election is still in place, it supports Unmanaged Group Encryption implicitely when importing the right Keys. With the removal of Key Election this property will be lost. Therefore, an explicit implementation for Group Encryption is necessary.
Design
Overview
The p≡p management DB will be used to hold management data for Groups. Group Identities, the identities of communication partners who are Group members, will be stored in the p≡p engine of the Group Manager. By executing API calls for creating a Group, adding Members, excluding Members and dissolving a Group the Group Manager is triggering corresponding transactions into the management DB, and sending membership information and Keys to the Group Members. When membership information arrives then a Group Member’s p≡p engine is storing the membership in their management DB, creating a Group Identity and assigning the incoming Group Key. By doing so distributed knowledge about the Group is being available to the Group Members and the Group Manager.
Dependencies
- Group Key Reset
- implementation of Group Management API in the p≡p adapters
Security considerations
p≡p network protocols solve the authentication problem by naming the authorized entity and either using a secure channel from this entity or by delivering authentication data like a signature with each message. Implementations must check the authentication based on these two pieces of information and ignore messages, which are not properly authenticated.
While in protocols like KeyReset the messages always come from the initiator of the Key Reset, Group Encryption messages may come from other sources in future implementation. This is why Group Encryption messages always name the Group Manager.
Pros and Cons
The Managed Group Encryption Protocol covers the case of an explicitly managed group completely. It cannot be used for multilateral communication, which defines a group of recipients instantanously by message like using multiple communication partners in To: and/or CC:
API
(Taken from group.h in branch ENGINE-606 in darthmama’s development fork (which should not be checked out and used by adapter and app devs - an updated version will be pushed to master when darthmama is back from being sick. Ownership descriptions are partially incomplete at this time and will be updated shortly.)
/*************************************************************************************************
* In-memory objects and functions for representation of groups
*************************************************************************************************/
/**
* @struct pEp_member
* @brief memory object for holding information about an invited group member
* and whether they have joined the group
* (groups are persistent and are stored in the management database)
*/
typedef struct _pEp_member {
pEp_identity *ident; //!< member identity
bool joined; //!< boolean for whether the member has accepted the invite
} pEp_member;
/**
* <!-- new_member() -->
*
* @brief allocate pEp_member struct. This struct only allocates the member object for
* group representation.
*
* @param[in] ident the pEp_identity object representing the member
*
* @retval pEp_member allocated member struct on success
* NULL if ident is not present or other failure occurs
*
* @ownership ownership of all parameters goes to the struct
*
* @warning This is only an in-memory object allocator and performs NONE of the
* database or key management functions for groups or members!
*
*/
DYNAMIC_API pEp_member *new_member(pEp_identity *ident);
/**
* <!-- free_member() -->
*
* @brief deallocate pEp_member struct and the identity it points to.
*
* @param[in] member member object to be freed
*
* @ownership ALL objects pointed to by the struct will be freed!
*
* @warning This is only an in-memory object deallocator and performs NONE of the
* database or key management functions for group members!
*
*/
DYNAMIC_API void free_member(pEp_member *member);
/**
* @struct member_list
* @brief list structure for pEp_member objects
* @see pEp_member
*/
typedef struct _member_list {
pEp_member *member; //!< member object containing the identity and joined status for this list node
struct _member_list *next; //!< pointer to next node in list
} member_list;
/**
* <!-- new_memberlist() -->
*
* @brief allocate member_list node struct. This struct only allocates the member_list object for
* group representation.
*
* @param[in] member the member to be associated with this member_list node
*
* @retval member_list allocated member_list struct on success
* NULL if failure occurs (typically: out of memory)
*
* @ownership ownership of all parameters goes to the struct
*
* @warning This is only an in-memory object allocator and performs NONE of the
* database or key management functions for groups or members!
*
*/
DYNAMIC_API member_list *new_memberlist(pEp_member *member);
/**
* <!-- free_memberlist() -->
*
* @brief deallocate the node pointed to by the list argument and all nodes following it in the list
* and their associated objects
*
* @param[in] list memberlist object to be freed
*
* @ownership ALL objects pointed to by the struct will be freed!
*
* @warning This is only an in-memory object deallocator and performs NONE of the
* database or key management functions for group members!
*
*/
DYNAMIC_API void free_memberlist(member_list *list);
/**
* <!-- memberlist_add() -->
*
* @brief add memberlist node containing this member to the end of the list
* pointed to by the list argument and return a pointer to the tail of the list
*
* @param[in,out] list node pointing to the list to add to (if this is NULL,
* a new list will be created and returned)
* @param[in] member member to add to the list
*
* @retval member_list tail of list on success (or pointer to new list if input list was NULL)
* NULL if failure occurs (typically: out of memory)
*
* @ownership ownership of all parameters goes to the callee
*
* @warning This is only an in-memory object allocator and performs NONE of the
* database or key management functions for groups or members!
*
*/
DYNAMIC_API member_list *memberlist_add(member_list *list, pEp_member *member);
/**
* @struct pEp group
* @brief memory object for holding all information about a group
* (groups are persistent and are stored in the management database)
*/
typedef struct _pEp_group {
pEp_identity *group_identity; //!< identity representing this group
pEp_identity *manager; //!< identity of the group manager
member_list *members; //!< list of members associated with group
bool active; //!< boolean true if group is marked as active, else false
} pEp_group;
/**
* <!-- new_group() -->
*
* @brief allocate pEp_group struct. This function does not create
* a group in the database, it only allocates the object for
* group representation.
*
* @param[in] group_identity the pEp_identity object representing the group
* @param[in] manager the pEp_identity object representing the group's manager
* @param[in] memberlist optional list of group members
*
* @retval group allocated group struct on success
* NULL if group_identity is not present or other failure occurs
*
* @ownership ownership of all parameters goes to the struct
*
* @warning This is only an in-memory object allocator and performs NONE of the
* database or key management functions for groups!
*
*/
DYNAMIC_API pEp_group *new_group(
pEp_identity *group_identity,
pEp_identity *manager,
member_list *memberlist
);
/**
* <!-- free_group() -->
*
* @brief deallocate pEp_group struct and all objects it points to.
* This function does not dissolve groups, only deallocates the memory object
* representing a group.
*
* @param[in] group group object to be freed
*
* @ownership ALL objects pointed to by the struct will be freed!
*
* @warning This is only an in-memory object deallocator and performs NONE of the
* database or key management functions for groups!
*
*/
DYNAMIC_API void free_group(pEp_group *group);
/*************************************************************************************************
* Group management functions
*************************************************************************************************/
/**
* <!-- group_create() -->
*
* @brief Create a group in the database with the input group_identity and manager and invite new members to the group
* if this is an own group (for the external API, this is always the case).
*
* This function sets up the actual database structures for a group and invites new members to the group.
*
* For the external API, it is used when creating an own group. The group is represented by the
* incoming group_identity, which contains the user_id and address for the group.
* If no key is present for the former, it will be generated - if there is already
* a default key for the group_identity in the database, that will be used instead.
* The manager
*
* @param[in] session associated session object
* @param[in] group_identity the pEp_identity object representing the group. Must contain at least
* a user_id and address
* @param[in] manager the pEp_identity object representing the group's manager. Must contain
* a user_id and address, and there must be a default key for the manager
* present in the database
* @param[in] memberlist list of group members
* @param[in,out] group Optional reference for pointer to group object
* representing the created group.
* (When input is NULL, no object is created)
*
* @retval PEP_STATUS_OK on success
* error on failure
*
* @ownership FIXME
*
*
*/
DYNAMIC_API PEP_STATUS group_create(
PEP_SESSION session,
pEp_identity *group_identity,
pEp_identity *manager,
member_list *memberlist,
pEp_group **group
);
/**
* <!-- group_join() -->
*
* @brief Join a group for which we have received an invitation, marking
* our own membership in the database for the group and sending the manager
* a confirmation of the acceptance of the invitation
*
* @param[in] session associated session object
* @param[in] group_identity the pEp_identity object representing the group. Must contain at least
* a user_id and address
* @param[in] as_member the pEp_identity object representing the own identity we want to use to
* join the group. This must match the identity which was invited to the group.
* Must contain a user_id and address.
*
* @retval PEP_STATUS_OK on success
* error on failure
*
* @ownership FIXME
*
*
*/
DYNAMIC_API PEP_STATUS group_join(
PEP_SESSION session,
pEp_identity *group_identity,
pEp_identity *as_member
);
/**
* <!-- group_dissolve() -->
*
* @brief Dissolve a group, revoke its key, notify all members of the dissolution and
* revocation, and mark the group as inactive in the database
*
* @param[in] session associated session object
* @param[in] group_identity the pEp_identity object representing the group. Must contain at least
* a user_id and address
* @param[in] manager the pEp_identity object representing the group's manager. Must contain
* a user_id and address, and there must be a default key for the manager
* present in the database
*
* @retval PEP_STATUS_OK on success
* error on failure
*
* @ownership FIXME
*
* @warning For recipients to accept the dissolution, the sender/manager key used must be a key that they
* have a trust entry for.
*/
DYNAMIC_API PEP_STATUS group_dissolve(
PEP_SESSION session,
pEp_identity *group_identity,
pEp_identity *manager
);
/**
* <!-- group_invite_member() -->
*
* @brief Invite a member to an extant group, marking the member as invited in the database and
* sending out an invitation to said member
*
* @param[in] session associated session object
* @param[in] group_identity the pEp_identity object representing the group. Must contain at least
* a user_id and address
* @param[in] group_member the pEp_identity object representing the member to invite. Must contain
* a user_id and address, and there must be a default key for the member
* present in the database
*
* @retval PEP_STATUS_OK on success
* error on failure
*
* @ownership FIXME
*
* @note This generates a GroupCreate message even though the group already exists - this is because
* this is the accepted message format for invitations to potential members
*
*/
DYNAMIC_API PEP_STATUS group_invite_member(
PEP_SESSION session,
pEp_identity *group_identity,
pEp_identity *group_member
);
/**
* <!-- group_remove_member() -->
*
* @brief Remove a member from a group, deleting the member from the member list and executing a key
* reset on the group identity
*
* @param[in] session associated session object
* @param[in] group_identity the pEp_identity object representing the group. Must contain at least
* a user_id and address
* @param[in] group_member the pEp_identity object representing the member to remove. Must contain
* a user_id and address
*
* @retval PEP_STATUS_OK on success
* error on failure
*
* @ownership FIXME
*
* @todo Revamp implementation and execute key reset
*
*/
PEP_STATUS group_remove_member(
PEP_SESSION session,
pEp_identity *group_identity,
pEp_identity *group_member
);
/**
* <!-- group_rating() -->
*
* @brief Get the rating for this group - if the caller is the manager, this will return the aggregate rating
* of group members. For members, this will return the rating of the group_identity
*
* @param[in] session associated session object
* @param[in] group_identity the pEp_identity object representing the group. Must contain at least
* a user_id and address
* @param[in] manager the pEp_identity object representing the member to remove. Must contain
* a user_id and address
* @param[out] rating the group rating
*
* @retval PEP_STATUS_OK on success
* error on failure
*
* @ownership FIXME
*
*/
DYNAMIC_API PEP_STATUS group_rating(
PEP_SESSION session,
pEp_identity *group_identity,
pEp_identity *manager,
PEP_rating *rating
);
Actions
Creating a Group
- The Group Manager creates the Group by defining a Group Identity and a set of Personal Identities as initial Group Members.
- The Group Identity is stored.
- A Group Key is generated.
- A Create Group Message is sent to all Members. The Message contains the Group Key.
- The Group is being marked as active.
Joining a Group
- CreateGroup is Received by a Member, which is not the Group Manager.
- The Group is joined by calling
group_join(). - GroupAdopted is sent to the Group Manager
- The Group is marked as active.
Adding a Group Member
The Group Manager can add Members.
- The Group Manager calls
group_add_member() - CreateGroup is sent to the new Member
Removing a Group Member
- The Group Manager calls
group_remove_member() - The former Member is removed from the Member List
- A Key Reset is executed on the Group Identity
Dissolving a Group
- The Group Manager calls
group_dissolve() - GroupDissolve is sent to all Group Members
- Group is marked as inactive
Calculation of a Rating for a Group Identity
- If a User is Group Manager then the Rating of the Group Identity is the Group Rating.
- If a User is not Group Manager then the Rating of the Group Identity is the straight Rating of this Identity.
Storage in management DB
The following information must be stored:
- group flag for identity for marking group identites
- the list of groups
- for each group a flag if it is active
- for a Member, for each group if it has been joined
- for a Manager, for each group the Members list
- for a Manager, for each group for each Member if he/she has joined the Group
Protocol
The definition of all p≡p protocols is in the FSM Y-language defined in pEpEngine/sync/fsm.yml2
p≡p network protocols are mapped into ASN.1. Hence, the documentation of actual message data is in ASN.1. See pEpEngine/asn.1/managedgroup.asn1
For the protocol overview See in branch ENGINE-822 pEpEngine/sync/distribution.fsm
fsm ManagedGroup 2 {
version 1, 0;
message GroupCreate 2 {
field Identity groupIdentity;
field Identity manager;
}
message GroupAdopted 3 {
field Identity groupIdentity;
field Identity member;
}
message GroupDissolve 4 {
field Identity groupIdentity;
field Identity manager;
}
}GroupCreate
Sent from the Group Manager to the Group
The message must contain:
- Group Identity
- Group Mananger sending the message
Sample
GroupCreate {
groupIdentity {
address "dev@pep-project.org",
fpr "E41DDCF6AFB12E6AD9869469D0A556A8D12F1002"
user-id "pEp_own_userId",
username "p≡p development",
comm-type 255,
lang "en"
}
manager {
address "vb@pep-project.org",
fpr "AAB978A882B9A6E793960B071ADFC82AC3586C14"
user-id "pEp_own_userId",
username "Volker Birk",
comm-type 255,
lang "en"
}
}Sending
- The Group Manager sends GroupCreate when creating a Group.
- The Group Manger sends GroupCreate to all Group Members have not yet joined the group when receiving a Group Message.
If a Group Member is not reachable on a channel, which is at least yellow then the GroupDissolve message is not sent to this Group Member.
Receiving
When receiving a GroupCreate from a Group Manager it must be answered with GroupAdopted.
If the message is not coming from a channel, which is at least yellow then the message is ignored.
If the Sender Key is not a Key from the Group Manager the message is ignored.
GroupAdopted
Sent from each Group Member to the Group Manager
The message must contain:
- Group Identity
- Group Member sending the message
Sample
GroupAdopted {
groupIdentity {
address "dev@pep-project.org",
fpr "E41DDCF6AFB12E6AD9869469D0A556A8D12F1002"
user-id "23",
username "p≡p development",
comm-type 127,
lang "en"
}
member {
address "alice@pep.foundation",
fpr "AABFB526AAB9A63437ECBB071555582AC3336BC2"
user-id "pEp_own_userId",
username "Alice Marquez",
comm-type 255,
lang "es"
}
}Sending
If the Group Manager is not reachable on a channel, which is at least yellow then the GroupAdopted message is not sent.
Receiving
When the Group Manager receives GroupAdopted then it marks the Group Member as having joined the Group.
If the Group does not exist or is inactive the GroupAdopted message is ignored.
If the message is not coming from a Group Member then the message is ignored.
If the message is not coming from a channel, which is at least yellow then the message is ignored.
If the Sender Key is not a Key of the Group Member then the message is ignored.
GroupDissolve
Sent from the Group Manager to the Group
The message must contain:
- Group Identity
- Group Manager sending the message
Sample
GroupDissolve {
groupIdentity {
address "dev@pep-project.org",
fpr "E41DDCF6AFB12E6AD9869469D0A556A8D12F1002"
user-id "pEp_own_userId",
username "p≡p development",
comm-type 255,
lang "en"
}
manager {
address "vb@pep-project.org",
fpr "AAB978A882B9A6E793960B071ADFC82AC3586C14"
user-id "pEp_own_userId",
username "Volker Birk",
comm-type 255,
lang "en"
}
}Sending
The Group Manager sends this message to all Group Members when the User decides to dissolve the Group.
If a Group Member is not reachable or the channel is not at least yellow then the GroupDissolve message is not sent to this Group Member.
Receiving
A Group Member receiving this message is marking the Group as being inactive.
Inactive Groups are not supported by the Managed Group Encryption implementation and will be furthermore ignored like they would not exist.
If the message is not coming from a channel, which is at least yellow then the message is ignored.
If the message is not coming from the Group Manager then the message is ignored.
If the Sender Key is not a Key from the Group Manager the message is ignored.
Group Message from inactive Group
When receiving a Group Message from a Member of an inactive Group the Group Mananger sends GroupDissolve to the Sender.
Security considerations
- The Sender of all GroupCreate and GroupDissolve must be the Group Manager
- Only the GroupManager reacts on GroupAdopted
- A Group Message must be encrypted with a Key of the Sender.
Known issues
For the initial implementation, we do not support generation of passphrase-encrypted keys for the group. Volker has ideas as to how to fix this, but at the moment this remains unspecified.
Service requirements
Service tooling
Service must have tooling to decode messages of this protocol. This may be part of Enterprise Toolkit (see there).
Logging
There must be switchable Service Log information for:
- Group Identity
- Manager / Member data
- Actions
- Message Content
Enterprise Toolkit
The Enterprise Toolkit must contain tools for:
- Display Groups
- Search Groups by Group Identity
- Provision Groups
- Manage Groups while runtime
Affected Teams
- Implementation: Engine team
Additionally:
- All p≡p for E-Mail Application teams for implementing UI
- All p≡p Adapter developers for implementing the extension to the p≡p API
- The p≡p Enterprise Toolkit developers
- The p≡p Service & QA team
See also
Distribution.ManagedGroup
TODO
- Rename GroupAdopted to GroupJoined
- Update discussion of passphrases
- Review document for correctness in comparison to implementation (and vice versa)
