FreeIPA Identity Management planet - technical blogs

July 15, 2019

William Brown

CPU atomics and orderings explained

CPU atomics and orderings explained

Sometimes the question comes up about how CPU memory orderings work, and what they do. I hope this post explains it in a really accessible way.

Short Version - I wanna code!

Summary - The memory model you commonly see is from C++ and it defines:

  • Relaxed
  • Acquire
  • Release
  • Acquire/Release (sometimes AcqRel)
  • SeqCst

There are memory orderings - every operation is “atomic”, so will work correctly, but there rules define how the memory and code around the atomic are influenced.

If in doubt - use SeqCst - it’s the strongest guarantee and prevents all re-ordering of operations and will do the right thing.

The summary is:

  • Relaxed - no ordering guarantees, just execute the atomic as is.
  • Acquire - all code after this atomic, will be executed after the atomic.
  • Release - all code before this atomic, will be executed before the atomic.
  • Acquire/Release - both Acquire and Release - ie code stays before and after.
  • SeqCst - Stronger consistency of Acquire/Release.

Long Version … let’s begin …

So why do we have memory and operation orderings at all? Let’s look at some code to explain:

let mut x = 0;
let mut y = 0;
x = x + 3;
y = y + 7;
x = x + 4;
x = y + x;

Really trivial example - now to us as a human, we read this and see a set of operations that are linear by time. That means, they execute from top to bottom, in order.

However, this is not how computers work. First, compilers will optimise your code, and optimisation means re-ordering of the operations to achieve better results. A compiler may optimise this to:

let mut x = 0;
let mut y = 0;
// Note removal of the x + 3 and x + 4, folded to a single operation.
x = x + 7
y = y + 7;
x = y + x;

Now there is a second element. Your CPU presents the illusion of running as a linear system, but it’s actually an asynchronous, out-of-order task execution engine. That means a CPU will reorder your instructions, and may even run them concurrently and asynchronously.

For example, your CPU will have both x + 7 and y + 7 in the pipeline, even though neither operation has completed - they are effectively running at the “same time” (concurrently).

When you write a single thread program, you generally won’t notice this behaviour. This is because a lot of smart people write compilers and CPU’s to give the illusion of linear ordering, even though both of them are operating very differently.

Now we want to write a multithreaded application. Suddenly this is the challenge:

We write a concurrent program, in a linear language, executed on a concurrent asynchronous machine.

This means there is a challenge is the translation between our mind (thinking about the concurrent problem), the program (which we have to express as a linear set of operations), which then runs on our CPU (an async concurrent device).

Phew. How do computers even work in this scenario?!

Why are CPU’s async?

CPU’s have to be async to be fast - remember spectre and meltdown? These are attacks based on measuring the side effects of CPU’s asynchronous behaviour. While computers are “fast” these attacks will always be possible, because to make a CPU synchronous is slow - and asynchronous behaviour will always have measurable side effects. Every modern CPU’s performance is an illusion of async black magic.

A large portion of the async behaviour comes from the interaction of the CPU, cache, and memory.

In order to provide the “illusion” of a coherent synchronous memory interface there is no seperation of your programs cache and memory. When the cpu wants to access “memory” the CPU cache is utilised transparently and will handle the request, and only on a cache miss, will we retrieve the values from RAM.

(Aside: in almost all cases more CPU cache, not frequency will make your system perform better, because a cache miss will mean your task stalls waiting on RAM. Ohh no!)

CPU -> Cache -> RAM

When you have multiple CPU’s, each CPU has it’s own L1 cache:

CPU1 -> L1 Cache -> |              |
CPU2 -> L1 Cache -> | Shared L2/L3 | -> RAM
CPU3 -> L1 Cache -> |              |
CPU4 -> L1 Cache -> |              |

Ahhh! Suddenly we can see where problems can occur - each CPU has an L1 cache, which is transparent to memory but unique to the CPU. This means that each CPU can make a change to the same piece of memory in their L1 cache without the other CPU knowing. To help explain, let’s show a demo.

CPU just trash my variables fam

We’ll assume we now have two threads - my code is in rust again, and there is a good reason for the unsafes - this code really is unsafe!

// assume global x: usize = 0; y: usize = 0;

THREAD 1                        THREAD 2

if unsafe { *x == 1 } {          unsafe {
    unsafe { *y += 1 }              *y = 10;
}                                   *x = 1;

At the end of execution, what state will X and Y be in? The answer is “it depends”:

  • What order did the threads run?
  • The state of the L1 cache of each CPU
  • The possible interleavings of the operations.
  • Compiler re-ordering

In the end the result of x will always be 1 - because x is only mutated in one thread, the caches will “eventually” (explained soon) become consistent.

The real question is y. y could be:

  • 10
  • 11
  • 1

10 - This can occur because in thread 2, x = 1 is re-ordered above y = 10, causing the thread 1 “y += 1” to execute, followed by thread 2 assign 10 directly to y. It can also occur because the check for x == 1 occurs first, so y += 1 is skipped, then thread 2 is run, causing y = 10. Two ways to achieve the same result!

11 - This occurs in the “normal” execution path - all things considered it’s a miracle :)

1 - This is the most complex one - The y = 10 in thread 2 is applied, but the result is never sent to THREAD 1’s cache, so x = 1 occurs and is made available to THREAD 1 (yes, this is possible to have different values made available to each cpu …). Then thread 1 executes y (0) += 1, which is then sent back trampling the value of y = 10 from thread 2.

If you want to know more about this and many other horrors of CPU execution, Paul McKenny is an expert in this field and has many talks at LCA and others on the topic. He can be found on twitter and is super helpful if you have questions.

So how does a CPU work at all?

Obviously your system (likely a multicore system) works today - so it must be possible to write correct concurrent software. Cache’s are kept in sync via a protocol called MESI. This is a state machine describing the states of memory and cache, and how they can be synchronised. The states are:

  • Modified
  • Exclusive
  • Shared
  • Invalid

What’s interesting about MESI is that each cache line is maintaining it’s own state machine of the memory addresses - it’s not a global state machine. To coordinate CPU’s asynchronously message each other.

A CPU can be messaged via IPC (Inter-Processor-Communication) to say that another CPU wants to “claim” exclusive ownership of a memory address, or to indicate that it has changed the content of a memory address and you should discard your version. It’s important to understand these messages are asynchronous. When a CPU modifies an address it does not immediately send the invalidation message to all other CPU’s - and when a CPU recieves the invalidation request it does not immediately act upon that message.

If CPU’s did “synchronously” act on all these messages, they would be spending so much time handling IPC traffic, they would never get anything done!

As a result, it must be possible to indicate to a CPU that it’s time to send or acknowledge these invalidations in the cache line. This is where barriers, or the memory orderings come in.

  • Relaxed - No messages are sent or acknowledged.
  • Release - flush all pending invalidations to be sent to other CPUS
  • Acquire - Acknowledge and process all invalidation requests in my queue
  • Acquire/Release - flush all outgoing invalidations, and process my incomming queue
  • SeqCst - as AcqRel, but with some other guarantees around ordering that are beyond this discussion.

Understand a Mutex

With this knowledge in place, we are finally in a position to understand the operations of a Mutex

// Assume mutex: Mutex<usize> = Mutex::new(0);

THREAD 1                            THREAD 2

{                                   {
    let guard = mutex.lock()            let guard = mutex.lock()
    *guard += 1;                        println!(*guard)
}                                   }

We know very clearly that this will print 1 or 0 - it’s safe, no weird behaviours. Let’s explain this case though:


    let guard = mutex.lock()
    // Acquire here!
    // All invalidation handled, guard is 0.
    // Compiler is told "all following code must stay after .lock()".
    *guard += 1;
    // content of usize is changed, invalid req is queue
// Release here!
// Guard goes out of scope, invalidation reqs sent to all CPU's
// Compiler told all proceeding code must stay above this point.

            THREAD 2

                let guard = mutex.lock()
                // Acquire here!
                // All invalidations handled - previous cache of usize discarded
                // and read from THREAD 1 cache into S state.
                // Compiler is told "all following code must stay after .lock()".
            // Release here!
            // Guard goes out of scope, no invalidations sent due to
            // no modifications.
            // Compiler told all proceeding code must stay above this point.

And there we have it! How barriers allow us to define an ordering in code and a CPU, to ensure our caches and compiler outputs are correct and consistent.

Benefits of Rust

A nice benefit of Rust, and knowing these MESI states now, we can see that the best way to run a system is to minimise the number of invalidations being sent and acknowledged as this always causes a delay on CPU time. Rust variables are always mutable or immutable. These map almost directly to the E and S states of MESI. A mutable value is always exclusive to a single cache line, with no contention - and immutable values can be placed into the Shared state allowing each CPU to maintain a cache copy for higher performance.

This is one of the reasons for Rust’s amazing concurrency story is that the memory in your program map to cache states very clearly.

It’s also why it’s unsafe to mutate a pointer between two threads (a global) - because the cache of the two cpus’ won’t be coherent, and you may not cause a crash, but one threads work will absolutely be lost!

Finally, it’s important to see that this is why using the correct concurrency primitives matter - it can highly influence your cache behaviour in your program and how that affects cache line contention and performance.

For comments and more, please feel free to email me!

Shameless Plug

I’m the author and maintainer of Conc Read - a concurrently readable datastructure library for Rust. Check it out on!

July 15, 2019 02:00 PM

July 09, 2019

William Brown

I no longer recommend FreeIPA

I no longer recommend FreeIPA

It’s probably taken me a few years to write this, but I can no longer recommend FreeIPA for IDM installations.

Why not?

The FreeIPA project focused on Kerberos and SSSD, with enough other parts glued on to look like a complete IDM project. Now that’s fine, but it means that concerns in other parts of the project are largely ignored. It creates design decisions that are not scalable or robust.

Due to these decisions IPA has stability issues and scaling issues that other products do not.

To be clear: security systems like IDM or LDAP can never go down. That’s not acceptable.

What do you recommend instead?

  • Samba with AD
  • AzureAD
  • 389 Directory Server

All of these projects are very reliable, secure, scalable. We have done a lot of work into 389 to improve our out of box IDM capabilities too, but there is more to be done too. The Samba AD team have done great things too, and deserve a lot of respect for what they have done.

Is there more detail than this?

Yes - buy me a drink and I’ll talk :)

Didn’t you help?

I tried and it was not taken on board.

So what now?

Hopefully in the next year we’ll see new IDM projects for opensource released that have totally different approachs to the legacy we currently ride upon.

July 09, 2019 02:00 PM

July 04, 2019

William Brown

Using 389ds with docker

Using 389ds with docker

I’ve been wanting to containerise 389 Directory Server for a long time - it’s been a long road to get here, but I think that our container support is getting very close to a production ready and capable level. It took so long due to health issues and generally my obsession to do everything right.

Today, container support along with our new command line tools makes 389 a complete breeze to administer. So lets go through an example of a deployment now.

Please note: the container image here is a git-master build and is not production ready as of 2019-07, hopefully this changes soon.

Getting the Container

docker pull firstyear/389ds:latest

If you want to run an ephemeral instance (IE you will LOSE all your data on a restart)

docker run firstyear/389ds:latest

If you want your data to persist, you need to attach a volume at /data:

docker volume create 389ds_data
docker run -v 389ds_data:/data firstyear/389ds:latest

The image exposes ports 3389 and 3636, so you may want to consider publishing these if you want external access.

The container should now setup and run an instance! That’s it, LDAP has never been easier to deploy!

Actually Adding Some Data …

LDAP only really matters if we have some data! So we’ll create a new backend. You need to run these instructions inside the current container, so I prefix these with:

docker exec -i -t <name of container> <command>
docker exec -i -t 389inst dsconf ....

This uses the ldapi socket via /data, and authenticates you based on your process uid to map you to the LDAP administrator account - basically, it’s secure, host only admin access to your data.

Now you can choose any suffix you like, generally based on your dns name (IE I use dc=blackhats,dc=net,dc=au).

dsconf localhost backend create --suffix dc=example,dc=com --be-name userRoot
> The database was sucessfully created

Now fill in the suffix details into your configuration of the container. You’ll need to find where docker stores the volume on your host for this (docker inspect will help you). My location is listed here:

vim /var/lib/docker/volumes/389ds_data/_data/config/container.inf

--> change
# basedn ...
--> to
basedn = dc=example,dc=com

Now you can populate data into that: The dsidm command is our tool to manage users and groups of a backend, and it can provide initialised data which has best-practice aci’s, demo users and groups and starts as a great place for you to build an IDM system.

dsidm localhost initialise

That’s it! You can now see you have a user and a group!

dsidm localhost user list
> demo_user
dsidm localhost group list
> demo_group

You can create your own user:

dsidm localhost user create --uid william --cn William --displayName 'William Brown' --uidNumber 1000 --gidNumber 1000 --homeDirectory /home/william
> Successfully created william
dsidm localhost user get william

It’s trivial to add an ssh key to the user:

dsidm localhost user modify william add:nsSshPublicKey:AAA...
> Successfully modified uid=william,ou=people,dc=example,dc=com

Or to add them to a group:

dsidm localhost group add_member demo_group uid=william,ou=people,dc=example,dc=com
> added member: uid=william,ou=people,dc=example,dc=com
dsidm localhost group members demo_group
> dn: uid=william,ou=people,dc=example,dc=com

Finally, we can even generale config templates for your applications:

dsidm localhost client_config sssd.conf
dsidm localhost client_config ldap.conf
dsidm localhost client_config display

I’m happy to say, LDAP administration has never been easier - we plan to add more functionality to enabled broader ranges of administrative tasks, especially in the IDM area and management of the configuration. It’s honestly hard to beleve that in a shortlist of commands you can now have a fully functional LDAP IDM solution working.

July 04, 2019 02:00 PM

June 01, 2019

Luc de Louw

OpenID and SAML authentication with Keycloak and FreeIPA

Not every web application can handle Kerberos SSO, but some provide OpenID and/or SAML. There is how Keycloak comes into the game. You can use Keycloak to federate users from different sources. This guide shows how to integrate Keyclock and … Continue reading

The post OpenID and SAML authentication with Keycloak and FreeIPA appeared first on Luc de Louw's Blog.

by Luc de Louw at June 01, 2019 08:43 PM

May 28, 2019

Fraser Tweedale

A Distinguished Name is not a string

A Distinguished Name is not a string

Distinguished Names (DNs) are used to identify entities in LDAP databases and X.509 certificates. Although DNs are often presented as strings, they have a complex structure. Because of the numerous formal and ad-hoc serialisations have been devised, and the prevalence of ad-hoc or buggy parsers, treating DNs as string in the interals of a program inevitably leads to errors. In fact, dangerous security issues can arise!

In this post I will explain the structure of DNs, review the common serialisation regimes, and review some DN-related bugs in projects I worked on. I’ll conclude with my best practices recommendations for working with DNs.

DN structure

DNs are defined by the ITU-T X.501 standard a ASN.1 objects:

Name ::= CHOICE {
  -- only one possibility for now --
  rdnSequence RDNSequence }

RDNSequence ::= SEQUENCE OF RelativeDistinguishedName

DistinguishedName ::= RDNSequence

RelativeDistinguishedName ::=
  SET SIZE (1..MAX) OF AttributeTypeAndValue

AttributeTypeAndValue ::= SEQUENCE {
  type  ATTRIBUTE.&id({SupportedAttributes}),
  value ATTRIBUTE.&Type({SupportedAttributes}{@type}),
  ... }

The AttributeTypeAndValue definition refers to some other definitions. It means that type is an object identifier (OID) of some supported attribute, and the syntax of value is determined by type. The term attribute-value assertion (AVA) is a common synonym for AttributeTypeAndValue.

Applications define a bounded set of supported attributes. For example the X.509 certificate standard suggests a minimal set of supported attributes, and an LDAP server’s schema defines all the attribute types understood by that server. Depending on the application, a program might fail to process a DN with an unrecognised attribute type, or it might process it just fine, treating the corresponding value as opaque data.

Whereas the order of AVAs within an RDN is insignificant (it is a SET), the order of RDNs within the DN is significant. If you view the list left-to-right, then the root is on the left. X.501 formalises it thus:

Each initial sub-sequence of the name of an object is also the name of an object. The sequence of objects so identified, starting with the root and ending with the object being named, is such that each is the immediate superior of that which follows it in the sequence.

This also means that the empty DN is a valid DN.

Comparing DNs

Testing DNs for equality is an important operation. For example, when constructing an X.509 certification path, we have to find a trusted CA certificate based on the certificate chain presented by an entity (e.g. a TLS server), then verify that the chain is complete by ensuring that each Issuer DN, starting from the end entity certificate, matches the Subject DN of the certificate “above” it, all the way up to a trusted CA certificate. (Then the signatures must be verified, and several more checks performed).

Continuing with this example, if an implementation falsely determines that two equal DNs (under X.500) are inequal, then it will fail to construct the certification path and reject the certificate. This is not good. But even worse would be if it decides that two unequal DNs are in fact equal! Similarly, if you are issuing certificates or creating LDAP objects or anything else, a user could exploit bugs in your DN handling code to cause you to issue certificates, or create objects, that you did not intend.

Having motivated the importance of correct DN comparison, well, how do you compare DNs correctly?

First, the program must represent the DNs according to their true structure: a list of sets (RDNs) of attribute-value pairs (AVAs). If the DNs are not already represented this way in the program, they must be parsed or processed—correctly.

Now that the structure is correct, AVAs can be compared for equality. Each attribute type defines an equality matching rule that says how values should be compared. In some cases this is just binary matching. In other cases, normalisation or other rules must be applied to the values. For example, some string types may be case insensitive.

A notable case is the DirectoryString syntax used by several attribute types in X.509:

DirectoryString ::= CHOICE {
    teletexString       TeletexString   (SIZE (1..MAX)),
    printableString     PrintableString (SIZE (1..MAX)),
    universalString     UniversalString (SIZE (1..MAX)),
    utf8String          UTF8String      (SIZE (1..MAX)),
    bmpString           BMPString       (SIZE (1..MAX)) }

DirectoryString supports a choice of string encodings. Values of use PrintableString orr UTF8String encoding must be preprocessed using the LDAP Internationalized String Preparation rules (RFC 4518), including case folding and insignificant whitespace compression.

Taking the DN as a whole, two DNs are equal if they have the same RDNs in the same order, and two RDNs are equal if they have the same AVAs in any order (i.e. sets of equal size, with each AVA in one set having a matching AVA in the other set).

Ultimately this means that, despite X.509 certificates using Distinguised Encoding Rules (DER) for serialisation, there can still be multiple ways to represent equivalent data (by using different string encodings). Therefore, binary matching of serialised DNs, or even binary matching of individual attribute values, is incorrect behaviour and may lead to failures.

String representations

Several string representations of DNs, both formally-specified and ad-hoc, are in widespread use. In this section I’ll list some of the more important ones.

Because DNs are ordered, one of the most obvious characteristics of a string representation is whether it lists the RDNs in forward or reverse order, i.e. with the root at the left or right. Some popular libraries and programs differ in this regard.

As we look at some of these common implementations, we’ll use the following DN as an example:

SEQUENCE (3 elem)
  SET (2 elem)
    SEQUENCE (2 elem)
      OBJECT IDENTIFIER countryName
      PrintableString AU
    SEQUENCE (2 elem)
      OBJECT IDENTIFIER stateOrProvinceName
      PrintableString Queensland
  SET (1 elem)
    SEQUENCE (2 elem)
      OBJECT IDENTIFIER organizationName
      PrintableString Acme, Inc.
  SET (1 elem)
    SEQUENCE (2 elem)
      OBJECT IDENTIFIER commonName
      PrintableString CA

RFC 4514

CN=CA,O=Acme\, Inc.,C=AU+ST=Queensland
CN=CA,O=Acme\2C Inc.,C=AU+ST=Queensland

RFC 4514 defines the string representation of distinguished names used in LDAP. As such, there is widespread library support for parsing and printing DNs in this format. The RDNs are in reverse order, separated by ,. Special characters are escaped using backslash (\), and can be represented using the escaped character itself (e.g. \,) or two hex nibbles (\2C). The AVAs within a multi-valued RDN are separated by +, in any order.

Due to the multiple ways of escaping special characters, this is not a distinguished encoding.

This format is used by GnuTLS, OpenLDAP and FreeIPA, among other projects.

RFC 1485

CN=CA,O="Acme, Inc.",C=AU+ST=Queensland

RFC 1485 is a predecessor of a predecessor (RFC 1779) of a predecessor (RFC 2253) of RFC 4514. There are some differences from RFC 4514. For example, special character escapes are not supported; quotes must be used. This format is still relevant today because NSS uses it for pretty-printing and parsing DNs.


OpenSSL prints DNs in its own special way. Unlike most other implementations, it works with DNs in forward order (root at left). The pretty-print looks like:

C = AU + ST = Queensland, O = "Acme, Inc.", CN = CA

The format when parsing is different again. Some commands need a flag to enable support for multi-valued RDNs; e.g. openssl req -multivalue-rdn ....

/C=AU+ST=Queensland/O=Acme, Inc./CN=CA

OpenSSL can also read DNs from a config file where AVAs are given line by line (see config and x509v3_config(5)). But this is not a DN string representation per se so I won’t cover it here.

Bugs, bugs, bugs

Here are three interesting bugs I discovered, related to DN string encoding.

389 DS #49543: certmap fails when Issuer DN has comma in name

389 DS supports TLS certificate authentication for binding to LDAP. Different certificate mapping (certmap) policies can be defined for different CAs. The issuer DN in the client certificate is used to look up a certmap configuration. Unfortunately, a string comparison was used to perform this lookup. 389 uses NSS, which serialised the DN using RFC 1485 syntax. If this disagreed with how the DN in the certmap configuration appeared (after normalisation), the lookup—hence the LDAP bind—would fail. The normalisation function was also buggy.

The fix was to parse the certmap DN string into an a NSS CertNAME using the CERT_AsciiToName routine, then compare the Issuer DN from the certificate against it using the NSS DN comparison routine (CERT_CompareName). The buggy normalisation routine was deleted.

Certmonger #90: incorrect DN in CSR

Certmonger stores tracking request configuration in a flat text file. This configuration includes the string representation of the DN, ostensibly in RFC 4514 syntax. When constructing a CSR for the tracking request, it parsed the DN then used the result to construct an OpenSSL X509_NAME, which would be used in OpenSSL routines to create the CSR.

Unfortunately, the DN parsing implementation—a custom routine in Certmonger itself—was busted. A DN string like:

CN=IPA RA,O=Acme\, Inc.,ST=Massachusetts,C=US

Resulted in a CSR with the following DN:

CN=IPA RA,CN=Inc.,O=Acme\\,ST=Massachusetts,C=US

The fix was to remove the buggy parser and use the OpenLDAP ldap_str2dn routine instead. This was a joint effort between Rob Crittenden and myself.

FreeIPA #7750: invalid modlist when attribute encoding can vary

FreeIPA’s LDAP library, ipaldap, uses python-ldap for handling low-level stuff and provides a lot of useful stuff on top. One useful thing it does is keeps track of the original attribute values for an object, so that we can perform changes locally and efficiently produce a list of modifications (modlist) for when we want to update the object at the server.

ipaldap did not take into account the possibility of the attribute encoding returned by python-ldap differing from the attribute encoding produced by FreeIPA. A disagreement could arise when DN attribute values contained special characters requiring escaping. For example, python-ldap escaped characters using hex encoding:

CN=CA,O=Red Hat\2C Inc.,L=Brisbane,C=AU

The representation produced by python-ldap is recorded as the original value of the attribute. However, if you wrote the same attribute value back, it would pass through FreeIPA’s encoding routine, which might encode it differently and record it as a new value:

CN=CA,O=Red Hat\, Inc.,L=Brisbane,C=AU

When you go to update the object, the modlist would look like:

[ (ldap.MOD_ADD, 'ipacaissuerdn',
    [b'CN=CA,O=Red Hat\, Inc.,L=Brisbane,C=AU'])
, (ldap.MOD_DELETE, 'ipacaissuerdn',
    [b'CN=CA,O=Red Hat\2C Inc.,L=Brisbane,C=AU'])

Though encoded differently, these are the same value but that in itself is not a problem. The problem is that the server also has the same value, and processing the MOD_ADD first results in an attributeOrValueExists error. You can’t add a value that’s already there!

The ideal fix for this would be to update ipaldap to record all values as ASN.1 data or DER, rather than strings. But that would be a large and risky change. Instead, we work around the issue by always putting deletes before adds in the modlist. LDAP servers process changes in the order they are presented (389 DS does so atomically). So deleting an attribute value then adding it straight back is a safe, albeit inefficient, workaround.


So you have to compare or handle some DNs. What do you do? My recommendations are:

  • If you need to print/parse DNs as strings, if possible use RFC 4514 because it has the most widespread library support.
  • Don’t write your own DN parsing code. This is where security vulnerabilities are most likely. Use existing library routines for parsing DNs. If you have no other choice, take extreme care and if possible use a parser combinator library or parser generator to make the definitions more declarative and reduce likelihood of error.
  • Always decode attribute values (if the DN parsing routine doesn’t do it for you). This avoids confusion where attribute values could be encoded in different ways (due to escaped characters or differing string encodings).
  • Use established library routines for comparing DNs using the internal DN structures, not strings.

Above all, just remember: a Distinguished Name is not a string, so don’t treat it like a string. For sure it’s more work, but DNs need special treatment or bugs will certainly arise.

That’s not to say that “native” DN parsing and comparison routines are bug-free. They are not. A common error is equal DNs comparing inequal due to differing attribute string encodings (e.g. PrintableString versus UTF8String). I have written about this in a previous post. In Dogtag we’ve enountered this kind of bug quite a few times. In these situations the DN comparison should be fixed, but it may be a satisfactory workaround to serialise both DNs and perform a string comparison.

Another common issue is lack of support for multi-valued RDNs. A few years ago we wanted to switch FreeIPA’s certificate handling from python-nss to the cryptography library. I had to add support for multi-valued RDNs before we could make the switch.

A final takeaway for authors of standards. Providing multiple ways to serialise the same value leads to incompatibilities and bugs. For sure, there is a tradeoff between usability, implementation complexity and risk of interoperability issues and bugs. RFC 4514 would be less human-friendly if it only permitted hex-escapes. But implementations would be simpler and the interop/bug risk would be reduced. It’s important to think about these tradeoffs and the consequences, especially for standards and protocols relating to security.

May 28, 2019 12:00 AM

May 24, 2019

Fraser Tweedale

Fixing expired system certificates in FreeIPA

Fixing expired system certificates in FreeIPA

In previous posts I outlined and demonstrated the pki-server cert-fix tool. This tool is part of Dogtag PKI. I also discussed what additional functionality would be needed to successfully use this tool in a FreeIPA environment.

This post details the result of the effort to make cert-fix useful for FreeIPA administrators. We implemented a wrapper program, ipa-cert-fix, which performs FreeIPA-specific steps in addition to executing pki-server cert-fix.

What does ipa-cert-fix do?

In brief, the steps performed by ipa-cert-fix are:

  1. Inspect deployment to work out which certificates need renewing. This includes both Dogtag system certificates, FreeIPA-specific certificates (HTTP, LDAP, KDC and IPA RA).
  2. Print intentions and await operator confirmation.
  3. Invoke pki-server cert-fix to renew expired certificates, including FreeIPA-specific certificates.
  4. Install renewed FreeIPA-specific certificates to their respective locations.
  5. If any shared certificates were renewed (Dogtag system certificates excluding HTTP, and IPA RA), import them to the LDAP ca_renewal subtree and set the caRenewalMaster configuration to be the current server. This allows CA replicas to pick up the renewed shared certificates.
  6. Restart FreeIPA (ipactl restart).


For this demonstration I used a deployment with the following characteristics:

  • Two servers, f29-0 and f29-1, with CA on both.
  • f29-0 is the current CA renewal master.
  • A KRA instance is installed on f29-1.
  • The deployment was created on 2019-05-24, so most of the certificates expire on or before 2021-05-24 (the exception being the CA certificate).

On both servers I disabled chronyd and put the clock forward 27 months, so that all the certificates (except the IPA CA itself) are expired:

[f29-1] ftweedal% sudo systemctl stop chronyd
[f29-1] ftweedal% date
Fri May 24 12:01:16 AEST 2019
[f29-1] ftweedal% sudo date 082412012021
Tue Aug 24 12:01:00 AEST 2021

We want to perform this step on all machines in the topology. After all, we are simulating the passage of time.

After ipactl restart the Dogtag CA did not start, and we cannot communicate with FreeIPA due to the expired HTTP certificate:

[f29-1] ftweedal% sudo ipactl status
Directory Service: RUNNING
krb5kdc Service: RUNNING
kadmin Service: RUNNING
httpd Service: RUNNING
ipa-custodia Service: RUNNING
pki-tomcatd Service: STOPPED
ipa-otpd Service: RUNNING
ipa: INFO: The ipactl command was successful

[f29-1] ftweedal% ipa user-find
ipa: ERROR: cannot connect to 'https://f29-1.ipa.local/ipa/json':
  [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed:
  certificate has expired (_ssl.c:1056)

Fixing the first server

I will repair f29-1 first, so that we can see why resetting the CA renewal master is an important step performed by ipa-cert-fix.

I ran ipa-cert-fix as root. It analyses the server, then prints a warning and the list of certificates to be renewed, and asks for confirmation:

[f29-1] ftweedal% sudo ipa-cert-fix


ipa-cert-fix is intended for recovery when expired certificates
prevent the normal operation of FreeIPA.  It should ONLY be used
in such scenarios, and backup of the system, especially certificates

The following certificates will be renewed:

Dogtag sslserver certificate:                                                                                                                                                                                [2/34]
  Subject: CN=f29-1.ipa.local,O=IPA.LOCAL 201905222205                                                                                                                                                             
  Serial:  13
  Expires: 2021-05-12 05:55:47

Dogtag subsystem certificate:
  Subject: CN=CA Subsystem,O=IPA.LOCAL 201905222205
  Serial:  4
  Expires: 2021-05-11 12:07:11

Dogtag ca_ocsp_signing certificate:
  Subject: CN=OCSP Subsystem,O=IPA.LOCAL 201905222205
  Serial:  2
  Expires: 2021-05-11 12:07:11

Dogtag ca_audit_signing certificate:
  Subject: CN=CA Audit,O=IPA.LOCAL 201905222205
  Serial:  5
  Expires: 2021-05-11 12:07:12

Dogtag kra_transport certificate:
  Subject: CN=KRA Transport Certificate,O=IPA.LOCAL 201905222205
  Serial:  268369921
  Expires: 2021-05-12 06:00:10

Dogtag kra_storage certificate:
  Subject: CN=KRA Storage Certificate,O=IPA.LOCAL 201905222205
  Serial:  268369922
  Expires: 2021-05-12 06:00:10

Dogtag kra_audit_signing certificate:
  Subject: CN=KRA Audit,O=IPA.LOCAL 201905222205
  Serial:  268369923
  Expires: 2021-05-12 06:00:11

IPA IPA RA certificate:
  Subject: CN=IPA RA,O=IPA.LOCAL 201905222205
  Serial:  7
  Expires: 2021-05-11 12:07:47

IPA Apache HTTPS certificate:
  Subject: CN=f29-1.ipa.local,O=IPA.LOCAL 201905222205
  Serial:  12
  Expires: 2021-05-23 05:54:11

IPA LDAP certificate:
  Subject: CN=f29-1.ipa.local,O=IPA.LOCAL 201905222205
  Serial:  11
  Expires: 2021-05-23 05:53:58

IPA KDC certificate:
  Subject: CN=f29-1.ipa.local,O=IPA.LOCAL 201905222205
  Serial:  14
  Expires: 2021-05-23 05:57:50

Enter "yes" to proceed:

Observe that the KRA certificates are included (we are on f29-1). I type “yes” and continue. After a few minutes the process has completed:

Renewed Dogtag sslserver certificate:
  Subject: CN=f29-1.ipa.local,O=IPA.LOCAL 201905222205
  Serial:  268369925
  Expires: 2023-08-14 02:19:33

... (9 certificates elided)

Renewed IPA KDC certificate:
  Subject: CN=f29-1.ipa.local,O=IPA.LOCAL 201905222205
  Serial:  268369935
  Expires: 2023-08-25 02:19:42

Becoming renewal master.
The ipa-cert-fix command was successful

As suggested by the expiry dates, it took about 11 seconds to renew all 11 certifiates. So why did it take so long? The pki-server cert-fix command, which is part of Dogtag and invoked by ipa-cert-fix, restarts the Dogtag instance as its final step. Although a new LDAP certificate was issued, it is not yet been installed in 389’s certificate database. Dogtag fails to start; it cannot talk to LDAP because of the expired certificate, and the restart operation hangs for a while. ipa-cert-fix knows to expect this and ignores the pki-server cert-fix failure when the LDAP certificate needs renewal.

ipa-cert-fix also reported that it was setting the renewal master (because shared certificates were renewed). Let’s check the server status and verify the configuration.

[f29-1] ftweedal% sudo ipactl status
Directory Service: RUNNING
krb5kdc Service: RUNNING
kadmin Service: RUNNING
httpd Service: RUNNING
ipa-custodia Service: RUNNING
pki-tomcatd Service: RUNNING
ipa-otpd Service: RUNNING
ipa: INFO: The ipactl command was successful

The server is up and running.

[f29-1] ftweedal% kinit admin
Password for admin@IPA.LOCAL:
Password expired.  You must change it now.
Enter new password:
Enter it again:

Passwords have expired (due to time-travel).

[f29-1] ftweedal% ipa config-show |grep renewal
  IPA CA renewal master: f29-1.ipa.local

f29-1 has indeed become the renewal master. Oh, and the HTTP and LDAP certifiate have been fixed.

[f29-1] ftweedal% ipa cert-show 1 | grep Subject
  Subject: CN=Certificate Authority,O=IPA.LOCAL 201905222205

And the IPA framework can talk to Dogtag. This proves that the IPA RA and Dogtag HTTPS and subsystem certificates are valid.

Fixing subsequent servers

Jumping back onto f29-0, let’s look at the Certmonger request statuses:

[f29-0] ftweedal% sudo getcert list \
                  | egrep '^Request|status:|subject:'
Request ID '20190522120745':
        status: CA_UNREACHABLE
        subject: CN=IPA RA,O=IPA.LOCAL 201905222205
Request ID '20190522120831':
        status: CA_UNREACHABLE
        subject: CN=CA Audit,O=IPA.LOCAL 201905222205
Request ID '20190522120832':
        status: CA_UNREACHABLE
        subject: CN=OCSP Subsystem,O=IPA.LOCAL 201905222205
Request ID '20190522120833':
        status: CA_UNREACHABLE
        subject: CN=CA Subsystem,O=IPA.LOCAL 201905222205
Request ID '20190522120834':
        status: MONITORING
        subject: CN=Certificate Authority,O=IPA.LOCAL 201905222205
Request ID '20190522120835':
        status: CA_UNREACHABLE
        subject: CN=f29-0.ipa.local,O=IPA.LOCAL 201905222205
Request ID '20190522120903':
        status: CA_UNREACHABLE
        subject: CN=f29-0.ipa.local,O=IPA.LOCAL 201905222205
Request ID '20190522120932':
        status: CA_UNREACHABLE
        subject: CN=f29-0.ipa.local,O=IPA.LOCAL 201905222205
Request ID '20190522120940':
        status: CA_UNREACHABLE
        subject: CN=f29-0.ipa.local,O=IPA.LOCAL 201905222205

The MONITORING request is the CA certificate. All the other requests are stuck in CA_UNREACHABLE.

The Certmonger tracking requests need to communicate with LDAP to retrieve shared certificates. So we have to ipactl restart with --force to ignore individual service startup failures (Dogtag will fail):

[f29-0] ftweedal% sudo ipactl restart --force
Skipping version check
Starting Directory Service
Starting krb5kdc Service
Starting kadmin Service
Starting httpd Service
Starting ipa-custodia Service
Starting pki-tomcatd Service
Starting ipa-otpd Service
ipa: INFO: The ipactl command was successful

[f29-0] ftweedal% sudo ipactl status
Directory Service: RUNNING
krb5kdc Service: RUNNING
kadmin Service: RUNNING
httpd Service: RUNNING
ipa-custodia Service: RUNNING
pki-tomcatd Service: STOPPED
ipa-otpd Service: RUNNING
ipa: INFO: The ipactl command was successful

Now Certmonger is able to renew the shared certificates by retrieving the new certificate from LDAP. The IPA-managed certificates are also able to be renewed by falling back to requesting them from another CA server (the already repaired f29-1). After a short wait, getcert list shows that all but one of the certificates have been renewed:

[f29-0] ftweedal% sudo getcert list \
                  | egrep '^Request|status:|subject:'
Request ID '20190522120745':
        status: MONITORING
        subject: CN=IPA RA,O=IPA.LOCAL 201905222205
Request ID '20190522120831':
        status: MONITORING
        subject: CN=CA Audit,O=IPA.LOCAL 201905222205
Request ID '20190522120832':
        status: MONITORING
        subject: CN=OCSP Subsystem,O=IPA.LOCAL 201905222205
Request ID '20190522120833':
        status: MONITORING
        subject: CN=CA Subsystem,O=IPA.LOCAL 201905222205
Request ID '20190522120834':
        status: MONITORING
        subject: CN=Certificate Authority,O=IPA.LOCAL 201905222205
Request ID '20190522120835':
        status: CA_UNREACHABLE
        subject: CN=f29-0.ipa.local,O=IPA.LOCAL 201905222205
Request ID '20190522120903':
        status: MONITORING
        subject: CN=f29-0.ipa.local,O=IPA.LOCAL 201905222205
Request ID '20190522120932':
        status: MONITORING
        subject: CN=f29-0.ipa.local,O=IPA.LOCAL 201905222205
Request ID '20190522120940':
        status: MONITORING
        subject: CN=f29-0.ipa.local,O=IPA.LOCAL 201905222205

The final CA_UNREACHABLE request is the Dogtag HTTP certificate. We can now run ipa-cert-fix on f29-0 to repair this certificate:

[f29-0] ftweedal% sudo ipa-cert-fix


ipa-cert-fix is intended for recovery when expired certificates
prevent the normal operation of FreeIPA.  It should ONLY be used
in such scenarios, and backup of the system, especially certificates

The following certificates will be renewed:

Dogtag sslserver certificate:
  Subject: CN=f29-0.ipa.local,O=IPA.LOCAL 201905222205
  Serial:  3
  Expires: 2021-05-11 12:07:11

Enter "yes" to proceed: yes
Renewed Dogtag sslserver certificate:
  Subject: CN=f29-0.ipa.local,O=IPA.LOCAL 201905222205
  Serial:  15
  Expires: 2023-08-14 04:25:05

The ipa-cert-fix command was successful

All done?

Yep. A subsequent execution of ipa-cert-fix shows that there is nothing to do, and exits:

[f29-0] ftweedal% sudo ipa-cert-fix
Nothing to do.
The ipa-cert-fix command was successful

Feature status

Against the usual procedure for FreeIPA (and Red Hat projects in general), ipa-cert-fix was developed “downstream-first”. It has been merged to the ipa-4-6 branch, but there might not even be another upstream release from that branch. But there might be a future RHEL release based on that branch (the savvy reader might infer a high degree of certainty, given we actually bothered to do that…)

In the meantime, work to forward-port the feature to master and newer branches is ongoing. I hope that it will be merged in the next week or so.

May 24, 2019 12:00 AM

May 07, 2019

Rob Crittenden

How do I revert back to using IPA-issued Web & LDAP certs?

For IPA v4.6.x.

So you have an IPA installation with a CA and you decided you don’t want your users to have to install the IPA CA certificate(s) so instead you want to use certificates for the Web and LDAP using some known 3rd party issuer. Sure, that works fine. You’d do something like:

Install the 3rd party CA chain and update your IPA master:

# ipa-cacert-manage install /path/to/root.pem -t CT,,
# ipa-cacert-manage install intermediate.cert.pem -t CT,,
# ipa-certupdate

Install the 3rd-party provided server certificate. In this case I have it as two separate files, the public cert and the private key.

# ipa-server-certinstall --dirman-password password -w -d --pin '' server.cert.pem server.cert.key root.pem \

Great. IPA is working fine and my users don’t need to import the IPA CA.

Two years later…

My 3rd party certs are expiring soon and I don’t want to pay for new ones and I want to switch back to using IPA-issued certificates. We can use certmonger for that. This assumes that your CA is still up and functioning properly.

I’d start by backing up the two NSS databases. It is safest to do this offline (ipactl stop). You need to copy *.db from /etc/dirsrv/slapd-EXAMPLE-TEST and /etc/httpd/alias someplace safe, then restart the world (ipactl start).

First the web server:

# ipa-getcert request -d /etc/httpd/alias -n Server-Cert -K HTTP/`hostname` -D `hostname` -C /usr/libexec/ipa/certmonger/restart_httpd -p /etc/httpd/alias/pwdfile.txt

Edit /etc/httpd/conf.d/nss.conf and replace the value of NSSNickname with Server-Cert.

Wait a bit to be sure the cert is issued. You can run this to see the status:

# ipa-getcert list -d /etc/httpd/alias -n Server-Cert

Now the LDAP server:

# ipa-getcert request -d /etc/dirsrv/slapd-EXAMPLE-TEST -n Server-Cert -D `hostname` -K ldap/`hostname` -C "/usr/libexec/ipa/certmonger/restart_dirsrv EXAMPLE-TEST" -p /etc/dirsrv/slapd-EXAMPLE-TEST/pwdfile.txt

Similarly wait for it to be issued. To track the status:

# ipa-getcert list -d /etc/dirsrv/slapd-EXAMPLE-TEST -n Server-Cert

Once it is issued run:

# ipactl stop

Now edit /etc/dirsrv/slapd-EXAMPLE-TEST/dse.ldif. We could do this while the server is online but we need to restart anyway and your favorite editor is easier than ldapmodify. Replace the value of nsSSLPersonalitySSL with Server-Cert

Now restart the world:

# ipactl start

Connect to each port if you want to confirm that the certificate and chain are correct, e.g.

# openssl s_client -host `hostname` -port 443
depth=1 O = EXAMPLE.TEST, CN = Certificate Authority
verify return:1
depth=0 O = EXAMPLE.TEST, CN = ipa.example.test
verify return:1
Certificate chain
0 s:/O=EXAMPLE.TEST/CN=ipa.example.test
i:/O=EXAMPLE.TEST/CN=Certificate Authority
1 s:/O=EXAMPLE.TEST/CN=Certificate Authority
i:/O=EXAMPLE.TEST/CN=Certificate Authority

by rcritten at May 07, 2019 02:43 AM

April 27, 2019

William Brown

Implementing Webauthn - a series of complexities …

Implementing Webauthn - a series of complexities …

I have recently started to work on a rust webauthn library, to allow servers to be implemented. However, in this process I have noticed a few complexities to an API that should have so much promise for improving the state of authentication. So far I can say I have not found any cryptographic issues, but the design of the standard does raise questions about the ability for people to correctly implement Webauthn servers.

Odd structure decisions

Webauth is made up of multiple encoding standards. There is a good reason for this, which is that the json parts are for the webbrowser, and the cbor parts are for ctap and the authenticator device.

However, I quickly noticed an issue in the Attestation Object, as described here . Can you see the problem?

The problem is that the Authenticator Data relies on hand-parsing bytes, and has two structures that are concatenated with no length. This means:

  • You have to hand parse bytes 0 -> 36
  • You then have to CBOR deserialise the Attested Cred Data (if present)
  • You then need to serialise the ACD back to bytes and record that length (if your library doesn’t tell you how long the amount of data is parsed was).
  • Then you need to CBOR deserialise the Extensions.

What’s more insulting about this situation is that the Authenticator Data literally is part of the AttestationObject which is already provided as CBOR! There seems to be no obvious reason for this to require hand-parsing, as the Authenticator Data which will be signature checked, has it’s byte form checked, so you could have the AttestationObject store authDataBytes, then you can CBOR decode the nested structure (allowing the hashing of the bytes later).

There are many risks here because now you have requirements to length check all the parameters which people could get wrong - when CBOR would handle this correctly for you you and provides a good level of correctness that the structure is altered. I also trust the CBOR parser authors to do proper length checks too compared to my crappy byte parsing code!

Confusing Naming Conventions and Layout

The entire standard is full of various names and structures, which are complex, arbitrarily nested and hard to see why they are designed this way. Perhaps it’s a legacy compatability issue? More likely I think it’s object-oriented programming leaking into the specification, which is a paradigm that is not universally applicable.

Regardless, it would be good if the structures were flatter, and named better. There are many confusing structure names throughout the standard, and it can sometimes be hard to identify what you require and don’t require.

Additionally, naming of fields and their use, uses abbrivations to save bandwidth, but makes it hard to follow. I did honestly get confused about the difference between rp (the relying party name) and rp_id, where the challenge provides rp, and the browser response use rp_id.

It can be easy to point fingers and say “ohh William, you’re just not reading it properly and are stupid”. Am I? Or is it that humans find it really hard to parse data like this, and our brains are better suited to other tasks? Human factors are important to consider in specification design both in naming of values, consistency of their use, and appropriate communication as to how they are used properly. I’m finding this to be a barrier to correct implementation now (especially as the signature verification section is very fragmented and hard to follow …).

Crypto Steps seem complex or too static

There are a lot of possible choices here - there are 6 attestation formats and 5 attestation types. As some formats only do some types, there are then 11 verification paths you need to implement for all possible authenticators. I think this level of complexity will lead to mistakes over a large number of possible code branch paths, or lacking support for some device types which people may not have access to.

I think it may have been better to limit the attestation format to one, well defined format, and within that to limit the attestation types available to suit a more broad range of uses.

It feels a lot like these choice are part of some internal Google/MS/Other internal decisions for high security devices, or custom deviges, which will be internally used. It’s leaked into the spec and it raises questions about the ability for people to meaningfully implement the full specification for all possible devices, let alone correctly.

Some parts even omit details in a cryptographic operation, such as here in verification step 2, it doesn’t even list what format the bytes are. (Hint: it’s DER x509).

What would I change?

  • Be more specific

There should be no assumptions about format types, what is in bytes. Be verbose, detailed and without ambiguity.

  • Use type safe, length checked structures.

I would probably make the entire thing a single CBOR structure which contains other nested structures as required. We should never have to hand-parse bytes in 2019, especially when there is a great deal of evidence to show the risks of expecting people to do this.

  • Don’t assume object orientation

I think simpler, flatter structures in the json/cbor would have helped, and been clearer to implement, rather than the really complex maze of types currently involved.


Despite these concerns, I still think webauthn is a really good standard, and I really do think it will become the future of authentication. I’m hoping to help make that a reality in opensource and I hope that in the future I can contribute to further development and promotion of webauthn.

April 27, 2019 02:00 PM

The Case for Ethics in OpenSource

The Case for Ethics in OpenSource

For a long time there have been incidents in technology which have caused negative effects on people - from leaks of private data, to interfaces that are not accessible, to even issues like UI’s doing things that may try to subvert a persons intent. I’m sure there are many more: and we could be here all day listing the various issues that exist in technology, from small to great.

The theme however is that these issues continue to happen: we continue to make decisions in applications that can have consequences to humans.

Software is pointless without people. People create software, people deploy software, people interact with software, and even software indirectly can influence people’s lives. At every layer people exist, and all software will affect them in some ways.

I think that today, we have made a lot of progress in our communities around the deployment of code’s of conduct. These are great, and really help us to discuss the decisions and actions we take within our communities - with the people who create the software. I would like this to go further, where we can have a framework to discuss the effect of software on people that we write: the people that deploy, interact with and are influenced by our work.


I’m not a specialist in ethics or morality: I’m not a registered or certified engineer in the legal sense. Finally, like all humans I am a product of my experiences which causes all my view points to be biased through the lens of my experience.

Additionally, I specialise in Identity Management software, so many of the ideas and issues I have encountered are really specific to this domain - which means I may overlook the issues in other areas. I also have a “security” mindset which also factors into my decisions too.

Regardless I hope that this is a starting point to recieve further input and advice from others, and a place where we can begin to improve.

The Problem

TODO: Discuss data handling practices

Let’s consider some issues and possible solutions in work that I’m familiar with - identity management software. Lets list a few “features”. (Please don’t email me about how these are wrong, I know they are …)

  • Storing usernames as first and last name
  • Storing passwords in cleartext.
  • Deleting an account sets a flag to mark deletion
  • Names are used as the primary key
  • We request sex on signup
  • To change account details, you have to use a command line tool

Now “technically”, none of these decisions are incorrect at all. There is literally no bad technical decision here, and everything is “technically correct” (not always the best kind of correct).

What do we want to achieve?

There are lots of different issues here, but really want to prevent harm to a person. What is harm? Well that’s a complex topic. To me, it could be emotional harm, disrespect of their person, it could be a feeling of a lack of control.

I don’t believe it’s correct to dictate a set of rules that people should follow. People will be fatigued, and will find the process too hard. We need to trust that people can learn and want to improve. Instead I believe it’s important we provide important points that people should be able to consider in a discussion around the development of software. The same way we discuss technical implementation details, we should discuss potential human impact in every change we have. To realise this, we need a short list of important factors that relate to humans.

I think the following points are important to consider when designing software. These relate to general principles which I have learnt and researched.

People should be respected to have:

  • Informed consent
  • Choice over how they are identified
  • Ability to be forgotten
  • Individual Autonomy
  • Free from Harmful Discrimination
  • Privacy
  • Ability to meaningfully access and use software

There is already some evidence in research papers to show that there are strong reasons for moral positions in software. For example, to prevent harm to come to people, to respect peoples autonomy and to conform to privacy legislation ( source ).

Let’s apply these

Given our set of “features”, lets now discuss these with the above points in mind.

  • Storing usernames as first and last name

This point clearly is in violation of the ability to choose how people are identified - some people may only have a single name, some may have multiple family names. On a different level this also violates the harmful discrimination rule due to the potential to disrespect individuals with cultures that have different name schemes compared to western/English societies.

A better way to approach this is “displayName” as a freetext UTF8 case sensitive field, and to allow substring search over the content (rather than attempting to sort by first/last name which also has a stack of issues).

  • Storing passwords in cleartext.

This one is a violation of privacy, that we risk the exposure of a password which may have been reused (we can’t really stop password reuse, we need to respect human behaviour). Not only that some people may assume we DO hash these correctly, so we actually are violating informed consent as we didn’t disclose the method of how we store these details.

A better thing here is to hash the password, or at least to disclose how it will be stored and used.

  • Deleting an account sets a flag to mark deletion

This violates the ability to be forgotten, because we aren’t really deleting the account. It also breaks informed consent, because we are being “deceptive” about what our software is actually doing compared to the intent of the users request

A better thing is to just delete the account, or if not possible, delete all user data and leave a tombstone inplace that represents “an account was here, but no details associated”.

  • Names are used as the primary key

This violates choice over identification, especially for women who have a divorce, or individuals who are transitioning or just people who want to change their name in general. The reason for the name change doesn’t matter - what matters is we need to respect peoples right to identification.

A better idea is to use UUID/ID numbers as a primary key, and have name able to be changed at any point in time.

  • We request sex on signup

Violates a privacy as a first point - we probably have no need for the data unless we are a medical application, so we should never ask for this at all. We also need to disclose why we need this data to satisfy informed consent, and potentially to allow them to opt-out of providing the data. Finally (if we really require this), to not violate self identification, we need to allow this to be a free-text field rather than a Male/Female boolean. This is not just in respect of individuals who are LGBTQI+, but the reality that there are biologically people who medically are neither. We also need to allow this to be changed at any time in the future. This in mind Sex and Gender are different concepts, so we should be careful which we request - Sex is the medical term of a person’s genetics, and Gender is who the person identifies as.

Not only this, because this is a very personal piece of information, we must disclose how we protect this information from access, who can see it, and if or how we’ll ever share it with other systems or authorities.

Generally, we probably don’t need to know, so don’t ask for it at all.

  • To change account details, you have to use a command line tool

This violates a users ability to meaningfully access and use software - remember, people come from many walks of life and all have different skill sets, but using command line tools is not something we can universally expect.

A proper solution here is at minimum a web/graphical self management portal that is easy to access and follows proper UX/UI design rules, and for a business deploying, a service desk with humans involved that can support and help people change details on their account on their behalf if the person is unable to self-support via the web service.


I think that OpenSource should aim to have a code of ethics - the same way we have a code of conduct to guide our behaviour internally to a project, we should have a framework to promote discussion of people’s rights that use, interact with and are affected by our work. We should not focus on technical matters only, but should be promoting people at the core of all our work. Every decision we make is not just technical, but social.

I’m sure that there are more points that could be considere than what I have listed here: I’d love to hear feedback to william at Thanks!

April 27, 2019 02:00 PM

Powered by Planet