Postgres 16 highlight - File inclusions in pg_hba.conf and pg_ident.conf

You liked this post or you have a comment? Contact information is available here and sponsoring is here.
29 Mar 2023
Tags: postgres, postgresql, 16, administration, configure

The third and last phase of the improvements done in PostgreSQL 16 for authentication configuration involves both pg_hba.conf and pg_ident.conf, mainly with this commit:

commit: a54b658ce77b6705eb1f997b416c2e820a77946c
author: Michael Paquier <michael@paquier.xyz>
date: Thu, 24 Nov 2022 13:51:34 +0900
Add support for file inclusions in HBA and ident configuration files

pg_hba.conf and pg_ident.conf gain support for three record keywords:
- "include", to include a file.
- "include_if_exists", to include a file, ignoring it if missing.
- "include_dir", to include a directory of files.  These are classified
by name (C locale, mostly) and need to be prefixed by ".conf", hence
following the same rules as GUCs.

[ ... ]

Author: Julien Rouhaud
Reviewed-by: Michael Paquier
Discussion: https://postgr.es/m/20220223045959.35ipdsvbxcstrhya@jrouhaud

This feature is at the core of making pg_hba.conf and pg_ident.conf more in-line with postgresql.conf, even if the parsing logic of the first two is not the same as the third one as GUCs require their own thing, while HBA and ident files need to handle full entries made of a sequence of items. And some of these items can be lists, additionally.

Up to PostgreSQL 15, inclusion of files is possible in pg_hba.conf within a list for databases and users, by defining a file path prefixed by ‘@’. This can be a relative path, compiled from the location of the HBA file being currently read, or even absolute. Note that a file has to exist, and not finding one in the path defined results in a loading failure. For example, here is a set of two files loaded by hba.c:

$ cat $PGDATA/pg_hba.conf
# TYPE  DATABASE                USER     ADDRESS   METHOD
local   db_1,@dblist.conf,db_2  user_1             trust
$ cat $PGDATA/dblist.conf
db_3,db_4
db_5 db_6
db_7

The tokenization of the second file is very flexible, following a few rules:

  • Comma-separated items listed on the same line are treated as a full list of multiple elements.
  • Items separated by whitespaces and newlines are separate elements.

Hence, using the previous example, pg_hba_file_rules reports this data, with seven databases listed (note the order of the items):

=# select type, database, user_name from pg_hba_file_rules;
 type  |               database               | user_name
-------+--------------------------------------+-----------
 local | {db_1,db_3,db_4,db_5,db_6,db_7,db_2} | {user_1}
(1 row)

Speaking of which, attempting a circling dependency with these files would cause a failure at reload. For example, see this file including itself:

$ cat $PGDATA/pg_hba.conf
# TYPE  DATABASE      USER     ADDRESS   METHOD
local   @pg_hba.conf  user_1             trust

The error generated is.. Surprising up to PostgreSQL 15 and older versions, and this has never been reported to the lists as far as I know:

FATAL:  exceeded maxAllocatedDescs (166) while trying to open file "/to/data/folder/pg_hba.conf"

In 16 and newer versions, things are improved a lot, with the same rules applied as for postgresql.conf, with a context line produced for each file read across the included chain. This way, it is possible to get a full log of the events leading down to the location of the error, (here the context leads to 10 times the same line as the file includes itself, respecting the depth limit of included files):

LOG:  listening on Unix socket "/socket/path/.s.PGSQL.5432"
LOG:  could not open file "/to/data/folder/pg_hba.conf": maximum nesting depth exceeded
CONTEXT:  line 2 of configuration file "/to/data/foler/pg_hba.conf"
  [ ... ]
  line 2 of configuration file "/to/data/folder/pg_hba.conf"

Now, back to the actual feature… The commit message mentioned above explains it all, with one argument on the thread discussing this feature that it can be useful in container configurations. This consists of three new clauses available in HBA and ident files. Each file included must be written with rules or maps compatible with the format of HBA or ident files. Here is a more advanced example for HBA rules, one record is incorrect (which one?), leading to a reload failure:

$ cat $PGDATA/pg_hba.conf
include_if_exists pg_hba_extra.conf
include_dir hba_conf
$ cat $PGDATA/hba_conf/001_hba.conf
# TYPE  DATABASE  USER     ADDRESS   METHOD
local   db_1      user_1             trust
$ cat $PGDATA/hba_conf/002_hba.conf
# TYPE  DATABASE  USER     ADDRESS   METHOD
local   db_0      user_0             trust
local   db_2      user_2             incorrect
$ cat $PGDATA/pg_hba_extra.conf
# TYPE  DATABASE  USER     ADDRESS   METHOD
local   db_3      user_3             trust

There are a few gotchas to be aware of.

First, the order of the files listed in a directory is strictly decided: these are ordered by name, after scanning the directory to get a full list of them. One thing I would recommend to do is adding numbers in front of their names, for example, to guarantee a strict order. An empty folder, or a directory made of files with no contents, leads to the same effect as an empty pg_hba.conf: if at the end of loading the whole no entries are found, the reload fails. A missing directory defined by “include_dir” causes a hard failure, and a directory having zero files suffixed by “.conf” is a valid flow.

“include” and “include_if_exists” differ in the error handling, with semantics similar to postgresql.conf. If a file listed by “include” is not found, the computation of the HBA or ident files is a hard failure, and the full set of new rules will not be reloaded. “include_if_exists” will not complain that a file is missing, and the rules will be able to reload if a file is missing. Note that parsing errors in a file listed by “include_if_exists” when the file exists causes a reload of the new rules to fail. Besides, the same rules as the files included via ‘@’ apply for the computation of the file paths with these commands:

  • If a relative path is defined, the file’s path is compiled using as base location the path of the file currently being loaded.
  • If an absolute path is defined, this is used as-is.

As an effect of this change, the system views pg_ident_file_mappings and pg_hba_file_rules have gained two columns each to allow users to find all the information needed to debug data in included files:

  • Name of the file, kind of the most essential piece.
  • A rule ordering number, named map_number for ident files and rule_number for HBA files. This indicates the order in which the rules are considered until one matches with a database and a role. In 15 and older versions, knowing only about the line number of each rule is enough for this purpose.

Taking the previous example, pg_hba_file_rules reports:

=# SELECT rule_number AS order, file_name, line_number AS line, database, user_name, error
     FROM pg_hba_file_rules ;
 order |          file_name          | line | database | user_name |                   error
-------+-----------------------------+------+----------+-----------+-------------------------------------------
     1 | /data/pg_hba_extra.conf     |    2 | {db_3}   | {user_3}  | null
     2 | /data/hba_conf/001_hba.conf |    2 | {db_1}   | {user_1}  | null
     3 | /data/hba_conf/002_hba.conf |    2 | {db_0}   | {user_0}  | null
  null | /data/hba_conf/002_hba.conf |    3 | null     | null      | invalid authentication method "incorrect"
(4 rows)

As mentioned above, one of the rules is failing, causing a reload failure so the new rules are not applied yet.

Have fun with this feature..

Search

Social

Github

RSS Feeds

Main

Postgres

Sponsor