v2.4.1 Supabase/PostgresML support, and other extensions!

Supabase & PostgresML support, and other new extensions!

Highlights

• Supabase support: run open-source Firebase alternative with external postgres managed by Pigsty: example config
• PostgresML support: Run LLMs, vector operations, classical Machine Learning in Postgres.
• GraphQL support: pg_graphql reflects a GraphQL schema from the existing SQL schema.
• Async HTTP Client support pg_net enables asynchronous (non-blocking) HTTP/HTTPS requests with SQL
• JWT support: pgjwt is the PostgreSQL implementation of JWT (JSON Web Tokens)
• Vault support: vault can store encrypted secrets in the Vault
• New component pg_filedump for pg 14 & 15, low-level data recovery tool for PostgreSQL
• New extension hydra the columnar available for PG 13 - 15. (not packaged due to conflict with citus columnar)
• Reduce offline packages size for el9 400MB by removing proj-data*
• Bump FerretDB version to v1.10
• Fix patroni v3.1.1 broken issue

7e3989a98b7b0cd213e7efa09a8e8ebc  pigsty-v2.4.1.tgz
efabe7632d8994f3fb58f9838b8f9d7d  pigsty-pkg-v2.4.1.el7.x86_64.tgz
ea78957e8c8434b120d1c8c43d769b56  pigsty-pkg-v2.4.1.el8.x86_64.tgz
4ef280a7d28872814e34521978b851bb  pigsty-pkg-v2.4.1.el9.x86_64.tgz

v2.4.0 PG16 & RDS Monitor & New Extensions

Get started with bash -c "$(curl -fsSL https://get.pigsty.cc/latest)".

Highlights

• PostgreSQL 16 support
• The first LTS version with business support and consulting service
• Monitoring existing PostgreSQL, RDS for PostgreSQL / PolarDB with PGRDS Dashboards
• New extension: Apache AGE, openCypher graph query engine on PostgreSQL
• New extension: zhparser, full text search for Chinese language
• New extension: pg_roaringbitmap, roaring bitmap for PostgreSQL
• New extension: pg_embedding, hnsw alternative to pgvector
• New extension: pg_tle, admin / manage stored procedure extensions
• New extension: pgsql-http, issue http request with SQL interface
• Add extensions: pg_auth_mon pg_checksums pg_failover_slots pg_readonly postgresql-unit pg_store_plans pg_uuidv7 set_user
• Redis enhancement: add monitoring panels for redis sentinel, and auto HA configuration for redis ms cluster.

API Change

• New Parameter: REDIS.redis_sentinel_monitor: specify masters monitor by redis sentinel cluster

MD5 (pigsty-v2.4.0.tgz) = d6d9e6cc3e79f208098f1c7be503d131
MD5 (pigsty-pkg-v2.4.0.el7.x86_64.tgz) = 257443e3c171439914cbfad8e9f72b17
MD5 (pigsty-pkg-v2.4.0.el8.x86_64.tgz) = 41ad8007ffbfe7d5e8ba5c4b51ff2adc
MD5 (pigsty-pkg-v2.4.0.el9.x86_64.tgz) = 9a950aed77a6df90b0265a6fa6029250

v2.3.1 HNSW Pgvector & PG16 RC1

HNSW Pgvector, PG16 RC1, CN Docs & Bug Fix

Get started with bash -c "$(curl -fsSL https://get.pigsty.cc/latest)".

Highlights

• PGVector 0.5 with HNSW index support
• PostgreSQL 16 RC1 for el8/el9
  ** Adding SealOS for kubernetes support

Bug Fix

• Fix infra.repo.repo_pkg task when downloading rpm with * in their names in repo_packages.
  • if /www/pigsty already have package name match that pattern, some rpm will be skipped.
• Change default value of vip_dns_suffix to '' empty string rather than .vip
• Grant sudo privilege for postgres dbsu when pg_dbsu_sudo = limit and patroni_watchdog_mode = required
  • /usr/bin/sudo /sbin/modprobe softdog: enable watchdog module before launching patroni
  • /usr/bin/sudo /bin/chown {{ pg_dbsu }} /dev/watchdog: chown watchdog before launching patroni

Documentation Update

• Add details to English documentation
• Add Chinese/zh-cn documentation

Software Upgrade

• PostgreSQL 16 RC1 on el8/el9
• PGVector 0.5.0 with hnsw index
• TimescaleDB 2.11.2
• grafana 10.1.0
• loki & promtail 2.8.4
• mcli-20230829225506 / minio-20230829230735
• ferretdb 1.9
• sealos 4.3.3
• pgbadger 1.12.2

MD5 (pigsty-v2.3.1.tgz) = 024f006ba0df0d2dae047cb2c6aec0fe
MD5 (pigsty-pkg-v2.3.1.el7.x86_64.tgz) = ce69791eb622fa87c543096cdf11f970
MD5 (pigsty-pkg-v2.3.1.el8.x86_64.tgz) = 495aba9d6d18ce1ebed6271e6c96b63a
MD5 (pigsty-pkg-v2.3.1.el9.x86_64.tgz) = 38b45582cbc337ff363144980d0d7b64

v2.3.0 (2023-08-20)

Node VIP, Mongo/ferretdb, nocodb, MySQL Stub, and CVE Fix

Get started with bash -c "$(curl -fsSL https://get.pigsty.cc/latest)"

Highlight

• INFRA: NODE/PGSQL VIP monitoring support
• NODE: Allow bind node_vip to node cluster with keepalived
• REPO: Dedicate yum repo, enable https for get.pigsty.cc and demo.pigsty.cc
• PGSQL: Fix CVE-2023-39417 with PostgreSQL 15.4, 14.9, 13.12, 12.16, bump patroni version to v3.1.0
• APP: Bump app/bytebase to v2.6.0, app/ferretdb version to v1.8, new application nocodb
• REDIS: bump to v7.2 and rework on dashboards
• MONGO: basic deploy & monitor support with FerretDB 1.8
• MYSQL: add prometheus/grafana/ca stub for future implementation.

API Change

Add 1 new section NODE.NODE_VIP with 8 new parameter

• NODE.VIP.vip_enabled : enable vip on this node cluster?
• NODE.VIP.vip_address : node vip address in ipv4 format, required if vip is enabled
• NODE.VIP.vip_vrid : required, integer, 1-255 should be unique among same VLAN
• NODE.VIP.vip_role : master/backup, backup by default, use as init role
• NODE.VIP.vip_preempt : optional, true/false, false by default, enable vip preemption
• NODE.VIP.vip_interface : node vip network interface to listen, eth0 by default
• NODE.VIP.vip_dns_suffix : node vip dns name suffix, .vip by default
• NODE.VIP.vip_exporter_port : keepalived exporter listen port, 9650 by default

MD5 (pigsty-v2.3.0.tgz) = 6b42f339ceeb686c4c4119cc5680fc25
MD5 (pigsty-pkg-v2.3.0.el7.x86_64.tgz) = 81db95f1c591008725175d280ad23615
MD5 (pigsty-pkg-v2.3.0.el8.x86_64.tgz) = 6f4d169b36f6ec4aa33bfd5901c9abbe
MD5 (pigsty-pkg-v2.3.0.el9.x86_64.tgz) = 4bc9ae920e7de6dd8988ca7ee681459d

v2.2.0 Dashboard Overhaul

Get started with bash -c "$(curl -fsSL http://get.pigsty.cc/latest)"

Release Note: https://vonng.github.io/pigsty/#/RELEASENOTE?id=v220

Highlight

• Monitoring Dashboards Overhaul: http://demo.pigsty.cc
• Vagrant Sandbox Overhaul: libvirt support and new templates
• Pigsty EL Yum Repo: Building simplified
• OS Compatibility: UOS-v20-1050e support
• New config template: prod simulation with 42 nodes
• Use official pgdg citus distribution for el7

Software Upgrade

• PostgreSQL 16 beta2
• Citus 12 / PostGIS 3.3.3 / TimescaleDB 2.11.1 / PGVector 0.44
• patroni 3.0.4 / pgbackrest 2.47 / pgbouncer 1.20
• grafana 10.0.3 / loki/promtail/logcli 2.8.3
• etcd 3.5.9 / haproxy v2.8.1 / redis v7.0.12
• minio 20230711212934 / mcli 20230711233044

Bug Fix

• Fix docker group ownership issue [29434bd]29434bd
• Append infra os group rather than set it as primary group
• Fix redis sentinel systemd enable status 5c96feb
• Loose bootstrap & configure if /etc/redhat-release not exists
• Fix grafana 9.x CVE-2023-1410 with 10.0.3
• Add PG 14 - 16 new command tags and error codes for pglog schema

API Change

Add 1 new parameter

• INFRA.NGINX.nginx_exporter_enabled : now you can disable nginx_exporter with this parameter

Default value changes:

• repo_modules: node,pgsql,infra : redis is removed from it
• repo_upstream:
  • add pigsty-el: distribution independent rpms: such as grafana, minio, pg_exporter, etc...
  • add pigsty-misc: distribution aware rpms: such as redis, prometheus stack binaries, etc...
  • remove citus repo since pgdg now have full official citus support (on el7)
  • remove remi , since redis is now included in pigsty-misc
  • remove grafana in build config for acceleration
• repo_packages:
  • ansible python3 python3-pip python3-requests python3.11-jmespath dnf-utils modulemd-tools # el7: python36-requests python36-idna yum-utils
  • grafana loki logcli promtail prometheus2 alertmanager karma pushgateway node_exporter blackbox_exporter nginx_exporter redis_exporter
  • redis etcd minio mcli haproxy vip-manager pg_exporter nginx createrepo_c sshpass chrony dnsmasq docker-ce docker-compose-plugin flamegraph
  • lz4 unzip bzip2 zlib yum pv jq git ncdu make patch bash lsof wget uuid tuned perf nvme-cli numactl grubby sysstat iotop htop rsync tcpdump
  • netcat socat ftp lrzsz net-tools ipvsadm bind-utils telnet audit ca-certificates openssl openssh-clients readline vim-minimal
  • postgresql13* wal2json_13* pg_repack_13* passwordcheck_cracklib_13* postgresql12* wal2json_12* pg_repack_12* passwordcheck_cracklib_12* postgresql16* timescaledb-tools
  • postgresql15 postgresql15* citus_15* pglogical_15* wal2json_15* pg_repack_15* pgvector_15* timescaledb-2-postgresql-15* postgis33_15* passwordcheck_cracklib_15* pg_cron_15*
  • postgresql14 postgresql14* citus_14* pglogical_14* wal2json_14* pg_repack_14* pgvector_14* timescaledb-2-postgresql-14* postgis33_14* passwordcheck_cracklib_14* pg_cron_14*
  • patroni patroni-etcd pgbouncer pgbadger pgbackrest pgloader pg_activity pg_partman_15 pg_permissions_15 pgaudit17_15 pgexportdoc_15 pgimportdoc_15 pg_statement_rollback_15*
  • orafce_15* mysqlcompat_15 mongo_fdw_15* tds_fdw_15* mysql_fdw_15 hdfs_fdw_15 sqlite_fdw_15 pgbouncer_fdw_15 multicorn2_15* powa_15* pg_stat_kcache_15* pg_stat_monitor_15* pg_qualstats_15 pg_track_settings_15 pg_wait_sampling_15 system_stats_15
  • plprofiler_15* plproxy_15 plsh_15* pldebugger_15 plpgsql_check_15* pgtt_15 pgq_15* pgsql_tweaks_15 count_distinct_15 hypopg_15 timestamp9_15* semver_15* prefix_15* rum_15 geoip_15 periods_15 ip4r_15 tdigest_15 hll_15 pgmp_15 extra_window_functions_15 topn_15
  • pg_background_15 e-maj_15 pg_catcheck_15 pg_prioritize_15 pgcopydb_15 pg_filedump_15 pgcryptokey_15 logerrors_15 pg_top_15 pg_comparator_15 pg_ivm_15* pgsodium_15* pgfincore_15* ddlx_15 credcheck_15 safeupdate_15 pg_squeeze_15* pg_fkpart_15 pg_jobmon_15
• repo_url_packages:
  • http://get.pigsty.cc/rpm/pev.html
  • http://get.pigsty.cc/rpm/chart.tgz
• node_default_packages:
  • lz4,unzip,bzip2,zlib,yum,pv,jq,git,ncdu,make,patch,bash,lsof,wget,uuid,tuned,nvme-cli,numactl,grubby,sysstat,iotop,htop,rsync,tcpdump
  • netcat,socat,ftp,lrzsz,net-tools,ipvsadm,bind-utils,telnet,audit,ca-certificates,openssl,readline,vim-minimal,node_exporter,etcd,haproxy,python3,python3-pip
• infra_packages
  • grafana,loki,logcli,promtail,prometheus2,alertmanager,karma,pushgateway
  • node_exporter,blackbox_exporter,nginx_exporter,redis_exporter,pg_exporter
  • nginx,dnsmasq,ansible,postgresql15,redis,mcli,python3-requests
• PGSERVICE in .pigsty is removed, replaced with PGDATABASE=postgres.

FHS Changes:

• bin/dns and bin/ssh now moved to vagrant/

MD5 (pigsty-v2.2.0.tgz) = ecec41c937ef7e9fa6e0bec364d668dd
MD5 (pigsty-pkg-v2.2.0.el7.x86_64.tgz) = 5fb6a449a234e36c0d895a35c76add3c
MD5 (pigsty-pkg-v2.2.0.el8.x86_64.tgz) = c7211730998d3b32671234e91f529fd0
MD5 (pigsty-pkg-v2.2.0.el9.x86_64.tgz) = 385432fe86ee0f8cbccbbc9454472fdd

v2.1.0 : PG 12 - 16 support

Install

bash -c "$(curl -fsSL http://download.pigsty.cc/get)"

Highlight

• PostgreSQL 16 beta support, and 12 ~ 15 support.
• Add PGVector for AI Embedding for 12 - 15
• Add 6 extra panel & datasource plugins for grafana
• Add bin/profile to profile remote process and generate flamegraph
• Add bin/validate to validate pigsty.yml configuration file
• Add bin/repo-add to add upstream repo files to /etc/yum.repos.d
• PostgreSQL 16 observability: pg_stat_io and corresponding dashboards

Software Upgrade

• PostgreSQL 15.3 , 14.8, 13.11, 12.15, 11.20, and 16 beta1
• pgBackRest 2.46
• pgbouncer 1.19
• Redis 7.0.11
• Grafana v9.5.3
• Loki / Promtail / Logcli 2.8.2
• Prometheus 2.44
• TimescaleDB 2.11.0
• minio-20230518000536 / mcli-20230518165900
• Bytebase v2.2.0

Enhancement

• Now use all id*.pub when installing local user's public key

Checksums

MD5 (pigsty-v2.1.0.tgz) = f77d49026c4a4bb4f292a47378ba2fe5
MD5 (pigsty-pkg-v2.1.0.el7.x86_64.tgz) = be8fbb255c2d09a1d02d46824121b304
MD5 (pigsty-pkg-v2.1.0.el8.x86_64.tgz) = 6c2e22c79159d47148423215f12c6bf9
MD5 (pigsty-pkg-v2.1.0.el9.x86_64.tgz) = 687933aa60819bdd7c3b22d102dce371

v2.0.2

Highlight

Store OpenAI embedding and search similar vectors with pgvector

• New extension pgvector
• MinIO CVE-2023-28432 fix, and upgrade to 20230324 with new policy API:

Changes

• New extension pgvector for storing OpenAI embedding and searching similar vectors.
• MinIO CVE-2023-28432 fix, and upgrade to 20230324 with new policy API.
• Add reload functionality to DNSMASQ systemd services
• Bump pev to v1.8
• Bump grafana to v9.4.7
• Bump MinIO and MCLI version to 20230324
• Bump bytebase version to v1.15.0
• Upgrade monitoring dashboards and fix dead links
• Upgrade aliyun terraform template image to rockylinux 9
• Adopt grafana provisioning API change since v9.4
• Add asciinema videos for various administration tasks
• Fix broken EL8 pgsql deps: remove anonymizer_15 faker_15 and pgloader

MD5 (pigsty-pkg-v2.0.2.el7.x86_64.tgz) = d46440a115d741386d29d6de646acfe2
MD5 (pigsty-pkg-v2.0.2.el8.x86_64.tgz) = 5fa268b5545ac96b40c444210157e1e1
MD5 (pigsty-pkg-v2.0.2.el9.x86_64.tgz) = c8b113d57c769ee86a22579fc98e8345

v2.0.1

Bug fix for v2.0.0 and security improvement.

Enhancement

• Replace the pig shape logo for compliance with the PostgreSQL trademark policy.
• Bump grafana version to v9.4 with better UI and bugfix.
• Bump patroni version to v3.0.1 with some bugfix.
• Change: rollback grafana systemd service file to rpm default.
• Use slow copy instead of rsync to copy grafana dashboards.
• Enhancement: add back default repo files after bootstrap
• Add asciinema video for various administration tasks.
• Security Enhance Mode: restrict monitor user privilege.
• New config template: dual.yml for two-node deployment.
• Enable log_connections and log_disconnections in crit.yml template.
• Enable $lib/passwordcheck in pg_libs in crit.yml template.
• Explicitly grant monitor view permission to pg_monitor role.
• Remove default dbrole_readonly from dbuser_monitor to limit monitor user privilege
• Now patroni listen on {{ inventory_hostname }} instead of 0.0.0.0
• Now you can control postgres/pgbouncer listen to address with pg_listen
• Now you can use placeholder ${ip}, ${lo}, ${vip} in pg_listen
• Bump Aliyun terraform image to rocky Linux 9 instead of centos 7.9
• Bump bytebase to v1.14.0

Bug Fixes

• Add missing advertise address for alertmanager
• Fix missing pg_mode error when adding postgres user with bin/pgsql-user
• Add -a password to redis-join task @ redis.yml
• Fix missing default value in infra-rm.yml.remove infra data
• Fix prometheus targets file ownership to prometheus
• Use admin user rather than root to delete metadata in DCS
• Fix Meta datasource missing database name due to grafana 9.4 bug.

Caveats

Official EL8 pgdg upstream is broken now, DO use it with caution!

Affected packages: postgis33_15, pgloader, postgresql_anonymizer_15*, postgresql_faker_15

How to Upgrade

cd ~/pigsty; tar -zcf /tmp/files.tgz files; rm -rf ~/pigsty    # backup files dir and remove      
cd ~; bash -c "$(curl -fsSL http://download.pigsty.cc/get)"    # get latest pigsty source
cd ~/pigsty; rm -rf files; tar -xf /tmp/files.tgz -C ~/pigsty  # restore files dir

Checksums

MD5 (pigsty-v2.0.1.tgz) = 7addc87bc4edb46c3eacf723163ffbd4
MD5 (pigsty-pkg-v2.0.1.el7.x86_64.tgz) = 5cfbe98fd9706b9e0f15c1065971b3f6
MD5 (pigsty-pkg-v2.0.1.el8.x86_64.tgz) = c34aa460925ae7548866bf51b8b8759c
MD5 (pigsty-pkg-v2.0.1.el9.x86_64.tgz) = 055057cebd93c473a67fb63bcde22d33

Special thanks to @cocoonkid for his feedback.

v2.0.0 Release

"PIGSTY" is now the abbr of "PostgreSQL in Great STYle"

Get pigsty v2.0.0 via the following command:

curl -fsSL http://download.pigsty.cc/get | bash

Download directly from GitHub Release

# get from GitHub
bash -c "$(curl -fsSL https://raw.githubusercontent.com/Vonng/pigsty/master/bin/get)"

# or download tarball directly with curl
curl -L https://github.com/Vonng/pigsty/releases/download/v2.0.0/pigsty-v2.0.0.tgz -o ~/pigsty.tgz                 # SRC
curl -L https://github.com/Vonng/pigsty/releases/download/v2.0.0/pigsty-pkg-v2.0.0.el9.x86_64.tgz -o /tmp/pkg.tgz  # EL9
curl -L https://github.com/Vonng/pigsty/releases/download/v2.0.0/pigsty-pkg-v2.0.0.el8.x86_64.tgz -o /tmp/pkg.tgz  # EL8
curl -L https://github.com/Vonng/pigsty/releases/download/v2.0.0/pigsty-pkg-v2.0.0.el7.x86_64.tgz -o /tmp/pkg.tgz  # EL7

Highlights

• PostgreSQL 15.2, PostGIS 3.3, Citus 11.2, TimescaleDB 2.10 now works together and unite as one.
• Now works on EL 7,8,9 for RHEL, CentOS, Rocky, AlmaLinux, and other EL compatible distributions
• Security enhancement with self-signed CA, full SSL support, scram-sha-256 pwd encryption, and more.
• Patroni 3.0 with native HA citus cluster support and dcs failsafe mode to prevent global DCS failures.
• Auto-Configured, Battery-Included PITR for PostgreSQL powered by pgbackrest, local or S3/minio.
• Dedicate module ETCD, which can be easily deployed and scaled in/out. Used as DCS instead of Consul.
• Dedicate module MINIO, local S3 alternative for the optional central backup repo for PGSQL PITR.
• Better config templates with adaptive tuning for Node & PG according to your hardware spec.
• Use AGPL v3.0 license instead of Apache 2.0 license due to Grafana & MinIO reference.

Compatibility

• Pigsty now works on EL7, EL8, EL9, and offers corresponding pre-packed offline packages.
• Pigsty now works on EL compatible distributions: RHEL, CentOS, Rocky, AlmaLinux, OracleLinux,...
• Pigsty now use RockyLinux 9 as default developing & testing environment instead of CentOS 7
• EL version, CPU arch, and pigsty version string are part of source & offline package names.
• PGSQL: PostgreSQL 15.2 / PostGIS 3.3 / TimescaleDB 2.10 / Citus 11.2 now works together.
• PGSQL: Patroni 3.0 is used as default HA solution for PGSQL, and etcd is used as default DCS.
  • Patroni 3.0 with DCS failsafe mode to prevent global DCS failures (demoting all primary)
  • Patroni 3.0 with native HA citus cluster support, with entirely open sourced v11 citus.
  • vip-manager 2.x with ETCDv3 API, ETCDv2 API is deprecated, so does patroni.
• PGSQL: pgBackRest v2.44 is introduced to provide battery-include PITR for PGSQL.
  • it will use local backup FS on primary by default for a two-day retention policy
  • it will use S3/minio as an alternative central backup repo for a two-week retention policy
• ETCD is used as default DCS instead of Consul, And V3 API is used instead of V2 API.
• NODE module now consist of node itself, haproxy, docker, node_exporter, and promtail
  • chronyd is used as default NTP client instead of ntpd
  • HAPROXY now attach to NODE instead of PGSQL, which can be used for exposing services
  • You can register PG Service to dedicate haproxy clusters rather than local cluster nodes.
  • You can expose ad hoc service in a NodePort manner with haproxy, not limited to pg services.
• INFRA now consist of dnsmasq, nginx, prometheus, grafana, loki
  • DNSMASQ is enabled on all infra nodes, and added to all nodes as the default resolver.
  • Add blackbox_exporter for ICMP probe, add pushgateway for batch job metrics.
  • Switch to official loki & promtail rpm packages. Use official Grafana Echarts Panel.
  • Add infra dashboards for self-monitoring, add patroni & pg15 metrics to the monitoring system
• Software Upgrade
  • PostgreSQL 15.2 / PostGIS 3.3 / TimescaleDB 2.10 / Citus 11.2
  • Patroni 3.0 / Pgbouncer 1.18 / pgBackRest 2.44 / vip-manager 2.1
  • HAProxy 2.7 / Etcd 3.5 / MinIO 20230222182345 / mcli 20230216192011
  • Prometheus 2.42 / Grafana 9.3 / Loki & Promtail 2.7 / Node Exporter 1.5

Security

• A full-featured self-signed CA enabled by default
• Redact password in postgres logs.
• SSL for Nginx (you have to trust the self-signed CA or use thisisunsafe to dismiss the warning)
• SSL for etcd peer/client traffics by @alemacci
• SSL for postgres/pgbouncer/patroni by @alemacci
• scram-sha-256 auth for postgres password encryption by @alemacci
• Pgbouncer Auth Query by @alemacci
• Use AES-256-CBC for pgbackrest encryption by @alemacci
• Adding a security enhancement config template which enforce global SSL
• Now all hba rules are defined in config inventory, no default rules.

Maintainability

• Adaptive tuning template for PostgreSQL & Patroni by @Vonng, @alemacci
• configurable log dir for Patroni & Postgres & Pgbouncer & Pgbackrest by @alemacci
• Replace fixed ip placeholder 10.10.10.10 with ${admin_ip} that can be referenced
• Adaptive upstream repo definition that can be switched according to EL ver, region & arch.
• Terraform Templates for AWS CN & Aliyun, which can be used for sandbox IaaS provisioning
• Vagrant Templates: meta, full, el7 el8, el9, build, minio, citus, etc...
• New playbook pgsql-monitor.yml for monitoring existing pg instance or RDS PG.
• New playbook pgsql-migration.yml for migrating existing pg instance to pigsty managed pg.
• New shell utils under bin/ to simplify the daily administration tasks.
• Optimize ansible role implementation. which can be used without default parameter values.
• Now you can define pgbouncer parameters on database & user level

API Changes

69 parameters were added, 16 parameters were removed, rename 14 parameters

Added Parameters

• INFRA.META.admin_ip : primary meta node ip address
• INFRA.META.region : upstream mirror region: default|china|europe
• INFRA.META.os_version : enterprise linux release version: 7,8,9
• INFRA.CA.ca_cn : ca common name, pigsty-ca by default
• INFRA.CA.cert_validity : cert validity, 20 years by default
• INFRA.REPO.repo_enabled : build a local yum repo on infra node?
• INFRA.REPO.repo_upstream : list of upstream yum repo definition
• INFRA.REPO.repo_home : home dir of local yum repo, usually same as nginx_home '/www'
• INFRA.NGINX.nginx_ssl_port : https listen port
• INFRA.NGINX.nginx_ssl_enabled : nginx https enabled?
• INFRA.PROMTETHEUS.alertmanager_endpoint : altermanager endpoint in (ip|domain):port format
• NODE.NODE_TUNE.node_hugepage_count : number of 2MB hugepage, take precedence over node_hugepage_ratio
• NODE.NODE_TUNE.node_hugepage_ratio : mem hugepage ratio, 0 disable it by default
• NODE.NODE_TUNE.node_overcommit_ratio : node mem overcommit ratio, 0 disable it by default
• NODE.HAPROXY.haproxy_service : list of haproxy service to be exposed
• PGSQL.PG_ID.pg_mode : pgsql cluster mode: pgsql,citus,gpsql
• PGSQL.PG_BUSINESS.pg_dbsu_password : dbsu password, empty string means no dbsu password by default
• PGSQL.PG_INSTALL.pg_log_dir : postgres log dir, /pg/data/log by default
• PGSQL.PG_BOOTSTRAP.pg_storage_type : SSD|HDD, SSD by default
• PGSQL.PG_BOOTSTRAP.patroni_log_dir : patroni log dir, /pg/log by default
• PGSQL.PG_BOOTSTRAP.patroni_ssl_enabled : secure patroni RestAPI communications with SSL?
• PGSQL.PG_BOOTSTRAP.patroni_username : patroni rest api username
• PGSQL.PG_BOOTSTRAP.patroni_password : patroni rest api password (IMPORTANT: CHANGE THIS)
• PGSQL.PG_BOOTSTRAP.patroni_citus_db : citus database managed by patroni, postgres by default
• PGSQL.PG_BOOTSTRAP.pg_max_conn : postgres max connections, auto will use recommended value
• PGSQL.PG_BOOTSTRAP.pg_shared_buffer_ratio : postgres shared buffer memory ratio, 0.25 by default, 0.1~0.4
• PGSQL.PG_BOOTSTRAP.pg_rto : recovery time objective, ttl to failover, 30s by default
• PGSQL.PG_BOOTSTRAP.pg_rpo : recovery point objective, 1MB data loss at most by default
• PGSQL.PG_BOOTSTRAP.pg_pwd_enc : algorithm for encrypting passwords: md5|scram-sha-256
• PGSQL.PG_BOOTSTRAP.pgbouncer_log_dir : pgbouncer log dir, /var/log/pgbouncer by default
• PGSQL.PG_BOOTSTRAP.pgbouncer_auth_query : if enabled, query pg_authid table to retrieve biz users instead of populating userlist
• PGSQL.PG_BOOTSTRAP.pgbouncer_sslmode : SSL for pgbouncer client: disable|allow|prefer|require|verify-ca|verify-full
• PGSQL.PG_BACKUP.pgbackrest_enabled : pgbackrest enabled?
• PGSQL.PG_BACKUP.pgbackrest_clean : remove pgbackrest data during init ?
• PGSQL.PG_BACKUP.pgbackrest_log_dir : pgbackrest log dir, /pg/log by default
• PGSQL.PG_BACKUP.pgbackrest_method : pgbackrest backup repo method, local or minio
• PGSQL.PG_BACKUP.pgbackrest_repo : pgbackrest backup repo config
• PGSQL.PG_SERVICE.pg_service_provider : dedicate haproxy node group name, or empty string for local nodes by default
• PGSQL.PG_SERVICE.pg_default_service_dest : default service destination if svc.dest='default'
• PGSQL.PG_SERVICE.pg_vip_enabled : enable a l2 vip for pgsql primary? false by default
• PGSQL...

v1.5.1 Release (PG14.4 BUGFIX)

Highlights

WARNING: CREATE INDEX|REINDEX CONCURRENTLY PostgreSQL 14.0 - 14.3 may lead to index data corruption!

Please upgrade postgres to 14.4 ASAP.

Software Upgrade

• upgrade postgres to 14.4 (important bug fix)
• upgrade citus to 11.0-2 (with enterprise features)
• upgrade timescaledb to 2.7 (more continuous aggregates)
• Upgrade patroni to 2.1.4 (new sync health-check)
• Upgrade haproxy to 2.6.0 (cli, reload, ssl,...)
• Upgrade grafana to 9.0.0 (new ui)
• Upgrade prometheus 2.36.0

Bug fix:

• Fix typo in pgsql-migration.yml
• remove pid file in haproxy config
• remove i686 packages when using repotrack under el7
• Fix redis service systemctl enabled issue
• Fix patroni systemctl service enabled=no by default issue
• stop vip-manager when purging existing postgres

API Changes

• Mark grafana_database and grafana_pgurl as obsolete
• Add some new etcd & pgsql alias (optional)

New Apps

• wiki.js : Local wiki with Postgres
• FerretDB : MongoDB API over Postgres

MD5 (pigsty.tgz) = 5a7403a85b20b2b56f874fa5304979f6
MD5 (pkg.tgz) = efdf59af503d49430c7989383a6b4b32
MD5 (app.tgz) = 7f5712993d68e09798e1d40cea6c6d87
MD5 (docker.tgz) = cdbd2cbd3fd4c026ca5055475bdf2067
MD5 (matrix.tgz) = 3d063437c482d94bd7e35df1a08bbc84