Setting up PGCluster

PGCluster is a fork of PostgreSQL that allows multi-master replication. This document explains how to set it up.

Notes

• All hosts in conf files are referenced by name - these must resolve properly to thier IPs.
• PGCluster xml conf files are case sensitive
• PGCluster 1.9.0 is based off PostgreSQL 8.3

Installation

First install the packages

 apk add pgcluster pgcluster-client

For recovery, you'll need ssh and rsync

 apk add openssh rsync

Configuration

Edit /etc/hosts file if necessary so all names can be resolved.

Check the data path in /etc/conf.d/pgcluster. Also check here to see which processes the init script will start on this host.

Create the database files (same as for postgresql)

 /etc/init.d/pgcluster setup

Edit the config files in the data path found above (usually /var/lib/posgresql/pgcluster)

postgres (postmaster)

postgres has 3 configuration files of interest here: postgresql.conf, pg_hba.conf, cluster.conf.

To start it manually:

 su postgres
 cd /var/lib/postgresql/pgcluster
 pg_ctl -D . start

or

 postgres -D .

postgresql.conf

This is the standard postgresql config file. Just make sure of this:

 listen_addresses='*'

You might also want to changed the logging settings.

pg_hba.conf

This is also a standard postgresql config file. Make sure all the hosts involved trust each other. Here's an example line you can use to get a test cluster running (it's a bit permissive for production use):

 host    all         all         192.168.0.0/16        trust

cluster.conf

This file is new with PGCLuster - it configures the patched database instance.

Add a block like this for each replication server:

 <Replicate_Server_Info>
 	<Host_Name>     pgrep1  </Host_Name>
 	<Port>          8001    </Port>
 	<Recovery_Port> 8101    </Recovery_Port>
 </Replicate_Server_Info>

Also add the following:

 <Host_Name>		pgdb1				</Host_Name>
 <Recovery_Port>	7001				</Recovery_Port>
 <Rsync_Path>		/usr/bin/rsync			</Rsync_Path>
 <Rsync_Option>	ssh -1				</Rsync_Option>
 <Rsync_Compress>	yes				</Rsync_Compress>
 <Rsync_Timeout>	10min				</Rsync_Timeout>
 <Rsync_Bwlimit>	0KB				</Rsync_Bwlimit>
 <Pg_Dump_Path>	/var/lib/pgcluster/pg_dump	</Pg_Dump_Path>
 <Ping_Path>		/bin/ping			</Ping_Path>
 <When_Stand_Alone>	read_only			</When_Stand_Alone>
 <Replication_Timeout>	1min				</Replication_Timeout>
 <LifeCheck_Timeout>	3s				</LifeCheck_Timeout>
 <LifeCheck_Interval>	11s				</LifeCheck_Interval>

And this bit is optional:

 <Not_Replicate_Info>
 	<DB_Name>		test_db		</DB_Name>
 #	<Table_Name>		log_table	</Table_Name>
 </Not_Replicate_Info>

pgreplicate

This process replicates to the other servers.

To start it manually:

 su postgres
 cd /var/lib/postgresql/pgcluster
 pgreplicate -D . 

If you want debug output change the last line to:

 pgreplicate -D . -n -v

pgreplicate.conf

Add a block like this for each database (also called cluster) instance:

 <Cluster_Server_Info>
    <Host_Name>pgdb1</Host_Name>
    <Port>5432</Port>
    <Recovery_Port>7001</Recovery_Port>
 </Cluster_Server_Info>

Add this block for your load balancer instance:

 <LoadBalance_Server_Info>
    <Host_Name>pglb1</Host_Name>
    <Recovery_Port>6001</Recovery_Port>
 </LoadBalance_Server_Info>

And add this to describe the replicator instance you're configuring:

 <Host_Name>pgrep1</Host_Name>
 <Replication_Port>8002</Replication_Port>
 <Recovery_Port>8101</Recovery_Port>
 <RLog_Port>8301</RLog_Port>
 <Response_Mode>normal</Response_Mode>
 <Use_Replication_Log>no</Use_Replication_Log>

Optional Logging info:

 <Log_File_Info>
         <File_Name>pg_log/pgreplicate.log</File_Name>
 </Log_File_Info>

pgreplicate

This process provides load balancing.

To start it manually:

 su postgres
 cd /var/lib/postgresql/pgcluster
 pglb -D . 

If you want debug output change the last line to:

 pglb -D . -n -v

pglb.conf

Similar to pgreplicate, add the following block for each database instance:

 <Cluster_Server_Info>
    <Host_name>pgdb1</Host_Name>
    <Port>5432</Port>
    <Max_Connect>30</Max_Connect>
 </Cluster_Server_Info>

Add the following for the load balancer:

 <Host_Name>pglb1</Host_Name>
 <Backend_Socket_Dir>/tmp</backend_socket_dir>
 <Receive_Port>5433</Receive_Port>
 <Recovery_Port>6001</Recovery_Port>
 <Max_Cluster_Num>128</Max_Cluster_Num>
 <Use_Connection_Pooling>no</Use_Connection_Pooling>

Optional Logging info:

 <Log_File_Info>
         <File_Name>pg_log/pgreplicate.log</File_Name>
 </Log_File_Info>

Recovery

...

References

http://pgfoundry.org/projects/pgcluster/ http://pgcluster.i-logic.hu/cgi-bin/trac.cgi http://pgcluster.projects.postgresql.org/ http://odyssi.blogspot.com/2007/08/postgresql-replication-with-pgcluster.html