Chapter 4. Using PostGIS

Table of Contents

4.1. GIS Objects
4.1.1. OpenGIS WKB and WKT
4.1.2. PostGIS EWKB, EWKT and Canonical Forms
4.2. Using OpenGIS Standards
4.2.1. The SPATIAL_REF_SYS Table
4.2.2. The GEOMETRY_COLUMNS Table
4.2.3. Creating a Spatial Table
4.3. Loading GIS Data
4.3.1. Using SQL
4.3.2. Using the Loader
4.4. Retrieving GIS Data
4.4.1. Using SQL
4.4.2. Using the Dumper
4.5. Building Indexes
4.5.1. GiST Indexes
4.5.2. Using Indexes
4.6. Complex Queries
4.6.1. Taking Advantage of Indexes
4.6.2. Examples of Spatial SQL
4.7. Using Mapserver
4.7.1. Basic Usage
4.7.2. Frequently Asked Questions
4.7.3. Advanced Usage
4.7.4. Examples
4.8. Java Clients (JDBC)
4.9. C Clients (libpq)
4.9.1. Text Cursors
4.9.2. Binary Cursors

4.1. GIS Objects

The GIS objects supported by PostGIS are a superset of the "Simple Features" defined by the OpenGIS Consortium (OGC). As of version 0.9, PostGIS supports all the objects and functions specified in the OGC "Simple Features for SQL" specification.

PostGIS extends the standard with support for 3DZ,3DM and 4D coordinates.

4.1.1. OpenGIS WKB and WKT

The OpenGIS specification defines two standard ways of expressing spatial objects: the Well-Known Text (WKT) form and the Well-Known Binary (WKB) form. Both WKT and WKB include information about the type of the object and the coordinates which form the object.

Examples of the text representations (WKT) of the spatial objects of the features are as follows:

  • POINT(0 0)

  • LINESTRING(0 0,1 1,1 2)

  • POLYGON((0 0,4 0,4 4,0 4,0 0),(1 1, 2 1, 2 2, 1 2,1 1))

  • MULTIPOINT(0 0,1 2)

  • MULTILINESTRING((0 0,1 1,1 2),(2 3,3 2,5 4))

  • MULTIPOLYGON(((0 0,4 0,4 4,0 4,0 0),(1 1,2 1,2 2,1 2,1 1)), ((-1 -1,-1 -2,-2 -2,-2 -1,-1 -1)))

  • GEOMETRYCOLLECTION(POINT(2 3),LINESTRING((2 3,3 4)))

The OpenGIS specification also requires that the internal storage format of spatial objects include a spatial referencing system identifier (SRID). The SRID is required when creating spatial objects for insertion into the database.

Input/Output of these formats are available using the following interfaces:

	bytea WKB = asBinary(geometry);
	text WKT = asText(geometry);
	geometry = GeomFromWKB(bytea WKB, SRID); 
	geometry = GeometryFromText(text WKT, SRID);
	

For example, a valid insert statement to create and insert an OGC spatial object would be:

	INSERT INTO SPATIALTABLE ( 
		  THE_GEOM, 
		  THE_NAME 
	) 
	VALUES ( 
		  GeomFromText('POINT(-126.4 45.32)', 312), 
		  'A Place' 
	)

4.1.2. PostGIS EWKB, EWKT and Canonical Forms

OGC formats only support 2d geometries, and the associated SRID is *never* embedded in the input/output representations.

Postgis extended formats are currently superset of OGC one (every valid WKB/WKT is a valid EWKB/EWKT) but this might vary in the future, specifically if OGC comes out with a new format conflicting with our extensions. Thus you SHOULD NOT rely on this feature!

Postgis EWKB/EWKT add 3dm,3dz,4d coordinates support and embedded SRID information.

Examples of the text representations (EWKT) of the extended spatial objects of the features are as follows:

  • POINT(0 0 0) -- XYZ

  • SRID=32632;POINT(0 0) -- XY with SRID

  • POINTM(0 0 0) -- XYM

  • POINT(0 0 0 0) -- XYZM

  • SRID=4326;MULTIPOINTM(0 0 0,1 2 1) -- XYM with SRID

  • MULTILINESTRING((0 0 0,1 1 0,1 2 1),(2 3 1,3 2 1,5 4 1))

  • POLYGON((0 0 0,4 0 0,4 4 0,0 4 0,0 0 0),(1 1 0,2 1 0,2 2 0,1 2 0,1 1 0))

  • MULTIPOLYGON(((0 0 0,4 0 0,4 4 0,0 4 0,0 0 0),(1 1 0,2 1 0,2 2 0,1 2 0,1 1 0)),((-1 -1 0,-1 -2 0,-2 -2 0,-2 -1 0,-1 -1 0)))

  • GEOMETRYCOLLECTIONM(POINTM(2 3 9),LINESTRINGM((2 3 4,3 4 5)))

Input/Output of these formats are available using the following interfaces:

	bytea EWKB = asEWKB(geometry);
	text EWKT = asEWKT(geometry);
	geometry = GeomFromEWKB(bytea EWKB);
	geometry = GeomFromEWKT(text EWKT);
	

For example, a valid insert statement to create and insert a PostGIS spatial object would be:

	INSERT INTO SPATIALTABLE ( 
		  THE_GEOM, 
		  THE_NAME 
	) 
	VALUES ( 
		  GeomFromEWKT('SRID=312;POINTM(-126.4 45.32 15)'), 
		  'A Place' 
	)

The "canonical forms" of a PostgreSQL type are the representations you get with a simple query (without any function call) and the one which is guaranteed to be accepted with a simple insert, update or copy. For the postgis 'geometry' type these are:

	- Output -
	binary: EWKB
	 ascii: HEXEWKB (EWKB in hex form)

	- Input -
	binary: EWKB
	 ascii: HEXEWKB|EWKT
	

For example this statement reads EWKT and returns HEXEWKB in the process of canonical ascii input/output:

	=# SELECT 'SRID=4;POINT(0 0)'::geometry;
			      geometry
	----------------------------------------------------
	 01010000200400000000000000000000000000000000000000
	(1 row)
	

4.2. Using OpenGIS Standards

The OpenGIS "Simple Features Specification for SQL" defines standard GIS object types, the functions required to manipulate them, and a set of meta-data tables. In order to ensure that meta-data remain consistent, operations such as creating and removing a spatial column are carried out through special procedures defined by OpenGIS.

There are two OpenGIS meta-data tables: SPATIAL_REF_SYS and GEOMETRY_COLUMNS. The SPATIAL_REF_SYS table holds the numeric IDs and textual descriptions of coordinate systems used in the spatial database.

4.2.1. The SPATIAL_REF_SYS Table

The SPATIAL_REF_SYS table definition is as follows:

CREATE TABLE SPATIAL_REF_SYS ( 
  SRID INTEGER NOT NULL PRIMARY KEY, 
  AUTH_NAME VARCHAR(256), 
  AUTH_SRID INTEGER, 
  SRTEXT VARCHAR(2048), 
  PROJ4TEXT VARCHAR(2048)
)

The SPATIAL_REF_SYS columns are as follows:

SRID

An integer value that uniquely identifies the Spatial Referencing System (SRS) within the database.

AUTH_NAME

The name of the standard or standards body that is being cited for this reference system. For example, "EPSG" would be a valid AUTH_NAME.

AUTH_SRID

The ID of the Spatial Reference System as defined by the Authority cited in the AUTH_NAME. In the case of EPSG, this is where the EPSG projection code would go.

SRTEXT

The Well-Known Text representation of the Spatial Reference System. An example of a WKT SRS representation is:

PROJCS["NAD83 / UTM Zone 10N", 
  GEOGCS["NAD83",
    DATUM["North_American_Datum_1983", 
      SPHEROID["GRS 1980",6378137,298.257222101]
    ], 
    PRIMEM["Greenwich",0], 
    UNIT["degree",0.0174532925199433] 
  ],
  PROJECTION["Transverse_Mercator"], 
  PARAMETER["latitude_of_origin",0],
  PARAMETER["central_meridian",-123], 
  PARAMETER["scale_factor",0.9996],
  PARAMETER["false_easting",500000], 
  PARAMETER["false_northing",0],
  UNIT["metre",1] 
]

For a listing of EPSG projection codes and their corresponding WKT representations, see http://www.opengis.org/techno/interop/EPSG2WKT.TXT. For a discussion of WKT in general, see the OpenGIS "Coordinate Transformation Services Implementation Specification" at http://www.opengis.org/techno/specs.htm. For information on the European Petroleum Survey Group (EPSG) and their database of spatial reference systems, see http://epsg.org.

PROJ4TEXT

PostGIS uses the Proj4 library to provide coordinate transformation capabilities. The PROJ4TEXT column contains the Proj4 coordinate definition string for a particular SRID. For example:

+proj=utm +zone=10 +ellps=clrk66 +datum=NAD27 +units=m

For more information about, see the Proj4 web site at http://www.remotesensing.org/proj. The spatial_ref_sys.sql file contains both SRTEXT and PROJ4TEXT definitions for all EPSG projections.

4.2.2. The GEOMETRY_COLUMNS Table

The GEOMETRY_COLUMNS table definition is as follows:

CREATE TABLE GEOMETRY_COLUMNS ( 
  F_TABLE_CATALOG VARCHAR(256) NOT NULL, 
  F_TABLE_SCHEMA VARCHAR(256) NOT NULL, 
  F_TABLE_NAME VARCHAR(256) NOT NULL, 
  F_GEOMETRY_COLUMN VARCHAR(256) NOT NULL,
  COORD_DIMENSION INTEGER NOT NULL, 
  SRID INTEGER NOT NULL, 
  TYPE VARCHAR(30) NOT NULL 
)

The columns are as follows:

F_TABLE_CATALOG, F_TABLE_SCHEMA, F_TABLE_NAME

The fully qualified name of the feature table containing the geometry column. Note that the terms "catalog" and "schema" are Oracle-ish. There is not PostgreSQL analogue of "catalog" so that column is left blank -- for "schema" the PostgreSQL schema name is used (public is the default).

F_GEOMETRY_COLUMN

The name of the geometry column in the feature table.

COORD_DIMENSION

The spatial dimension (2, 3 or 4 dimensional) of the column.

SRID

The ID of the spatial reference system used for the coordinate geometry in this table. It is a foreign key reference to the SPATIAL_REF_SYS.

TYPE

The type of the spatial object. To restrict the spatial column to a single type, use one of: POINT, LINESTRING, POLYGON, MULTIPOINT, MULTILINESTRING, MULTIPOLYGON, GEOMETRYCOLLECTION or corresponding XYM versions POINTM, LINESTRINGM, POLYGONM, MULTIPOINTM, MULTILINESTRINGM, MULTIPOLYGONM, GEOMETRYCOLLECTIONM. For heterogeneous (mixed-type) collections, you can use "GEOMETRY" as the type.

Note

This attribute is (probably) not part of the OpenGIS specification, but is required for ensuring type homogeneity.

4.2.3. Creating a Spatial Table

Creating a table with spatial data is done in two stages:

  • Create a normal non-spatial table.

    For example: CREATE TABLE ROADS_GEOM ( ID int4, NAME varchar(25) )

  • Add a spatial column to the table using the OpenGIS "AddGeometryColumn" function.

    The syntax is:

    AddGeometryColumn(<schema_name>, <table_name>,
                <column_name>, <srid>, <type>,
                <dimension>)

    Or, using current schema:

    AddGeometryColumn(<table_name>,
                <column_name>, <srid>, <type>,
                <dimension>)

    Example1: SELECT AddGeometryColumn('public', 'roads_geom', 'geom', 423, 'LINESTRING', 2)

    Example2: SELECT AddGeometryColumn( 'roads_geom', 'geom', 423, 'LINESTRING', 2)

Here is an example of SQL used to create a table and add a spatial column (assuming that an SRID of 128 exists already):

CREATE TABLE parks ( PARK_ID int4, PARK_NAME varchar(128), PARK_DATE date, PARK_TYPE varchar(2) );
SELECT AddGeometryColumn('parks', 'park_geom', 128, 'MULTIPOLYGON', 2 );

Here is another example, using the generic "geometry" type and the undefined SRID value of -1:

CREATE TABLE roads ( ROAD_ID int4, ROAD_NAME varchar(128) ); 
SELECT AddGeometryColumn( 'roads', 'roads_geom', -1, 'GEOMETRY', 3 );

4.3. Loading GIS Data

Once you have created a spatial table, you are ready to upload GIS data to the database. Currently, there are two ways to get data into a PostGIS/PostgreSQL database: using formatted SQL statements or using the Shape file loader/dumper.

4.3.1. Using SQL

If you can convert your data to a text representation, then using formatted SQL might be the easiest way to get your data into PostGIS. As with Oracle and other SQL databases, data can be bulk loaded by piping a large text file full of SQL "INSERT" statements into the SQL terminal monitor.

A data upload file (roads.sql for example) might look like this:

BEGIN;
INSERT INTO ROADS_GEOM (ID,GEOM,NAME ) VALUES (1,GeomFromText('LINESTRING(191232 243118,191108 243242)',-1),'Jeff Rd'); 
INSERT INTO ROADS_GEOM (ID,GEOM,NAME ) VALUES (2,GeomFromText('LINESTRING(189141 244158,189265 244817)',-1),'Geordie Rd'); 
INSERT INTO ROADS_GEOM (ID,GEOM,NAME ) VALUES (3,GeomFromText('LINESTRING(192783 228138,192612 229814)',-1),'Paul St'); 
INSERT INTO ROADS_GEOM (ID,GEOM,NAME ) VALUES (4,GeomFromText('LINESTRING(189412 252431,189631 259122)',-1),'Graeme Ave'); 
INSERT INTO ROADS_GEOM (ID,GEOM,NAME ) VALUES (5,GeomFromText('LINESTRING(190131 224148,190871 228134)',-1),'Phil Tce'); 
INSERT INTO ROADS_GEOM (ID,GEOM,NAME ) VALUES (6,GeomFromText('LINESTRING(198231 263418,198213 268322)',-1),'Dave Cres');
COMMIT;

The data file can be piped into PostgreSQL very easily using the "psql" SQL terminal monitor:

psql -d [database] -f roads.sql

4.3.2. Using the Loader

The shp2pgsql data loader converts ESRI Shape files into SQL suitable for insertion into a PostGIS/PostgreSQL database. The loader has several operating modes distinguished by command line flags:

-d

Drops the database table before creating a new table with the data in the Shape file.

-a

Appends data from the Shape file into the database table. Note that to use this option to load multiple files, the files must have the same attributes and same data types.

-c

Creates a new table and populates it from the Shape file. This is the default mode.

-D

Creates a new table and populates it from the Shape file. This uses the PostgreSQL "dump" format for the output data and is much faster to load than the default "insert" SQL format. Use this for very large data sets.

-s <SRID>

Creates and populates the geometry tables with the specified SRID.

-k

Keep idendifiers case (column, schema and attributes). Note that attributes in Shapefile are all UPPERCASE.

-i

Coerce all integers to standard 32-bit integers, do not create 64-bit bigints, even if the DBF header signature appears to warrant it.

An example session using the loader to create an input file and uploading it might look like this:

# shp2pgsql shaperoads myschema.roadstable > roads.sql 
# psql -d roadsdb -f roads.sql

A conversion and upload can be done all in one step using UNIX pipes:

# shp2pgsql shaperoads myschema.roadstable | psql -d roadsdb

4.4. Retrieving GIS Data

Data can be extracted from the database using either SQL or the Shape file loader/dumper. In the section on SQL we will discuss some of the operators available to do comparisons and queries on spatial tables.

4.4.1. Using SQL

The most straightforward means of pulling data out of the database is to use a SQL select query and dump the resulting columns into a parsable text file:

db=# SELECT id, AsText(geom) AS geom, name FROM ROADS_GEOM; 
id | geom                                    | name 
---+-----------------------------------------+-----------
 1 | LINESTRING(191232 243118,191108 243242) | Jeff Rd  
 2 | LINESTRING(189141 244158,189265 244817) | Geordie Rd 
 3 | LINESTRING(192783 228138,192612 229814) | Paul St 
 4 | LINESTRING(189412 252431,189631 259122) | Graeme Ave 
 5 | LINESTRING(190131 224148,190871 228134) | Phil Tce 
 6 | LINESTRING(198231 263418,198213 268322) | Dave Cres 
 7 | LINESTRING(218421 284121,224123 241231) | Chris Way 
(6 rows)

However, there will be times when some kind of restriction is necessary to cut down the number of fields returned. In the case of attribute-based restrictions, just use the same SQL syntax as normal with a non-spatial table. In the case of spatial restrictions, the following operators are available/useful:

&&

This operator tells whether the bounding box of one geometry intersects the bounding box of another.

~=

This operators tests whether two geometries are geometrically identical. For example, if 'POLYGON((0 0,1 1,1 0,0 0))' is the same as 'POLYGON((0 0,1 1,1 0,0 0))' (it is).

=

This operator is a little more naive, it only tests whether the bounding boxes of to geometries are the same.

Next, you can use these operators in queries. Note that when specifying geometries and boxes on the SQL command line, you must explicitly turn the string representations into geometries by using the "GeomFromText()" function. So, for example:

SELECT 
  ID, NAME 
FROM ROADS_GEOM 
WHERE 
  GEOM ~= GeomFromText('LINESTRING(191232 243118,191108 243242)',-1);

The above query would return the single record from the "ROADS_GEOM" table in which the geometry was equal to that value.

When using the "&&" operator, you can specify either a BOX3D as the comparison feature or a GEOMETRY. When you specify a GEOMETRY, however, its bounding box will be used for the comparison.

SELECT 
  ID, NAME 
FROM ROADS_GEOM 
WHERE 
  GEOM && GeomFromText('POLYGON((191232 243117,191232 243119,191234 243117,191232 243117))',-1);

The above query will use the bounding box of the polygon for comparison purposes.

The most common spatial query will probably be a "frame-based" query, used by client software, like data browsers and web mappers, to grab a "map frame" worth of data for display. Using a "BOX3D" object for the frame, such a query looks like this:

SELECT 
  AsText(GEOM) AS GEOM 
FROM ROADS_GEOM 
WHERE 
  GEOM && GeomFromText('BOX3D(191232 243117,191232 243119)'::box3d,-1);

Note the use of the SRID, to specify the projection of the BOX3D. The value -1 is used to indicate no specified SRID.

4.4.2. Using the Dumper

The pgsql2shp table dumper connects directly to the database and converts a table (possibly defined by a query) into a shape file. The basic syntax is:

pgsql2shp [<options>] <database> [<schema>.]<table>
pgsql2shp [<options>] <database> <query>

The commandline options are:

-f <filename>

Write the output to a particular filename.

-h <host>

The database host to connect to.

-p <port>

The port to connect to on the database host.

-P <password>

The password to use when connecting to the database.

-u <user>

The username to use when connecting to the database.

-g <geometry column>

In the case of tables with multiple geometry columns, the geometry column to use when writing the shape file.

-b

Use a binary cursor. This will make the operation faster, but will not work if any NON-geometry attribute in the table lacks a cast to text.

-r

Raw mode. Do not drop the gid field, or escape column names.

-d

For backward compatibility: write a 3-dimensional shape file when dumping from old (pre-1.0.0) postgis databases (the default is to write a 2-dimensional shape file in that case). Starting from postgis-1.0.0+, dimensions are fully encoded.

4.5. Building Indexes

Indexes are what make using a spatial database for large data sets possible. Without indexing, any search for a feature would require a "sequential scan" of every record in the database. Indexing speeds up searching by organizing the data into a search tree which can be quickly traversed to find a particular record. PostgreSQL supports three kinds of indexes by default: B-Tree indexes, R-Tree indexes, and GiST indexes.

  • B-Trees are used for data which can be sorted along one axis; for example, numbers, letters, dates. GIS data cannot be rationally sorted along one axis (which is greater, (0,0) or (0,1) or (1,0)?) so B-Tree indexing is of no use for us.

  • R-Trees break up data into rectangles, and sub-rectangles, and sub-sub rectangles, etc. R-Trees are used by some spatial databases to index GIS data, but the PostgreSQL R-Tree implementation is not as robust as the GiST implementation.

  • GiST (Generalized Search Trees) indexes break up data into "things to one side", "things which overlap", "things which are inside" and can be used on a wide range of data-types, including GIS data. PostGIS uses an R-Tree index implemented on top of GiST to index GIS data.

4.5.1. GiST Indexes

GiST stands for "Generalized Search Tree" and is a generic form of indexing. In addition to GIS indexing, GiST is used to speed up searches on all kinds of irregular data structures (integer arrays, spectral data, etc) which are not amenable to normal B-Tree indexing.

Once a GIS data table exceeds a few thousand rows, you will want to build an index to speed up spatial searches of the data (unless all your searches are based on attributes, in which case you'll want to build a normal index on the attribute fields).

The syntax for building a GiST index on a "geometry" column is as follows:

CREATE INDEX [indexname] ON [tablename] 
  USING GIST ( [geometryfield] GIST_GEOMETRY_OPS ); 

Building a spatial index is a computationally intensive exercise: on tables of around 1 million rows, on a 300MHz Solaris machine, we have found building a GiST index takes about 1 hour. After building an index, it is important to force PostgreSQL to collect table statistics, which are used to optimize query plans:

VACUUM ANALYZE [table_name] [column_name];

-- This is only needed for PostgreSQL 7.4 installations and below
SELECT UPDATE_GEOMETRY_STATS([table_name], [column_name]);

GiST indexes have two advantages over R-Tree indexes in PostgreSQL. Firstly, GiST indexes are "null safe", meaning they can index columns which include null values. Secondly, GiST indexes support the concept of "lossiness" which is important when dealing with GIS objects larger than the PostgreSQL 8K page size. Lossiness allows PostgreSQL to store only the "important" part of an object in an index -- in the case of GIS objects, just the bounding box. GIS objects larger than 8K will cause R-Tree indexes to fail in the process of being built.

4.5.2. Using Indexes

Ordinarily, indexes invisibly speed up data access: once the index is built, the query planner transparently decides when to use index information to speed up a query plan. Unfortunately, the PostgreSQL query planner does not optimize the use of GiST indexes well, so sometimes searches which should use a spatial index instead default to a sequence scan of the whole table.

If you find your spatial indexes are not being used (or your attribute indexes, for that matter) there are a couple things you can do:

  • Firstly, make sure statistics are gathered about the number and distributions of values in a table, to provide the query planner with better information to make decisions around index usage. For PostgreSQL 7.4 installations and below this is done by running update_geometry_stats([table_name, column_name]) (compute distribution) and VACUUM ANALYZE [table_name] [column_name] (compute number of values). Starting with PostgreSQL 8.0 running VACUUM ANALYZE will do both operations. You should regularly vacuum your databases anyways -- many PostgreSQL DBAs have VACUUM run as an off-peak cron job on a regular basis.

  • If vacuuming does not work, you can force the planner to use the index information by using the SET ENABLE_SEQSCAN=OFF command. You should only use this command sparingly, and only on spatially indexed queries: generally speaking, the planner knows better than you do about when to use normal B-Tree indexes. Once you have run your query, you should consider setting ENABLE_SEQSCAN back on, so that other queries will utilize the planner as normal.

    Note

    As of version 0.6, it should not be necessary to force the planner to use the index with ENABLE_SEQSCAN.

  • If you find the planner wrong about the cost of sequencial vs index scans try reducing the value of random_page_cost in postgresql.conf or using SET random_page_cost=#. Default value for the parameter is 4, try setting it to 1 or 2. Decrementing the value makes the planner more inclined of using Index scans.

4.6. Complex Queries

The raison d'etre of spatial database functionality is performing queries inside the database which would ordinarily require desktop GIS functionality. Using PostGIS effectively requires knowing what spatial functions are available, and ensuring that appropriate indexes are in place to provide good performance.

4.6.1. Taking Advantage of Indexes

When constructing a query it is important to remember that only the bounding-box-based operators such as && can take advatage of the GiST spatial index. Functions such as distance() cannot use the index to optimize their operation. For example, the following query would be quite slow on a large table:

SELECT the_geom FROM geom_table
WHERE distance( the_geom, GeomFromText( 'POINT(100000 200000)', -1 ) ) < 100

This query is selecting all the geometries in geom_table which are within 100 units of the point (100000, 200000). It will be slow because it is calculating the distance between each point in the table and our specified point, ie. one distance() calculation for each row in the table. We can avoid this by using the && operator to reduce the number of distance calculations required:

SELECT the_geom FROM geom_table
WHERE the_geom && 'BOX3D(90900 190900, 100100 200100)'::box3d
  AND distance( the_geom, GeomFromText( 'POINT(100000 200000)', -1 ) ) < 100

This query selects the same geometries, but it does it in a more efficient way. Assuming there is a GiST index on the_geom, the query planner will recognize that it can use the index to reduce the number of rows before calculating the result of the distance() function. Notice that the BOX3D geometry which is used in the && operation is a 200 unit square box centered on the original point - this is our "query box". The && operator uses the index to quickly reduce the result set down to only those geometries which have bounding boxes that overlap the "query box". Assuming that our query box is much smaller than the extents of the entire geometry table, this will drastically reduce the number of distance calculations that need to be done.

4.6.2. Examples of Spatial SQL

The examples in this section will make use of two tables, a table of linear roads, and a table of polygonal municipality boundaries. The table definitions for the bc_roads table is:

  Column    |       Type        |   Description
------------+-------------------+-------------------
 gid        | integer           | Unique ID
 name       | character varying | Road Name
 the_geom   | geometry          | Location Geometry (Linestring)

The table definition for the bc_municipality table is:

  Column   |       Type        |   Description
-----------+-------------------+-------------------
 gid       | integer           | Unique ID
 code      | integer           | Unique ID
 name      | character varying | City / Town Name
 the_geom  | geometry          | Location Geometry (Polygon)
4.6.2.1.1. What is the total length of all roads, expressed in kilometers?
4.6.2.1.2. How large is the city of Prince George, in hectares?
4.6.2.1.3. What is the largest municipality in the province, by area?
4.6.2.1.4. What is the length of roads fully contained within each municipality?
4.6.2.1.5. Create a new table with all the roads within the city of Prince George.
4.6.2.1.6. What is the length in kilometers of "Douglas St" in Victoria?
4.6.2.1.7. What is the largest municipality polygon that has a hole?
4.6.2.1.1. What is the total length of all roads, expressed in kilometers?
4.6.2.1.2. How large is the city of Prince George, in hectares?
4.6.2.1.3. What is the largest municipality in the province, by area?
4.6.2.1.4. What is the length of roads fully contained within each municipality?
4.6.2.1.5. Create a new table with all the roads within the city of Prince George.
4.6.2.1.6. What is the length in kilometers of "Douglas St" in Victoria?
4.6.2.1.7. What is the largest municipality polygon that has a hole?
4.6.2.1.1.

What is the total length of all roads, expressed in kilometers?

You can answer this question with a very simple piece of SQL:

postgis=# SELECT sum(length(the_geom))/1000 AS km_roads FROM bc_roads;
     km_roads
------------------
 70842.1243039643
(1 row)
4.6.2.1.2.

How large is the city of Prince George, in hectares?

This query combines an attribute condition (on the municipality name) with a spatial calculation (of the area):

postgis=# SELECT area(the_geom)/10000 AS hectares FROM bc_municipality 
          WHERE name = 'PRINCE GEORGE';
     hectares
------------------
 32657.9103824927
(1 row) 
4.6.2.1.3.

What is the largest municipality in the province, by area?

This query brings a spatial measurement into the query condition. There are several ways of approaching this problem, but the most efficient is below:

postgis=# SELECT name, area(the_geom)/10000 AS hectares 
          FROM bc_municipality 
          ORDER BY hectares DESC 
          LIMIT 1;
     name      |    hectares
---------------+-----------------
 TUMBLER RIDGE | 155020.02556131
(1 row)

Note that in order to answer this query we have to calculate the area of every polygon. If we were doing this a lot it would make sense to add an area column to the table that we could separately index for performance. By ordering the results in a descending direction, and them using the PostgreSQL "LIMIT" command we can easily pick off the largest value without using an aggregate function like max().

4.6.2.1.4.

What is the length of roads fully contained within each municipality?

This is an example of a "spatial join", because we are bringing together data from two tables (doing a join) but using a spatial interaction condition ("contained") as the join condition rather than the usual relational approach of joining on a common key:

postgis=# SELECT m.name, sum(length(r.the_geom))/1000 as roads_km 
          FROM bc_roads AS r,bc_municipality AS m 
          WHERE r.the_geom && m.the_geom 
          AND contains(m.the_geom,r.the_geom) 
          GROUP BY m.name 
          ORDER BY roads_km;

            name            |     roads_km
----------------------------+------------------
 SURREY                     | 1539.47553551242
 VANCOUVER                  | 1450.33093486576
 LANGLEY DISTRICT           | 833.793392535662
 BURNABY                    | 773.769091404338
 PRINCE GEORGE              |  694.37554369147
 ...

This query takes a while, because every road in the table is summarized into the final result (about 250K roads for our particular example table). For smaller overlays (several thousand records on several hundred) the response can be very fast.

4.6.2.1.5.

Create a new table with all the roads within the city of Prince George.

This is an example of an "overlay", which takes in two tables and outputs a new table that consists of spatially clipped or cut resultants. Unlike the "spatial join" demonstrated above, this query actually creates new geometries. An overlay is like a turbo-charged spatial join, and is useful for more exact analysis work:

postgis=# CREATE TABLE pg_roads as
          SELECT intersection(r.the_geom, m.the_geom) AS intersection_geom, 
                 length(r.the_geom) AS rd_orig_length, 
                 r.* 
          FROM bc_roads AS r, bc_municipality AS m 
          WHERE r.the_geom && m.the_geom 
          AND intersects(r.the_geom, m.the_geom) 
          AND m.name = 'PRINCE GEORGE';
4.6.2.1.6.

What is the length in kilometers of "Douglas St" in Victoria?

postgis=# SELECT sum(length(r.the_geom))/1000 AS kilometers 
          FROM bc_roads r, bc_municipality m 
          WHERE r.the_geom && m.the_geom 
          AND r.name = 'Douglas St' 
          AND m.name = 'VICTORIA';
    kilometers
------------------
 4.89151904172838
(1 row)
4.6.2.1.7.

What is the largest municipality polygon that has a hole?

postgis=# SELECT gid, name, area(the_geom) AS area 
          FROM bc_municipality 
          WHERE nrings(the_geom) > 1 
          ORDER BY area DESC LIMIT 1;
 gid |     name     |       area
-----+--------------+------------------
  12 | SPALLUMCHEEN | 257374619.430216
(1 row)

4.7. Using Mapserver

The Minnesota Mapserver is an internet web-mapping server which conforms to the OpenGIS Web Mapping Server specification.

4.7.1. Basic Usage

To use PostGIS with Mapserver, you will need to know about how to configure Mapserver, which is beyond the scope of this documentation. This section will cover specific PostGIS issues and configuration details.

To use PostGIS with Mapserver, you will need:

  • Version 0.6 or newer of PostGIS.

  • Version 3.5 or newer of Mapserver.

Mapserver accesses PostGIS/PostgreSQL data like any other PostgreSQL client -- using libpq. This means that Mapserver can be installed on any machine with network access to the PostGIS server, as long as the system has the libpq PostgreSQL client libraries.

  1. Compile and install Mapserver, with whatever options you desire, including the "--with-postgis" configuration option.

  2. In your Mapserver map file, add a PostGIS layer. For example:

    LAYER
      CONNECTIONTYPE postgis
      NAME "widehighways"
      # Connect to a remote spatial database
      CONNECTION "user=dbuser dbname=gisdatabase host=bigserver"
      # Get the lines from the 'geom' column of the 'roads' table
      DATA "geom from roads"
      STATUS ON
      TYPE LINE
      # Of the lines in the extents, only render the wide highways
      FILTER "type = 'highway' and numlanes >= 4"
      CLASS
        # Make the superhighways brighter and 2 pixels wide
        EXPRESSION ([numlanes] >= 6)
        COLOR 255 22 22      
        SYMBOL "solid"
        SIZE 2
      END
      CLASS
        # All the rest are darker and only 1 pixel wide
        EXPRESSION ([numlanes] < 6)
        COLOR 205 92 82      
      END
    END

    In the example above, the PostGIS-specific directives are as follows:

    CONNECTIONTYPE

    For PostGIS layers, this is always "postgis".

    CONNECTION

    The database connection is governed by the a 'connection string' which is a standard set of keys and values like this (with the default values in <>):

    user=<username> password=<password> dbname=<username> hostname=<server> port=<5432>

    An empty connection string is still valid, and any of the key/value pairs can be omitted. At a minimum you will generally supply the database name and username to connect with.

    DATA

    The form of this parameter is "<column> from <tablename>" where the column is the spatial column to be rendered to the map.

    FILTER

    The filter must be a valid SQL string corresponding to the logic normally following the "WHERE" keyword in a SQL query. So, for example, to render only roads with 6 or more lanes, use a filter of "num_lanes >= 6".

  3. In your spatial database, ensure you have spatial (GiST) indexes built for any the layers you will be drawing.

    CREATE INDEX [indexname]
      ON [tablename] 
      USING GIST ( [geometrycolumn] GIST_GEOMETRY_OPS );
  4. If you will be querying your layers using Mapserver you will also need an "oid index".

    Mapserver requires unique identifiers for each spatial record when doing queries, and the PostGIS module of Mapserver uses the PostgreSQL oid value to provide these unique identifiers. A side-effect of this is that in order to do fast random access of records during queries, an index on the oid is needed.

    To build an "oid index", use the following SQL:

    CREATE INDEX [indexname] ON [tablename] ( oid );

4.7.2. Frequently Asked Questions

4.7.2.1.1. When I use an EXPRESSION in my map file, the condition never returns as true, even though I know the values exist in my table.
4.7.2.1.2. The FILTER I use for my Shape files is not working for my PostGIS table of the same data.
4.7.2.1.3. My PostGIS layer draws much slower than my Shape file layer, is this normal?
4.7.2.1.4. My PostGIS layer draws fine, but queries are really slow. What is wrong?
4.7.2.1.1. When I use an EXPRESSION in my map file, the condition never returns as true, even though I know the values exist in my table.
4.7.2.1.2. The FILTER I use for my Shape files is not working for my PostGIS table of the same data.
4.7.2.1.3. My PostGIS layer draws much slower than my Shape file layer, is this normal?
4.7.2.1.4. My PostGIS layer draws fine, but queries are really slow. What is wrong?
4.7.2.1.1.

When I use an EXPRESSION in my map file, the condition never returns as true, even though I know the values exist in my table.

Unlike shape files, PostGIS field names have to be referenced in EXPRESSIONS using lower case.

EXPRESSION ([numlanes] >= 6)
4.7.2.1.2.

The FILTER I use for my Shape files is not working for my PostGIS table of the same data.

Unlike shape files, filters for PostGIS layers use SQL syntax (they are appended to the SQL statement the PostGIS connector generates for drawing layers in Mapserver).

FILTER "type = 'highway' and numlanes >= 4"
4.7.2.1.3.

My PostGIS layer draws much slower than my Shape file layer, is this normal?

In general, expect PostGIS layers to be 10% slower than equivalent Shape files layers, due to the extra overhead involved in database connections, data transformations and data transit between the database and Mapserver.

If you are finding substantial draw performance problems, it is likely that you have not build a spatial index on your table.

postgis# CREATE INDEX geotable_gix ON geotable USING GIST ( geocolumn );
postgis# SELECT update_geometry_stats();  -- For PGSQL < 8.0
postgis# VACUUM ANALYZE;                  -- For PGSQL >= 8.0
4.7.2.1.4.

My PostGIS layer draws fine, but queries are really slow. What is wrong?

For queries to be fast, you must have a unique key for your spatial table and you must have an index on that unique key.

You can specify what unique key for mapserver to use with the USING UNIQUE clause in your DATA line:

DATA "the_geom FROM geotable USING UNIQUE gid"

If your table does not have an explicit unique column, you can "fake" a unique column by using the PostgreSQL row "oid" for your unique column. "oid" is the default unique column if you do not declare one, so enhancing your query speed is a matter of building an index on your spatial table oid value.

postgis# CREATE INDEX geotable_oid_idx ON geotable (oid);

4.7.3. Advanced Usage

The USING pseudo-SQL clause is used to add some information to help mapserver understand the results of more complex queries. More specifically, when either a view or a subselect is used as the source table (the thing to the right of "FROM" in a DATA definition) it is more difficult for mapserver to automatically determine a unique identifier for each row and also the SRID for the table. The USING clause can provide mapserver with these two pieces of information as follows:

DATA "the_geom FROM (SELECT table1.the_geom AS the_geom, table1.oid AS oid, table2.data AS data
 FROM table1 LEFT JOIN table2 ON table1.id = table2.id) AS new_table USING UNIQUE oid USING SRID=-1"
USING UNIQUE <uniqueid>

Mapserver requires a unique id for each row in order to identify the row when doing map queries. Normally, it would use the oid as the unique identifier, but views and subselects don't automatically have an oid column. If you want to use Mapserver's query functionality, you need to add a unique column to your view or subselect, and declare it with USING UNIQUE. For example, you could explicitly select one of the table's oid values for this purpose, or any other column which is guaranteed to be unique for the result set.

The USING statement can also be useful even for simple DATA statements, if you are doing map queries. It was previously recommended to add an index on the oid column of tables used in query-able layers, in order to speed up the performance of map queries. However, with the USING clause, it is possible to tell mapserver to use your table's primary key as the identifier for map queries, and then it is no longer necessary to have an additional index.

Note

"Querying a Map" is the action of clicking on a map to ask for information about the map features in that location. Don't confuse "map queries" with the SQL query in a DATA definition.

USING SRID=<srid>

PostGIS needs to know which spatial referencing system is being used by the geometries in order to return the correct data back to mapserver. Normally it is possible to find this information in the "geometry_columns" table in the PostGIS database, however, this is not possible for tables which are created on the fly such as subselects and views. So the USING SRID= option allows the correct SRID to be specified in the DATA definition.

Warning

The parser for Mapserver PostGIS layers is fairly primitive, and is case sensitive in a few areas. Be careful to ensure that all SQL keywords and all your USING clauses are in upper case, and that your USING UNIQUE clause precedes your USING SRID clause.

4.7.4. Examples

Lets start with a simple example and work our way up. Consider the following Mapserver layer definition:

LAYER
 CONNECTIONTYPE postgis
 NAME "roads"
 CONNECTION "user=theuser password=thepass dbname=thedb host=theserver"
 DATA "the_geom FROM roads"
 STATUS ON
 TYPE LINE
 CLASS
  COLOR 0 0 0
 END
END

This layer will display all the road geometries in the roads table as black lines.

Now lets say we want to show only the highways until we get zoomed in to at least a 1:100000 scale - the next two layers will acheive this effect:

LAYER
 CONNECTION "user=theuser password=thepass dbname=thedb host=theserver"
 DATA "the_geom FROM roads"
 MINSCALE 100000
 STATUS ON
 TYPE LINE
 FILTER "road_type = 'highway'"
 CLASS
  COLOR 0 0 0
 END
END

LAYER
 CONNECTION "user=theuser password=thepass dbname=thedb host=theserver"
 DATA "the_geom FROM roads"
 MAXSCALE 100000
 STATUS ON
 TYPE LINE
 CLASSITEM road_type
 CLASS
  EXPRESSION "highway"
  SIZE 2
  COLOR 255 0 0
 END
 CLASS
  COLOR 0 0 0
 END
END

The first layer is used when the scale is greater than 1:100000, and displays only the roads of type "highway" as black lines. The FILTER option causes only roads of type "highway" to be displayed.

The second layer is used when the scale is less than 1:100000, and will display highways as double-thick red lines, and other roads as regular black lines.

So, we have done a couple of interesting things using only mapserver functionality, but our DATA SQL statement has remained simple. Suppose that the name of the road is stored in another table (for whatever reason) and we need to do a join to get it and label our roads.

LAYER
 CONNECTION "user=theuser password=thepass dbname=thedb host=theserver"
 DATA "the_geom FROM (SELECT roads.oid AS oid, roads.the_geom AS the_geom, road_names.name as name
   FROM roads LEFT JOIN road_names ON roads.road_name_id = road_names.road_name_id) AS named_roads
   USING UNIQUE oid USING SRID=-1"
 MAXSCALE 20000
 STATUS ON
 TYPE ANNOTATION
 LABELITEM name
 CLASS
  LABEL
   ANGLE auto
   SIZE 8
   COLOR 0 192 0
   TYPE truetype
   FONT arial
  END
 END
END

This annotation layer adds green labels to all the roads when the scale gets down to 1:20000 or less. It also demonstrates how to use an SQL join in a DATA definition.

4.8. Java Clients (JDBC)

Java clients can access PostGIS "geometry" objects in the PostgreSQL database either directly as text representations or using the JDBC extension objects bundled with PostGIS. In order to use the extension objects, the "postgis.jar" file must be in your CLASSPATH along with the "postgresql.jar" JDBC driver package.

import java.sql.*; 
import java.util.*; 
import java.lang.*; 
import org.postgis.*; 

public class JavaGIS { 
  public static void main(String[] args) 
  { 
    java.sql.Connection conn; 
    try 
    { 
      /* 
      * Load the JDBC driver and establish a connection. 
      */  
      Class.forName("org.postgresql.Driver"); 
      String url = "jdbc:postgresql://localhost:5432/database"; 
      conn = DriverManager.getConnection(url, "postgres", ""); 
    
      /* 
      * Add the geometry types to the connection. Note that you 
      * must cast the connection to the pgsql-specific connection * implementation before calling the addDataType() method. 
      */
      ((org.postgresql.Connection)conn).addDataType("geometry","org.postgis.PGgeometry");
      ((org.postgresql.Connection)conn).addDataType("box3d","org.postgis.PGbox3d");

      /* 
      * Create a statement and execute a select query. 
      */ 
      Statement s = conn.createStatement(); 
      ResultSet r = s.executeQuery("select AsText(geom) as geom,id from geomtable"); 
      while( r.next() ) 
      { 
        /* 
        * Retrieve the geometry as an object then cast it to the geometry type. 
        * Print things out. 
        */ 
        PGgeometry geom = (PGgeometry)r.getObject(1); 
        int id = r.getInt(2);
        System.out.println("Row " + id + ":"); 
        System.out.println(geom.toString()); 
      }
      s.close(); 
      conn.close(); 
    } 
    catch( Exception e ) 
    { 
      e.printStackTrace(); 
    }  
  }
}

The "PGgeometry" object is a wrapper object which contains a specific topological geometry object (subclasses of the abstract class "Geometry") depending on the type: Point, LineString, Polygon, MultiPoint, MultiLineString, MultiPolygon.

PGgeometry geom = (PGgeometry)r.getObject(1); 
if( geom.getType() = Geometry.POLYGON ) 
{ 
  Polygon pl = (Polygon)geom.getGeometry();
  for( int r = 0; r < pl.numRings(); r++ ) 
  { 
    LinearRing rng = pl.getRing(r);
    System.out.println("Ring: " + r); 
    for( int p = 0; p < rng.numPoints(); p++ ) 
    { 
      Point pt = rng.getPoint(p); 
      System.out.println("Point: " + p);
      System.out.println(pt.toString()); 
    } 
  } 
}

The JavaDoc for the extension objects provides a reference for the various data accessor functions in the geometric objects.

4.9. C Clients (libpq)

...

4.9.1. Text Cursors

...

4.9.2. Binary Cursors

...