next_inactive up previous Implementation of OGC WPS standard: PyWPS Jachym Cepicky Copyright (c)2006-2009 PyWPS Development Team Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license is included in the section entitled "GNU Free Documentation License". In this file, you can found the description of installation and configuration of PyWPS script. At the and, you can learn, how to add your own process. This document describes most recent version of PyWPS (2.0.0), available in subversion respository. PyWPS project has been started on April 2006 with support of DBU - Deutsche Bundesstiftung Umwelt1 and with help of GDF-Hannover2 and Help Service Remote Sensing3 companies. Initial author is Jachym Cepicky4. Contents * Introduction * How it works * Quick install * Know issues * Installation * Installation the quick 'n' dirty way * Installation the 'clean' way * Configuration * Write your own processes * Process initialization and configuration * Process Programming * Using GRASS GIS * Testing your new process Introduction PyWPS (Python Web Processing Service) is implementation of Web Processing Service 1.0.x standard from Open Geospatial Consortium5. It has been started on Mai 2006 as project supported by DBU. It offers environment for programming own process (geofunctions or models) which can be accessed from the public. The main advantage of PyWPS is, that it has been written with native support for GRASS GIS6. Access GRASS modules via web interace should be as easy as possible. However, not only GRASS GIS is supported. Usage of other programs, like R package or GDAL or PROJ tools is possible as well. PyWPS is written in Python programming language, your processes must use this language too. PyWPS Homepage can be found at http://pywps.wald.intevation.org. PyWPS Wiki is hosted on http://pywps.ominiverdi.org/wiki. How it works PyWPS is an translator application between client (Web Browser, Desktop GIS, command line tool, ...) and working tool installed on the server. PyWPS does no process the data by it self. As working tool, GRASS GIS, GDAL, PROJ, R and other programs can be used. Figure 1: How does PyWPS work: GRASS GIS is in this case working tool Image pywps-schema Quick install 1. Install PyWPS, see page [*] for details 2. NOTE: Rename original files (process examples, configuration files) with .py-dist suffix to .py, when you see them. 3. Edit configuration files in pywps/etc/ directory. See page [*] for details. 4. Create or edit __init__.py file in pywps/processes directory. Add available process names to __all__ array. 5. Add your processes to pywps/processes directory. See page [*] for details. 6. Run PyWPS with ./wps.py command, see page [*] for details. Know issues Known bugs and limitations to UL> * Translations do not work for GetCapabilities. They only work for DescribeProcess request types. * If inputs are of type LiteralValue and it's type is string, it could be security problem. Take care on your inputs and do not use it directly in scripts to avoid your server to be hacked. Please report all problems or unexpected handeling either via pywps mailing list7or using PyWPS bugtracker8. Installation Required packages: * python * python-xml * python-htmltmpl Recommended packages: * Web Server (e.g. Apache) - http://httpd.apache.org - You will need an web server, to be able to execute processes from remote computers. * GIS GRASS - http://grass.itc.it - Geographical Resources Analysis Support System (GRASS) is Open Source GIS, which provides more then 350 modules for raster and vector (2D, 3D) data analysis. PyWPS is written with native support for GRASS and it's functions. * PROJ.4 - http://proj.maptools.org - Cartographic Projections library used in various Open Source projects, such as GRASS, UMN MapServer, QGIS and others. It can be used e.g. for data transformation. * GDAL/OGR - http://gdal.org - translator library for raster geospatial data formats, is used in various projects for importing, exporting and transformation between various raster and vector data formats. * R - http://www.r-project.org - is a language and environment for statistical computing and graphics. Installation the quick 'n' dirty way For installing pywps to your server simply unzip the archive to the directory, where cgi programs are allowed to run. You can also use current repository version. $ cd /usr/lib/cgi-bin/ $ tar xvzf /tmp/pywps-VERSION.tar.gz $ pywps/wps.py Installation the 'clean' way Unzip the package $ tar -xzf pywps-VERSION.tar.gz and run $ python setup.py install adjust the configuration file $ vim /etc/pywps.cfg permint write access to templates directory # chmod -R 777 /usr/lib/python2.5/site-packages/pywps/Templates Several binary packages for Linux distributios (RPM,DEB) are also avaliable on PyWPS homepage9. Configuration Before you start to tune your PyWPS installation, you should get your copy of OpenGIS(R) Web Processing Service document (OGC 05-007r7) version 1.0.010. NOTE: Note, that the configuration option are CASE SENSITIVE Pywps configuration takes place in pywps.cfg file located in /etc/pywps.cfg or pywps/etc/pywps.cfg. Default configuration file is located in pywps/default.cfg, you can always make a copy of this file and start the configuration from scratch. Several sections are in the file. * Section [wps] contains general WPS settings, which are: * encoding - Language encoding (utf-8, iso-8859-2, windows-1250, ...) * title - Server title * version - WPS version (1.0.0) * abstract - Server anstract * fees - Possible fees * constraints - Possible constraints * serveraddress - WPS script address: http://foo/bar/wps.py * keywords - Comma-separated list of kyewords * lang - Default langue (eng) * Section [provider] contains informations about you * providerName - Name of your company * individualName - Your name * positionName * role * deliveryPoint - Street * city * postalCode * country * electronicMailAddress - foo@bar * providerSite - http://foo.bar * phoneVoice * phoneFacsimile * administrativeArea * Section [server] contains server settings * maxoperations - Maximal number of parallel running processes. If set to 0, then there is no limit. * maxinputparamlength - Maximal length of string input parameter. * maxfilesize - Maximal input file size (raster or vector). The size can be determined as follows: 1GB, 5MB, 3kB, 1000b. * tempPath - Direcotory for temporary files (mostly temporary GRASS locations). * outputUrl - Url where the outputs are stored. * outputPath - Path. where output files are stored. * debug - true/false * Section [grass] - GRASS GIS settings * path - $PATH variable, e.g. /usr/lib/grass/bin * addonPath - $GRASS_ADDONS variable * version - GRASS version * gui - Should be "text" * gisbase - Path to GRASS GIS_BASE directory (/usr/lib/grass) * ldLibraryPath - Path of GRASS Libs (/usr/lib/grass/lib) File example follows: [wps] encoding=utf-8 title=PyWPS Server version=1.0.0 abstract=See http://pywps.wald.intevation.org and http://www.opengeospatial.org/standards/wps fees=None constraints=none serveraddress=http://localhost/cgi-bin/wps keywords=GRASS,GIS,WPS lang=eng [provider] providerName=Your Company Name individualName=Your Name positionName=Your Position role=Your role deliveryPoint=Street city=City postalCode=000 00 country=eu electronicMailAddress=login@server.org providerSite=http://foo.bar phoneVoice=False phoneFacsimile=False administrativeArea=False [server] maxoperations=3 maxinputparamlength=1024 maxfilesize=3mb tempPath=/tmp outputUrl=http://localhost/wps/wpsoutputs outputPath=/var/www/wps/wpsoutputs debug=true [grass] path=/usr/lib/grass/bin/:/usr/lib/grass/scripts/ addonPath= version=6.2.1 gui=text gisbase=/usr/lib/grass/ ldLibraryPath=/usr/lib/grass/lib subsectionTesting after installation For test, just run wps.py in your command line: $ ./wps.py "service=wps&request=getcapabilities" INIT DONE LOADING PRECOMPILED TEMPLATE: UPTODATE PRECOMPILED: UPTODATE Content-type: text/xml PyWPS Development Server ... If you got something like this, (Capabilities response), everything looks fine. If you got some other message, like e.g.: Traceback (most recent call last): File "/usr/bin/wps.py", line 221, in wps = WPS() File "/usr/bin/wps.py", line 140, in __init__ self.performRequest() File "/usr/bin/wps.py", line 188, in performRequest from pywps.WPS.GetCapabilities import GetCapabilities File "/usr/lib/python2.5/site-packages/pywps/WPS/GetCapabilities.py", line 26, in from Response import Response File "/usr/lib/python2.5/site-packages/pywps/WPS/Response.py", line 28, in from htmltmpl import TemplateManager, TemplateProcessor ImportError: No module named htmltmpl Than something is wrong with your Python installation or with the program. This message means, that the python-htmltmpl package is not installed in your system. Write your own processes All processes are stored in the pywps/processes directory. You can create custom directory anywhere in your system and set $PYTHON_PROCESS environment variabl (how to do this for the web server, refer to your Server documentation). Following example will describe buffering process. Several example processes are distributed along with PyWPS source code. Create file exampleBufferProcess.py in PYWPS_PROCESSES directory. Each process is stand-alone python script with one class Process, which has two methods: __init__, execute. It is possible also to add as many your functions/methods, as you wish. Process initialization and configuration 1 from pywps.Process.Process import WPSProcess 2 class Process(WPSProcess): 3 """Main process class""" 4 def __init__(self): 5 """Process initialization""" 7 # init process 8 WPSProcess.__init__(self, 9 identifier = "exampleBufferProcess", 10 title="Buffer", 11 version = "0.2", 12 storeSupported = "true", 13 statusSupported = "true", 14 abstract="Create a buffer around an input vector file", 15 grassLocation = True) We defined new process called exampleBufferProcess. The process is allowed to store it's output data on the server (storeSupported) and it is also possible to run it in asynchronous mode (statusSupported). The process will run within GRASS GIS environment (grassLocation = True). Metadata defition is stored in array self.Metadata in __init__ method. You can add new Medatada using self.AddMetadata() method: self.AddMetadata(identifier="point",type="point", textContent="Click in the map") Data Inputs Three types of data inputs are defined: * Literal Input - Basic literal input - single number or text value * ComplexValue Input - Mostly vector file embded in input XML request or reference (URL) to such file. * BoundingBox Input - Coordinates for lower-left and upper-right corner. ComplexInput Complex input can be raster or vector file, to be processed. 18 self.dataIn = self.addComplexInput(identifier="data", 19 title = "Input data") 20 LiteralInput With literal input, you can obtain any type of character string. 21 self.widthIn = self.addLiteralInput(identifier = "width", 22 title = "Width") 23 For further documentation, refere example processes distributed with the source code as well as pydoc pywps/Wps/Process.py. This help is also available in process.html11 file distributed along with PyWPS source code. Data Outputs Data outputs can be defined in similar way. * Literal Output * ComplexValue Outout * BoundingBox Output ComplexValue Output The complex value can be raster or vector file (or any other binary or text file). 24 self.bufferOut = self.addComplexOutput(identifier="buffer", 25 title="Output buffer file") 26 Literal Output If you want to output any text string. 27 self.textOut = self.addLiteralOutput(identifier="text", 28 title="just some text") 29 Process Programming The process must be defined in the execute(self) method. Basicly, you want to get input values and set output values. For this purpose, you can use getValue(input_identifier) and setValue(output_identifier,value) methods of the input/output objects (see lower). If you need to execute some shell command, you should use self.cmd(command,["string for standard input"]) instead of e.g. os.system() or os.popen() functions. Calculation progress can be set using self.status(string message, number percent) method. Example follows: 33 def execute(self): 34 """Execute process. 35 36 Each command will be executed and output values will be set 37 """ 38 39 # run some command from the command line 40 self.cmd("g.region -d") 41 42 # set status value 43 self.status.set("Importing data",20) 44 self.cmd("v.in.ogr dsn=%s output=data" %\ 45 (self.getInputValue('data'))) 46 47 self.status.set("Buffering",50) 48 self.cmd("v.buffer input=data output=data_buff buffer=%s scale=1.0 tolerance=0.01" %\ 49 (self.getInputValue('width'))) 50 51 self.status.set("Exporting data",90) 52 53 self.cmd("v.out.ogr type=area format=GML input=data_buff dsn=out.xml olayer=path.xml") 54 55 self.bufferOut.setValue("out.xml") 56 self.textOut.setValue("ahoj, svete") 57 return Error handling At the end of the execute function, None value should be returned. Any other value means, that the calculation will be stopped and error report will be returned back to the client, example: def execute(self): ... return "Ups, something failed!" Using GRASS GIS Configuration is done using standard pywps configuration file (see page[*]). If you want to use GRASS GIS commands in your process, and there is no GRASS Location to be used, you have to set grassLocation=True in process definition: WPSProcess.__init__(self, identifier = "exampleBufferProcess", .... grassLocation = True) In this case, temporary GRASS Location will be created and after the process is done, it will be deleted again. By default, no GRASS Location is created. You can also work in existing GRASS Location, then just set the location path: WPSProcess.__init__(self, identifier = "exampleBufferProcess", .... grassLocation = "/home/grass/grassdata/spearfish60") Testing your new process To test your PyWPS installation, you run it either as Webserver cgi-application or in the command line directly. It is always good to start with the command line test, so do not have to check error.log of the web server. * GetCapabilities request (webserver) ./wps.py "service=wps&request=getcapabilities" wget -nv -q -O - "http://localhost/cgi-bin/wps.py?\ service=Wps&request=getcapabilities" * DescribeProcess request: ./wps.py "version=1.0.0&service=Wps&request=DescribeProcess&\ Identifier=bufferExampleProcess" wget -nv -q -O - "http://localhost/cgi-bin/wps.py?\ version=0.4.0&service=Wps&request=DescribeProcess&\ Identifier=exampleBufferProcess" * Execute request: For data inputs encoding, using HTTP Get method, see OGC 05-007r712, page 38 "Execute HTTP GET request KVP encoding`` ./wps.py "version=1.0.0&service=Wps&\ request=Execute&Identifier=exampleBufferProcess&\ datainputs=data=http://foo/bar/roads.gml;width=0.5" Some examples of XML request econding are available in doc/examples directory. Before testing WPS via HTTP POST, you have to set REQUEST_METHOD environment variable, then you can redirect input XML into wps.py script via standard input: $ export REQUEST_METHOD=POST $ cat doc/wps_execute_request-responsedocument.xml|./wps.py About this document ... Implementation of OGC WPS standard: PyWPS This document was generated using the LaTeX2HTML translator Version 2002-2-1 (1.71) Copyright (c) 1993, 1994, 1995, 1996, Nikos Drakos, Computer Based Learning Unit, University of Leeds. Copyright (c) 1997, 1998, 1999, Ross Moore, Mathematics Department, Macquarie University, Sydney. The command line arguments were: latex2html -split 0 pywps-howto.tex The translation was initiated by Jachym Cepicky on 2008-08-28 ---------------------------------------------------------------------- Footnotes ... Umwelt1 http://dbu.de ... GDF-Hannover2 http://gdf-hannover.de ... Sensing3 http://www.bnhelp.cz ... Cepicky4 http://les-ejk.cz ... Consortium5 http://www.opengeospatial.org/standards/requests/28 ... GIS6 http://grass.itc.it ... list7 http://wald.intevation.org/mailman/listinfo/pywps-develPyWPS - development list ... bugtracker8 http://wald.intevation.org/tracker/?atid=174&group_id=22&func=browsePyWPS Bug tracker ... homepage9 http://pywps.wald.intevation.org ... 1.0.010 http://www.opengeospatial.org/standards/wps ...process.html11 http://wald.intevation.org/plugins/scmsvn/viewcvs.php/*checkout*/trunk/doc/process.html?rev=369&root=pywpsDocumentation to Process.py module ... 05-007r712 http://opengeospatial.org/standards/wps/ ---------------------------------------------------------------------- next_inactive up previous Jachym Cepicky 2008-08-28