An International Nonindigenous Species Database Network

Information for Developers

Download a pdf of the NISbase Documentation

Introduction:

During the past decade dozens of non-indigenous species (NIS) database workshops have been held around the world. At the end of those meetings the general consensus is almost always the same; we need to develop better ways to share our data. If you’ve attended one of these workshops recently, it is likely you have heard terms like “XML” and “distributed databases” touted as the answer to our prayers. Well, NISbase is a relatively simple lightweight XML based distributed database system designed for NIS information by the Smithsonian Environmental Research Center and the USGS.
To test NISbase yourself, go to the current version of the portal website.

How NISbase works:

The major concepts of NISbase were inspired by other distributed database systems being developed, particularly DiGIR (Distributed Generic Information Retrieval). DiGIR is currently being developed to share museum specimen collections by programs such as the Species Analyst and the MaNIS (Mammal Networked Information System). A DiGIR implementation of NISbase is currently under consideration and promises to add many improvements to the current system.

DiGIR http://digir.net
Species Analyst http://speciesanalyst.net/index.html
MaNIS http://elib.cs.berkeley.edu/manis/

NISbase, like DiGIR, functions as a distributed system of portals and data providers (Figure 1).

Figure 1. Basic NISbase schematic.

Figure 1 NISbase schematic

The portal’s web interface allows the user to search records on more than one data provider at a time. Think of the analogy of searching one of the many travel planning sites (a.k.a. portals) for the best airfare from multiple airlines (a.k.a. data providers), this method is much more efficient than searching each airline’s website individually. With NISbase, a user would search for NIS with the portal (Figure 2) and retrieve information from multiple NIS databases simultaneously. Search results are returned as a single table (Figure 3) with links to further information from the original data providers (Figure 4).

Figure 2. The NISbase Portal Search Form.

Figure 2 Portal Search Form

Figure 3. NISbase merged results table.

Figure 3 merged results table

Figure 4. A species summary web page example.

Figure 4 species summary

The current implementation of NISbase takes advantage of the way many websites display their NIS information, one species at a time and in the form of species summaries or fact sheets (see Figure 4). Furthermore, websites that have search capabilities for their NIS information allow the user to search other simple fields such as common name, species name and taxonomic group.

Some online NIS databases contain more searchable fields than NISbase. To accommodate these databases we are developing an advanced version of NISbase to allow the user to search on these advanced fields such as invasion pathway, habitat invaded, and date of first observation.

NISbase Provider Information

If your website can dynamically create an html table from your NIS database, you should be able to create the XML results needed to participate in NISbase. If you’re unfamiliar with XML, a visit to the following website can be helpful. http://www.w3schools.com/xml.

Provider Requirements:

  1. NIS information
  2. A web server with the ability to dynamically create html and xml through some scripting language (php, asp, perl, jsp, coldfusion, etc.)
  3. A database that the dynamic webpages can draw data from (ms-sql, access, mysql, oracle, postgress, etc..)
  4. Ability to query database with the current NISbase search criteria
    a. Taxonomic Group
    b. Genus
    c. Species
    d. Common Name
  5. Ability to limit the returned result set based on the record limit parameter.
  6. Ability to return query results in XML following the NISbase format.
    -See http://invasions.si.edu/nemesis/SpQueryResults.dtd
  7. A static IP address on the server
  8. Creation of metadata for your database provider
  9. Acceptance by the NISbase charter members
    a. Verification that other provider requirements (1-8) have been successfully meet
    b. Acceptable of server response time
    c. Acceptable content

Once a query is submitted to the NISbase portal all selected database providers will be sent an http request of the following generic format.

  http://providerIPaddress/../providerpage?Parameter1=X&PArameter2=Y

The current parameters passed to the provider by the portal are…

  1. ComName - “Common Name” this should be treated as though it is surrounded by wildcards when querying your database.
  2. Genus – “Genus”
  3. Species – “Species”
  4. State – “State” This is optional. If a state parameter is sent and you don’t have State specific information, return a null result set to the portal. We will probably implement a “Country” parameter in a similar fashion.
  5. Group – “Taxonomic Group” This is a predetermined list of taxonomic groups hosted on the NISbase server. (It is currently under revision.)
  6. “Record set limit” – To prevent searches that return too many records to be reasonably processed, this establishes the maximum number of records each server may return.

The following example is a search request to the USGS-NAS database provider for all fish with a common name containing the word “oscar”

  http://nas.er.usgs.gov/queries/SpResultsXML.asp?ComName=oscar&Genus=&Spec
	ies=&State=&Group=&Size=10

If other providers were selected they would receive similar requests from the portal. In this example, the database provider page “SpResultsXML.asp” is required to interpret the search parameters and return valid XML results in the NISbase format. The following is the XML returned from the USGS-NAS database provider for the previously mentioned search. 

 <?xml version="1.0" encoding="iso-8859-1" ?> 
  <!DOCTYPE X SYSTEM "http://invasions.si.edu/nemesis/SpQueryResults.dtd">
   - <X>
   - <TAXON>
      <Taxa_SciName>Astronotus ocellatus</Taxa_SciName> 
      <Taxa_CommonName>oscar</Taxa_CommonName> 
      <Taxa_ITIS>169772</Taxa_ITIS> 
      <Taxa_Group>Fishes</Taxa_Group> 
   - <Record>
         <Record_Source>USGS-NAS</Record_Source> 
         <Record_ID>436</Record_ID> 
         <Factsheet_URL>http://nas.er.usgs.gov/queries/SpFactSheet.asp?
					speciesID=436</Factsheet_URL> 
         <Factsheet_Name>USGS-NAS FactSheet</Factsheet_Name> 
         <Collection_URL>http://nas.er.usgs.gov/queries/spCollections.asp?
					SpeciesID=436</Collection_URL> 
         <Collection_Name>USGS-NAS Collection Info</Collection_Name> 
      </Record>
    </TAXON>
  </X>

Provider Metadata

A provider is added to the NISbase portal by adding some basic metadata to an XML file called InvServers.xml. The portal search page accesses this information so that users can select the databases they wish to search and to ensure that the data is sent to the correct URLs. The current InvServers.xml file can be viewed at the following URL.
http://invasions.si.edu/nemesis/merge/InvServers.xml
Here is the current listing for the USGS-NAS database. 

 - <DATA_SOURCE>
    <NAME>USGS-NAS</NAME> 
    <NAME_LONG>NAS Database (Nonidigenous Aquatic Species Database)</NAME_LONG>
    <URL>http://nas.er.usgs.gov/queries/SpResultsXML.asp</URL> 
    <HOME>http://nas.er.usgs.gov/</HOME> 
    <OWNER>United States Geological Service</OWNER> 
    <LOGO>http://nas.er.usgs.gov/images/c_usgsid.gif</LOGO>
    <ECOSYSTEM>Freshwater</ECOSYSTEM> 
    <SCOPE>Animals</SCOPE> 
    <SPATIAL>All USA</SPATIAL> 
    <DATATYPES>Fact sheets; Collection Records; Maps; Images</DATATYPES> 
 </DATA_SOURCE>
  1. NAME – a short name for the database less than 12 characters long
  2. NAME_LONG – the full name of the database
  3. URL – the base url for the provider XML script page
  4. HOME – the home page for the database or project
  5. OWNER – the name of the organization that owns the database
  6. LOGO – a URL to a logo for the owner’s program
  7. ECOSYSTEM – keyword(s) describing the ecosystem that the database covers
  8. SCOPE – keywords(s) describing the general scope of the database (animals, plants, weeds, fish, etc…)
  9. SPATIAL – the spatial scope of the database (a region, country, etc.)
  10. DATATYPES – the types of information provided by the database.

Basic Provider Implementation

  1. Make species summaries and/or collections data accessible on the web as static files (static html, pdf files, etc.).
  2. Create an NIS database that contains only the information needed to search and return the NISbase required XML including the URLs to the species summaries and collection web pages.
  3. Write a basic script that retrieves the search parameters from the portal, queries the database, and returns XML in the NISbase structure.

Advanced Provider Implementation

  1. Species summaries and/or collections data are dynamically generated for the web.
  2. Database contains many tables but a query can be written to access the information needed to search and return the NISbase required XML.
  3. A custom script that retrieves the search parameters, queries the database, and returns the XML.

NISbase Portal Information

Portal Requirements:

  1. Apache Tomcat or other servlet server
  2. NISbase.war

The portal is responsible for creating the search page based on a list of available data providers. Searches from the portal are passed to each provider and when XML results are returned the portal merges the results from the providers and formats it into one HTML table of results. The portal is currently developed using JSP but other scripting languages may also be used.

Portal Installation:

  1. Install Apache Tomcat 4.X (java installation may be required)
  2. Place NISbase.war in the webapps directory of the Tomcat installation
  3. Point a browser to http://yourserver:8080/NISbase/

Portal Mirrors:

We encourage others to install the NISbase portal on their servers as it potentially lessens the load on the original portal server. We will make available a list of all known mirrored portals to allow users to find an alternative selection.

Thematic Portals:

We also encourage the development of thematic NISbase portals. An example thematic portal currently under consideration is NISbase-Pacific which will bring together NIS databases from Hawaii and other pacific islands. Other potential NISbase thematic portals might include NISbase-Weeds, NISbase-Fish, NISbase-Asia.

Thematic portals will likely have slightly different looks, sets of data providers, and modified search terms.

Collaborators | NISbase News| Structure & Operation | Documentation | Current Status | Next Steps | Citation | Disclaimer |