Pathway Tools FAQ

Table of Contents:

0. What are the system requirements -- hardware and software -- to use Pathway Tools?
1. Display Problems
   • 1.1. Motif version: Problems with window resizing, fonts, and breaks
   • 1.2. X-Display Emulation on Microsoft Windows PCs
   • 1.3. Debian Linux Error: Cannot open the display
2. Pathway Tools Hang or Freeze
   • 2.1. All-purpose Abort character Control-Z
   • 2.2. Unresponsive GUI
3. Running Pathway Tools as a WWW Server
   • 3.1. Graphics do not get generated
   • 3.2. Running server without staying logged in
4. PGDB Maintenance/Editing/Transfer
   • 4.1. Copying PGDBs from one computer to another
   • 4.2. Importing a .ocelot format PGDB into Pathway Tools
   • 4.3. How can I edit or otherwise update PGDBs built into Pathway Tools?
   • 4.4. Making a copy of a PGDB on one computer
   • 4.5. How can I convert my locally created file PGDB to a MySQL PGDB? (i.e. change the OrgID)
5. Windows (XP, NT, 2000) Specific Problems
   • 5.1. Heap Problems
6. Using the Lisp API
   • 6.1. Setting up Emacs to interact with Pathway Tools in Lisp Mode
7. User-Provided Ortholog Data
   • 7.1. I have ortholog data. How do I get it into Pathway Tools?
8. Configuring Pathway Tools
   • 8.1. Configuration for multiple curators
   • 8.2Configuration for PGDBs stored in multiple MySQL servers
   • 8.3. Configuring Proxy Server Access to Circumvent a Firewall for Patch Downloading
   • 8.4. Configuring for off-line, standalone mode (suppressing Internet access)
9. Running multiple instances of Pathway Tools
   • Command line arguments to run multiple instances in batch mode w/o using X11

0. What are the system requirements -- hardware and software -- to use Pathway Tools?

Please see the "Requirements" Section in the Pathway Tools Installation Guide.

Q: On Linux, I am having problems with windows: subwindows are the wrong size and cannot be resized, fonts look funny, and my system sometimes freezes when I try to use the mouse with Pathway Tools. What can I do?

A: The Unix Pathway Tools need Motif, which is a library of GUI widgets. Under Linux, the correct version needs to have been installed to avoid numerous problems. The symptoms described above can all be caused by using Lesstif, a clone of the Motif window toolkit, rather than OpenMotif, which is a free version of the official Motif library. Ptools-8.0 expects Motif to be available as the dynamically loadable library libXm.so.2 . Make sure that libXm.so.2 (located in either /usr/lib or /usr/X11R6/lib ) points to the OpenMotif version, not Lesstif.

Older Linux distributions tended to ship with Lesstif, in which case you need to download and install OpenMotif yourself. This will require access as the root user. Download OpenMotif from http://www.opengroup.org/openmotif/ . The version with the most bug fixes appears to be metrolink/openmotif-2.1.30-4_MLI.i386.rpm .

After installation, if you end up having both libXm.so.2 and libXm.so.3 , you probably need to delete the Lesstif libXm.so.2 , and make a symbolic link from libXm.so.3 to libXm.so.2 ( ln -s libXm.so.3 libXm.so.2 ).

Redhat-9.x Linux appears to be shipped with OpenMotif installed already. However, the version of the dynamically loadable library is different from what the Pathway Tools program expects. The following is an example of the failure symptom, when starting up the Pathway Tools:

Warning: Loading sys:climxm.so failed with error:
        libXm.so.2: cannot open shared object file: No such file or directory.

If the Pathway Tools image wants libXm.so.2 but the OpenMotif installation does not have that particular version link, please try to make a symbolic link to "fake" that version, to see whether that will work. Some variation of this example should do it (you will need to be the root user):

cd /usr/X11R6/lib
ln -s libXm.so.2 libXm.so

Then restart the Pathway Tools. This fix is known to work on Redhat-9.x.

1.2. X-Display Emulation on Microsoft Windows PCs

Q: How can I get the Unix Pathway Tools to display on my Microsoft Windows Laptop ?

A: The Unix Pathway Tools use the X-Windows windowing system that allows other Unix computers to display the Pathway Tools running on a central server. X Emulators exist that allow this remote display to work on Microsoft Windows computers as well. Not all emulators work equally well, though.

• Hummingbird Exceed is a commercial product. A Demo version can be downloaded, to be tried for free for 30 days. With version 7.0, SRI has had good luck.
• Reflection X is a commercial product. One of our users said it works well.
• Cygwin X/Free86 is free, but in fall 2003, SRI had bad luck with this one.
• Xwin32 (??)(any opinions, anyone ?)

Q: On Debian Linux, when I try to run Pathway Tools, I get an error message of the form "Error: Cannot open the display: <myhostname>:0". What is going on?

A: By default, the X Windows system on Debian Linux does not allow connections to the X server via TCP. You need to search through the config files in /etc/X11 and its subdirectories, and wherever you find the string "-nolisten tcp", delete it.

2. Pathway Tools Hang or Freeze

Q:I typed characters in the Navigator listener pane (at the bottom of the screen) and the Navigator command menus no longer function -- what do I do?

A: Type ^Z (hold down the Control key and then strike the "Z" key). The ^Z character is a convenient way to abort out of certain menus as well.

2.2. Unresponsive GUI

Q: Pathway Tools does not respond. How do I continue ?

A: If you find that the software hangs and does not respond to your input, try the following:

• Wait 30 seconds -- the software may be computing or performing memory management.

• Verify that there are no hidden popup windows waiting for input.

• Check if the NUMLOCK mode on your keyboard has been selected by pressing the NUMLOCK key. Selection of NUMLOCK mode interferes with almost all Pathway Tools operations.

• On some computers, your mouse pointer must lie within the window to which you wish your typing to be directed, or you must click on the window to which you wish your typing to be otherwise that window will not receive your typing.

• Look in the window in which you first started the Pathway Tools (the window with the Unix shell). If you see a listing similar to the following, then an error has occurred. Please file a bug report to tell SRI about the error.

  Restart actions (select using :continue): 
   0: Return to Pathway Tools command level 
   1: Pathway Tools top level 
   2: Exit Pathway Tools 
  [changing package from "USER" to "ECOCYC"] 
  [1] EC(3): 
  

  The software is in a state called a "break" and you are interacting with a debugger. You can probably resume operation by typing :continue followed by the number of the line on which the text "Pathway Tools top level" occurs. In the case shown here you would type :continue 1 followed by Return.

  If restarting did not work, try typing (exit) to exit from the Pathway Tools (if you exit, you will lose any KB updates in the current session that you have not already saved; to attempt to save those changes, type (save-kb) before typing (exit)). You will now be back to the operating system, and you can execute the Pathway Tools anew.

• You can try to interrupt Pathway Tools by typing the Control-C character in the original window in which you invoked the software (you must move the mouse to that window). Once the Pathway Tools receives the interrupt, you will see a listing similar to that shown in (3), and you can restart or halt the software as described under (3).

3. Running the Pathway Tools as a WWW Server

Q: My Ptools website doesn't display graphics.

A: The web server serves up graphics from the /tmp directory. If that directory doesn't exist, it needs to be created. The user account that runs Pathway Tools needs to have read-write access to the /tmp directory.

3.2. Running server without staying logged in

Q: I want to run the web interface of Pathway Tools without having to stay logged in. Please help.

A: Please see: How to logout after starting the web interface of Pathway Tools.

4. PGDB Maintenance

Q: How does one copy a file PGDB from one computer to another ?

Note that an easier way of moving PGDBs from one computer to another is to register the PGDB in the Pathway Tools registry from the first computer, and to then download the PGDB from the registry using the second computer. See the Pathway Tools User's Guide for details.

A: Please see: How to Copy a PGDB Between Computers.

4.2. Importing a .ocelot Format PGDB into Pathway Tools

Q: How do you obtain a PGDB from the BioCyc data file distribution, and import it into Pathway Tools?

Note that an easier way of importing PGDBs into Pathway Tools is to download them from the PGDB registry as described in the Pathway Tools User's Guide.

A: For instructions on loading .ocelot files directly into Pathway Tools, please see: How to Import a .ocelot Format PGDB into Pathway Tools.

4.3. How can I edit or otherwise update PGDBs built into Pathway Tools?

PGDBs that are bundled into the Pathway Tools distribution are usually "built-in" to the Pathway Tools binary executable. It is not possible to edit or otherwise update such PGDBs directly. Such updates cannot be saved, and if they were to be saved, they could not overwrite the built-in versions of each PGDB.

A PGDB that you either build yourself, get from someone as a flat-file copy, or obtain from the Pathway Tools Registry, will be editable.

However, one way that you can update a built-in PGDB is to save the PGDB out as a separate file PGDB, which can be edited. See xxx

4.4 Making a copy of a PGDB on one computer (i.e. change the OrgID)

• Sometimes you want to make changes to a PGDB, yet keep an unmodified copy of the PGDB around, so that you can compare the two versions, such as by running Pathway Tools in web mode, and using the Comparative Analysis tools.

• If you want to make a copy of any PGDB, including built-in PGDBs like MetaCyc, EcoCyc, or any of the Tier 2 PGDBS, make sure to heed the warning about the default-version file as found in the external documentation about copying a PGDB from one computer to another.
• You have to make sure to extend the shell variable PTOOLS_ORG_ROOTS to include the path where you are placing any newly-copied PGDBs.
• If you want to have the original PGDB and the newly-copied PGDB to be loaded into the same session of PTools, then you'll need to give the newly-copied PGDB a unique name & org-id. You'll have to:
  • If the original PGDB is a file KB, make sure that it has been saved to disk recently. If the original PGDB is a RDBMS KB, make sure to run (backup-ekb) via the PTools Lisp API with the original PGDB as the current KB.
  • Copy the original PGDB's directory structure to the new location
  • Change the directory name from foocyc to footestcyc.
  • Change the org-id in the footestcyc/[version]/input/organism.dat file to something unique, such as FOOTEST.
  • Modify the NAME & ABBREV-NAME attributes in the same file to signify that they are different than the original. This helps avoid confusion.
  • Don't forget to modify the STORAGE attribute in the same file to be FILE, not MYSQL or ORACLE.
  • Go into the footestcyc/[version]/kb directory, and move foobase.ocelot file to footestbase.ocelot .
  • Open up the footestbase.ocelot file, and change the :name attribute in the first s-expression (that starts with :OCELOT-KB) to the org-id (from above, FOOTEST) that you selected earlier.
  • In the footestbase.ocelot file, look for a top-level s-expression that starts a line with (FOO NIL (, where foo is the old org-id. Change it to the new org-id of the copied PGDB (i.e., FOOTEST).
  • In the data directory for the PGDB, you'll need to rename some files in order to have the nucleotide and protein sequence data display correctly in PTools. Effectively, whenever you see a file with the prefix foobase, it needs to be replaced with the prefix footestbase. Also, there should be a file with a suffix of .fsa, and a prefix of the Organism ID (i.e., foo). You'll need to change the prefix to the new Organism ID, such as footest.

A: MySQL is a free, open-source relational database management system. Using MySQL to store your PGDB instead of a file enables multi-user concurrent access, change logging, and in some cases faster load and save times. Because of the format in which the data is stored, however, it will not allow you to access the data in your PGDB by writing SQL queries. If you want to query a PGDB via SQL, you can load the PGDB into an instance of BioWarehouse using the BioCyc Loader. If you wish to programmatically access the data in Pathway Tools directly, you can use one of our Lisp, Perl or Java APIs.

MySQL can be obtained from http://www.mysql.com. We do not provide any support for obtaining or installing the base MySQL software. Instructions for configuring Pathway Tools to access MySQL and for installing the Pathway Tools schema in your MySQL database can be found here. There is also a link to these instructions from the Pathway Tools Installation Guide.

Once MySQL is installed and correctly configured, start up the Pathway Tools Navigator, and invoke PathoLogic from the Tools menu. To convert a particular PGDB to use MySQL select it in PathoLogic and then invoke the command Organism->Convert File DB to MySQL DB.

5. Windows (XP, NT, 2000) Specific Problems

These explanations apply to Allegro Common Lisp, which is the underlying software used by Pathway Tools. Pathway Tools may require a large amount of computer memory (RAM) to run some commands, to access large PGDBs, or just to start up. The data used by Pathway Tools is stored in a location called "the heap". When Pathway Tools is started it tries to reserve one large block of memory for its heap, probably around 700MB. If it cannot reserve this much memory, that value is automatically lowered. In such a case you may see a message of the form

Temporarily scaling back lisp reserved region from x to y bytes

lisp heap being relocated by yyyyy bytes

In other cases, you may see an error message of the form

Error: An allocation request for x bytes caused tenuring and a
       need for y more bytes of heap. The operating system will
       not make the space available because the address space reserved
       for the heap could not be increased.

• Check to see how much RAM and virtual memory you have. This can be done by going to the Windows Control Panel (usually from the start menu/settings), selecting "System" and then the "General" panel. This panel should show the amount of RAM. If it is less then 256MB, it may not be posible to run Pathway Tools with some PGDBs. Preferably it is 1GB or more (over 1000MB). If it is less, you can try to increase the virtual memory by going to the "Advanced" panel, then to the "Performance" section. You should see the amount of virtual memory currently selected. If it is less then 400MB, it should be increased to at least 700MB.
• Try to close down all other unneeded applications. This should reduce memory usage. The Windows Task Manager may help you to determine which applications are using large amount of memory. (The Task Manager is available by right-clicking the task bar, usually located at the bottom of your screen, then selecting it from the menu.)
• You should restart Pathway Tools after doing the previous two steps. If the heap problem persists, try applying the following step.
• The cause for lack of heap space may be due to one or more dynamic libraries -- known as DLLs -- that Windows cannot relocate. Unfortunately, on Windows, some DLLs may specify that they cannot be relocated in memory (some do this even when it is not the case). Pathway Tools requires a large block of contiguous memory for its heap. If a DLL specifies that it resides in the area Pathway Tools wants to use, Pathway Tools may give the above error message. Therefore, your first step is to find such DLLs (hopefully only one) and either deactivate the applications that use them (which could be applications started when you boot your computer) or permanently relocate the DLLs. The second option is more complicated and should only be done if you really need the application active while using Pathway Tools. To find the application(s) that have DLLs that conflict with Pathway Tools, please follow the instructions at the Franz FAQ. On that page, there is a discussion of a utility called the Process Explorer, provided on the Microsoft web site, that should be downloaded and installed on your Windows computer. Using this utility, you should locate any application which uses memory addresses too close to Pathway Tools, as described here. Relocating a DLL is not recommended and should only be done if the application cannot be deactivated. For more information on how to relocate a DLL, please follow the directions from the Franz FAQ.

Pathway Tools has a built-in Lisp environment that you can use to programmatically query and modify your Pathway / Genome Database. You can access the Lisp listener by starting up Pathway Tools like so: pathway-tools -lisp. Interacting with it via Emacs provides you with a fully-integrated IDE that provides debugging support, syntax high-lighting, and syntax-aware navigation commands. The recommended Common Lisp interaction platform for Pathway Tools users is SLIME, the Superior Lisp Interaction Mode for Emacs.

These instructions were generously provided by Jeremy Zucker.

1. Download SLIME: slime-current.tgz
2. Add this to your .emacs file:

   (setq slime-lisp-implementations
        '((ptools ("/usr/local/bin/pathway-tools" "-api" "-lisp") :init slime-init-command)))
   
   (add-to-list 'load-path "/path/to/slime")

3. Now, if you try to run M-x slime, you will get an error like this:
   Condition: Can't locate the module "SCM"
4. The current workaround for this is to download Allegro Common Lisp (ACL), and copy files.bu into the pathway-tools exe directory (shown below).
   Note, you do not need a license to download ACL. You only need one if you are planning to run ACL.
5. This is how you would do this on a Mac:
   cp /Applications/AllegroCL/files.bu /usr/local/pathway-tools/aic-export/pathway-tools/ptools/12.5/exe
6. Now, running M-x slime in emacs should just work.

First, we recommend that each PGDB that will be curated be stored in a single relational database management system (DBMS). Oracle and MySQL are the two DBMSs currently supported by Pathway Tools. All PGDBs can be stored within one DBMS server.

This configuration allows multiple curators to access and update the PGDB in parallel. The Pathway Tools that they run will access the DBMS through network queries.

Curators can run and access Pathway Tools in several possible ways:

• A separate instance of Pathway Tools can be installed on each curator's workstation.

• Curators can access a single shared instance of Pathway Tools via X-Windows. For example, a curator could run an X-window desktop on their Windows-NT computer, with Pathway Tools running on a separate Unix computer, and the Pathway Tools windows are transmitted via the network to the Windows-NT desktop.

• Curators with NFS access to a Pathway Tools instance can each run Pathway Tools on their own workstation, but can access a single Pathway Tools installation. This option is probably the optimal one since it requires only a single installation of Pathway Tools, yet it allows each curator to run Pathway Tools on a separate computer.

These institutions tend to offer an alternate mechanism for WWW access, going through a proxy server. Pathway Tools can download patches through a proxy server, if configured correctly.

To activate proxy server access, the shell environment variable PROXY has to be set to point to the proxy server and port, prior to launching Pathway Tools. For UNIX, it is best to do this in one of the user's shell init scripts such as .cshrc or .login. As an example, in csh syntax:

setenv PROXY example.hostname.com:8888

For Pathway Tools 27.0 , it should be possible to disable most of the connections described below, by calling the Lisp function (run-in-standalone-mode) . This can be accomplished by supplying the -eval '(run-in-standalone-mode)' commandline argument to the pathway-tools invocation script.

• Upon startup, Ptools will look for any new patches to download and install, to keep Ptools up to date. This can be prevented by supplying the -no-patch-download commandline argument to the pathway-tools invocation script. However, we recommend having an alternate access to the patches. For example, if a user runs into a software bug in Ptools, we will usually issue a patch for the fix, which then would have to manually downloaded and installed.

  The patches can be downloaded under the following URL: https://bioinformatics.ai.sri.com/ptools/VERSION/PLATFORM/patches/ , whereby VERSION is the release version of the Pathway Tools software. The operating system of the computer, on which Ptools is running, needs to be specified by PLATFORM, and the following choices exist:

  • Linux-64
  • MacOSX
  • MSWindows

  An example URL for the 27.0 version of Linux would be this: https://bioinformatics.ai.sri.com/ptools/27.0/Linux-64/patches/

  After the patch files have been downloaded, they have to be copied into the patches directory under the Ptools installation directory. The patches directory has a sub-directory, the name of which starts with bin-. The *.fasl patches need to go inside of this sub-directory. An example path for the 27.0 version of Linux would be this: pathway-tools/aic-export/pathway-tools/ptools/27.0/patches/bin-acl-10-1-amd64-linux/

• When running certain comparative operations, SRI's public ortholog server will be accessed, to obtain the ortholog data. This can be suppressed by uncommenting the parameter called Get-Orthologs-From-SRI and setting it to N. This parameter is found in the ptools-init.dat file. However, comparisons that rely on ortholog data will thereafter no longer run properly, unless a user supplies their own ortholog database. Such a MySQL database would first have to be prepared, and the corresponding access parameters in the ptools-init.dat file will have to be configured. See section 7.1. for more information.

• The PGDB Registry can be accessed, to check for newer versions of already installed PGDBs. This can be prevented by supplying the -standalone commandline argument to the pathway-tools invocation script.

• If curation is performed on a PGDB and Pubmed ID citations are added to comments in the editing tools, there are attempts to download the full citations (including authors, title, etc.) from Pubmed.

• The Genome Context server can be accessed from a gene page, if this was enabled in the ptools-init.dat file.

• When building new PGDBs by the "Pathologic" functionality, a server at SRI is queried initially, to obtain a unique ID for the new PGDB (to avoid potential namespace collisions). If there is no network access, this query should simply time out, and the user will need to pick their own ID, when prompted.

• -no-cel-overview (see Section 2.3)
• -no-web-cel-overview (see Section 2.3)
• -disable-metadata-saving (for Pathway Tools version 18.5+) -- eliminates contention for the metadata-kb. If you do provide this switch, building the metadata kb after building all PGDBs will speed Pathway Tools startup. To build the metadata kb, invoke Pathway Tools with the command-line argument -update-pgdb-metadata (for Pathway Tools version 22.5+), or invoke Pathway Tools with the -lisp command-line argument and then enter (update-pgdb-metadata-kb) at the lisp prompt.
• -nologfile (turns off an extra debugging log which can cause contention between processes)