p3d README

Version information:
  This is the release version 1.1 of p3d. All parts of the tool work, although
  some parts require precise operation in order to work as intended.

  $Id: README 159 2010-04-16 14:26:46Z christersandin $


Table of Contents:
------------------

1.0) Introduction
 * 1.1 About p3d
 * 1.2 Installing p3d

2.0) Contact
 * 2.1 Funding acknowledgments
 * 2.2 Collaborators
 * 2.3 Reporting Bugs
 * 2.4 Feature Requests

3.0) Software Platform
 * 3.1 Coding Philosophy

4.0) Supported Instruments

5.0) Features

6.0) Recompiling the binary metafile for the IDL Virtual Machine


1.0) Introduction:
------------------

1.1) About p3d:
---------------

1.2) Installing p3d:
--------------------
Installing p3d requires a few steps depending on if you only intend to use it
with the IDL Virtual Machine (see 3.0) or if you want to use it and develop it
using your own IDL license.

In any case you need to have IDL, version>=6.2, installed on your system.
IDL can be downloaded from the following web page: http://www.ittvis.com.
See the relevant IDL documents on how to install IDL, and on how to acquire
and setup an IDL license (if you require one).

 i) Acquiring the p3d software

 You can either download one of the prepared releases of p3d from the program
 files area (https://sourceforge.net/projects/p3d/files/) or you can download
 the latest bleeding edge version using subversion. The latter approach is
 preferred if you want new functionality, which has not yet been included in
 the released versions. Although note that there is no guarantee that the code
 of subversion will be bug free (it is being corrected as time permits).

 In order to download p3d from the subversion repository at sourceforge.net the
 first time you first open a terminal and change directory to where you would
 like to have the software. Thereafter you type the following
   svn checkout https://p3d.svn.sourceforge.net/svnroot/p3d/trunk p3d
 whereby all files are placed in the directory 'p3d/' (you can of course use
 any name on this directory).

 In order to update p3d you should at any time change directory to p3d/ (or
 whatever name you have used) and type 'svn update'. For any additional
 functionality of subversion please see its web page
 http://subversion.tigris.org/.

 ii) Using the IDL Virtual Machine

 There is a (bash-)script provided to run p3d using the IDL Virtual Machine on
 all UNIX-type platforms. Although, before you can use the script you have to
 prepare your system in a few steps.

 After you have installed IDL you have to setup your environment to tell
 the system where it can find IDL. The environmental variable IDL_DIR must be
 set to the name of the directory where your IDL installation resides.
 Depending on your system shell you can add the following line to your
 ~/.bashrc (bash shell):

        export IDL_DIR=/path/to/IDL/

 or ~/.cshrc (C shell):

        setenv IDL_DIR /path/to/IDL/

 where the tilde character (~) denotes your home-directory.

 If you have python installed (with the name 'python') the p3d startup script
 will check if you have the right version of IDL installed (>=6.2).

 The environmental variable p3d_path must be set to the name of the directory
 where you have copied the p3d distribution. Additionally you might want to
 set the environmental variable p3d_data_path to the name of the (base)
 directory where you keep the data that you will work with. You may also want
 to add the vm/ directory to you path variable so that you don't have to enter
 the full path to where p3d resides every time you want to run it. Depending on
 your system shell you can add the following lines to a suitable place in your
 ~/.bashrc file (bash shell; Bourne Shell type environment):

        export p3d_path=/path/to/p3d/
        export p3d_data_path=/path/to/IFU/data/   # Optional
        export PATH=$PATH:${p3d_path}/vm          # Optional

 or ~/.cshrc (C shell):

        setenv p3d_path /path/to/p3d/
        setenv p3d_data_path /path/to/IFU/data/   # Optional
        setenv PATH $PATH:${p3d_path}/vm          # Optional

 In order to start p3d you thereafter enter the following in a terminal
 (if you added ${p3d_dir}/vm to your system path): 

        p3d_vm.sh

 or, if you didn't add ${p3d_dir}/vm to your path:

        ${p3d_dir}/vm/p3d_vm.sh

 Note! The bash-script p3d_vm.sh checks if the required IDL binary meta file
       p3d.sav is present in the ${p3d_path}/vm/ directory. If it exists IDL
       is started using that binary file.

 Note! If you have an IDL license, and want to recompile the binary for use
       with the IDL Virtual Machine you can use the provided script
       ${p3d_dir}/vm/p3d_tool_makevm.sh. See below (6.0) for further
       instructions.


 iii) Using an IDL license

 In order to run p3d using a license it is necessary to tell IDL where it can
 find all required source files. In addition to all files of the p3d
 distribution the files of the following sources are required:

   Astro-lib (http://idlastro.gsfc.nasa.gov/homepage.html)
     All files (actually a subset, but downloading all is easier [for us]).

   Mpfit (http://purl.com/net/mpfit)
     mpfit.pro and mpcurvefit.pro

 These files can be put anywhere, but if you would like to use the provided
 scripts when generating a binary metafile for use with the IDL Virtual
 Machine (if you made any changes to the code) it is recommended to download
 these files to ${p3d_path}/contrib/astro-lib (for the files of Astro-lib) and
 ${p3d_path}/contrib/ (all other contributing files). In order to download the
 files to these directories automatically use the provided script:
   ${p3d_path}/vm/p3d_acquire_xtools.sh


 When all files are downloaded, or if they are already present on the system,
 it is necessary to setup IDL to find these files. The most straightforward
 way is to set the environmental variable IDL_STARTUP to the name of an IDL
 program where the necessary information is defined. The following lines can
 be used to do this (the filename IDL_STARTUP.PRO can of course be chosen
 freely):

 For a Bourne Shell type environment, add these lines to your ~/.bashrc
     export IDL_STARTUP=/path/to/IDL_STARTUP.PRO # Modify to fit your system
     export p3d_data_path=/path/to/IFU/data/     # Optional
 
 For a C Shell type environment, add these lines to your ~/.cshrc
     setenv IDL_STARTUP /path/to/IDL_STARTUP.PRO # Modify to fit your system
     setenv p3d_data_path /path/to/IFU/data/     # Optional

 Thereafter add the following lines to IDL_STARTUP.PRO:
     astrolib_path='/path/to/astrolib/root/directory' # Modify to fit your sys.
     markwardt_path='/path/to/markwardt/files/directory' # Modify...
     p3d_path='/path/to/p3d'                          # Modify to fit your sys.
     !path=markwardt_path+':'+astrolib_path+'/pro:'+!path
     !path=p3d_path+':'+p3d_path+'/routines:'+!path

 If you chose to put the additionally required files under the root directory
 of p3d, as we recommended above, you could add the following lines to
 IDLSTARTUP.PRO instead:
       p3d_path='/path/to/p3d'                       # Modify to fit your sys.
       !path=p3d_path+'/contrib:'+p3d_path+'/contrib/astro-lib/pro:'+!path
       !path=p3d_path+':'+p3d_path+'/routines:'+!path

 Note! The exact order that paths are specified in is important. Also, observe
       that no extra directory separator ('/') should be used in the final
       position of the specification of the paths (i.e. '/path/to/p3d' not
       '/path/to/p3d/').


 p3d can now be started from the idl prompt using the command
       IDL> p3d

 If you'd like to know the available options you could just have a look in
 the program file (using your favorite editor, or e.g. more or less), or type:
       IDL> p3d,/help


2.0) Contact:
-------------

All collaborators, unless noted otherwise, have the following affiliation:

    Astrophysikalisches Institut Potsdam (AIP)
    Instrumentation
    An der Sternwarte 16
    DE-14482 Potsdam
    GERMANY  
    Tel: +49-(0)331-74990
    http://www.aip.de

p3d project web site:

    http://p3d.sourceforge.net

2.1) Funding acknowledgments:
-----------------------------
This project has been made possible through funds from the following sources:
   BMBF: 03Z2A51, 05A08BA1

2.2) Collaborators:
-------------------

Lead Programmer:
   Christer Sandin   o If you have comments of bugs or feature requests, which
                       are directly related to p3d, and that you cannot add to
                       the tracker at the project web site (see below for
                       instructions) then please send them to:
                          Christer Sandin <CSandin@aip.de>

Additional Programming:
   Joris Gerssen     o Improved the treatment of the wavelength calibration in
                       order to make it more intuitive than it used to be.
                     o Included support for reading and writing output in the
                       e3d-format.
                     o Started the work to write code to fit profiles for the
                       optimal extraction.

   Peter Weilbacher  o Included an option to calibrate the dispersion mask
                       against sky emission lines.

Testing of Software:

   Ana Monreal-Ibero (ESO)
      PMAS/LARR

Original Programmers/Authors:
   Thomas Becker     o Thomas wrote the first version of many parts of p3d for
                       a large number of instruments. This public release
                       version is based upon the earlier variant of p3d that
                       was called p3d_online.

   Petra Böhm        o Maintained all previous versions of p3d (including
                       p3d_online, ppak_online, virus_online, p3d, etc.), she
                       also fixed bugs and wrote various supporting documents.

   Martin Matthias Roth
                     o Martin worked with software design of early versions of
                       p3d.

Additional acknowledgments:

   Luzma Cairós      o Tested the dispersion mask creation tool thoroughly and
                       provided many tips on its improvement.
   Ole Streicher     o Has provided p3d with a first version of the tutorials
                       at the project web site.
   Stefano Ciroi (Padova)
                     o Figured out how to setup a basic configuration of p3d to
                       be used with the BTA/MPFS (3kx2k CCD) IFU.

2.3) Reporting Bugs:
--------------------
In order to report any kind of bug we urge you to use the bug tracker that is
available at the p3d project web site (Develop/Tracker/Bugs). This way your
bug will be seen by all users and programmers - and it won't be forgotten.

At the bug tracker page you can search previously reported bugs, to see if
your bug has already been reported. Or you can open your own bug description
(by clicking on "Add new"). Doing this, please make sure you are using the
latest version on the subversion trunk of p3d (or specify the version of the
release you are using). Thereafter specify a category for the bug, and provide
a short summary of the problem, and most importantly, describe the problem with
enough detail that somebody who does not use your setup and data files is able
to follow your arguments. Bug reports without a useful description will be
assigned a low priority, or even be deleted. If you think it would help us,
attach all relevant data files to your report (please compress fits-files using
tar/gzip or zip to save space). Additionally, please specify a reasonable
priority to your bug. While our intention is to correct serious bugs, which
prevent p3d from producing spectra, as soon as possible, we will assign GUI-
related issues a lower priority.

Important note. When reporting a bug, turn on the debugging mode (/DEBUG) of
p3d and paste the resulting output from the IDL terminal/window (once the bug
has been reached) into your bug description. The reason we ask you to do this
is that the information that is returned by the error handler when the debug
keyword is unset only provides the name of the file where the problem occurred,
and an IDL-specific number. Such sparse information is much more difficult to
interpret than a full output.

2.4) Feature requests:
----------------------
If p3d is missing some functionality that you would very much like to have
included then you can make a feature request using the tracker on the p3d
project web site (Develop/Tracker/Feature request). Following the description
on how to add a new bug description we ask you, also in this case, to be as
detailed as possible. Please understand that we are a team with limited
resources. If your request concerns a huge modification, which requires many
days, or even months, to code it might not happen at all. On the other hand, we
are highly interested in taking on project members who are interested in
contributing to p3d with their own time and coding. Please drop us a note if
you think this could apply to you.

3.0) Software Platform:
-----------------------
p3d is fully based on the language of the proprietary tool Interactive Data
Language (IDL; cf. http://www.ittvis.com) and cannot be used if IDL is not
installed on the system. Although it is necessary to have an IDL license in
order to modify code, p3d can be run without an IDL-license using the supplied
binary metafile ('vm/p3d.sav'). In this case the so-called IDL Virtual Machine
is used to run p3d. p3d has been implemented in such a way that it can be used
with full functionality with the Virtual Machine.

Since p3d is based fully on IDL it can be used on any platform that is
supported by IDL. Note, however, that the installation procedure differs
for an installation on a Windows machine compared to that on the other
platforms supported (the windows version of the installation instructions is
not yet implemented).

3.1) Coding Philosophy:
-----------------------
p3d is a large, and growing, code. In order to keep it manageable we have
applied rules of coding practice; use of this practice has so far made
debugging easy as bugs are found quickly.

These are the rules, which we would ask any project programming member to
follow:
  o All files must have filenames which conform to: p3d_task_name.pro
  o Code layout
     + All files must have a fully explained header.
     + All routines use the options: compile_opt hidden,IDL2
     + The maximum number of character columns per line is 79.
     + All blocks are indented by 2 characters.
     + Comment lines begin with ';;'
     + Integers are always specified as long with L, e.g. '0L'.
         Exceptions: VERBOSE and ERROR variables use integers.
     + All commands and statements are written with small characters.
  o Common blocks are only allowed for variables which obviously
    will never ever change, i.e. pretty much never.
  o An error handler is setup for every routine, unless the user keyword DEBUG
    is set.
  o Logging comment statements must always use 'p3d_misc_logger'.
  o Input parameters should be checked for consistency in terms
    of type, size, value and range, as far as possible.
  o All command keywords are for clarity specified in full, i.e.
    no abbreviations (for example, /XSTYLE not /XST).
  o Text widgets must include error checking to ensure that the
    entered values are of the right type and range.
  o All widgets in GUI tools with a status line must include the
    UNAME field that specifies what the widget in questions does.

Issues related to the subversion repository of p3d:
  o No file on the trunk should be committed before the modifications are
    confirmed to work as intended.
  o All commits to the repository should include a describing message:
       Example: svn ci --message "Corrected/modified this and that." <file>
  o Larger modifications should use a branch, and not the trunk.
  o The idea is to copy the trunk to branch/ in preparations for a
    release. Then we only make bug fixes on that branch before we
    copy the final release version to tags/.


4.0) Supported Instruments:
---------------------------
We have developed p3d to make it a general tool. General in the sense that
it can be used with any Integral Field Unit (IFU). However, before p3d can be
used with any instrument it must be properly setup. p3d has been setup, tested,
and found to work well with the following instruments:

---The following needs editing!---

  PMAS - is mounted on the Calar Alto 3.5m telescope
              PMAS :: http://www.aip.de/groups/opti/pmas/OptI_pmas.html
        Calar Alto :: http://www.caha.es

    LARR (Lens Array) :: 16x16 square-shaped elements that is used with either
     the 0.5"x0.5" or the 1.0"x1.0" magnification settings.

    PPAK () :: 331 circular elements, each of 2.8" diameter.

    Since October 2009 PMAS has been equipped with a new 4kx4k CCD, replacing
    the previous 2kx4k chip. p3d supports data of both CCDs. In order to use
    either setup you can select them in the menu upon startup of p3d, or
    use the input parameters: /larr2k, /larr4k, /ppak2k, or /ppak4k.


  VIRUS-P - is mounted on the Harlan J. Smith 2.7m telescope
     McDonald Observatory ::
                   http://www.as.utexas.edu/mcdonald/facilities/2.7m/2.7.html

     VIRUS-P is a prototype for the VIRUS instrument that is made for the
            Hobby Eberly Telescope.

     VIRUS-P (bundle 2) :: has 246 circular spatial elements
     VIRUS-P (bundle 1) :: has 247 circular spatial elements

  SPIRAL

  VIMOS

  FLAMES/ARGUS

  IMACS

  GMOS

  ...


5.0) Features
-------------


6.0) Recompiling the binary metafile for the IDL Virtual Machine:
-----------------------------------------------------------------
The simplest way to (re-)compile the binary metafile that is used with the IDL
Virtual Machine is to use the provided scripts in ${p3d_path}/vm/. Note that in
order to recompile p3d you must have an IDL license. You could use the
following steps from a terminal:

 i) Set the p3d root directory and IDL_DIR environmental variables. In a Bourne
    Shell type environment (e.g. bash) type:
        export p3d_path=/path/to/p3d/
        export IDL_DIR=/path/to/IDL/ # IDL version >=6.2 is required!

    In a C Shell type environment (e.g. csh) type:
        setenv p3d_path /path/to/p3d/
	setenv IDL_DIR /path/to/IDL/ # IDL version >=6.2 is required!

 ii) Download the required auxiliary files. You could, most conveniently, use
     the provided script:
        ${p3d_path}/vm/p3d_acquire_xtools.sh

     Note! The binary metafile generation script requires the additional files
           to be present in ${p3d_path}/contrib/astro-lib (with all .pro-files
           in ${p3d_path}/contrib/astro-lib/pro/) and ${p3d_path}/contrib/.

 iii) Change directory to ${p3d_path} and generate the binary metafile
      ${p3d_path}/vm/p3d.sav. You could, most conveniently, use the provided
      script:
        pushd ${p3d_path} > /dev/null
        ${p3d_path}/vm/p3d_tool_makevm.sh
        popd > /dev/null

 Note! The binary metafile generation script must be executed from the p3d
       directory ${p3d_path}!

 Note! If there is any problem in the binary metafile generation process please
       check the output of the scripts to find out what the problem is.
