The software used in the development and operation of COBRA prototype has two components. The first comprises of a set of programs used for testing data acquisition and communication between nodes. The second concerns the software which is used to operate the system. These are described separately in the sections following this overview.
The test programs were initially developed to study the capabilities of the system and specify the data acquisition hardware. They will be useful as diagnostic aids in future if the system malfunctions. They can also be adapted for further development of the system. For example, the data acquisition test programs can be used to test a new card which may have enhanced data acquisition features.
A different set of programs is used to operate COBRA prototype. These programs are still in a developmental stage and will evolve with time. The cluster is divided into five functional unit from an operational point of view. The cluster is controlled by a program called COBRA MASTER. The data acquisition is carried out by a program called SERVER, which also farms out data to the nodes which carry out the bulk of signal processing. These nodes run a program called CLIENT. The processed data from the client is assembled in a suitable form by COLLECTOR. For ease of operation, the user interacts with the machine (the entire cluster as a whole) through a graphical user interface (COBRA GUI). A brief description of these programs is included in the following sections.
This section provides a brief description of programs used for testing the data acquisition system and the communications. These are a set of independent programs which can be run on any node(s) of the cluster for a specific task.
Data acquisition test programs
The following is a brief list of available programs followed by a brief description of each of the program.
This program sets up, initialises an EDT card and starts continuous data acquisition from a PCI EDT card. The data is written to a ring buffer using DMA. The data is then written to a temporary file and re-read and is displayed as a time series. The program is used for testing acquisition using EDT and is used with a counter test rig connected to EDT.
This program sets up, initialises an acquisition card and starts continuous data acquisition. The data is then compared with an expected count to check if the acquisition is taking place correctly. If a byte is missed, it is signalled by the program. If 1 in 10000 byte is missed, the program exits. This program can be used to test the efficacy of the acquisition card as well as a one sided data transfer across the PCI bus. It is used with a counter test rig connected to the acquisition card. There are versions available for both EDT and ADLINK cards and it can easily be adapted for a new card.
This program sets up, initialises an EDT card and starts continuous data acquisition from a PCI EDT card. It also sets up COBRA prototype sampler card for acquisition. The observing bandpass is then obtained by computing the Fast Fourier Transform of the complex sampled data acquired through COBRA prototype sampler. This program is used with a signal generator. The sampler card operation can be tested by shifting the frequency of signal generator which is reflected in the output bandpass.
This program sets up, initialises an EDT card and starts continuous data acquisition from a PCI EDT card. It also sets up COBRA prototype sampler card for acquisition. The data is then written to an output file. This program can be used to record data from 42 ft. Together with 1 sec cal signal, it can be used to acquire useful data on a strong pulsar such as PSR B0329+54. This data can be subsequently used for testing the signal processing software.
This program carries out continuous data acquisition from a PCI card. The data is then sent to another PC using Ethernet TCP/IP socket at real time. The program is used for testing real time acquisition and transmission.
This program is used to test acquisition and transfer of data via a TCP/IP socket. It is used in conjunction with ACQ_SEND. It connects as a client socket to a server PC acquiring data. The data is read and written to an output file.
TCP/IP cluster communication programs
This is a set of programs which runs on machines designated USER, MASTER, SERVER and CLIENT. These programs establish basic TCP/IP connectivity between these machines and set up communication. Each program has a control section, a communication handler and individual communication controller for the other machine. These tasks are implemented as threads in these programs. These programs are called COBRA_COMMAND_TEST, COBRA_MASTER_TEST, SERVER_TEST and CLIENT_TEST. These implement a rudimentary chain of command from the user interface to individual components of software (which does nothing except display of messages). They are useful in testing the basic communication infraware which forms the backbone of COBRA prototype operation software. In a sense, this implements a low level MPI protocol.
COBRA prototype Operation software
These programs are used to operate the COBRA prototype as an integrated backend for the 42 ft telescope. These are still under development and some of these may evolve with time. The structure of this software was briefly described in the overview. The individual component are described here.
This program is the SERVER module for control and operation of the COBRA. It runs on the server machine and carries out data acquisition under MASTER control. It consists of a control thread, appropriate threads implementing communication with master and clients and the acquisition thread. First, TCP/IP sockets with the master and clients are set up and communication threads are initialized. Then, the acquisition thread is started which reads UTC on the server (UTC is maintained within 128 ms by NTP server) and initializes the headers for data blocks. The acquisition begins on the next 10 sec tick and the time is then kept on the sampling clock which is derived from the observatory frequency standard. The commands from master are interpreted by the control threads which also sends requests for clients to MASTER when a data block is available. Finally, the control thread directs blocks of data with appropriate wings to the client indicated by the MASTER. Each data block is appended with a header including the time stamp. It is launched as daemon on boot up.
In future, the TCP/IP communication will be replaced by MPI calls simplifying the structure somewhat. The server can then be fired from the master node instead of running as a daemon. The acquisition thread will become an independent root process starting at boot time sharing data through shared memory. The MPI part is under development.
This program is the CLIENT module for control and operation of the COBRA. It runs on the client machines and carries out data analysis under MASTER control. It consists of a control thread, appropriate threads implementing communication with master and server and the signal processing threads. First, TCP/IP sockets with the master and server are set up and communication threads are initialized. Then, the signal processing and control threads are initialized. Before an observation is started, the control thread receives set up information from MASTER. This is used to set up the signal processing. At present, one can select either coherent de-dispersion and fold or filter-bank and incoherent de-dispersion and fold. The latter is under development. Once, the observation is started, the data is received from servers and passed to appropriate signal processing module. A brief description of these modules is given below.
Coherent De-dispersion and Fold
This module implements an inverse filter to remove the dispersion smear introduced by inter-stellar medium. During set up, the observing frequency, the bandwidth and DM is used to calculate the dispersion time across the passband, which is given by the following formula
In order to eliminate the dispersion smear, a time slice twice the dispersion smear is convolved with a filter function given by
where the signal is assumed to be complex sampled. Thus, this filter function can be computed from the observational parameters, which is done at set-up time.
The input data is broken up into overlapping segments and each segment is convolved with the filter function. The convolution is carried out in the frequency domain. Each segment is transformed and multiplied by the filter function followed by an inverse transform. The transform operation are carried out using Fast Fourier Transform in West (FFTW) algorithm which has a built in optimisation for a given hardware. There is an option in the software to split the band into sub-bands before the inverse transform. This feature is useful for zapping data with narrow-band RFI.
The polynomial coefficients generated using the ephemeris of pulsar are transmitted by the master during the setup phase. These are interpolated to obtain the phase and period of the pulsar at the observing frequency and the coherently de-dispersed data is binned accordingly by a routine. The four product profiles are produced for each data block processed by the signal processing module and stored on the disk with the time stamps.
Filter-bank and incoherent de-dispersion.
This module implements an incoherent de-dispersion algorithm using a software filter-bank approach. The data stream is split into N frequency channels by carrying out a Fast Fourier transform. The signal is then detected and the powers so obtained in N Channels represent N dispersed time series. These are shifted by appropriate samples to align them and are then collapsed. The output is folded in a manner similar to that described in the last section.
This modules coordinates the operation of the entire cluster. It accepts operational commands from observation control ( COBRA GUI) in Arthur. It also receives set up and timing information from Arthur. These commands and information is then routed appropriately to data servers and front end control nodes (1 in this case). Likewise, the timing information is routed to Process nodes. It also keeps the status information of all the compute nodes and a request queue. Once the data acquisition is begun, it supplies the ID of next available compute node to a data server. The required timing information and parameters are sent to each compute node in the set up phase by this program. It can read input commands either from a file (to implement file based communication) or TCP/IP sockets.
COBRA GUI or command module
This is a test version implementing three commands - SETUP, BEGIN OBSERVATION and END OBSERVATION. It is used for developmental purposes only. It will be eventually replaced by a TCL/TK interface which is described elsewhere.
This document was generated by Duncan Lorimer on November 14, 2001